512 lines
12 KiB
Plaintext
512 lines
12 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Exercise 1"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import nltk\n",
|
|
"from nltk import word_tokenize, pos_tag"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Classifiers\n",
|
|
"note: for model1 and model3 you can try different classifiers: Hidden Markov Model, Logistic Regression, Maximum Entropy Markov Models, Decision Trees, Naive Bayes, etc.. __choose one!__"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from sklearn.tree import DecisionTreeClassifier\n",
|
|
"from sklearn.feature_extraction import DictVectorizer\n",
|
|
"from sklearn.pipeline import Pipeline"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1. model1 = your POS tagger model (english)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{'word': 'bims', 'length': 4, 'is_capitalized': False, 'prefix-1': 'b', 'suffix-1': 's', 'prev_word': 'i', 'next_word': 'der'}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def features(sentence, index):\n",
|
|
" return {\n",
|
|
" 'word': sentence[index],\n",
|
|
" 'length': len(sentence[index]),\n",
|
|
" 'is_capitalized': sentence[index][0].upper() == sentence[index][0],\n",
|
|
" 'prefix-1': sentence[index][0],\n",
|
|
" 'suffix-1': sentence[index][-1],\n",
|
|
" 'prev_word': '' if index == 0 else sentence[index - 1],\n",
|
|
" 'next_word': '' if index == len(sentence) - 1 else sentence[index + 1]\n",
|
|
" }\n",
|
|
"\n",
|
|
"print(features(\"halli hallo i bims der Programmierer\".strip().split(\" \"), 3))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2. model2 = pre-trained POS tagger model using NLTK (maxentropy english)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 3. model3.x = rule-based classifiers (x = 1 to 5)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 4. model4 = your POS tagger model (not english)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 5. model5 = pre-trained POS tagger model using RDRPOSTagger 1 or TreeTagger 2 (not english)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Corpora\n",
|
|
"note: data split for training/test = 0.8/0.2 (sequencial)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### 1. X1 = nltk.corpus.treebank (english)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[nltk_data] Downloading package treebank to\n",
|
|
"[nltk_data] /Users/Carsten/nltk_data...\n",
|
|
"[nltk_data] Package treebank is already up-to-date!\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"nltk.download('treebank')\n",
|
|
"x1 = nltk.corpus.treebank"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### 2. X2 = nltk.corpus.brown (english)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[nltk_data] Downloading package brown to /Users/Carsten/nltk_data...\n",
|
|
"[nltk_data] Package brown is already up-to-date!\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"nltk.download('brown')\n",
|
|
"x2 = nltk.corpus.brown"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### 3. X3 = other language (not english)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"#nltk.download('brown')\n",
|
|
"#x3 = other language"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Task 1\n",
|
|
"* get results for english (plot a graph with all classifiers x results)\n",
|
|
" * performance 1.1 = model1 in X1"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"##### Generate Training and Testdata\n",
|
|
"1. split annotaed sentences into training and testdata\n",
|
|
"2. split trainingdata into input data and teacherdata\n",
|
|
" *input is the feature vector of each word\n",
|
|
" *output is a list of POS tags for each word and sentences"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"got 3131 training sentences and 783 test sentences\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"#to generate trainingsdata, delete the assigned tags as a function\n",
|
|
"def untag(tagged_sentence):\n",
|
|
" return [w for w, t in tagged_sentence]\n",
|
|
"\n",
|
|
"#object including the annotated sentences\n",
|
|
"annotated_sent = nltk.corpus.treebank.tagged_sents()\n",
|
|
"\n",
|
|
"#to split the data, calculate the borders for ratio\n",
|
|
"cutoff = int(.8 * len(annotated_sent))\n",
|
|
"training_sentences = annotated_sent[:cutoff]\n",
|
|
"test_sentences = annotated_sent[cutoff:]\n",
|
|
"\n",
|
|
"#show the amount of sentences\n",
|
|
"print(\"got \",len(training_sentences),\" training sentences and \", len(test_sentences), \" test sentences\")\n",
|
|
"\n",
|
|
"#for training split sentences with its tags into y (for a sentences its resulting tags for each word) and transform sentences and x as a list of the features extracet for echt word in the sentences\n",
|
|
"def transform_to_dataset(tagged_sentences):\n",
|
|
" X, y = [], []\n",
|
|
" for tagged_sentence in tagged_sentences:\n",
|
|
" for index in range(len(tagged_sentence)):\n",
|
|
" X.append(features(untag(tagged_sentence), index))\n",
|
|
" y.append(tagged_sentence[index][1]) \n",
|
|
" return X, y\n",
|
|
"\n",
|
|
"#trainings inputset X and training teacher set y\n",
|
|
"X, y = transform_to_dataset(training_sentences)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"source": [
|
|
"#### Implementing a classifier\n",
|
|
"relevant imports\n",
|
|
"* decision tree as the AI for classfing\n",
|
|
"* dict vercorizer transforms the feature dictionary into a vector as the input for the tree"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from sklearn.tree import DecisionTreeClassifier\n",
|
|
"from sklearn.feature_extraction import DictVectorizer\n",
|
|
"from sklearn.pipeline import Pipeline"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Pipeline manages vectorizer and classifier"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"clf = Pipeline([\n",
|
|
" ('vectorizer', DictVectorizer(sparse=False)),\n",
|
|
" ('classifier', DecisionTreeClassifier(criterion='entropy'))\n",
|
|
"])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"##### Calculate performance 1.1 \n",
|
|
"* fit the decision tree for a limited amount (size) of training \n",
|
|
"* test data and compare with score function on testdata"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"training OK\n",
|
|
"Accuracy: 0.880832376865\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"size=10000\n",
|
|
"clf.fit(X[:size], y[:size])\n",
|
|
" \n",
|
|
"print('training OK')\n",
|
|
" \n",
|
|
"X_test, y_test = transform_to_dataset(test_sentences)\n",
|
|
"\n",
|
|
"performance1_1 = clf.score(X_test, y_test)\n",
|
|
"\n",
|
|
"print(\"Accuracy:\", performance1_1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"##### Calculate other performances"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"performance1_2 = 0\n",
|
|
"performance1_3 = 0\n",
|
|
"performance1_4 = 0\n",
|
|
"performance1_5 = 0\n",
|
|
"performance1_6 = 0"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Using the classifier\n",
|
|
"for results the link of pos_tags:\n",
|
|
"https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"3.6.3\n",
|
|
"checking...\n",
|
|
"[('Hello', 'NNP'), ('world', 'VBD'), (',', ','), ('lets', 'NNS'), ('do', 'VB'), ('something', 'VBG'), ('awesome', 'NN'), ('today', 'NN'), ('!', 'CD')]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def pos_tag(sentence):\n",
|
|
" print('checking...')\n",
|
|
" tagged_sentence = []\n",
|
|
" tags = clf.predict([features(sentence, index) for index in range(len(sentence))])\n",
|
|
" return zip(sentence, tags)\n",
|
|
"\n",
|
|
"import platform\n",
|
|
"print(platform.python_version())\n",
|
|
"\n",
|
|
"print(list(pos_tag(word_tokenize('Hello world, lets do something awesome today!'))))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Results for Task 1\n",
|
|
"* get results for english (plot a graph with all classifiers x results)\n",
|
|
" * performance 1.1 = model1 in X1\n",
|
|
" * performance 1.2 = model2 in X1\n",
|
|
" * performance 1.3.x = model3.x in X1\n",
|
|
" * performance 1.4 = model1 in X2\n",
|
|
" * performance 1.5 = model2 in X2\n",
|
|
" * performance 1.6.x = model3.x in X2"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 22,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"High five! You successfully sent some data to your account on plotly. View your plot in your browser at https://plot.ly/~carsten95/0 or inside your plot.ly account where it is named 'basic-bar'\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<iframe id=\"igraph\" scrolling=\"no\" style=\"border:none;\" seamless=\"seamless\" src=\"https://plot.ly/~carsten95/0.embed\" height=\"525px\" width=\"100%\"></iframe>"
|
|
],
|
|
"text/plain": [
|
|
"<plotly.tools.PlotlyDisplay object>"
|
|
]
|
|
},
|
|
"execution_count": 22,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import plotly\n",
|
|
"plotly.tools.set_credentials_file(username='carsten95', api_key='vElf5IOxiFheQdjTxjXW')\n",
|
|
"plotly.__version__\n",
|
|
"import plotly.plotly as py\n",
|
|
"import plotly.graph_objs as go\n",
|
|
"\n",
|
|
"data = [go.Bar(\n",
|
|
" x=['performance 1.1', 'performance 1.2', 'performance 1.3', 'performance 1.4', 'performance 1.5' , 'performance 1.6'],\n",
|
|
" y=[performance1_1, performance1_2, performance1_3, performance1_4, performance1_5, performance1_6]\n",
|
|
" )]\n",
|
|
"\n",
|
|
"py.iplot(data, filename='basic-bar')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|