2018-05-09 10:47:51 +02:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"# NLP-LAB Exercise 02 by Jonas Weinz (2571421)\n",
2018-05-09 16:50:59 +02:00
"## links:\n",
"\n",
"* Article: https://miguelmalvarez.com/2017/03/23/how-can-machine-learning-and-ai-help-solving-the-fake-news-problem/\n",
" * corresponding code: https://github.com/kjam/random_hackery/blob/master/Attempting%20to%20detect%20fake%20news.ipynb\n",
"\n",
"* Tutorial on Datacamp: https://www.datacamp.com/community/tutorials/scikit-learn-fake-news\n",
"\n",
"* liar dataset paper: https://www.cs.ucsb.edu/~william/papers/acl2017.pdf\n",
2018-05-17 10:38:34 +02:00
" * dataset: https://www.cs.ucsb.edu/~william/data/liar_dataset.zip\n",
"\n",
"* Mex Vocabulary: http://jens-lehmann.org/files/2015/semantics_mex.pdf"
2018-05-09 19:13:08 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-09 16:50:59 +02:00
"## Dependencies for this Notebook:\n",
2018-05-09 10:47:51 +02:00
"* library [rdflib](https://github.com/RDFLib/rdflib)\n",
2018-05-09 19:13:08 +02:00
" * install: `pip3 install rdflib`\n"
2018-05-09 10:47:51 +02:00
]
},
{
"cell_type": "code",
2018-05-09 19:13:08 +02:00
"execution_count": 1,
2018-05-09 10:47:51 +02:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
2018-05-11 12:19:53 +02:00
"%pylab inline"
2018-05-09 16:50:59 +02:00
]
},
{
"cell_type": "code",
2018-05-11 12:19:53 +02:00
"execution_count": 2,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import itertools\n",
"import sklearn.utils as sku\n",
2018-05-09 19:13:08 +02:00
"from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer, HashingVectorizer\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.linear_model import PassiveAggressiveClassifier\n",
"from sklearn.naive_bayes import MultinomialNB\n",
"from sklearn import metrics\n",
"import matplotlib.pyplot as plt\n",
2018-05-13 12:44:36 +02:00
"from pprint import pprint as pp\n",
"from IPython.display import display, Markdown, Latex\n",
"import collections\n",
"import traceback\n",
2018-05-17 12:06:34 +02:00
"import os\n"
2018-05-09 16:50:59 +02:00
]
},
2018-05-09 19:13:08 +02:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tools used later"
]
},
{
"cell_type": "code",
2018-05-11 12:19:53 +02:00
"execution_count": 3,
2018-05-09 19:13:08 +02:00
"metadata": {},
"outputs": [],
"source": [
"def plot_confusion_matrix(cm, classes,\n",
" title,\n",
" normalize=False,\n",
" cmap=plt.cm.Blues):\n",
" fig_1, ax_1 = plt.subplots()\n",
" \"\"\"\n",
" See full source and example: \n",
" http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html\n",
" \n",
" This function prints and plots the confusion matrix.\n",
" Normalization can be applied by setting `normalize=True`.\n",
" \"\"\"\n",
" plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
2018-05-13 13:36:21 +02:00
" plt.title('Confusion Matrix for:\\n' + title)\n",
2018-05-09 19:13:08 +02:00
" plt.colorbar()\n",
" tick_marks = np.arange(len(classes))\n",
" plt.xticks(tick_marks, classes, rotation=45)\n",
" plt.yticks(tick_marks, classes)\n",
"\n",
" if normalize:\n",
" cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
" print(\"Normalized confusion matrix\")\n",
" else:\n",
" print('Confusion matrix, without normalization')\n",
"\n",
" thresh = cm.max() / 2.\n",
2018-05-17 10:38:34 +02:00
" \n",
" pp(cm)\n",
" \n",
2018-05-09 19:13:08 +02:00
" for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
" plt.text(j, i, cm[i, j],\n",
" horizontalalignment=\"center\",\n",
" color=\"white\" if cm[i, j] > thresh else \"black\")\n",
"\n",
" plt.tight_layout()\n",
" plt.ylabel('True label')\n",
" plt.xlabel('Predicted label')"
]
},
2018-05-13 10:12:19 +02:00
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
2018-05-13 12:44:36 +02:00
"def test_classifier(labels, title, Xt, yt, clf):\n",
" pred = clf.predict(Xt)\n",
" score = metrics.accuracy_score(yt, pred)\n",
" pp(\"score: \" + str(score))\n",
" cm = metrics.confusion_matrix(yt, pred, labels=labels)\n",
2018-05-22 16:40:46 +02:00
" plot_confusion_matrix(cm, classes=labels, title=title)\n",
" return cm"
2018-05-13 10:12:19 +02:00
]
},
2018-05-09 16:50:59 +02:00
{
2018-05-17 12:06:34 +02:00
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 5,
2018-05-09 16:50:59 +02:00
"metadata": {},
2018-05-17 12:06:34 +02:00
"outputs": [
{
"data": {
"text/plain": [
"\"\\nfrom rdflib import Graph, Literal, BNode, RDF, Namespace\\nfrom rdflib.namespace import FOAF, DC, XSD\\n\\nmexcore = Namespace('http://mex.aksw.org/mex-core#')\\nmexperf = Namespace('http://mex.aksw.org/mex-perf#')\\nmexalgo = Namespace('http://mex.aksw.org/mex-algo#')\\nprov = Namespace('http://www.w3.org/ns/prov#')\\n\\ndef create_mex_graph():\\n graph = Graph()\\n graph.bind(mexcore)\\n graph.bind(mexperf)\\n graph.bind(mexalgo)\\n graph.bind(prov)\\n graph.bind(FOAF)\\n graph.bind(DC)\\n graph.bind(XSD)\\n \\n return graph\\n\\ndef mex_performance(experiment, model, dataset, performance, phase='Train', graph=create_mex_graph()):\\n \\n p = BNode()\\n \\n\""
]
},
2018-05-22 16:40:46 +02:00
"execution_count": 5,
2018-05-17 12:06:34 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
2018-05-09 16:50:59 +02:00
"source": [
2018-05-17 12:06:34 +02:00
"'''\n",
"from rdflib import Graph, Literal, BNode, RDF, Namespace\n",
"from rdflib.namespace import FOAF, DC, XSD\n",
2018-05-09 16:50:59 +02:00
"\n",
2018-05-17 12:06:34 +02:00
"mexcore = Namespace('http://mex.aksw.org/mex-core#')\n",
"mexperf = Namespace('http://mex.aksw.org/mex-perf#')\n",
"mexalgo = Namespace('http://mex.aksw.org/mex-algo#')\n",
"prov = Namespace('http://www.w3.org/ns/prov#')\n",
"\n",
"def create_mex_graph():\n",
" graph = Graph()\n",
" graph.bind(mexcore)\n",
" graph.bind(mexperf)\n",
" graph.bind(mexalgo)\n",
" graph.bind(prov)\n",
" graph.bind(FOAF)\n",
" graph.bind(DC)\n",
" graph.bind(XSD)\n",
" \n",
" return graph\n",
"\n",
"def mex_performance(experiment, model, dataset, performance, phase='Train', graph=create_mex_graph()):\n",
" \n",
" p = BNode()\n",
" \n",
"'''"
2018-05-09 16:50:59 +02:00
]
},
2018-05-17 12:06:34 +02:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generate/Download Datasets we are working on\n",
"\n",
"* running bash script to download all needed data and store it into the `data` subfolder"
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 6,
2018-05-17 12:06:34 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"================================================================================\n",
"checking whether unzip is installed\n",
"================================================================================\n",
"UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.\n",
"\n",
"Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;\n",
"see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.\n",
"\n",
"Compiled with gcc 6.3.0 20170415 for Unix (Linux ELF).\n",
"\n",
"UnZip special compilation options:\n",
" ACORN_FTYPE_NFS\n",
" COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)\n",
" SET_DIR_ATTRIB\n",
" SYMLINKS (symbolic links supported, if RTL and file system permit)\n",
" TIMESTAMP\n",
" UNIXBACKUP\n",
" USE_EF_UT_TIME\n",
" USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)\n",
" USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)\n",
" UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)\n",
" LARGE_FILE_SUPPORT (large files over 2 GiB supported)\n",
" ZIP64_SUPPORT (archives using Zip64 for large files supported)\n",
" USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)\n",
" VMS_TEXT_CONV\n",
" WILD_STOP_AT_DIR\n",
" [decryption, version 2.11 of 05 Jan 2007]\n",
"\n",
"UnZip and ZipInfo environment options:\n",
" UNZIP: [none]\n",
" UNZIPOPT: [none]\n",
" ZIPINFO: [none]\n",
" ZIPINFOOPT: [none]\n",
"================================================================================\n",
"successfully finished action: checking whether unzip is installed\n",
"================================================================================\n",
"================================================================================\n",
"downloading and unpacking https://www.cs.ucsb.edu/~william/data/liar_dataset.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
"successfully finished action: downloading and unpacking https://www.cs.ucsb.edu/~william/data/liar_dataset.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
"downloading and unpacking https://raw.githubusercontent.com/GeorgeMcIntire/fake_real_news_dataset/master/fake_or_real_news.csv.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
"successfully finished action: downloading and unpacking https://raw.githubusercontent.com/GeorgeMcIntire/fake_real_news_dataset/master/fake_or_real_news.csv.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
"downloading Helper script: script_dataset3.py\n",
"================================================================================\n",
"================================================================================\n",
"successfully finished action: downloading Helper script: script_dataset3.py\n",
"================================================================================\n"
]
}
],
2018-05-09 16:50:59 +02:00
"source": [
"%%bash\n",
"./Task_2_gen_data.sh"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"----\n",
"## configuration 1"
2018-05-09 16:50:59 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 7,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [],
"source": [
"df_1 = pd.read_csv('data/fake_or_real_news.csv').set_index('Unnamed: 0')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* display first 10 entries"
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 8,
2018-05-09 16:50:59 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"data": {
"text/plain": [
"(6335, 3)"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>title</th>\n",
" <th>text</th>\n",
" <th>label</th>\n",
" </tr>\n",
" <tr>\n",
" <th>Unnamed: 0</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>8476</th>\n",
" <td>You Can Smell Hillary’ s Fear</td>\n",
" <td>Daniel Greenfield, a Shillman Journalism Fello...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10294</th>\n",
" <td>Watch The Exact Moment Paul Ryan Committed Pol...</td>\n",
" <td>Google Pinterest Digg Linkedin Reddit Stumbleu...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3608</th>\n",
" <td>Kerry to go to Paris in gesture of sympathy</td>\n",
" <td>U.S. Secretary of State John F. Kerry said Mon...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10142</th>\n",
" <td>Bernie supporters on Twitter erupt in anger ag...</td>\n",
" <td>— Kaydee King (@KaydeeKing) November 9, 2016 T...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>875</th>\n",
" <td>The Battle of New York: Why This Primary Matters</td>\n",
" <td>It's primary day in New York and front-runners...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6903</th>\n",
" <td>Tehran, USA</td>\n",
" <td>\\nI’ m not an immigrant, but my grandparents ...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7341</th>\n",
" <td>Girl Horrified At What She Watches Boyfriend D...</td>\n",
" <td>Share This Baylee Luciani (left), Screenshot o...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>95</th>\n",
" <td>‘ Britain’ s Schindler’ Dies at 106</td>\n",
" <td>A Czech stockbroker who saved more than 650 Je...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4869</th>\n",
" <td>Fact check: Trump and Clinton at the 'commande...</td>\n",
" <td>Hillary Clinton and Donald Trump made some ina...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2909</th>\n",
" <td>Iran reportedly makes new push for uranium con...</td>\n",
" <td>Iranian negotiators reportedly have made a las...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" title \\\n",
"Unnamed: 0 \n",
"8476 You Can Smell Hillary’ s Fear \n",
"10294 Watch The Exact Moment Paul Ryan Committed Pol... \n",
"3608 Kerry to go to Paris in gesture of sympathy \n",
"10142 Bernie supporters on Twitter erupt in anger ag... \n",
"875 The Battle of New York: Why This Primary Matters \n",
"6903 Tehran, USA \n",
"7341 Girl Horrified At What She Watches Boyfriend D... \n",
"95 ‘ Britain’ s Schindler’ Dies at 106 \n",
"4869 Fact check: Trump and Clinton at the 'commande... \n",
"2909 Iran reportedly makes new push for uranium con... \n",
"\n",
" text label \n",
"Unnamed: 0 \n",
"8476 Daniel Greenfield, a Shillman Journalism Fello... FAKE \n",
"10294 Google Pinterest Digg Linkedin Reddit Stumbleu... FAKE \n",
"3608 U.S. Secretary of State John F. Kerry said Mon... REAL \n",
"10142 — Kaydee King (@KaydeeKing) November 9, 2016 T... FAKE \n",
"875 It's primary day in New York and front-runners... REAL \n",
"6903 \\nI’ m not an immigrant, but my grandparents ... FAKE \n",
"7341 Share This Baylee Luciani (left), Screenshot o... FAKE \n",
"95 A Czech stockbroker who saved more than 650 Je... REAL \n",
"4869 Hillary Clinton and Donald Trump made some ina... REAL \n",
"2909 Iranian negotiators reportedly have made a las... REAL "
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-09 16:50:59 +02:00
"source": [
"display(df_1.shape)\n",
"display(df_1[:10])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* create test dataset"
2018-05-09 10:47:51 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 9,
2018-05-09 10:47:51 +02:00
"metadata": {},
"outputs": [],
2018-05-09 16:50:59 +02:00
"source": [
2018-05-13 13:36:21 +02:00
"X1, Xt1, y1, yt1 = train_test_split(df_1.drop('label', axis=1)['text'], df_1.label, test_size=0.25, random_state=4222)"
2018-05-09 16:50:59 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 10,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [],
2018-05-09 19:13:08 +02:00
"source": [
2018-05-17 10:38:34 +02:00
"vectorizer_1 = TfidfVectorizer(stop_words='english', max_df=0.7)\n",
2018-05-13 18:29:18 +02:00
"vec_train_1 = vectorizer_1.fit_transform(X1)\n",
"vec_test_1 = vectorizer_1.transform(Xt1)"
2018-05-09 19:13:08 +02:00
]
},
{
2018-05-17 10:38:34 +02:00
"cell_type": "markdown",
2018-05-09 19:13:08 +02:00
"metadata": {},
2018-05-11 12:19:53 +02:00
"source": [
2018-05-17 10:38:34 +02:00
"* trying a Random Forest classifier "
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 11,
2018-05-11 12:19:53 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.9960008419280152'\n",
"Confusion matrix, without normalization\n",
"array([[2342, 3],\n",
" [ 16, 2390]])\n",
"'score: 0.8478535353535354'\n",
"Confusion matrix, without normalization\n",
"array([[737, 82],\n",
" [159, 606]])\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUcAAAEmCAYAAADx4VKUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xm8VHX9x/HX+4IgCokIuAGSprmVqLhkppim4L6vqZjmktli+cssFbeyMjNzzdwt1DLL3chEc0FFQ8VUEMUEkT0DEYTL5/fH+V4cxrl3Bpi5M3Pv+8njPO7M95w55zMzl8/9fr/ne75HEYGZmS2todoBmJnVIidHM7MCnBzNzApwcjQzK8DJ0cysACdHM7MCnBzbGEldJN0r6X1Jf1yB/Rwl6W/ljK0aJD0o6djlfO2FkmZIeq/ccVntc3KsEklHShotaa6kKek/8Y5l2PXBwJrAGhFxyPLuJCJ+HxG7lyGepUgaJCkk3Z1XvkUqH1nifoZJuq3YdhExJCJuXo44+wHfAzaNiLWW9fVW/5wcq0DS6cBlwE/IElk/4CpgvzLsfj1gXEQsKsO+KmU68AVJa+SUHQuMK9cBlFmR3+9+wMyImLYcx+64Ase1WhERXlpxAVYD5gKHtLBNZ7Lk+W5aLgM6p3WDgElktZppwBTguLTuPOAjYGE6xvHAMOC2nH33BwLomJ4PBd4E5gBvAUfllD+R87odgOeA99PPHXLWjQQuAJ5M+/kb0LOZ99YU/zXAqamsAzAZOAcYmbPtr4F3gP8BzwNfSuWD897nizlxXJTi+BD4TCo7Ia2/GrgrZ/8/Ax4BlBfjbun1i9P+b0rl+wKvAP9N+90k5zUTgR8ALwELmj5fL/W7VD2A9rak/9iLWvrPA5wPjAJ6A72Ap4AL0rpB6fXnAysBewLzgNXT+vxk2GxyBFZNieezad3awGbp8ZLkCPQAZgNHp9cdkZ6vkdaPBCYAGwFd0vOLm3lvTclxB+CZVLYn8DBwQl5y/CqwRjrm94D3gJULva+cOP4DbJZes1JeclyFrHY6FPgSMAPo01KcOc83Aj4AvpL2+3/AG0CntH4iMAboC3RJZVcBV1X7d87L8i1uVre+NYAZ0XKz9yjg/IiYFhHTyWqER+esX5jWL4yIB8hqN59dzngWA5tL6hIRUyLilQLb7AWMj4hbI2JRRAwHXgP2ydnmxogYFxEfAncCA1o6aEQ8BfSQ9FngGOCWAtvcFhEz0zF/SVajLvY+b4qIV9JrFubtbx7Z53gpcBtwWkRMKrK/JocB90fEiLTfS8j+EOyQs83lEfFO+gyIiG9ExDdK3L/VGCfH1jcT6FmkX2od4O2c52+nsiX7yEuu84CuyxpIRHxA9p/+ZGCKpPslbVxCPE0xrZvzPPeMbqnx3Ap8E9gFuDt/paTvS3o1nXn/L1mXRM8i+3ynpZUR8QxZN4LIkniplvoMImJxOlbuZ9Disa2+ODm2vqfJ+qT2b2Gbd8lOrDTpl8qWxwdkzckmS515jYiHI+IrZE3q14DrSoinKabJyxlTk1uBbwAPpFrdEpK+RNZ0PZSsy6A7WX+nmkJvZp8tTjMl6VSyGui7af+lWuozkCSyJnTuZ+AprtoQJ8dWFhHvk514uFLS/pJWkbSSpCGSfp42Gw78WFIvST3T9kWHrTRjDLCTpH6SVgN+2LRC0pqS9pO0KlnCnkvWzM73ALBRGn7UUdJhwKbAfcsZEwAR8RawM/CjAqu7kfWtTgc6SjoH+FTO+qlA/2U5Iy1pI+BCsr7Mo4H/k9Ri8z/HncBeknaVtBJZH+gCsv5ga4OcHKsg9Z+dDvyY7D//O2TNy7+kTS4ERpOd+XwZeCGVLc+xRgB3pH09z9IJrSHF8S4wiyxRnVJgHzOBvckSwkyyGtfeETFjeWLK2/cTEVGoVvww8BDZCZS3gfks3WxtGuA+U9ILxY6TujFuA34WES9GxHjgLOBWSZ1LiPN1sqT6G7ITOfsA+0TERy0c8xpJ1xTbt9UmRbglYGaWzzVHM7MCnBzNzApwcjQzK8DJ0cysACfHMsmfKqzWpvySdJak31U7jtYkaaKk3UrYrn+aEahuJ4xIZ8bPrnYcbUm7S46tNVVYVGjKr1KkacGWuiwuIn4SESdU4FhrS7pH0rspwfQv9zHaulKTeEsi4uSIuKBcMVk7S45tYaqwMkzFVW6LycYjHlTtQNqqeq7R1rVqz3zRWgutP1XYUJae8mt34HWyS+CuAh7j49lihtHytGIj+eRUXMcBr5JNEfYmcFLadlWWnm5rLtl1wfnHKDb91vfJBo6/TzaIfOUin2/HFHP/ZfhOmt7ncWQDvGeTXee9TTr2f4ErcrZvIBs4/3b6Dm4BVstZf3RaN5PsqpuJwG45rz2TbPagmWRXvPQo9HkXiLPpdXOAfwMHlPH38tb0XX2Yvqv/y4nneLJZhh5P2/6R7Br294HHSTMopXU3ARcW+131sgzfTbUDaLU32vpThQ3l4ym/epJNDXZgSiLfJkuky5Ic86fi2gvYgOxa451TLFvlxDop770tOQalTb/1LFlS7UGWhE8u8vmuSHK8BliZ7A/IfLIrhXqTTeowDdg5bf+1FOf6ZBNb/Bm4Na3blCy57ET2R+7S9H01Jcdvp++2T1p/LTC80OddIM5D0mfRQDZRxwfA2mX83ZzYFGdePLeQ/bHrkvP+u/HxH/ExOa+5iaWTY7O/q15KW2qpeVZp1ZwqbE/glYj4czr+5Sw9i00pboqcqbgi4v6ImBCZx8gmmP1SifsqdfqtdyNiFnAvRaYgW0EXRMT8iPgbWeIZnr6DycA/gS3TdkcBl0bEmxExl+w68cNTs/Ng4L6IeDwiFgBns/R14icDP4qISWn9MODgUpqsEfHH9Fksjog7gPHAtmV55y0bFhEfxMdToN0QEXNy4t8iXS9fSDmntWuX2lNyrOZUYeuQc11wZH/eS51HsMlS02GliSpGSZqVpvPak+LTeeXGU2z6reWZgmx5Tc15/GGB503HLvT9dCTrP87/jD8g+86brAfcLem/6fN6FWhMr22RpGMkjcl57eY081mnE31NS790Frnp+VnFjpVnyfuR1EHSxZImSPofWW2T5uKgTNPatWftKTm29lRhuaaQNeeAJdNd9clZ3+K0YsmSi+DTRAl3kdX41oxsOq8HKD6dV5NSpt+qRYW+n0VkyXQK2XsAQNIqZK2FJu8AQyKie86ycqqdNkvSemTTuH2TbCRCd2AsH3/WS4mIrjnLfyI7i9z0/CfNHKaU6deOJDtxuBtZ/3n/phBbit+WX7tJjtH6U4Xluh/4XDpuR+BUlk6AzU4r1oxOZP1O04FFkoaQ9dc1mQqs0UKTq6zTb0laOcUD0Dk9b1o3TCXeUbAEw4HvSvq0pK5kow7uSDWkPwF7S9pRUiey/rbc3+9rgItSsiN9x6WMUliVLElNT687jqzmWE5TyfpRW9KN7DuaSfaHtLlEa2XSbpIjtO5UYXnHnUHWqf9zsl/uTdNxFqT1LU0rVmh/c4BvkSW52WS1inty1r9GlkjeTE3BdfJev8zTbxXRdKYVsglzP8xZ15fsLHs53EB2dvdxspuBzQdOA4js9g6nAn8gq0XOZumui1+TfUZ/kzSH7OTMdsUOGBH/Bn5J1vKYCnyO8r2fJj8l+6P8X0nfb2abW8i6ESaTnTEfVeYYLI+nLKuCNE5xEtmd/h6tdjyVJGkMsGtkc0Ka1Y12VXOsJkl7SOqe+gvPIusravN//SNigBOj1SMnx9bzBbKBxE3N2P2bhmiYWe1xs9rMrADXHM3MCmizF7SrY5dQ5+ZGsli1bLlxn+IbWat6++2JzJgxo6zjJTt8ar2IRcV7jeLD6Q9HxOByHrtc2m5y7LwanTc7qtphWJ4nn/pltUOwPF/cbmDZ9xmLPqTzZw8tut38MVeWelVXq2uzydHMqkiChg7VjmKFODmaWWXU1LSjy87J0cwqQ/V92beTo5lVgFxzNDP7BOE+RzOzT5Kb1WZmBblZbWZWgGuOZmZ5PM7RzKwZblabmeXzUB4zs8Ia3Od
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c28134da0>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAAEmCAYAAAATPUntAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XecVOX1x/HPd+mIghQJUi3YYkFFxI5iAWxoFFEjaDCosURN+Rk1RhNNTLMQoybRCGqCkhgjKhERNZaICgqI0UgRAoh0EVmQdn5/PM/AsNndGZa5OzM7583rvnbm3jv3nplhzz7PLeeRmeGcc6WoLN8BOOdcvngCdM6VLE+AzrmS5QnQOVeyPAE650qWJ0DnXMnyBFjHSGoi6WlJKyT9ZRu2c76k53MZWz5I+oekwTV87a2Slkj6NNdxucLgCTBPJJ0naaKkLyQtiL+oR+Zg02cBbYFWZnZ2TTdiZn8ysxNzEM8WJPWSZJKerDD/gDj/5Sy3c7OkRzOtZ2Z9zWxEDeLsBHwH2MfMvrK1r3fFwRNgHki6FrgL+CkhWXUC7gVOz8HmOwMfmdn6HGwrKYuBwyS1Sps3GPgoVztQsC3/vzsBS81sUQ32XX8b9utqk5n5VIsT0Bz4Aji7mnUaERLkJ3G6C2gUl/UC5hFaJ4uABcBFcdktwFpgXdzHEOBm4NG0bXcBDKgfn18IzAJWAh8D56fNfy3tdYcDbwMr4s/D05a9DPwEeD1u53mgdRXvLRX//cDlcV49YD5wE/By2rp3A3OBz4FJwFFxfp8K73NKWhy3xThWA7vHeRfH5fcBT6Rt/+fAeEAVYjw+vn5j3P7wOP804H3gs7jdvdNeMxv4P2Aq8GXq8/WpsKe8B1BqU/zlXV/dLwjwY2ACsBPQBvgX8JO4rFd8/Y+BBkA/oBzYMS6vmPCqTIDAdjG57BmXtQO+Gh9vSoBAS2A5cEF83bnxeau4/GVgJrAH0CQ+v72K95ZKgIcDb8Z5/YCxwMUVEuDXgVZxn98BPgUaV/a+0uL4L/DV+JoGFRJgU0Ir80LgKGAJ0KG6ONOe7wGsAk6I2/0+MANoGJfPBiYDHYEmcd69wL35/j/nU9WTd4FrXytgiVXfRT0f+LGZLTKzxYSW3QVpy9fF5evMbAyhlbJnDePZCOwrqYmZLTCz9ytZ52Rgupk9YmbrzWwk8CFwato6D5nZR2a2GhgFdKtup2b2L6ClpD2BQcDDlazzqJktjfv8NaFlnOl9Djez9+Nr1lXYXjnhc7wDeBS40szmZdheyjnAs2Y2Lm73V4Rkf3jaOsPMbG78DDCzb5nZt7LcvssDT4C1bynQOsNxop2BOWnP58R5m7ZRIYGWA822NhAzW0X4xb4UWCDpWUl7ZRFPKqb2ac/Tz5RmG88jwBXAscCTFRdK+q6kD+IZ7c8Ihw9aZ9jm3OoWmtmbhC6/CIk6W1t8Bma2Me4r/TOodt+u8HgCrH1vEI4R9a9mnU8IJzNSOsV5NbGK0PVL2eKMppmNNbMTCN3fD4E/ZBFPKqb5NYwp5RHgW8CY2DrbRNJRhG7mAEL3vgXh+KNSoVexzWrLG0m6nNCS/CRuP1tbfAaSROjupn8GXlqpyHgCrGVmtoJwsP+3kvpLaiqpgaS+kn4RVxsJ3CipjaTWcf2Ml3xUYTJwtKROkpoDP0gtkNRW0umStiMk5S8IXeKKxgB7xEt36ks6B9gHeKaGMQFgZh8DxwA3VLJ4e8KxzsVAfUk3ATukLV8IdNmaM72S9gBuJRxbvAD4vqRqu+ppRgEnS+otqQHhmOSXhOOzrkh5AsyDeDzrWuBGwi/4XEJX8O9xlVuBiYQziu8B78R5NdnXOODxuK1JbJm0ymIcnwDLCMnoskq2sRQ4hfBLv5TQcjrFzJbUJKYK237NzCpr3Y4FniOctJgDrGHLLmbqIu+lkt7JtJ94yOFR4OdmNsXMpgPXA49IapRFnP8hJM7fEE6enAqcamZrq9nn/ZLuz7Rtlz8y81a7c640eQvQOVeyPAE650qWJ0DnXMnyBOicK1meABNQsSRVoZWWknS9pAfyHUdtkjRb0vFZrNclVqXxggYloKQTYG2VpLKESktlI5af2uJ2LzP7qZldnMC+2kkaLemTmES65HofdV22iTqL7Vwo6bVcxFSXlWwCrAslqXJQ8inXNhKu3ftavgNxLiv5rsaQj4naL0l1IVuWljoR+A/h1q57gX+yuWLJzVRfvupl/rfk00XAB4RSVLOAS+K627FlWacvCPe0VtxHpjJP3yVcSL2CcFF14wyfb/0Yc5et+E5S7/MiwgXPywn3KB8S9/0ZcE/a+mWEC8nnxO/gYaB52vIL4rKlhDtNZgPHp732OkIFm6WEuzxaVvZ5VxJn6nUrgX8DZ+Tw/+Uj8btaHb+r78f5PQl3nHwGTAF6pb3mQiqUMwP2Jlw4viFu57N8/84V6pT3APLypmu/JNWFbC4t1ZpQgurMmCi+TUiWW5MAK5Z8OhnYjXCf7DExloPSYp1X4b1t2gfZlXl6i5A4WxIS7aUZPt9tSYD3A40JfyTWEO6O2YlQdGARcExc/xsxzl0JhRf+BjwSl+0Tf/GPJvwhuyN+X6kE+O343XaIy38HjKzs864kzrPjZ1FGKCSxCmiXw/+bs1NxxuftCUm6X9znCfF5G7IsZ+ZT1VMhdZ9qUz5LUvUD3jezv8X9D2PLSirZGG5pJZ/M7Fkzm2nBPwkFSY/KclvZlnn6xMyWAU+TodTVNvqJma0xs+cJyWVk/A7mA68CB8b1zgfuMLNZZvYF4R7ngfHkxVnAM2b2ipl9CfyQLe9xvhS4wczmxeU3A2dlc+LDzP4SP4uNZvY4MB3okZN3XrmvE4pFjIn7HEe4TbJfXJ5NOTNXhVJNgPksSbUzafe0WvhznW1NupQtyi7FQgoTJC2LZaP6kblsVHo8mco81aTUVU0tTHu8upLnqX1X9v3UJxzPrfgZryJ85ymdgSclfRY/rw8I3cW2mYKTNEjS5LTX7ksVn3U8uZaaOsV7g1PPr8+0r7RYz07tL+7zSEKrM9tyZq4KpZoAa7skVboFhK4XsKmsUoe05dWWr4o23cAdb+R/gtBya2uhbNQYMpeNSsmmzFMhquz7WU9ImAsI7wEASU0Jrf6UuUBfM2uRNjWOrcwqSepMKBd2BeEMfwtgGps/6y2YWbO06b9mdmna859WsZuK39dcQtc+PdbtzOz2uI+qypn5Tf5ZKMkEaLVfkirds8B+cb/1gcvZMslVWb6qCg0Jx7EWA+sl9SUcP0tZCLSK26pMTss8SWoc4wFoFJ+nlt2sLEd9y8JI4BpJu0hqRjib/3hslf8VOEXSkZIaEo7Vpv9fvx+4LSY04neczdn/7QiJZXF83UWEFmAuLSQc10x5FDhV0kmS6klqHC9t6pChnNlCoEN8/64KJZkAoXZLUlXY7xLCgfRfELpl+8T9fBmXV1e+qrLtrQSuIiSy5cB5wOi05R8SksWs2IXaucLrt7rMUwapM5gQWiSr05Z1JJy9zoU/Es6avkI4+7kGuBIgHge7HPgzoTW4nC0PM9xN+Iyel7SScELk0Ew7NLN/A78m9CAWAvuRu/eT8jPCH97PJH3XzOYSLs26ns3/T79H+N2trpzZi4Qz+59K2uayZXWVl8PKs3gd3zzCaGwv5TueJEmaDPS2UF/Qubwr2RZgPsXuTIt4/O56wjGkCXkOK3Fm1s2TnyskngDz4zDCxbSpLmd/iyOJOedqj3eBnXMly1uAzrmSVWdL/qh+E1PD7fMdhqvgwL075TsEV8GcObNZsmRJpdcy1lS9HTqbrc98VMdWLx5rZn1yue+tUXcTYMPtabTngHyH4Sp4/c178h2Cq+CIQ7vnfJu2fnVWv39rJv822zuWElFnE6BzLo8kKKuX7ygy8gTonEtGQZWqrJwnQOdcMpTTw4qJ8ATonEuAvAXonCtRwo8BOudKlbwL7JwrYUXQBS78CJ1zxUnKPFX7cu0Zq2+nps8lXS2ppaRxkqbHnzvG9SVpmKQZkqZKOihTiJ4AnXO5l7oOMNN
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c280d7a90>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-11 12:19:53 +02:00
"source": [
2018-05-17 10:38:34 +02:00
"from sklearn.ensemble import RandomForestClassifier as RFC\n",
"clf_a = RFC(criterion='entropy', random_state=4222)\n",
"max_size=10000\n",
"clf_a.fit(vec_train_1[:max_size], y1[:max_size])\n",
2018-05-13 18:29:18 +02:00
"test_classifier(labels=[\"FAKE\",\"REAL\"], title=\"Configuration 1, model a -- train\", Xt=vec_train_1,yt=y1, clf=clf_a)\n",
2018-05-22 16:40:46 +02:00
"cm_1 = test_classifier(labels=[\"FAKE\",\"REAL\"], title=\"Configuration 1, model a -- test\", Xt=vec_test_1,yt=yt1, clf=clf_a)"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"----\n",
2018-05-13 10:35:15 +02:00
"## configuration 2\n",
2018-05-13 10:12:19 +02:00
"\n",
"* read data"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 12,
2018-05-11 12:19:53 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"data": {
"text/markdown": [
"----\n",
"#### Train Data:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>id</th>\n",
" <th>label</th>\n",
" <th>statement</th>\n",
" <th>subjects</th>\n",
" <th>speaker</th>\n",
" <th>job</th>\n",
" <th>state</th>\n",
" <th>party</th>\n",
" <th>#barely_true</th>\n",
" <th>#false</th>\n",
" <th>#half_true</th>\n",
" <th>#mostly_true</th>\n",
" <th>#pants_on_fire</th>\n",
" <th>context</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>2635.json</td>\n",
" <td>false</td>\n",
" <td>Says the Annies List political group supports ...</td>\n",
" <td>abortion</td>\n",
" <td>dwayne-bohac</td>\n",
" <td>State representative</td>\n",
" <td>Texas</td>\n",
" <td>republican</td>\n",
" <td>0.0</td>\n",
" <td>1.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>a mailer</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1123.json</td>\n",
" <td>false</td>\n",
" <td>Health care reform legislation is likely to ma...</td>\n",
" <td>health-care</td>\n",
" <td>blog-posting</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>none</td>\n",
" <td>7.0</td>\n",
" <td>19.0</td>\n",
" <td>3.0</td>\n",
" <td>5.0</td>\n",
" <td>44.0</td>\n",
" <td>a news release</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>12465.json</td>\n",
" <td>true</td>\n",
" <td>The Chicago Bears have had more starting quart...</td>\n",
" <td>education</td>\n",
" <td>robin-vos</td>\n",
" <td>Wisconsin Assembly speaker</td>\n",
" <td>Wisconsin</td>\n",
" <td>republican</td>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>2.0</td>\n",
" <td>5.0</td>\n",
" <td>1.0</td>\n",
" <td>a an online opinion-piece</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>5947.json</td>\n",
" <td>false</td>\n",
" <td>When Mitt Romney was governor of Massachusetts...</td>\n",
" <td>history,state-budget</td>\n",
" <td>mitt-romney</td>\n",
" <td>Former governor</td>\n",
" <td>Massachusetts</td>\n",
" <td>republican</td>\n",
" <td>34.0</td>\n",
" <td>32.0</td>\n",
" <td>58.0</td>\n",
" <td>33.0</td>\n",
" <td>19.0</td>\n",
" <td>an interview with CBN News</td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>620.json</td>\n",
" <td>true</td>\n",
" <td>McCain opposed a requirement that the governme...</td>\n",
" <td>federal-budget</td>\n",
" <td>barack-obama</td>\n",
" <td>President</td>\n",
" <td>Illinois</td>\n",
" <td>democrat</td>\n",
" <td>70.0</td>\n",
" <td>71.0</td>\n",
" <td>160.0</td>\n",
" <td>163.0</td>\n",
" <td>9.0</td>\n",
" <td>a radio ad</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" id label statement \\\n",
"0 2635.json false Says the Annies List political group supports ... \n",
"3 1123.json false Health care reform legislation is likely to ma... \n",
"5 12465.json true The Chicago Bears have had more starting quart... \n",
"12 5947.json false When Mitt Romney was governor of Massachusetts... \n",
"16 620.json true McCain opposed a requirement that the governme... \n",
"\n",
" subjects speaker job \\\n",
"0 abortion dwayne-bohac State representative \n",
"3 health-care blog-posting NaN \n",
"5 education robin-vos Wisconsin Assembly speaker \n",
"12 history,state-budget mitt-romney Former governor \n",
"16 federal-budget barack-obama President \n",
"\n",
" state party #barely_true #false #half_true #mostly_true \\\n",
"0 Texas republican 0.0 1.0 0.0 0.0 \n",
"3 NaN none 7.0 19.0 3.0 5.0 \n",
"5 Wisconsin republican 0.0 3.0 2.0 5.0 \n",
"12 Massachusetts republican 34.0 32.0 58.0 33.0 \n",
"16 Illinois democrat 70.0 71.0 160.0 163.0 \n",
"\n",
" #pants_on_fire context \n",
"0 0.0 a mailer \n",
"3 44.0 a news release \n",
"5 1.0 a an online opinion-piece \n",
"12 19.0 an interview with CBN News \n",
"16 9.0 a radio ad "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"----\n",
"#### Test Data:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>id</th>\n",
" <th>label</th>\n",
" <th>statement</th>\n",
" <th>subjects</th>\n",
" <th>speaker</th>\n",
" <th>job</th>\n",
" <th>state</th>\n",
" <th>party</th>\n",
" <th>#barely_true</th>\n",
" <th>#false</th>\n",
" <th>#half_true</th>\n",
" <th>#mostly_true</th>\n",
" <th>#pants_on_fire</th>\n",
" <th>context</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>11972.json</td>\n",
" <td>true</td>\n",
" <td>Building a wall on the U.S.-Mexico border will...</td>\n",
" <td>immigration</td>\n",
" <td>rick-perry</td>\n",
" <td>Governor</td>\n",
" <td>Texas</td>\n",
" <td>republican</td>\n",
" <td>30</td>\n",
" <td>30</td>\n",
" <td>42</td>\n",
" <td>23</td>\n",
" <td>18</td>\n",
" <td>Radio interview</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>11685.json</td>\n",
" <td>false</td>\n",
" <td>Wisconsin is on pace to double the number of l...</td>\n",
" <td>jobs</td>\n",
" <td>katrina-shankland</td>\n",
" <td>State representative</td>\n",
" <td>Wisconsin</td>\n",
" <td>democrat</td>\n",
" <td>2</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>a news conference</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>11096.json</td>\n",
" <td>false</td>\n",
" <td>Says John McCain has done nothing to help the ...</td>\n",
" <td>military,veterans,voting-record</td>\n",
" <td>donald-trump</td>\n",
" <td>President-Elect</td>\n",
" <td>New York</td>\n",
" <td>republican</td>\n",
" <td>63</td>\n",
" <td>114</td>\n",
" <td>51</td>\n",
" <td>37</td>\n",
" <td>61</td>\n",
" <td>comments on ABC's This Week.</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>5962.json</td>\n",
" <td>true</td>\n",
" <td>Over the past five years the federal governmen...</td>\n",
" <td>federal-budget,pensions,retirement</td>\n",
" <td>brendan-doherty</td>\n",
" <td>NaN</td>\n",
" <td>Rhode Island</td>\n",
" <td>republican</td>\n",
" <td>1</td>\n",
" <td>2</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>a campaign website</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>7070.json</td>\n",
" <td>true</td>\n",
" <td>Says that Tennessee law requires that schools ...</td>\n",
" <td>county-budget,county-government,education,taxes</td>\n",
" <td>stand-children-tennessee</td>\n",
" <td>Child and education advocacy organization.</td>\n",
" <td>Tennessee</td>\n",
" <td>none</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>in a post on Facebook.</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" id label statement \\\n",
"0 11972.json true Building a wall on the U.S.-Mexico border will... \n",
"1 11685.json false Wisconsin is on pace to double the number of l... \n",
"2 11096.json false Says John McCain has done nothing to help the ... \n",
"5 5962.json true Over the past five years the federal governmen... \n",
"6 7070.json true Says that Tennessee law requires that schools ... \n",
"\n",
" subjects speaker \\\n",
"0 immigration rick-perry \n",
"1 jobs katrina-shankland \n",
"2 military,veterans,voting-record donald-trump \n",
"5 federal-budget,pensions,retirement brendan-doherty \n",
"6 county-budget,county-government,education,taxes stand-children-tennessee \n",
"\n",
" job state party \\\n",
"0 Governor Texas republican \n",
"1 State representative Wisconsin democrat \n",
"2 President-Elect New York republican \n",
"5 NaN Rhode Island republican \n",
"6 Child and education advocacy organization. Tennessee none \n",
"\n",
" #barely_true #false #half_true #mostly_true #pants_on_fire \\\n",
"0 30 30 42 23 18 \n",
"1 2 1 0 0 0 \n",
"2 63 114 51 37 61 \n",
"5 1 2 1 1 0 \n",
"6 0 0 0 0 0 \n",
"\n",
" context \n",
"0 Radio interview \n",
"1 a news conference \n",
"2 comments on ABC's This Week. \n",
"5 a campaign website \n",
"6 in a post on Facebook. "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"----\n",
"#### Valid Data:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>id</th>\n",
" <th>label</th>\n",
" <th>statement</th>\n",
" <th>subjects</th>\n",
" <th>speaker</th>\n",
" <th>job</th>\n",
" <th>state</th>\n",
" <th>party</th>\n",
" <th>#barely_true</th>\n",
" <th>#false</th>\n",
" <th>#half_true</th>\n",
" <th>#mostly_true</th>\n",
" <th>#pants_on_fire</th>\n",
" <th>context</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>7891.json</td>\n",
" <td>false</td>\n",
" <td>Says Having organizations parading as being so...</td>\n",
" <td>campaign-finance,congress,taxes</td>\n",
" <td>earl-blumenauer</td>\n",
" <td>U.S. representative</td>\n",
" <td>Oregon</td>\n",
" <td>democrat</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>a U.S. Ways and Means hearing</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>9416.json</td>\n",
" <td>false</td>\n",
" <td>Says when armed civilians stop mass shootings ...</td>\n",
" <td>guns</td>\n",
" <td>jim-rubens</td>\n",
" <td>Small business owner</td>\n",
" <td>New Hampshire</td>\n",
" <td>republican</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>in an interview at gun shop in Hudson, N.H.</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>6861.json</td>\n",
" <td>true</td>\n",
" <td>Says Tennessee is providing millions of dollar...</td>\n",
" <td>education,state-budget</td>\n",
" <td>andy-berke</td>\n",
" <td>Lawyer and state senator</td>\n",
" <td>Tennessee</td>\n",
" <td>democrat</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>a letter to state Senate education committee c...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>1122.json</td>\n",
" <td>false</td>\n",
" <td>The health care reform plan would set limits s...</td>\n",
" <td>health-care</td>\n",
" <td>club-growth</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>none</td>\n",
" <td>4</td>\n",
" <td>5</td>\n",
" <td>4</td>\n",
" <td>2</td>\n",
" <td>0</td>\n",
" <td>a TV ad</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>13138.json</td>\n",
" <td>true</td>\n",
" <td>Says Donald Trump started his career back in 1...</td>\n",
" <td>candidates-biography,diversity,housing</td>\n",
" <td>hillary-clinton</td>\n",
" <td>Presidential candidate</td>\n",
" <td>New York</td>\n",
" <td>democrat</td>\n",
" <td>40</td>\n",
" <td>29</td>\n",
" <td>69</td>\n",
" <td>76</td>\n",
" <td>7</td>\n",
" <td>the first presidential debate</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" id label statement \\\n",
"2 7891.json false Says Having organizations parading as being so... \n",
"5 9416.json false Says when armed civilians stop mass shootings ... \n",
"6 6861.json true Says Tennessee is providing millions of dollar... \n",
"7 1122.json false The health care reform plan would set limits s... \n",
"8 13138.json true Says Donald Trump started his career back in 1... \n",
"\n",
" subjects speaker \\\n",
"2 campaign-finance,congress,taxes earl-blumenauer \n",
"5 guns jim-rubens \n",
"6 education,state-budget andy-berke \n",
"7 health-care club-growth \n",
"8 candidates-biography,diversity,housing hillary-clinton \n",
"\n",
" job state party #barely_true #false \\\n",
"2 U.S. representative Oregon democrat 0 1 \n",
"5 Small business owner New Hampshire republican 1 1 \n",
"6 Lawyer and state senator Tennessee democrat 0 0 \n",
"7 NaN NaN none 4 5 \n",
"8 Presidential candidate New York democrat 40 29 \n",
"\n",
" #half_true #mostly_true #pants_on_fire \\\n",
"2 1 1 0 \n",
"5 0 1 0 \n",
"6 0 0 0 \n",
"7 4 2 0 \n",
"8 69 76 7 \n",
"\n",
" context \n",
"2 a U.S. Ways and Means hearing \n",
"5 in an interview at gun shop in Hudson, N.H. \n",
"6 a letter to state Senate education committee c... \n",
"7 a TV ad \n",
"8 the first presidential debate "
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-11 12:19:53 +02:00
"source": [
2018-05-13 10:12:19 +02:00
"names = [\n",
" \"id\",\n",
" \"label\",\n",
" \"statement\",\n",
" \"subjects\",\n",
" \"speaker\",\n",
" \"job\",\n",
" \"state\",\n",
" \"party\",\n",
" \"#barely_true\",\n",
" \"#false\",\n",
" \"#half_true\",\n",
" \"#mostly_true\",\n",
" \"#pants_on_fire\",\n",
" \"context\"\n",
"]\n",
"\n",
"df_2_train = pd.read_csv(\"data/train.tsv\", delimiter='\\t', names=names)\n",
"df_2_test = pd.read_csv(\"data/test.tsv\", delimiter='\\t', names=names)\n",
2018-05-13 13:36:21 +02:00
"df_2_valid= pd.read_csv(\"data/valid.tsv\", delimiter='\\t', names=names)\n",
2018-05-13 10:12:19 +02:00
"\n",
"# use only 'False' and 'True' statements\n",
"df_2_train = df_2_train[df_2_train['label'].isin([\"false\",\"true\"])]\n",
"df_2_test = df_2_test[df_2_test['label'].isin([\"false\",\"true\"])]\n",
2018-05-13 13:36:21 +02:00
"df_2_valid = df_2_valid[df_2_valid['label'].isin([\"false\",\"true\"])]\n",
2018-05-13 10:12:19 +02:00
"\n",
2018-05-13 12:44:36 +02:00
"display(Markdown(\"----\\n#### Train Data:\"))\n",
2018-05-13 10:12:19 +02:00
"display(df_2_train.head())\n",
2018-05-13 12:44:36 +02:00
"display(Markdown(\"----\\n#### Test Data:\"))\n",
2018-05-13 13:36:21 +02:00
"display(df_2_test.head())\n",
"display(Markdown(\"----\\n#### Valid Data:\"))\n",
"display(df_2_valid.head())"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"#### tdidf vectorizer on new dataset\n"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 13,
2018-05-11 12:19:53 +02:00
"metadata": {},
2018-05-13 10:12:19 +02:00
"outputs": [],
"source": [
2018-05-13 10:35:15 +02:00
"X2 = df_2_train['statement']\n",
"y2 = df_2_train['label']\n",
"Xt2 = df_2_test['statement']\n",
2018-05-13 12:44:36 +02:00
"yt2 = df_2_test['label']\n",
2018-05-13 13:36:21 +02:00
"Xv2 = df_2_valid['statement']\n",
"yv2 = df_2_valid['label']\n"
2018-05-13 10:12:19 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 14,
2018-05-13 10:12:19 +02:00
"metadata": {},
"outputs": [],
"source": [
2018-05-17 10:38:34 +02:00
"vectorizer_2 = TfidfVectorizer(stop_words='english', max_df=0.7)\n",
2018-05-13 18:29:18 +02:00
"vec_train_2 = vectorizer_2.fit_transform(X2)\n",
"vec_test_2 = vectorizer_2.transform(Xt2)"
2018-05-13 10:12:19 +02:00
]
},
{
2018-05-13 10:35:15 +02:00
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 15,
2018-05-17 10:38:34 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Object `MLPClassifier` not found.\n"
]
}
],
2018-05-17 10:38:34 +02:00
"source": [
"?MLPClassifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* trying a MLP as classifier "
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 16,
2018-05-13 10:12:19 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"data": {
"text/plain": [
"MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,\n",
" beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
" hidden_layer_sizes=(100,), learning_rate='constant',\n",
" learning_rate_init=0.001, max_iter=200, momentum=0.9,\n",
" nesterovs_momentum=True, power_t=0.5, random_state=4222,\n",
" shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,\n",
" verbose=False, warm_start=False)"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
2018-05-13 10:12:19 +02:00
"source": [
2018-05-17 10:38:34 +02:00
"from sklearn.neural_network import MLPClassifier\n",
"clf_b = MLPClassifier(hidden_layer_sizes=(100,), random_state=4222)\n",
2018-05-13 18:29:18 +02:00
"clf_b.fit(vec_train_2, y2)"
2018-05-13 10:12:19 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 17,
2018-05-13 10:12:19 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.9997275946608554'\n",
"Confusion matrix, without normalization\n",
"array([[1675, 1],\n",
" [ 0, 1995]])\n",
"'score: 0.5776805251641138'\n",
"Confusion matrix, without normalization\n",
"array([[109, 99],\n",
" [ 94, 155]])\n",
"'score: 0.6041666666666666'\n",
"Confusion matrix, without normalization\n",
"array([[ 92, 77],\n",
" [ 94, 169]])\n"
]
},
{
"data": {
"text/plain": [
"array([[ 92, 77],\n",
" [ 94, 169]])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUYAAAEmCAYAAAAeIzmqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcneP9//HXe5IIlRCElETEFhq+te+CtihK0ZZStdRedNON8q2lpbrQX9VWvvVFaSzFlxJVXbSEICKWWGMJiZTEvkRI8vn9cV2TnDmZOefM5J45c2beT4/7kXOue/ucGecz13Xd931digjMzGyhpnoHYGbW3TgxmpmVcWI0MyvjxGhmVsaJ0cysjBOjmVkZJ8YeRNJSkv4s6S1J1y3GcQ6Q9NciY6sHSbdJOriD+/5U0ixJ/yk6Luv+nBjrQNJXJE2Q9K6kGfkLvG0Bh/4SMARYISL26ehBIuKqiNi5gHhakLSDpJB0Y1n5Brn8zhqPc6qkK6ttFxG7RsTlHYhzOPBdYFREfLy9+1vjc2LsYpKOB/4fcCYpiQ0HLgD2LODwqwFPR8TcAo7VWWYCW0laoaTsYODpok6gZHH+3x4OvBYRr3bg3H0X47zWXUSEly5agGWBd4F9KmzTn5Q4X87L/wP653U7ANNItZlXgRnA1/K604APgY/yOQ4DTgWuLDn2CCCAvvn9IcBzwDvA88ABJeV3l+y3NfAA8Fb+d+uSdXcCPwHG5eP8FRjcxmdrjv8i4Nhc1geYDvwYuLNk298ALwFvAw8Co3P5LmWf8+GSOM7IccwG1splh+f1FwLXlxz/58DfAZXFuGPef34+/mW5/PPAZODNfNxPlOzzAvBD4BFgTvPP10vjLnUPoDct+Us9t9IXBzgdGA+sBKwI3AP8JK/bIe9/OtAP2A14H1gury9PhG0mRmDpnHTWyetWBtbLrxckRmB54A3gwLzf/vn9Cnn9ncCzwEhgqfz+rDY+W3Ni3Bq4L5ftBtwOHF6WGL8KrJDP+V3gP8CSrX2ukjheBNbL+/QrS4wfI9VKDwFGA7OAYZXiLHk/EngP2Ckf9wfAFGCJvP4FYBKwKrBULrsAuKDe/8956djipnTXWgGYFZWbugcAp0fEqxExk1QTPLBk/Ud5/UcRMZZUq1mng/HMB9aXtFREzIiIya1s8zngmYj4Q0TMjYgxwJPAHiXb/G9EPB0Rs4FrgQ0rnTQi7gGWl7QOcBBwRSvbXBkRr+Vznk2qSVf7nJdFxOS8z0dlx3uf9HM8B7gS+EZETKtyvGZfBm6NiDvycX9F+iOwdck250bES/lnQEQcExHH1Hh862acGLvWa8DgKv1QqwBTS95PzWULjlGWWN8HBrQ3kIh4j/SFPxqYIelWSevWEE9zTENL3pdeua01nj8AxwGfAm4sXynpe5KeyFfY3yR1QwyucsyXKq2MiPtIXQciJfBatfgZRMT8fK7Sn0HFc1tjcWLsWveS+qD2qrDNy6SLKM2G57KOeI/UhGzW4gprRNweETuRmtFPApfUEE9zTNM7GFOzPwDHAGNzbW4BSaNJzdV9Sd0Eg0j9m2oOvY1jVhwqStKxpJrny/n4tWrxM5AkUrO59GfgYap6ECfGLhQRb5EuMpwvaS9JH5PUT9Kukn6RNxsDnCxpRUmD8/ZVb01pwyRgO0nDJS0LnNi8QtIQSXtKWpqUrN8lNa3LjQVG5luM+kr6MjAKuKWDMQEQEc8D2wMntbJ6IKkvdSbQV9KPgWVK1r8CjGjPlWdJI4GfkvouDwR+IKlik7/EtcDnJH1GUj9Sn+ccUv+v9UBOjF0s95cdD5xM+uK/RGpS/l/e5KfABNIVzkeBibmsI+e6A7gmH+tBWiazphzHy8DrpCT19VaO8RqwOykZvEaqae0eEbM6ElPZse+OiNZqw7cDfyFdLJkKfEDLpmrzzeuvSZpY7Ty56+JK4OcR8XBEPAP8CPiDpP41xPkUKaH+lnTRZg9gj4j4sMI5L5J0UbVjW/ekCLcAzMxKucZoZlbGidHMrIwTo5lZGSdGM7MyTowNJg+Q8L+S3pB0v6TRkp6qd1zNesqQZR0labKkHeodhy0eX5VuMPnm5zGkZ5zfq3MsI0iDT/Sr8phjEefakjRYxSbAPNJz0N+MiBkFHX8EXfRZrPtzjbHxrAa80BVJUVKfzj5HOywHXEwaCGM10kg+/9uVAXhIsV6k3qNY9OSF9NjYDaQbuV8DzsvlTaQbvKeShg+7Alg2rxtBerzsYNJoMbOAk/K6w0g3O88jPalyGouOBLMx8BApcVxHusH7p3ndIZQMJ5bLAlgrv76MNDzXWNLjhDuSBpF4iDQSz0vAqSX7vpj3fzcvW5Wfg4KGLGvlZ7sx8E6Bv6u2Pss44Nf59/dTYE3gH/n9LOAqYFDJcV4AdsyvTyU9NXNF/nyTgU3r/f+ll+qLa4ydJNe2biElvxGkAQeuzqsPycungDVIgy6cV3aIbUmjyXwG+LGkT0TE70mDPtwbEQMi4pSycy5BGpDhMtJwYWOAvdsZ+ldI4xoOBO4mJciDgEGkJPl1Sc3Pem+X/x2U47m3LJ7lgVuBc0kjC50D3Fo2SO1XgK+RhllbAvhejXFuR0o0RWnrs2xBGnhiCOnnIuBnpIElPkH643dqheN+nvR7HwTczKK/Z+uGnBg7z+akL8/3I+K9iPggIu7O6w4AzomI5yLiXdIzzPuVNdVOi4jZEfEw8DCwQQ3n3JI0FuG5kYYluwG4v51x3xQR4yJifo75zoh4NL9/hJRst6/xWIUPWQYg6ZOkZ8i/365P1jEvR8Rvc/yzI2JKpOHH5kQaFu4cKv887o6IsRExjzRwRi2/R6szJ8bOsyowNVrvyG9taLG+pFpJs44M5bUKMD0iSq+otXc4rBbbS9pC0j8lzZT0FqnGWm34r9J4Ch2yTNJawG3AtyLirja2GZ3n03lX0uRcNrmkbHSN8cOiP48hkq6WNF3S26RnsCv9PMo/35Luq+z+nBg7z0vA8Da+BK0NLTaXNGrM4pgBDM3DYjVbteR1i2HIJLU20VP5bQp/JDUBV42IZUnTElQb/qtZoUOWSVoN+BtpRPM/tLVdRNyVm8MDImK9XLZeSVlrCbXWoczOzGX/FRHLkAaX0CJ7WUNzYuw895MS1VmSlpa0pKRt8roxwHckrS5pAOnLdk0btcv2uJd0Yea4PETYnqQmfbOHgfUkbShpSSr3jTUbCLweER9I2pzUJ9hsJmmosjXa2LewIcskDSVd9DgvIjpj1Jpqn6XZQNLFmbdyTF3RnLcu5sTYSXKf0h6kSZleJM118uW8+lJSf9O/SffOfQB8o4Bzfgh8gXT1+k1SbeYW0tiBRMTTpPli/gY8Q7q4Us0xwOmS3iH16y0Y+TrSALNnAOMkvZnvNSyNp8ghyw4nJa1TS5rE73bgOK2q9llKnEa6Iv4W6cLSDUXFYN2Hb/Du4STdB1wUEV16z59ZI3ONsYeRtL2kj+em68HAJ0mDvppZjXx1rOdZh9TcXZp0/92XoqDH5sx6CzelzczKuCltZlamxzWlm5YcGE1Lr1jvMKzMBiOWr3cI1oaJEx+cFRGFfWn6LLNaxNzZVbeL2TNvj4hdijpvkXpeYlx6RQbscnq9w7Ay4y7/ar1DsDYs1U/lTyctlpg7m/7r7Ft1uw8mnV/rE1RdrsclRjOrMwmautOIde3nxGhmxVNjX75wYjSz4qmxHx93YjSzgsk1RjOzFoT7GM3MWpKb0mZmi3BT2sysjGuMZmYlfB+jmVkr3JQ2Myvl23XMzBbV5D5GM7OFfB+jmVk5N6XNzBbl23XMzMq4xmhmVsL3MZqZtcJNaTOzUr74Yma2qAavMTZ2Wjez7keCpr7Vl6qH0aWSXpX0WEnZNZIm5eUFSZNy+QhJs0vWXVSyzyaSHpU0RdK5UvWs7RqjmRWvmBrjZcB5wBXNBRHx5YWn0NnAWyXbPxsRG7ZynAuBI4D7gLHALsBtlU7sGqOZFU9N1ZcqIuLfwOutHj7V+vYFxlQMQ1oZWCYixkdEkJLsXtXO7cRoZsWTqi8
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c27ff2fd0>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAAEmCAYAAAATPUntAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcXFWZ//HPtzp7AgQIm4EQhbAPu6goixsCgkFG/IEoIDjIqo47wgiyieOIIyIyYcSwyeIIDsqmogyLICD7TlgTCGTDkD3p7uf3xzmdVDrdXdWd6q6uvt93XvfVVefeOveprq4n55x777mKCMzMiqhU7wDMzOrFCdDMCssJ0MwKywnQzArLCdDMCssJ0MwKywlwAJE0XNLvJM2V9OvVqOdwSX+oZWz1IOkWSUf28LVnS5ol6Y1ax2X9hxNgHUj6jKQHJc2XND1/UT9Qg6o/BWwArBsRh/S0koi4KiL2qUE8K5G0t6SQdEO78h1y+R1V1nOGpCsrbRcR+0XEZT2IcxzwNWCbiNiwu6+3xuEE2MckfRX4T+BcUrIaB1wETKxB9ZsCz0VEcw3q6i0zgfdJWres7EjguVrtQMnq/G2PA2ZHxIwe7HvQauzX+lpEeOmjBVgLmA8c0sU2Q0kJ8vW8/CcwNK/bG5hGap3MAKYDn8/rvgcsBZblfRwDnAFcWVb3eCCAQfn5UcCLwDzgJeDwsvK7y163O/AAMDf/3L1s3R3AWcA9uZ4/AGM6eW9t8V8MnJjLmoDXgO8Cd5Rt+xNgKvA28Hdgj1y+b7v3+WhZHOfkOBYBm+eyL+T1Pwd+U1b/D4DbAbWL8SP59a25/sm5/BPAk8A/cr1bl73mZeBbwGPAkrbfr5f+v9Q9gCIt+cvb3NUXBDgTuA9YH1gP+CtwVl63d379mcBgYH9gIbB2Xt8+4XWaAIGROblsmddtBGybHy9PgMA6wFvA5/LrDsvP183r7wBeALYAhufn53Xy3toS4O7A33LZ/sBtwBfaJcDPAuvmfX4NeAMY1tH7KovjVWDb/JrB7RLgCFIr8yhgD2AWsHFXcZY93wJYAHw01/tNYAowJK9/GXgE2AQYnssuAi6q99+cl64Xd4H71rrArOi6i3o4cGZEzIiImaSW3efK1i/L65dFxM2kVsqWPYynFdhO0vCImB4RT3awzceB5yPiiohojoirgWeAA8u2+WVEPBcRi4DrgB272mlE/BVYR9KWwBHA5R1sc2VEzM77/BGpZVzpfU6OiCfza5a1q28h6fd4PnAlcHJETKtQX5v/B9wUEX/M9f4HKdnvXrbNBRExNf8OiIgTIuKEKuu3OnEC7FuzgTEVxoneAbxS9vyVXLa8jnYJdCEwqruBRMQC0hf7OGC6pJskbVVFPG0xjS17Xn6ktNp4rgBOAj4I3NB+paSvS3o6H9H+B2n4YEyFOqd2tTIi/kbq8ouUqKu10u8gIlrzvsp/B13u2/onJ8C+dS9pjOigLrZ5nXQwo824XNYTC0hdvzYrHdGMiNsi4qOk7u8zwCVVxNMW02s9jKnNFcAJwM25dbacpD1I3cxPk7r3o0njj2oLvZM6u5zaSNKJpJbk67n+aq30O5AkUne3/HfgaZUakBNgH4qIuaTB/p9JOkjSCEmDJe0n6d/zZlcDp0laT9KYvH3FUz468Qiwp6RxktYCTmlbIWkDSRMljSQl5fmkLnF7NwNb5FN3Bkn6f8A2wO97GBMAEfESsBdwager1yCNdc4EBkn6LrBm2fo3gfHdOdIraQvgbNLY4ueAb0rqsqte5jrg45I+LGkwaUxyCWl81hqYE2Afy+NZXwVOI33Bp5K6gr/Nm5wNPEg6ovg48FAu68m+/ghcm+v6OysnrVKO43VgDikZHd9BHbOBA0hf+tmkltMBETGrJzG1q/vuiOiodXsbcCvpoMUrwGJW7mK2neQ9W9JDlfaThxyuBH4QEY9GxPPAd4ArJA2tIs5nSYnzp6SDJwcCB0bE0i72ebGkiyvVbfWlCLfczayY3AI0s8JyAjSzwnICNLPCcgI0s8JyAmxg+aL/X0p6S9L9kvaQ9Gy942ozUKbVsoHLR4EbWD5h+GrS9bwL6hzLeNKECoMrXOpXi329lzQBwy5AC+ma3y9FxPQa1T+eGr0XSZNJ1xWftvqRWa25BdjYNgVe7ovkJ6mpt/fRDWsDk0iTO2xKmoXml/UMyBpUvWdjKMpCunTqetLJz7OBC3N5iXRS9CukKa4uB9bK68aTLrE6kjTTySzg1LzuGNIJwi2kqzi+x6qzmOwMPExKEL8mnRR9dl53FGVTXuWyADbPjyeTppC6mXRJ3UdIEyM8TJpFZipwRtlrX82vn5+X97XfBzWaVquD3+3OwLwaflarvJdcfjTwNGk2nNuATXO5gB/nz+9t0gns2wHHkiavWJrr+V29/w69tPus6x1AERbSnHeP5i/JSGAY8IG87mjS1ErvIk0icD1wRV7XlgAvIc0+sgPpEqyt8/r2CWZ5AgSGkJLql0lTOB2cv4jdSYBzgfeTkvSwXP8/5efbky5JO6hdrIPK6lu+D2o4rVYHv9+vAPfV8PPq6L1MzJ/T1jn+04C/5nUfI11pMzonw62Bjcp+j2fX+2/QS8eLu8B9YzfSjCLfiIgFEbE4Iu7O6w4Hzo+IFyNiPul63UPbzRjzvYhYFBGPkhLpDlXs872kL+oFkabOuh64v5tx/29E3BMRrTnmOyLi8fz8MdL4415V1lXzabUAJG1Pul76G916Z913HPD9iHg60rjgucCOkjYltfLWALYijas/HTUaj7Te5QTYNzYBXomOB9Q7mv5qEGm6/DY9mW7qHcBrEVF+lKu7UzattL2k90j6i6SZkuaSkkKlKarK46nptFqSNgduAb4cEXd1ss0e+d4r8yU9mcueLCvbo8r4NwV+IukfeXquOaTW3tiI+DNwIfAzYIakSZLW7KIu6yecAPvGVGBcJ/MAdjT9VTOpe7k6pgNj89RNbTYpe7zSVFmSOrr5T/tTBH4F3AhsEhFrkaa2rzRFVZuaTquVW15/Is2WfUVn20XEXRExKi/b5rJty8o6SpwdvZepwBcjYnTZMjzS5K5ExAURsQtpppwtWNEi9WkW/ZgTYN+4n5SQzpM0UtIwSe/P664G/lXSOyWNInWtru2ktdgd95IOkJyUp7GaSOqKt3kU2FbSjpKGkaaZr2QNYE5ELJa0G/CZsnUzSdNpvauT19ZsWi1JY4E/kw4k9caMKx29l4uBUyRtm2NYS9Ih+fG7c+t4MOk/lsWsmFrsTTr/nVidOQH2gYhoIY11bU46wjiNNBszwKWkyUHvJJ17thg4uQb7XEo68HEM6UY+nyUlmyV5/XOke4v8CXgeuLvjmlZyAnCmpHmkcbflsypHmtT0HOCe3E18b7t4ajmt1hdISeWMsq7s/B7U06GO3ktE3EC6kdI1kt4GngD2yy9Zk3Sg6i1St3428MO87hfANrme32L9ik+ELhBJfwMujgifM2eGW4ADmqS9JG2Yu5xHkk5dubXecZn1F76J88C2JambOpJ0M6BP+fQMsxXcBTazwnIX2MwKa8B1gUevs25sNHZcvcOwdoYM8v+1/dVjjzw0KyLWq1V9TWtuGtG8qOJ2sWjmbRGxb6322xMDLgFuNHYcv7zhL/UOw9oZv96IyhtZXWw0emj7K3RWSzQvYuiWn6643eJHflbtVUS9ZsAlQDOrMwlK/Wn2tM45AZpZ7VV/z/q6cgI0s9pb6RL0/ssJ0MxqTG4BmllBCY8BmllRyV1gMyswd4HNrLDcAjSzQvJ5gGZWaO4Cm1kx+TQYMyuyUmOMATZGmjazxtF2HmClpVI10qWSZkh6oqzsDEmvSXokL/uXrTtF0hRJz0r6WDWhOgGaWY3lLnClpbLJQEfTZf04InbMy80AkrYBDgW2za+5SFLFLOsEaGa1J1VeKoiIO0k3oK/GROCaiFgSES8BU1j5NrAdcgI0s9qrrgU4RtKDZcuxVdZ+kqTHchd57Vw2lnTz+jbTclmXfBDEzGqr+vMAZ0XErt2s/efAWUDknz8
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c26d88048>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAAEmCAYAAAATPUntAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcHFW5//HPd2ayJ2QFhJAQBAICP0EERJRNvQiKgl5QFgUuKLKIC+KCoGyiuOGVi4ioyKYRUFSUTVCRRXZki7IEEEgIZIGELEOSyTy/P+pM6Awz0z2T6unuqe+bV73Sfaq66ukJ8+ScU1VPKSIwMyuiploHYGZWK06AZlZYToBmVlhOgGZWWE6AZlZYToBmVlhOgAOIpGGS/ihpoaQr12A/B0v6c56x1YKk6yQd2sfPfkPSPEkv5B2X1Q8nwBqQdJCkeyUtljQ7/aK+M4dd7wesC4yPiP37upOI+GVE7JFDPKuRtJukkPS7Tu1bp/abK9zPqZIuK7ddROwVERf3Ic7JwBeALSLiDb39vDUOJ8B+Jul44H+Bb5Ilq8nAecA+Oex+Q+DxiGjLYV/VMhd4u6TxJW2HAo/ndQBl1uT/7cnA/IiY04djt6zBca2/RYSXflqA0cBiYP8ethlCliCfT8v/AkPSut2AmWS9kznAbOB/0rrTgOXAinSMI4BTgctK9j0FCKAlvT8MeApYBDwNHFzSflvJ53YC7gEWpj93Kll3M3AGcHvaz5+BCd18t474zweOTW3NwCzg68DNJdv+EHgOeAW4D9g5te/Z6Xs+WBLHmSmOVmCT1PaJtP7HwG9L9v9t4C+AOsX4nvT59rT/i1L7B4HpwIK03zeVfOY/wJeBh4BlHT9fL/W/1DyAIi3pl7etp18Q4HTgTmAdYG3gH8AZad1u6fOnA4OA9wFLgbFpfeeE120CBEak5LJZWrcesGV6vSoBAuOAl4GPp88dmN6PT+tvBp4EpgLD0vuzuvluHQlwJ+Cu1PY+4AbgE50S4MeA8emYXwBeAIZ29b1K4ngW2DJ9ZlCnBDicrJd5GLAzMA/YoKc4S95PBZYA/5X2+yVgBjA4rf8P8AAwCRiW2s4Dzqv1/3Neel48BO5f44F50fMQ9WDg9IiYExFzyXp2Hy9ZvyKtXxER15L1UjbrYzztwFaShkXE7IiY3sU27weeiIhLI6ItIqYBjwIfKNnmFxHxeES0AlcA2/R00Ij4BzBO0mbAIcAlXWxzWUTMT8f8PlnPuNz3vCgipqfPrOi0v6VkP8ezgcuA4yJiZpn9dfgocE1E3Jj2+z2yZL9TyTbnRMRz6WdARBwTEcdUuH+rESfA/jUfmFBmnmh94JmS98+ktlX76JRAlwIjextIRCwh+8U+Cpgt6RpJm1cQT0dME0vel54prTSeS4FPA7sDv+u8UtIJkv6dzmgvIJs+mFBmn8/1tDIi7iIb8ossUVdqtZ9BRLSnY5X+DHo8ttUnJ8D+dQfZHNG+PWzzPNnJjA6TU1tfLCEb+nVY7YxmRNwQEf9FNvx9FPhpBfF0xDSrjzF1uBQ4Brg29c5WkbQz2TDzI2TD+zFk84/qCL2bffZY2kjSsWQ9yefT/iu12s9AksiGu6U/A5dVakBOgP0oIhaSTfb/SNK+koZLGiRpL0nfSZtNA06WtLakCWn7spd8dOMBYBdJkyWNBk7sWCFpXUn7SBpBlpQXkw2JO7sWmJou3WmR9FFgC+BPfYwJgIh4GtgVOKmL1aPI5jrnAi2Svg6sVbL+RWBKb870SpoKfINsbvHjwJck9ThUL3EF8H5J75Y0iGxOchnZ/Kw1MCfAfpbms44HTib7BX+ObCj4+7TJN4B7yc4oPgzcn9r6cqwbgcvTvu5j9aTVlOJ4HniJLBkd3cU+5gN7k/3SzyfrOe0dEfP6ElOnfd8WEV31bm8Aric7afEM8CqrDzE7LvKeL+n+csdJUw6XAd+OiAcj4gngq8ClkoZUEOdjZInz/8hOnnwA+EBELO/hmOdLOr/cvq22FOGeu5kVk3uAZlZYToBmVlhOgGZWWE6AZlZYToANJt3o/wtJL0u6W9LOkh6rdVwdBkoprd4orU6TLjlaLKm53LZWe06AjeedZPekbhARO0TErRHR11vh1oikKamM1ao7W6J6pbR2lHSjpJckzZV0paT18j7OmoqIZyNiZESsrHUsVp4TYOPZEPhPupWtqrrrxdTIWOACsoIOG5JVnvlFLQOyxucEWEWSJkm6KvVY5ks6N7U3STpZ0jOS5ki6JN2pUdqrOlTSs6kq8Ulp3RHAz8jq6S2WdFoqMjqz5JjbSvqnpEWpl3S5pG+kdYdJuq1TjCFpk/T6Ikk/lnStpCXA7pLen/b3iqTnJJ1a8vFb0p8LUjxv73wMSTtJuifd03uPpJ1K1t0s6QxJt6d4/5zufnmdiLguIq6MiFfSrXPnAu/o29/M6ykrSvvpTm0PSvpwev3D9P1fkXRful2vq/2s1iuWtJGkv6fvdyPl72e2fuQEWCWp9/QnsjsZppDdOP/rtPqwtOwOvJGseMC5nXbxTrLqJ+8Gvi7pTRHxc7LiBXekYdYpnY45mKywwEVkZaymAR/qZegHkdXVGwXcRnY/8SHAGLLKMEdL6riXeZf055gUzx2d4hkHXAOcQ1YJ52zgGq1eDPUg4H/Iyn8NBk6oMM5dyOrz5WUaWakvACRtQdbTvCY13UNW5WYc8CvgSklDK9jvr8juwplAVjexTyX6rTqcAKtnB7IqIl+MiCUR8WpEdPSMDgbOjoinImIx2T26B2j1KjGnRURrRDwIPAhsXcExdySrhXdOKpd1FXB3L+P+Q0TcHhHtKeabI+Lh9P4hskSxa4X7yr2UFoCkN5PdI/3FXn2znv0O2EZSR9GDg4GrImIZ9K08l7LS+tsDX4uIZRFxC/DHHGO2NeQEWD2TgGe6qf3XVcmrFrIS+R36UmJqfWBWrH5/Y2/LNK22vaS3SfpbGsYvJOuBVjqMy72UVhquXwd8NiJu7WabndOQfLGk6alteknb64avEbGIrLd3QGo6EPhlyT77Up5rfeDlTvO1nX8eVkNOgNXzHDBZXdf+66rkVRtZlZM1MRuYKEklbZNKXq9WHktSVw/86Xxz+K+Aq4FJETGarJx9ubJUHXItpZV6ZzeRVci+tLvt0pnxkWnZMrVtWdLWZeIkDYMlvR0YCvwtHbdcea7uzAbGKqu402Fy+W9q/cUJsHruJvsFOEvSCElDJXVM2k8DPp8myEeSPSDp8jKVoitxB7AS+LSy0lX7kA3FOzwIbClpmzR/dWoF+xwFvBQRr0ragWzOrsNcshJab+zms7mV0pI0EfgrcG5EVKvKyrVkCft0sr+PjvJg5cpzdSkiniGr7HOapMHKnvz3gTIfs37kBFgl6TqwD5A9nOdZsmdhfDStvpCsIOgtZA8jehU4LodjLgc+TPZApAVkJZz+RFa7joh4nOyX+ybgCbKTHOUcA5wuaRHZvNuqSsrpbOyZwO2SFkjasVM8eZbS+gRZoj21ZCi7uA/76Vaa77uK7MFIvypZVa48V08OAt5GVnLsFLoo/2+143JYA5yku4DzI8LXzJl14h7gACNpV0lvSEPOQ4E3k/VezKwTP8R54NmMbJg6guwBQPtFxOzahmRWnzwENrPC8hDYzAprwA2Bx46bEBMn+VKrejOkxf/W1qv7779vXkSsndf+mtfaMKKttex20Tr3hojYM6/j9sWAS4ATJ03mN9d3d52r1cqUtUeU38hqYtgg5Xp3SrS1MmSzj5Td7tUHflTzwhADLgGaWY1J0FRPldS65wRoZvmr/Jn1NeUEaGb5U7nbpOuDE6CZ5UzuAZpZQQnPAZpZUclDYDMrMA+Bzayw3AM0s0LydYBmVmgeAptZMfkyGDMrsibPAZpZEfk6QDMrLg+BzazIGuQymMZI02bWWNRUfim3C+lCSXMkPdKp/ThJj0qaLuk7Je0nSpoh6TFJ760kTPcAzSxf+V0HeBFwLiXPUpa0O7APsHVELJO0TmrfAjgA2BJYH7hJ0tT0fO5uuQdoZvmTyi9
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c26d15fd0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-11 12:19:53 +02:00
"source": [
2018-05-13 18:29:18 +02:00
"test_classifier(labels=[\"true\", \"false\"], title=\"configuration 2 -- train\", Xt=vec_train_2, yt=y2, clf=clf_b)\n",
2018-05-22 16:40:46 +02:00
"cm_2 = test_classifier(labels=[\"true\", \"false\"], title=\"configuration 2 -- test\", Xt=vec_test_2, yt=yt2, clf=clf_b)\n",
2018-05-13 18:29:18 +02:00
"test_classifier(labels=[\"true\", \"false\"], title=\"configuration 2 -- valid\", Xt=vectorizer_2.transform(Xv2), yt=yv2, clf=clf_b)"
2018-05-13 10:35:15 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----\n",
"## configuration 3"
2018-05-09 19:13:08 +02:00
]
},
2018-05-13 12:44:36 +02:00
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 18,
2018-05-13 12:44:36 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.5142231947483589'\n",
"Confusion matrix, without normalization\n",
"array([[ 7, 201],\n",
" [ 21, 228]])\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUEAAAEmCAYAAAD8/yLTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XecVOX1x/HPd0E6Cgpio4iKNYpiSTQqRmzYjWLvxhKNMSYmRv3ZoomJJcYYNRq7BiXBQhALEokVC0rRCCgtgnQRQTqc3x/PM3gZd3dmd+fu7OycN6/7YubeO/eemZ09+zy3nEdmhnPOlauKYgfgnHPF5EnQOVfWPAk658qaJ0HnXFnzJOicK2ueBJ1zZc2TYCMjqaWkf0laIOkfddjOyZJeKmRsxSDpeUmn1/K1N0iaK2lmoeNyDYcnwSKRdJKk9yQtkjQj/rJ+vwCbPhboBGxgZsfVdiNm9riZHViAeNYiqbckk/R01vyd4vzheW7nWkmP5VrPzA4xs4drEWcX4OfAdma2UU1f70qHJ8EikHQpcDvwW0LC6gLcBRxZgM13BSaY2coCbCstc4DvSdogMe90YEKhdqCgLt/vLsA8M5tdi303rcN+XX0zM5/qcQLWAxYBx1WzTnNCkvw8TrcDzeOy3sA0QitlNjADODMuuw5YDqyI+zgbuBZ4LLHtboABTePzM4BJwEJgMnByYv7ridftCbwLLIj/75lYNhz4DfBG3M5LQIcq3lsm/nuAC+O8JsB04GpgeGLdPwGfAV8BI4G94/yDs97n6EQcN8Y4lgBbxnnnxOV3AwMT2/89MAxQVox94utXx+0/FOcfAXwEfBm3u23iNVOAXwFjgGWZz9enhj8VPYBym+Iv8MrqfkmA64ERwIZAR+BN4DdxWe/4+uuBdYC+wGKgfVyenfSqTIJA65hgto7LNga2j4/XJEFgfWA+cGp83Ynx+QZx+XBgItADaBmf31TFe8skwT2Bt+O8vsCLwDlZSfAUYIO4z58DM4EWlb2vRBz/A7aPr1knKwm2IrQ2zwD2BuYCm1UXZ+J5D+Br4IC43V8CnwLN4vIpwCigM9AyzrsLuKvY3zmfqp+8O1z/NgDmWvXd1ZOB681stpnNIbTwTk0sXxGXrzCzIYTWyta1jGc1sIOklmY2w8w+qmSdQ4FPzOxRM1tpZv2BccDhiXUeNLMJZrYEGAD0rG6nZvYmsL6krYHTgEcqWecxM5sX93kroYWc630+ZGYfxdesyNreYsLneBvwGPATM5uWY3sZxwPPmdnQuN1bCAl/z8Q6d5jZZ/EzwMx+bGY/znP7rkg8Cda/eUCHHMeNNgGmJp5PjfPWbCMriS4G2tQ0EDP7mvDLfT4wQ9JzkrbJI55MTJsmnifPoOYbz6PARcB+wNPZCyX9QtLH8Uz3l4RDCR1ybPOz6haa2duE7r8IyTpfa30GZrY67iv5GVS7b9cweRKsf28RjhkdVc06nxNOcGR0ifNq42tCNzBjrTOdZvaimR1A6AqPA+7LI55MTNNrGVPGo8CPgSGxlbaGpL0JXc5+hK5+O8LxSGVCr2Kb1ZZFknQhoUX5edx+vtb6DCSJ0PVNfgZekqkEeRKsZ2a2gHAC4C+SjpLUStI6kg6R9Ie4Wn/gKkkdJXWI6+e8HKQKo4B9JHWRtB7w68wCSZ0kHSmpNSExLyJ0j7MNAXrEy3qaSjoe2A4YXMuYADCzycC+wJWVLG5LOPY5B2gq6Wpg3cTyWUC3mpwBltQDuIFwrPFU4JeSqu22JwwADpW0v6R1CMcolxGO17oS5kmwCOLxrUuBqwi/5J8RuoXPxFVuAN4jnGkcC7wf59VmX0OBJ+O2RrJ24qqIcXwOfEFISBdUso15wGGEX/x5hBbUYWY2tzYxZW37dTOrrJX7IvAC4UTGVGApa3c3MxeCz5P0fq79xMMPjwG/N7PRZvYJcAXwqKTmecQ5npA8/0w4oXI4cLiZLa9mn/dIuifXtl1xycxb8M658uUtQedcWfMk6Jwra54EnXNlzZOgc66slX0SjDfaPyhpvqR3JO0taXyx48poLCWtaiLfCjFx3eGSzqnFPppLGiepY80jrPG+zpD0etr7cbVT9kkQ+D7hftDNzGx3M3vNzGp7C1qdSOoWy0mtuZvE0itptV0s5TU/Ti9L2q7Q+2mozGwZ8ABwebFjSarJH4A09xP/SNwvaaqkhZJGSTok7biKwZNguAtgSryFLFWSmqS9jxr4nFB7cH3CrWiDgCeKGlH9+ztwej7XCZahpoTrMvcl3K54FTBAUrcixpSOYldwqMlEuE3pKcIFxvOAO+P8CsIPaSqhvNQjwHpxWTfC7UynEyqMzAWujMvOJlyEu4pwt8R1fLt6yC7AB4QSUf8gXHh8Q1x2BolyU3GeAVvGxw8RyjcNIdy+1odQjOADQvWWz4BrE6/9X3z9ojh9L3sfFKikVVbMTYELgcU1+FkMJ1zA/WaM9V+E4hCPx/f2LtAtz7g3B/4TYx4K3MnalW++G/fzJTAa6J0VxzlVxLg74TbFLwklx+4kVn1JrPMJsG8Nv4fKsXwDwh+Vr4B34s8k+TOsaYmwM4GP4+czCTgvsa0OhAvgvyRc8P4aUBGXbQIMJPy+TAYurm4/ebzvMcAPi50HCj0VPYAafPGaxF+APxJKQLUAvh+XnUUoa9SdcOP+U8CjcVk3QmK5j1D1YyfC7U7bxuVnZH1BexOTINCMkFh/SiifdEz88tQkCS4A9iIk6hZx+9+Jz3ck3P51VFasTRPbW7MPCljSKrH9Lwm3p60GrkrMPwkYU83rhsfPfAtCS+G/hLs7+sTYHiFUlskn7rcIlV2aA/sQftkfi8s2JfzB6xs/swPi846JOKpKgr0ICbRp/Gw/Bi7JWmcQMTnk+T1sSkjIu1azzhOE2+xaAzsQ7i9OfsdqWiLs0Pg5i9AyWwzsEpf9jlCbcZ047R3XqyAk2KsJ3+PuhAR6UFX7yfG+OxEaDNsUOxcUeiql7vDuhL9sl5nZ12a21MwyB5tPBm4zs0lmtohwf+wJWZVarjOzJWY2mpBMd8pjn5lfoDsslK16ivCXvSaeNbM3zGx1jHm4mY2Nz8cQ7hPeN89tpVHSqh0hiV1EaKFm5v/dzHbMEc+DZjbRwv3QzwMTzexlCxVu/gHsnCvuWMZ+N+D/zGyZmb1KaFVmnEIosDAkfmZDCbcU9s0RG2Y20sxGxH1OAf7Ktz/rhUC7XNtKbHMloXDrYEm7Zi+Phzx+CFwdv6cfAg9nbaNGJcLM7Ln4OZuZ/YfQwt87Ll5BKH7RNX5HX7OQtXYj/KG43syWm9kkQkPghHzfa+I9rUNo4T9sZuNq+vqGrpSSYGdgqlVeh6+y0lNNCX+9MmpT6mkTYHr8UmXUtFzSWutL2kPSK5LmSFpAKGOVqzxUMp6Cl7SycDz0HuARSRvmGQuEVmzGkkqeZ/ZdXdybAPNt7WOyyXW7AsdJ+jIzEU5mbZwrOEk9JA2WNFPSV4ThDLI/67aE1nBlr58ZT1StNRG6n52A+yt5WUe+OZ5W2fupcYmwWFxjhKQv4vp9E+vfTGiRvyRpkqTMiZ6uwCZZn9sVrP07kVMsUPEooQd0UU1eWypKKQl+BnSpog5fZaWnVrL2L2VtzAA2jWWTMjonHq9VpkpSZQPyZN+c/XdCF6yzma1HSD65ykNlpFXSCsJ3oRVrJ9RCqS7uGUD7WMkmuSzjM8KhjXaJqbWZ3ZTHfu8mtDi3MrN1CUlAWetsS+gZfIuZbWRmyp4IxSRmEY4pZ5tD+O4lvydr3k9NS4TFkzYDCUVcO8X1h2TWN7OFZvZzM+tOKP9/qaT9CZ/b5KzPra2Z9a1sP5WJ3/v7CYnzh5ZVpLaxKKUk+A7hF+YmSa0ltZC0V1zWH/iZpM0ltSH8xX+yilZjTbxFOGlyUSwhdSShW54xGtheUk9JLQjHWXJpC3xhZksl7U449pYxh3BsrnsVry1YSStJB0jaWVITSesSjsnNJxw3K7Qq4zazqYTu7XWSmimMuJfs3j9G6DYfFGNtoTBi3WZ57Lct4eTDIoV
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c26d88630>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-13 12:44:36 +02:00
"source": [
"yt2_c3 = yt2.copy()\n",
"yt2_c3[yt2_c3 == \"true\"] = \"REAL\"\n",
"yt2_c3[yt2_c3 == \"false\"] = \"FAKE\"\n",
"\n",
2018-05-22 16:40:46 +02:00
"cm_3b=test_classifier(labels=[\"REAL\", \"FAKE\"], \n",
2018-05-13 12:44:36 +02:00
" title=\"configuration 3: model a) → dataset 2\",\n",
2018-05-13 18:29:18 +02:00
" Xt=vectorizer_1.transform(Xt2),\n",
2018-05-13 12:44:36 +02:00
" yt=yt2_c3, clf=clf_a)"
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 19,
2018-05-13 12:44:36 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.5391414141414141'\n",
"Confusion matrix, without normalization\n",
"array([[274, 491],\n",
" [239, 580]])\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAAEmCAYAAAATPUntAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XecFPX9x/HX+46mVBVEpIgNwQY2bBhQoyIW0Fgw2InYSOwao/kpKkaNJTG2mBCNYo0lErDE2BKxgAqigAUUBESagHQpn98f3+/h3nG3u3fO3d7efp485sHuzOzMZ/f2Pvf9fmfmMzIznHOuEBXlOgDnnMsVT4DOuYLlCdA5V7A8ATrnCpYnQOdcwfIE6JwrWJ4A6xBJG0n6l6TFkv7xI7YzQNK/k4wtFyS9IOm0Kr72BknzJX2TdFyu9vAEmAOSfi7pPUlLJc2Ov6g9Etj0cUBrYDMzO76qGzGzR8zs0ATiKUVSL0km6dky87vG+a9nuZ1rJQ3PtJ6ZHW5mf69CnB2AS4AdzWyLyr7e5Q9PgDVM0sXAH4AbCcmqA3AP0DeBzW8FfGZmaxLYVnWZB+wrabOUeacBnyW1AwU/5rvdAVhgZnOrsO96P2K/rqaZmU81NAHNgaXA8WnWaUhIkF/H6Q9Aw7isFzCT0DqZC8wGzojLhgDfA6vjPgYC1wLDU7bdETCgXnx+OvAFsAT4EhiQMv/NlNftB4wFFsf/90tZ9jpwPTA6buffQMsK3ltJ/PcB58d5xcAs4P+A11PW/SMwA/gOeB84IM7vXeZ9fpgSx9AYxwpguzjvF3H5vcDTKdu/GXgFUJkYfxpfvy5u/8E4/2hgIrAobrdLymumAVcAE4BVJZ+vT7V/ynkAhTTFX9416X5BgOuAd4DNgVbAW8D1cVmv+PrrgPpAH2A5sElcXjbhVZgAgcYxuewQl7UBdoqP1ydAYFNgIXBKfN1J8flmcfnrwFSgE7BRfH5TBe+tJAHuB7wb5/UBXgJ+USYBngxsFvd5CfAN0Ki895USx1fATvE19cskwI0JrczTgQOA+UC7dHGmPO8ELAMOidu9HJgCNIjLpwHjgfbARnHePcA9uf7O+ZR+8i5wzdoMmG/pu6gDgOvMbK6ZzSO07E5JWb46Ll9tZs8TWik7VDGedcDOkjYys9lmNrGcdY4APjezh81sjZk9BnwCHJWyzgNm9pmZrQCeBLql26mZvQVsKmkH4FTgoXLWGW5mC+I+byO0jDO9zwfNbGJ8zeoy21tO+BxvB4YDvzSzmRm2V+JEYJSZvRy3eysh2e+Xss6dZjYjfgaY2Xlmdl6W23c54gmwZi0AWmYYJ9oSmJ7yfHqct34bZRLocqBJZQMxs2WEX+xzgNmSRknqnEU8JTG1TXmeeqQ023geBgYDBwLPll0o6VJJk+MR7UWE4YOWGbY5I91CM3uX0OUXIVFnq9RnYGbr4r5SP4O0+3a1kyfAmvU2YYyoX5p1viYczCjRIc6rimWErl+JUkc0zewlMzuE0P39BPhLFvGUxDSrijGVeBg4D3g+ts7Wk3QAoZt5AqF734Iw/qiS0CvYZtrSRpLOJ7Qkv47bz1apz0CSCN3d1M/AyyrlIU+ANcjMFhMG+++W1E/SxpLqSzpc0i1xtceAqyW1ktQyrp/xlI8KjAd+IqmDpObAlSULJLWW1FdSY0JSXkroEpf1PNApnrpTT9KJwI7AyCrGBICZfQn0BK4qZ3FTwljnPKCepP8DmqUsnwN0rMyRXkmdgBsIY4unAJdLSttVT/EkcISkgyXVJ4xJriKMz7o85gmwhsXxrIuBqwm/4DMIXcF/xlVuAN4jHFH8CPggzqvKvl4Gnojbep/SSasoxvE18C0hGZ1bzjYWAEcSfukXEFpOR5rZ/KrEVGbbb5pZea3bl4AXCQctpgMrKd3FLDnJe4GkDzLtJw45DAduNrMPzexz4DfAw5IaZhHnp4TE+SfCwZOjgKPM7Ps0+7xP0n2Ztu1yS2becnfOFSZvATrnCpYnQOdcwfIE6JwrWJ4AnXMFq6ATYLxo/gFJCyWNkXSApE9zHVeJulKWqjKyrfQS131d0i+qsp34896pqnFmK1bAyfaKE1fDCjoBAj0I13e2M7PuZvY/M6vqZWU/iqSOsSTU+qtErPrKUu0Yy3EtjNN/JO2Y9H5quVsJ11TXGpJOl/RmbdiPpBMkvSVpebZlyvJRoSfArYBp8bKwaiWpuLr3UQlfE2oHbkq4vGwE8HhOI6p5I4ADJXm9v/J9S6hEdFOuA6lOeZMAJbWX9IykeZIWSLorzi+SdLWk6ZLmSnooXvWQ2qo6TdJXscLvVXHZQOCvhNp0SyUNKdtdkbS7pHGSlkj6h6QnJN0Ql23wVzTua7v4+EFJ90p6XtIywi/bEXF730maIenalJf/N/6/KMazb9l9SNpP0th4fexYSfulLHtd0vWSRsd4/x2vJNmAmS0ys2kWTgIVsJZQPirbn8XrChWT34qx/kvSZpIeie9trKSOWca9taQ3YswvU+Z6X0n7xP0skvShpF7Zxgk0ij+zJZI+kNQ15TNYSTg5/LBKbK/kMrh0yzeKP/uFkiYBe5VZ/mtJU2NMkyQdE+d3IZQJK/k+LorzK/zOSGokaXj8fVgUP9vWcVlzScMUCu7Oij+v4or2U5aZ/cfMnqTql2Hmh1yXo8lmItSM+xC4g1DGqRHQIy47k1CaaBvCRfjPAA/HZR0J12j+hVC9oyvhEqYucfnplK5714tYBgloQLgK4QJCCaRjCXXobijvtXGeAdvFxw8Srl/dn/CHplHc/i7x+a6ES7r6lYm1Xsr21u+DBMtSpWx/EeGSs3XA1Snzfw5MSPO61+Nnvi2hSMEkwlUbP42xPUSoEJNN3G8TKrQ0BH5CqCk4PC5rS7j6pE/8zA6Jz1ulxPGLCmK8llA557j487uUUPOwfso6dwK3V/K7+C/giDTLbwL+F993e+BjSpfWOp5QXKGIUIxiGdAmzXcq3Xfm7BjPxoTfkT2AZnHZs8CfCb8vmwNjgLMr2k+a91OqTFldm/KlBdid8KW5zMyWmdlKMytpGQ0gfIm/MLOlhOtd+6t0xZUhZrbCzD4kJNKuZLYP4Rf2Tgulp54hfIkq4zkzG21m62LMr5vZR/H5BMJ1vz2z3FZ1lKVqQUhgg4FxKfMfNbNdM8TzgJlNtXB98wvAVAuthjWES9V2yxS3Qun5vYDfmtkqM/sv4Re6xMmEYgnPx8/sZcJlgn0yxFbifTN7ykIJq9sJf4T2SVm+BGiR5bZKDAGGSTqiguUnAEPN7Fszm0FIsuuZ2T/M7Ov4fp4APid8v8uV4TuzmlBibTszW2tm75vZd7EV2Ae4MP6+zCU0HvpX8r3WeflSvrs9MN3Kr6NXXvmoeoRy8yWqUq5pS2CWxT+DUWVLHpVaX9LehBbCzoQWZkN+uK41m3gSL0tlZssUrlmdJ6mLZV8Gfk7K4xXlPC/Zd7q4twQWWukx2OmEnzeEMdrjJaUm+frAa1nGuP7zN7N1cXgjtbRYU0IreAOS3gH2TrPtRwl/PMraktI/91LvXdKphGuwO8ZZTUhT5ivDd+Zhwmf1uKQWhOudryJ8bvUJZc5KNlWEl+zaQL60AGcAHVR+Hb3yyketofQvZFXMBtqWGfNpn/K4VKkplT+YXvZC60cJg+/tzaw5YSwmU4mnEtVVlgrC92BjSifTpKSLezawiUJFmtRlJWYQhjNapEyNzSzbgfn1Py+FyjHtKD2m1YXQI9iAme1jZio7EVqscwjDBOWZTenvyfr3I2krwnDMYMIQQAtCFzndd6DC70zsmQwxsx0JxVmPJBSYnUEY6mmZ8rk1M7OS0368AECULwlwDOGLdZOkxnHwd/+47DHgojiY3oRws6EnKmgtVsbbhIMDgxXKQPWldFflQ2AnSd0kNSKMOWXSFPjWzFZK6k7pX6J5hLG4bSp4bWJlqSQdImm3OCjejNA9XAhMruy2slBh3GY2ndClHSKpgcKd8VJbe8MJXeXDYqyNFA5Utcty33tIOjb+4byQkBTegXAAgTBm9nI
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c246e2fd0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-13 12:44:36 +02:00
"source": [
"yt1_c3 = yt1.copy()\n",
"yt1_c3[yt1_c3 == \"REAL\"] = \"true\"\n",
"yt1_c3[yt1_c3 == \"FAKE\"] = \"false\"\n",
"\n",
2018-05-22 16:40:46 +02:00
"cm_3a=test_classifier(labels=[\"true\", \"false\"], \n",
2018-05-13 12:44:36 +02:00
" title=\"configuration 3: model b) → dataset 1\",\n",
2018-05-13 18:29:18 +02:00
" Xt=vectorizer_2.transform(Xt1),\n",
2018-05-13 12:44:36 +02:00
" yt=yt1_c3, clf=clf_b)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----\n",
"## configuration 4)"
]
},
2018-05-13 13:36:21 +02:00
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 20,
2018-05-13 13:36:21 +02:00
"metadata": {},
"outputs": [],
"source": [
"def get_dataset3_split(dataset1_in, dataset2_in):\n",
" try:\n",
" print('processing datasets')\n",
" print('ds1=', dataset1_in)\n",
" print('ds2=', dataset2_in)\n",
"\n",
" print('-- fake news')\n",
" df1 = pd.read_csv(dataset1_in, sep=',', usecols=['title','text','label'])\n",
" df1['claim'] = df1[['title', 'text']].apply(lambda x: '. '.join(x), axis=1)\n",
" del df1['title']\n",
" del df1['text']\n",
" df1.rename(index=str, columns={'label': 'y'}, inplace=True)\n",
" print(df1.keys())\n",
" print(len(df1[df1['y']=='REAL']))\n",
" print(len(df1[df1['y']=='FAKE']))\n",
" df1['y'] = np.where(df1['y'] == 'FAKE', 'false', 'true')\n",
" print(len(df1))\n",
"\n",
" print('-- liar liar')\n",
" df2 = pd.read_csv(dataset2_in, sep='\\t', header=None, usecols=[1,2], names=['y', 'claim'])\n",
" print(df2.keys())\n",
" print(set(df2.y), len(df2))\n",
" print(len(df2[df2['y'] == 'true']))\n",
" print(len(df2[df2['y'] == 'false']))\n",
" df2=df2[(df2['y'] == 'true') | (df2['y'] == 'false')]\n",
" print(set(df2.y), len(df2))\n",
"\n",
" df3=pd.concat([df1, df2], ignore_index=True)\n",
"\n",
" print(df3['y'].value_counts())\n",
" print('done')\n",
2018-05-17 10:38:34 +02:00
" return train_test_split(df3['claim'], df3['y'], test_size=0.3, random_state=4222)\n",
2018-05-13 13:36:21 +02:00
" except Exception as e:\n",
" print(e)"
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 21,
2018-05-13 13:36:21 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"processing datasets\n",
"ds1= data/fake_or_real_news.csv\n",
"ds2= data/train.tsv\n",
"-- fake news\n",
"Index(['y', 'claim'], dtype='object')\n",
"3171\n",
"3164\n",
"6335\n",
"-- liar liar\n",
"Index(['y', 'claim'], dtype='object')\n",
"{'half-true', 'false', 'pants-fire', 'mostly-true', 'true', 'barely-true'} 10240\n",
"1676\n",
"1995\n",
"{'false', 'true'} 3671\n",
"false 5159\n",
"true 4847\n",
"Name: y, dtype: int64\n",
"done\n"
]
}
],
2018-05-13 13:36:21 +02:00
"source": [
"X3, Xt3, y3, yt3 = get_dataset3_split('data/fake_or_real_news.csv', 'data/train.tsv')"
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 22,
2018-05-13 13:36:21 +02:00
"metadata": {},
"outputs": [],
"source": [
2018-05-17 10:38:34 +02:00
"vectorizer_3 = TfidfVectorizer(stop_words='english', max_df=0.7)\n",
2018-05-13 18:29:18 +02:00
"vec_train_3 = vectorizer_3.fit_transform(X3)\n",
"vec_test_3 = vectorizer_3.transform(Xt3)"
2018-05-13 13:36:21 +02:00
]
},
2018-05-17 10:38:34 +02:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* using MLP again"
]
},
2018-05-13 13:36:21 +02:00
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 23,
2018-05-13 13:36:21 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"data": {
"text/plain": [
"MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,\n",
" beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
" hidden_layer_sizes=(16, 16), learning_rate='constant',\n",
" learning_rate_init=0.001, max_iter=200, momentum=0.9,\n",
" nesterovs_momentum=True, power_t=0.5, random_state=4222,\n",
" shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,\n",
" verbose=False, warm_start=False)"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
2018-05-13 13:36:21 +02:00
"source": [
2018-05-17 10:38:34 +02:00
"clf_3 = MLPClassifier(hidden_layer_sizes=(16,16), random_state=4222)\n",
2018-05-13 18:29:18 +02:00
"clf_3.fit(vec_train_3, y3)"
2018-05-13 13:36:21 +02:00
]
},
{
"cell_type": "code",
2018-05-22 16:40:46 +02:00
"execution_count": 24,
2018-05-13 13:36:21 +02:00
"metadata": {},
2018-05-22 16:40:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.9997144488863506'\n",
"Confusion matrix, without normalization\n",
"array([[3367, 1],\n",
" [ 1, 3635]])\n",
"'score: 0.7714856762158561'\n",
"Confusion matrix, without normalization\n",
"array([[1136, 343],\n",
" [ 343, 1180]])\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUYAAAEmCAYAAAAeIzmqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XecVdW5//HPdxhahEi1gcaGLd6IvVx7IbZETTTXEsUWe7xeNdFoij2aGJOYRI3+JGpMLNfEKyqGYO8GUSzYABUFUUGUiCJSnt8few2eOcyccxj2zJkz83372i/OWbustQfnYa1dnqWIwMzMvlBX7QaYmbU3DoxmZkUcGM3MijgwmpkVcWA0MyviwGhmVsSBsQOR1FPSnZJmS/rfZTjOIZL+mWfbqkHSPZKGt3DfCyTNlPRu3u2y9s+BsQokHSzpaUlzJE1Pv8Db5nDo/YEVgf4RcUBLDxIRf4mIYTm0pxFJO0oKSbcXlW+Uyh+s8DjnSLqx3HYRsUdEXN+Cdq4GnAZsEBErLe3+VvscGNuYpFOB3wAXkQWx1YArgH1yOPxXgNciYkEOx2otM4CtJfUvKBsOvJZXBcosy//bqwEfRMT7Lai7fhnqtfYiIry00QIsD8wBDiixTXeywPlOWn4DdE/rdgSmkvVm3gemA0ekdecCnwPzUx1HAecANxYce3UggPr0/XDgdeBj4A3gkILyRwv22wYYC8xOf25TsO5B4HzgsXScfwIDmjm3hvZfBZyYyroA04CfAg8WbPtb4G3g38A4YLtUvnvReT5X0I4LUzvmAmunsqPT+iuBvxUc/xLgPkBFbdw17b8oHf+6VP5NYALwUTru+gX7vAmcATwPzGv4+Xqp3aXqDehMS/qlXlDqFwc4D3gSWAEYCDwOnJ/W7Zj2Pw/oCuwJfAr0TeuLA2GzgRFYLgWdddO6lYGvps+LAyPQD/gQODTtd1D63j+tfxCYDKwD9EzfL27m3BoC4zbAU6lsT2A0cHRRYPwu0D/VeRrwLtCjqfMqaMdbwFfTPl2LAuOXyHqlhwPbATOBwaXaWfB9HeATYLd03B8Ck4Buaf2bwHhgVaBnKrsCuKLa/895adnioXTb6g/MjNJD3UOA8yLi/YiYQdYTPLRg/fy0fn5EjCLr1azbwvYsAjaU1DMipkfEhCa22QuYGBF/jogFEXET8ArwjYJt/hQRr0XEXOBWYGipSiPicaCfpHWBw4Abmtjmxoj4INX5K7KedLnzvC4iJqR95hcd71Oyn+NlwI3A9yNiapnjNfgv4O6IGJOOeynZPwLbFGxzeUS8nX4GRMQJEXFChce3dsaBsW19AAwocx1qFWBKwfcpqWzxMYoC66dAr6VtSER8QvYLfxwwXdLdktaroD0NbRpU8L3wzm2l7fkzcBKwE3B78UpJp0t6Od1h/4jsMsSAMsd8u9TKiHiK7NKByAJ4pRr9DCJiUaqr8GdQsm6rLQ6MbesJsmtQ+5bY5h2ymygNVktlLfEJ2RCyQaM7rBExOiJ2IxtGvwJcU0F7Gto0rYVtavBn4ARgVOrNLSZpO7Lh6nfILhP0Ibu+qYamN3PMkqmiJJ1I1vN8Jx2/Uo1+BpJENmwu/Bk4TVUH4sDYhiJiNtlNhj9I2lfSlyR1lbSHpF+kzW4CfixpoKQBafuyj6Y0YzywvaTVJC0P/KhhhaQVJe0jaTmyYD2HbGhdbBSwTnrEqF7SfwEbAHe1sE0ARMQbwA7A2U2s7k12LXUGUC/pp8CXC9a/B6y+NHeeJa0DXEB27fJQ4IeSSg75C9wK7CVpF0ldya55ziO7/msdkANjG0vXy04Ffkz2i/822ZDy/9ImFwBPk93hfAF4JpW1pK4xwC3pWONoHMzqUjveAWaRBanjmzjGB8DeZMHgA7Ke1t4RMbMlbSo69qMR0VRveDTwD7KbJVOAz2g8VG14eP0DSc+UqyddurgRuCQinouIicBZwJ8lda+gna+SBdTfkd20+QbwjYj4vESdV0m6qtyxrX1ShEcAZmaF3GM0MyviwGhmVsSB0cysiAOjmVkRB8YaVZxirL2lCpN0lqT/V+12VEN6PGqOpC7Vbou1jANjK2urFGPRSqnCKpHSiTV6vS4iLoqIo1u53hEpXdnaOR7zcEmPLssxIuKtiOgVEQvzape1LQfGVtQRUozlkMKrVaR/XNaqUt3uCXZ01c5i0VEX2j7F2OE0ThU2DHiV7FW6K4CH+CLTzDmUTkf2IEum8DoCeJkstdjrwLFp2+VonKZrDtm7xcV1lEvbdTrZg+izyR5K71Hi51YPPAt8LbV77Zz+ztYne5h8YTqPj1L5dWRpy0aRvWa5K1lyjWfJMhS9DZxT5udZUWo2L+1jaXc9gQ5ka6AHTSRIKHA2sBVZNpqNgC3I3ohpsBJZgB1EFvz+IKlvRPyMrBd6S2RDtmsLD5peJbyN7BXA/mQBsjATTCUOBY4hez1vCllw3pvs1bwjgF9L2iSyZBR7AO+ktvSKordZ0ut4NwGnkKVSGwXcKalbwWbfIUvLtgZZwDu8RNv+B3g4Ip5fynMqKSJeJkuq8UQ6jz4Fqw8m+8eiN/AoWYA8DOhDFiSPl1TqHfiDyX5uKwDdyP4hsHbKgbH1VDPF2J7AhIj4e6r/chpnwKnEdVGQwisi7o6IyZF5iKzXs12Fx6o0bdc7ETELuJNmUpdJWhU4luwd8rZ0R0Q8FhGLIuKziHgwIl5I358nC/w7lNj/T7EUqdmsuhwYW081U4ytQsG7xRERZMPypdEojVZKdPGkpFkpDdielE8DVtiecmm7Kk1d9huyfyxml6s03Rmfk5arCu4Wz5E0p8K2Nyj+eWwp6QFJMyTNJutplvp5tCQ1m1WJA2PraesUY4WmA4MbvqQ0WYML1pdMR5Ysfok+JVr4G1lPb8U0xBxF+TRgDSpJ21WpXYBfSnq3YAa/JyQdvMQJZHfGG4b3x8UXd4t7RURzganSlGZ/BUYCq0bE8mTTNWiJvawmOTC2kmj7FGOF7gb+I9VbD5xI4+DXbDqyZnQju1E0A1ggaQ+ymzsN3gP6p2M1Jc+0XeuQXY8dyhfD0W9Q+lru0ngPGFx0/bMpvYFZEfGZpC3IriFaB+HA2IqiDVOMFdU7EzgA+AXZkH6DVM+8tL5UOrKmjvcxcDJZgPuQLAiMLFj/ClmQf13SR5JWKdp/qdN2lWjL+xHxbsOSimema3d5uJ/s7vm7kkqlVjsBOE/Sx2T/oC1NRnBr55x2rBNIzyFOJZsF8IFqt8esvXOPsYOS9HVJfdL1wbPIrn89WeVmmdUEB8aOa2uyaU0bhq775jjcNOvQPJQ2MyviHqOZWZFSDx/XJHVbLtSjb7WbYUU2HtLUo5LWHjzzzLiZETEwr+N1+fJXIhaUv2oTc2eMjojd86o3Tx0vMPboS/fNT6x2M6zIY6PLPSpp1dKzq6aU36pysWAu3df9TtntPhv/h0rfnGpzHS4wmlmVSVBX25nZHBjNLH/tL4XnUnFgNLP8qbZfG6/tsG5m7ZCyHmO5pdxRpB6S/iXpOUkTJJ2byq+T9Iak8WkZmsol6XJJkyQ9L2mTgmMNlzQxLcPL1e0eo5nlS+R1jXEesHNEzEnJRx6VdE9a94OIuK1o+z2AIWnZkizr+paS+gE/AzYjy5I0TtLIiPiwuYrdYzSznCkbSpdbykhJkRvyZnZNS6k3UvYBbkj7PQn0kbQy8HVgTETMSsFwDFm2+GY5MJpZ/iobSg9IM2g2LMcscRipi6TxZFNrjImIp9KqC9Nw+dcpHwBkiY8LEwpPTWXNlTfLQ2kzy19lN19mRsRmpTaIbAraoZL6ALdL2pAsf+i7ZHlCrwbOAM5btgY35h6jmeWr4TnGcstSiIiPgAeA3SNiehouzwP+RDaJHGQZ4Vct2G1wKmuuvFkOjGaWv3zuSg9MPUUk9QR2A15J1w0bpsjYF3gx7TISOCzdnd4KmB0R04HRwDBJfSX1Jcs+P7pU3R5Km1nOlNcD3isD10vqQtaJuzUi7pJ0v6SBWUWMJ5uIDLJ5iPYEJpFNOHYEQETMknQ+MDZ
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c2429df28>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUYAAAEmCAYAAAAeIzmqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucXdPdx/HPN3cRJBJSQhqVuLcI4lIqrq27p0W16tKqa7RVdeepupZWtTxEHkqptkpdHlTc6lKlkiYhVBCZkFQQuUiCJEgmv+ePvSY5czKXM2PPnDkz33de+zVnr73O2uvMZH6z1l57r6WIwMzMVuhU7gqYmbU1DoxmZkUcGM3MijgwmpkVcWA0MyviwGhmVsSBsR2RtIqkByQtkPSXz1DOEZIezbNu5SDpIUlHN/O9l0iaI2lm3vWyts+BsQwkfVvSeEkfSXo3/QLvnEPRhwD9gb4RcWhzC4mIP0bE3jnUpxZJwyWFpHuL0rdM6U+VWM7PJP2hsXwRsU9E3NqMeg4EfgJsFhGfa+r7rfI5MLYySacBvwEuIwtiA4GRwEE5FP954PWIWJpDWS1lNrCjpL4FaUcDr+d1AmU+y//tgcDciJjVjHN3+QzntbYiIry10gasAXwEHNpAnu5kgfOdtP0G6J6ODQdmkLVmZgHvAt9Nxy4EPgWWpHMcC/wM+ENB2YOAALqk/WOAN4APgTeBIwrSnyl4307AOGBB+rpTwbGngIuBZ1M5jwL96vlsNfUfBYxIaZ2Bt4GfAk8V5L0aeAv4AJgA7JLSv1b0OV8sqMelqR6LgcEp7fvp+PXA3QXlXwE8Dqiojnum9y9L5d+S0g8EJgHzU7mbFrxnGnAW8BLwSc3311vlbmWvQEfa0i/10oZ+cYCLgDHA2sBawD+Bi9Ox4en9FwFdgX2BRUCfdLw4ENYbGIFVU9DZOB1bB9g8vV4eGIE1gXnAkel930r7fdPxp4CpwEbAKmn/8no+W01g3AkYm9L2BR4Bvl8UGL8D9E3n/AkwE+hR1+cqqMd/gM3Te7oWBcaeZK3SY4BdgDnAeg3Vs2B/I2AhsFcq90ygCuiWjk8DJgLrA6uktJHAyHL/n/PWvM1d6dbVF5gTDXd1jwAuiohZETGbrCV4ZMHxJen4kogYTdaq2biZ9VkGbCFplYh4NyIm1ZFnP2BKRNwWEUsj4nbgNeCAgjy/i4jXI2IxcCewVUMnjYh/AmtK2hg4Cvh9HXn+EBFz0zl/RdaSbuxz3hIRk9J7lhSVt4js+3gV8AfgBxExo5HyanwTeDAiHkvlXkn2R2CngjzXRMRb6XtARJwcESeXWL61MQ6MrWsu0K+R61DrAtML9qentOVlFAXWRUCvplYkIhaS/cKfCLwr6UFJm5RQn5o6DSjYLxy5LbU+twGnALsB9xYflHS6pFfTCPt8sssQ/Rop862GDkbEWLJLByIL4KWq9T2IiGXpXIXfgwbPbZXFgbF1PUd2DergBvK8QzaIUmNgSmuOhWRdyBq1Rlgj4pGI2IusG/0acGMJ9amp09vNrFON24CTgdGpNbecpF3IuquHkV0m6E12fVM1Va+nzAanipI0gqzl+U4qv1S1vgeSRNZtLvweeJqqdsSBsRVFxAKyQYbrJB0sqaekrpL2kfSLlO124HxJa0nql/I3emtKPSYCX5E0UNIawDk1ByT1l3SQpFXJgvVHZF3rYqOBjdItRl0kfRPYDPhrM+sEQES8CewKnFfH4dXIrqXOBrpI+imwesHx94BBTRl5lrQRcAnZtcsjgTMlNdjlL3AnsJ+kPSR1Jbvm+QnZ9V9rhxwYW1m6XnYacD7ZL/5bZF3K/0tZLgHGk41w/ht4PqU151yPAXeksiZQO5h1SvV4B3ifLEidVEcZc4H9yYLBXLKW1v4RMac5dSoq+5mIqKs1/AjwMNlgyXTgY2p3VWtuXp8r6fnGzpMuXfwBuCIiXoyIKcC5wG2SupdQz8lkAfV/yAZtDgAOiIhPGzjnKEmjGivb2iZFuAdgZlbILUYzsyIOjGZmRRwYzcyKODCamRVxYKxwxVONtbUpwySdK+m35a6HWVM4MLaS1ppqLFpoyrBSpGnFaj1mFxGXRcT3W/i8N6dpywbnWOYxkp7JqaxpkvbMoyxrHQ6MraA9TDWWw1ReLSL9cdmw3PWwdqbcs1i0943Wn2rsGGpPGbY3MJnskbqRwN9ZMePMz2h4WrKnWHkqr+8Cr5JNMfYGcELKuyq1p+v6iOwZ4+JzNDZ91+lkN6QvILs5vUcD37cuwAvAl1K9B+f0M9uU7Kby6vQ55hf8nK4km8XnPbLp02pm0+lHdgP9fLIb5v9B1vC4LX1PFqeyziz3/0lvjW9trgXQDu0I9KCOiRIKnAfsQDYrzZbAMLInY2p8jizADiALftdJ6hMRF5C1Qu+IiF4RcVNhoemRwrvIHgXsSxYgC2eEKcWRwPFkj+lNJwvO+5M9ovdd4NeShkY2KcU+wDupLr2i6KmW9Fje7cCpZFOqjQYekNStINthZNOzbUAW8I5poG4/Bp6OiJea+JkaFBGvkk2u8Vz6HL3TocvJpiDbiuyPxACyRzYh+8M1g+xz9Sd7siYi4kiyQHpAKusXWJvnwNjyyjnV2L7ApIi4J53/GmrPhFOKW6JgKq+IeDAipkbm72QT0+5SYlmlTt/1TkS8DzxAPVOYSVofOIEVgalFpYkjjgd+HBHvR8SHZH+UDk9ZlpBNxvH59H36R0T4sbIK5cDY8so51di6FDxjnH5RS52DsEat6bTShBdjJL2fpgPbl8anAyusT2PTd5U6hdlvyP5YLGjspGlk/KO0jUqTatTsf1Ri3dcim6logqT56bM/nNIBfkk2ee2jkt6QdHaJ5Vob5MDY8lp7qrFC7wLr1eykVs96BccbnJYsWd7qSRMu3E3W0uufupijaXw6sBqlTN9Vqj2AX0qaWbCS33OSvr3SB8hGxmu69ydGxH8K9usLvMWfZQ7ZdcLNI6J32taoeX9EfBgRP4mIL5BdRz1N0h71lGVtnANjC4vWn2qs0IPAF9N5uwAjqB386p2WrB7dyAYgZgNLJe1DNrhT4z2gbyqrLnlO37UR2fXYrVjR3T6Ahq/lNsV7wHo11z9T6/ZGsmuqawNIGiDpq+n1/pIGp2C/gGzgZllBWV/IqV7WChwYW0G04lRjReedAxwK/IKsS79ZOs8n6XhD05LVVd6HwA/JAtw84NvA/QXHXyML8m+k7ua6Re9v8vRdDdRlVkTMrNlS8pxISwvk4Amy0fOZkmqmWDuLrLs8RtIHwN9Yca13SNr/iKyXMDIinkzHfk72h2++pNNzqp+1IE871oGk+xBnkK0G+GRj+c06KrcY2zlJX5XUO10fPJfseuCYMlfLrE1zYGz/diRb3rSm63pwjt1Ns3bJXWkzsyJuMZqZFWnopuOKpK49Qz16N57RWtXWG63beCYri+efnzAnItZqPGdpOq/++YiljV+ticWzH4mIr+V13jy1v8DYozfdtz6u3NWwIs8+cWG5q2D1WKWrpjeeq3SxdDHdNz6s0XwfT7yu1CemWl27C4xmVmYSdOpc7lp8Jg6MZpa/tjd1Z5M4MJpZ/qTG87RhDoxmljO5xWhmVovwNUYzs9rkrrSZ2UrclTYzK+IWo5lZAd/HaGZWB3elzcwK+XYdM7OVdfI1RjOzFXwfo5lZMXelzcxWVuG361R2WDeztkmdGt8aK0K6WdIsSS8XpB0qaZKkZZK2Lcp/jqQqSZNr1vtO6V9LaVWSzi6l+g6MZpavmvsYG9sadwtQPMP3y8DXgadrn1KbAYcDm6f3jJTUWVJn4DpgH7J11b+V8jbIXWkzy18OXemIeFrSoKK0V7PiVyr/IODPEfEJ8KakKmBYOlYVEW+k9/055X2loXM7MJpZzkoefOknaXzB/g0RcUMzTzqA2uulz0hpAG8VpW/fWGEOjGaWv9JajHMiYtvGs7U+B0Yzy5cEnVo9tLwNrF+wv15Ko4H0ennwxczyJzW+5et+4HBJ3SVtAAwB/gWMA4ZI2kBSN7IBmvsbK8wtRjP
"text/plain": [
"<matplotlib.figure.Figure at 0x7f4c2809f940>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
2018-05-13 13:36:21 +02:00
"source": [
2018-05-13 18:29:18 +02:00
"test_classifier(labels=[\"true\",\"false\"], title=\"Configuration 4 -- train\", Xt=vec_train_3, yt=y3, clf=clf_3)\n",
2018-05-22 16:40:46 +02:00
"cm_4=test_classifier(labels=[\"true\",\"false\"], title=\"Configuration 4 -- test\", Xt=vec_test_3, yt=yt3, clf=clf_3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### exporting"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"def recall(cm):\n",
" return cm[0,0] / (cm[0,0] + cm[0,1])\n",
"def precision(cm):\n",
" return cm[0,0] / (cm[0,0] + cm[1,0])\n",
"\n",
"def accuracy(cm):\n",
" return (cm[0,0] + cm[1,1]) / np.sum(cm)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"from rdflib import Namespace, Graph, Literal\n",
"from rdflib.namespace import DCTERMS, RDF"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"nsp = {\n",
" \"this\": Namespace(\"http://mex.aksw.org/examples/\"),\n",
" \"xsd\": Namespace(\"http://www.w3.org/2001/XMLSchema#\"),\n",
" \"rdfs\": Namespace(\"http://www.w3.org/2000/01/rdf-schema#\"),\n",
" \"mexcore\": Namespace(\"http://mex.aksw.org/mex-core#\"),\n",
" \"mexperf\": Namespace(\"http://mex.aksw.org/mex-perf#\"),\n",
" \"mexalgo\": Namespace(\"http://mex.aksw.org/mex-algo#\"),\n",
" \"prov\": Namespace(\"http://www.w3.org/ns/prov#\"),\n",
" \"dct\": DCTERMS,\n",
" \"rdf\": RDF\n",
"}\n",
"g = Graph()\n",
"\n",
"for i in nsp.items():\n",
" g.bind(i[0],i[1])\n",
"\n",
"g.add((nsp[\"this\"].jonas_weinz_task_2, nsp[\"rdf\"].type, nsp[\"mexcore\"].Experiment))\n",
"\n",
"g.add((nsp[\"this\"].ds1, nsp[\"rdf\"].type, nsp[\"mexcore\"].dataset))\n",
"g.add((nsp[\"this\"].ds1, nsp[\"rdfs\"].label, Literal(\"Dataset01\")))\n",
"\n",
"g.add((nsp[\"this\"].ds2, nsp[\"rdf\"].type, nsp[\"mexcore\"].dataset))\n",
"g.add((nsp[\"this\"].ds2, nsp[\"rdfs\"].label, Literal(\"Dataset02\")))\n",
"\n",
"g.add((nsp[\"this\"].ds3, nsp[\"rdf\"].type, nsp[\"mexcore\"].dataset))\n",
"g.add((nsp[\"this\"].ds3, nsp[\"rdfs\"].label, Literal(\"Dataset03\")))\n",
"\n",
"g.add((nsp[\"this\"].modela,nsp[\"rdf\"].type,nsp[\"mexalgo\"].Algorithm))\n",
"g.add((nsp[\"this\"].modela,nsp[\"rdfs\"].label,Literal('model a')))\n",
"g.add((nsp[\"this\"].modela,nsp[\"dct\"].identifier,Literal('RandomForestClassifier')))\n",
"\n",
"g.add((nsp[\"this\"].modelb,nsp[\"rdf\"].type,nsp[\"mexalgo\"].Algorithm))\n",
"g.add((nsp[\"this\"].modelb,nsp[\"rdfs\"].label,Literal('model b')))\n",
"g.add((nsp[\"this\"].modelb,nsp[\"dct\"].identifier,Literal('MLPClassifier')))\n",
"\n",
"g.add((nsp[\"this\"].modelc,nsp[\"rdf\"].type,nsp[\"mexalgo\"].Algorithm))\n",
"g.add((nsp[\"this\"].modelc,nsp[\"rdfs\"].label,Literal('model c')))\n",
"g.add((nsp[\"this\"].modelc,nsp[\"dct\"].identifier,Literal('MLPClassifier')))\n",
"\n",
"g.add((nsp[\"this\"].conf1,nsp[\"rdf\"].type,nsp[\"mexcore\"].ExperimentConfiguration))\n",
"g.add((nsp[\"this\"].conf1,nsp[\"rdfs\"].label,Literal('configuration 1')))\n",
"g.add((nsp[\"this\"].conf1,nsp[\"prov\"].used, nsp[\"this\"].modela))\n",
"g.add((nsp[\"this\"].conf1,nsp[\"prov\"].used, nsp[\"this\"].ds1))\n",
"g.add((nsp[\"this\"].conf1,nsp[\"prov\"].wasStartedBy, nsp[\"this\"].jonas_weinz_task_2))\n",
"\n",
"g.add((nsp[\"this\"].conf2,nsp[\"rdf\"].type,nsp[\"mexcore\"].ExperimentConfiguration))\n",
"g.add((nsp[\"this\"].conf2,nsp[\"rdfs\"].label,Literal('configuration 2')))\n",
"g.add((nsp[\"this\"].conf2,nsp[\"prov\"].used, nsp[\"this\"].modelb))\n",
"g.add((nsp[\"this\"].conf2,nsp[\"prov\"].used, nsp[\"this\"].ds2))\n",
"g.add((nsp[\"this\"].conf2,nsp[\"prov\"].wasStartedBy, nsp[\"this\"].jonas_weinz_task_2))\n",
"\n",
"g.add((nsp[\"this\"].conf3a,nsp[\"rdf\"].type,nsp[\"mexcore\"].ExperimentConfiguration))\n",
"g.add((nsp[\"this\"].conf3a,nsp[\"rdfs\"].label,Literal('configuration 3a')))\n",
"g.add((nsp[\"this\"].conf3a,nsp[\"prov\"].used, nsp[\"this\"].modelb))\n",
"g.add((nsp[\"this\"].conf3a,nsp[\"prov\"].used, nsp[\"this\"].ds1))\n",
"g.add((nsp[\"this\"].conf3a,nsp[\"prov\"].wasStartedBy, nsp[\"this\"].jonas_weinz_task_2))\n",
"\n",
"g.add((nsp[\"this\"].conf3b,nsp[\"rdf\"].type,nsp[\"mexcore\"].ExperimentConfiguration))\n",
"g.add((nsp[\"this\"].conf3b,nsp[\"rdfs\"].label,Literal('configuration 3b')))\n",
"g.add((nsp[\"this\"].conf3b,nsp[\"prov\"].used, nsp[\"this\"].modela))\n",
"g.add((nsp[\"this\"].conf3b,nsp[\"prov\"].used, nsp[\"this\"].ds2))\n",
"g.add((nsp[\"this\"].conf3b,nsp[\"prov\"].wasStartedBy, nsp[\"this\"].jonas_weinz_task_2))\n",
" \n",
"g.add((nsp[\"this\"].conf4,nsp[\"rdf\"].type,nsp[\"mexcore\"].ExperimentConfiguration))\n",
"g.add((nsp[\"this\"].conf4,nsp[\"rdfs\"].label,Literal('configuration 4')))\n",
"g.add((nsp[\"this\"].conf4,nsp[\"prov\"].used, nsp[\"this\"].modelc))\n",
"g.add((nsp[\"this\"].conf4,nsp[\"prov\"].used, nsp[\"this\"].ds3))\n",
"g.add((nsp[\"this\"].conf4,nsp[\"prov\"].wasStartedBy, nsp[\"this\"].jonas_weinz_task_2))\n",
"\n",
"g.add((nsp[\"this\"].measure1,nsp[\"rdf\"].type,nsp[\"mexcore\"].PerformanceMeasure))\n",
"g.add((nsp[\"this\"].measure1,nsp[\"mexperf\"].precision,Literal(str(precision(cm_1)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure1,nsp[\"mexperf\"].recall,Literal(str(recall(cm_1)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure1,nsp[\"mexperf\"].accuracy,Literal(str(accuracy(cm_1)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure1,nsp[\"prov\"].wasGeneratedBy,nsp[\"this\"].conf1))\n",
"\n",
"g.add((nsp[\"this\"].measure2,nsp[\"rdf\"].type,nsp[\"mexcore\"].PerformanceMeasure))\n",
"g.add((nsp[\"this\"].measure2,nsp[\"mexperf\"].precision,Literal(str(precision(cm_2)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure2,nsp[\"mexperf\"].recall,Literal(str(recall(cm_2)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure2,nsp[\"mexperf\"].accuracy,Literal(str(accuracy(cm_2)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure2,nsp[\"prov\"].wasGeneratedBy,nsp[\"this\"].conf2))\n",
"\n",
"g.add((nsp[\"this\"].measure3a,nsp[\"rdf\"].type,nsp[\"mexcore\"].PerformanceMeasure))\n",
"g.add((nsp[\"this\"].measure3a,nsp[\"mexperf\"].precision,Literal(str(precision(cm_3a)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure3a,nsp[\"mexperf\"].recall,Literal(str(recall(cm_3a)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure3a,nsp[\"mexperf\"].accuracy,Literal(str(accuracy(cm_3a)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure3a,nsp[\"prov\"].wasGeneratedBy,nsp[\"this\"].conf3a))\n",
"\n",
"g.add((nsp[\"this\"].measure3b,nsp[\"rdf\"].type,nsp[\"mexcore\"].PerformanceMeasure))\n",
"g.add((nsp[\"this\"].measure3b,nsp[\"mexperf\"].precision,Literal(str(precision(cm_3b)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure3b,nsp[\"mexperf\"].recall,Literal(str(recall(cm_3b)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure3b,nsp[\"mexperf\"].accuracy,Literal(str(accuracy(cm_3b)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure3b,nsp[\"prov\"].wasGeneratedBy,nsp[\"this\"].conf3b))\n",
"\n",
"g.add((nsp[\"this\"].measure4,nsp[\"rdf\"].type,nsp[\"mexcore\"].PerformanceMeasure))\n",
"g.add((nsp[\"this\"].measure4,nsp[\"mexperf\"].precision,Literal(str(precision(cm_4)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure4,nsp[\"mexperf\"].recall,Literal(str(recall(cm_4)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure4,nsp[\"mexperf\"].accuracy,Literal(str(accuracy(cm_4)),datatype=nsp[\"xsd\"].float)))\n",
"g.add((nsp[\"this\"].measure4,nsp[\"prov\"].wasGeneratedBy,nsp[\"this\"].conf4))\n"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"g.serialize(destination='output.ttl', format='turtle')"
2018-05-13 13:36:21 +02:00
]
},
2018-05-09 19:13:08 +02:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
2018-05-09 10:47:51 +02:00
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}