nlp-lab/Jonas_Solutions/Task_02_JonasWeinz.ipynb

1348 lines
139 KiB
Plaintext
Raw Normal View History

2018-05-09 10:47:51 +02:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"# NLP-LAB Exercise 02 by Jonas Weinz (2571421)\n",
2018-05-09 16:50:59 +02:00
"## links:\n",
"\n",
"* Article: https://miguelmalvarez.com/2017/03/23/how-can-machine-learning-and-ai-help-solving-the-fake-news-problem/\n",
" * corresponding code: https://github.com/kjam/random_hackery/blob/master/Attempting%20to%20detect%20fake%20news.ipynb\n",
"\n",
"* Tutorial on Datacamp: https://www.datacamp.com/community/tutorials/scikit-learn-fake-news\n",
"\n",
"* liar dataset paper: https://www.cs.ucsb.edu/~william/papers/acl2017.pdf\n",
2018-05-09 19:13:08 +02:00
" * dataset: https://www.cs.ucsb.edu/~william/data/liar_dataset.zip"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-09 16:50:59 +02:00
"## Dependencies for this Notebook:\n",
2018-05-09 10:47:51 +02:00
"* library [rdflib](https://github.com/RDFLib/rdflib)\n",
2018-05-09 19:13:08 +02:00
" * install: `pip3 install rdflib`\n"
2018-05-09 10:47:51 +02:00
]
},
{
"cell_type": "code",
2018-05-09 19:13:08 +02:00
"execution_count": 1,
2018-05-09 10:47:51 +02:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
2018-05-11 12:19:53 +02:00
"%pylab inline"
2018-05-09 16:50:59 +02:00
]
},
{
"cell_type": "code",
2018-05-11 12:19:53 +02:00
"execution_count": 2,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import itertools\n",
"import sklearn.utils as sku\n",
2018-05-09 19:13:08 +02:00
"from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer, HashingVectorizer\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.linear_model import PassiveAggressiveClassifier\n",
"from sklearn.naive_bayes import MultinomialNB\n",
"from sklearn import metrics\n",
"import matplotlib.pyplot as plt\n",
2018-05-13 12:44:36 +02:00
"from pprint import pprint as pp\n",
"from IPython.display import display, Markdown, Latex\n",
"import collections\n",
"import traceback\n",
2018-05-09 16:50:59 +02:00
"import os"
]
},
2018-05-09 19:13:08 +02:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tools used later"
]
},
{
"cell_type": "code",
2018-05-11 12:19:53 +02:00
"execution_count": 3,
2018-05-09 19:13:08 +02:00
"metadata": {},
"outputs": [],
"source": [
"def plot_confusion_matrix(cm, classes,\n",
" title,\n",
" normalize=False,\n",
" cmap=plt.cm.Blues):\n",
" fig_1, ax_1 = plt.subplots()\n",
" \"\"\"\n",
" See full source and example: \n",
" http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html\n",
" \n",
" This function prints and plots the confusion matrix.\n",
" Normalization can be applied by setting `normalize=True`.\n",
" \"\"\"\n",
" plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
" plt.title('Confusion Matrix for: ' + title)\n",
" plt.colorbar()\n",
" tick_marks = np.arange(len(classes))\n",
" plt.xticks(tick_marks, classes, rotation=45)\n",
" plt.yticks(tick_marks, classes)\n",
"\n",
" if normalize:\n",
" cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
" print(\"Normalized confusion matrix\")\n",
" else:\n",
" print('Confusion matrix, without normalization')\n",
"\n",
" thresh = cm.max() / 2.\n",
" for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
" plt.text(j, i, cm[i, j],\n",
" horizontalalignment=\"center\",\n",
" color=\"white\" if cm[i, j] > thresh else \"black\")\n",
"\n",
" plt.tight_layout()\n",
" plt.ylabel('True label')\n",
" plt.xlabel('Predicted label')"
]
},
2018-05-13 10:12:19 +02:00
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
2018-05-13 12:44:36 +02:00
"def test_classifier(labels, title, Xt, yt, clf):\n",
" pred = clf.predict(Xt)\n",
" score = metrics.accuracy_score(yt, pred)\n",
" pp(\"score: \" + str(score))\n",
" cm = metrics.confusion_matrix(yt, pred, labels=labels)\n",
" plot_confusion_matrix(cm, classes=labels, title=title)"
2018-05-13 10:12:19 +02:00
]
},
2018-05-09 16:50:59 +02:00
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generate/Download Datasets we are working on\n",
"\n",
"* running bash script to download all needed data and store it into the `data` subfolder"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 5,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"================================================================================\n",
2018-05-11 12:19:53 +02:00
"checking whether unzip is installed\n",
2018-05-09 16:50:59 +02:00
"================================================================================\n",
"UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.\n",
"\n",
"Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;\n",
"see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.\n",
"\n",
"Compiled with gcc 6.3.0 20170415 for Unix (Linux ELF).\n",
"\n",
"UnZip special compilation options:\n",
" ACORN_FTYPE_NFS\n",
" COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)\n",
" SET_DIR_ATTRIB\n",
" SYMLINKS (symbolic links supported, if RTL and file system permit)\n",
" TIMESTAMP\n",
" UNIXBACKUP\n",
" USE_EF_UT_TIME\n",
" USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)\n",
" USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)\n",
" UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)\n",
" LARGE_FILE_SUPPORT (large files over 2 GiB supported)\n",
" ZIP64_SUPPORT (archives using Zip64 for large files supported)\n",
" USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)\n",
" VMS_TEXT_CONV\n",
" WILD_STOP_AT_DIR\n",
" [decryption, version 2.11 of 05 Jan 2007]\n",
"\n",
"UnZip and ZipInfo environment options:\n",
" UNZIP: [none]\n",
" UNZIPOPT: [none]\n",
" ZIPINFO: [none]\n",
" ZIPINFOOPT: [none]\n",
"================================================================================\n",
2018-05-11 12:19:53 +02:00
"successfully finished action: checking whether unzip is installed\n",
"================================================================================\n",
"================================================================================\n",
"downloading and unpacking https://www.cs.ucsb.edu/~william/data/liar_dataset.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
2018-05-09 16:50:59 +02:00
"successfully finished action: downloading and unpacking https://www.cs.ucsb.edu/~william/data/liar_dataset.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
"downloading and unpacking https://raw.githubusercontent.com/GeorgeMcIntire/fake_real_news_dataset/master/fake_or_real_news.csv.zip if not already existing\n",
"================================================================================\n",
"================================================================================\n",
"successfully finished action: downloading and unpacking https://raw.githubusercontent.com/GeorgeMcIntire/fake_real_news_dataset/master/fake_or_real_news.csv.zip if not already existing\n",
2018-05-13 10:12:19 +02:00
"================================================================================\n",
"================================================================================\n",
"downloading Helper script: script_dataset3.py\n",
"================================================================================\n",
"================================================================================\n",
"successfully finished action: downloading Helper script: script_dataset3.py\n",
2018-05-09 16:50:59 +02:00
"================================================================================\n"
]
}
],
"source": [
"%%bash\n",
"./Task_2_gen_data.sh"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"----\n",
"## configuration 1"
2018-05-09 16:50:59 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 6,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [],
"source": [
"df_1 = pd.read_csv('data/fake_or_real_news.csv').set_index('Unnamed: 0')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* display first 10 entries"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 7,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(6335, 3)"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>title</th>\n",
" <th>text</th>\n",
" <th>label</th>\n",
" </tr>\n",
" <tr>\n",
" <th>Unnamed: 0</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>8476</th>\n",
" <td>You Can Smell Hillarys Fear</td>\n",
" <td>Daniel Greenfield, a Shillman Journalism Fello...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10294</th>\n",
" <td>Watch The Exact Moment Paul Ryan Committed Pol...</td>\n",
" <td>Google Pinterest Digg Linkedin Reddit Stumbleu...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3608</th>\n",
" <td>Kerry to go to Paris in gesture of sympathy</td>\n",
" <td>U.S. Secretary of State John F. Kerry said Mon...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10142</th>\n",
" <td>Bernie supporters on Twitter erupt in anger ag...</td>\n",
" <td>— Kaydee King (@KaydeeKing) November 9, 2016 T...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>875</th>\n",
" <td>The Battle of New York: Why This Primary Matters</td>\n",
" <td>It's primary day in New York and front-runners...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6903</th>\n",
" <td>Tehran, USA</td>\n",
" <td>\\nIm not an immigrant, but my grandparents ...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7341</th>\n",
" <td>Girl Horrified At What She Watches Boyfriend D...</td>\n",
" <td>Share This Baylee Luciani (left), Screenshot o...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>95</th>\n",
" <td>Britains Schindler Dies at 106</td>\n",
" <td>A Czech stockbroker who saved more than 650 Je...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4869</th>\n",
" <td>Fact check: Trump and Clinton at the 'commande...</td>\n",
" <td>Hillary Clinton and Donald Trump made some ina...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2909</th>\n",
" <td>Iran reportedly makes new push for uranium con...</td>\n",
" <td>Iranian negotiators reportedly have made a las...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" title \\\n",
"Unnamed: 0 \n",
"8476 You Can Smell Hillarys Fear \n",
"10294 Watch The Exact Moment Paul Ryan Committed Pol... \n",
"3608 Kerry to go to Paris in gesture of sympathy \n",
"10142 Bernie supporters on Twitter erupt in anger ag... \n",
"875 The Battle of New York: Why This Primary Matters \n",
"6903 Tehran, USA \n",
"7341 Girl Horrified At What She Watches Boyfriend D... \n",
"95 Britains Schindler Dies at 106 \n",
"4869 Fact check: Trump and Clinton at the 'commande... \n",
"2909 Iran reportedly makes new push for uranium con... \n",
"\n",
" text label \n",
"Unnamed: 0 \n",
"8476 Daniel Greenfield, a Shillman Journalism Fello... FAKE \n",
"10294 Google Pinterest Digg Linkedin Reddit Stumbleu... FAKE \n",
"3608 U.S. Secretary of State John F. Kerry said Mon... REAL \n",
"10142 — Kaydee King (@KaydeeKing) November 9, 2016 T... FAKE \n",
"875 It's primary day in New York and front-runners... REAL \n",
"6903 \\nIm not an immigrant, but my grandparents ... FAKE \n",
"7341 Share This Baylee Luciani (left), Screenshot o... FAKE \n",
"95 A Czech stockbroker who saved more than 650 Je... REAL \n",
"4869 Hillary Clinton and Donald Trump made some ina... REAL \n",
"2909 Iranian negotiators reportedly have made a las... REAL "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(df_1.shape)\n",
"display(df_1[:10])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* create test dataset"
2018-05-09 10:47:51 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 8,
2018-05-09 10:47:51 +02:00
"metadata": {},
"outputs": [],
2018-05-09 16:50:59 +02:00
"source": [
2018-05-09 19:13:08 +02:00
"def create_training_and_test_set(dset, cutoff=0.7):\n",
" shuffled = sku.shuffle(dset) # shuffle dataset\n",
2018-05-09 16:50:59 +02:00
" y = shuffled.label\n",
2018-05-09 19:13:08 +02:00
" shuffled = shuffled.drop('label', axis=1)['text']\n",
" size = int(cutoff * shuffled.shape[0])\n",
" return shuffled[:size], y[:size], shuffled[size:], y[size:]\n",
2018-05-09 16:50:59 +02:00
" "
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 9,
2018-05-09 16:50:59 +02:00
"metadata": {},
"outputs": [],
"source": [
2018-05-13 10:35:15 +02:00
"X1,y1, Xt1,yt1 = create_training_and_test_set(df_1)"
2018-05-09 19:13:08 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 10,
2018-05-09 19:13:08 +02:00
"metadata": {},
"outputs": [],
"source": [
2018-05-13 10:12:19 +02:00
"count_vectorizer_1 = CountVectorizer(stop_words='english')\n",
2018-05-13 10:35:15 +02:00
"count_train_1 = count_vectorizer_1.fit_transform(X1)\n",
"count_test_1 = count_vectorizer_1.transform(Xt1)"
2018-05-09 19:13:08 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 11,
2018-05-09 19:13:08 +02:00
"metadata": {},
"outputs": [],
"source": [
2018-05-13 10:12:19 +02:00
"tfidf_vectorizer_1 = TfidfVectorizer(stop_words='english', max_df=0.7)\n",
2018-05-13 10:35:15 +02:00
"tfidf_train_1 = tfidf_vectorizer_1.fit_transform(X1)\n",
"tfidf_test_1 = tfidf_vectorizer_1.transform(Xt1)"
2018-05-09 16:50:59 +02:00
]
2018-05-09 19:13:08 +02:00
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 12,
2018-05-09 19:13:08 +02:00
"metadata": {},
2018-05-13 10:12:19 +02:00
"outputs": [],
2018-05-09 19:13:08 +02:00
"source": [
2018-05-13 12:44:36 +02:00
"#display(count_vectorizer.get_feature_names()[0:10])\n",
"#display(count_vectorizer.get_feature_names()[-10:])\n"
2018-05-09 19:13:08 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 13,
2018-05-09 19:13:08 +02:00
"metadata": {},
2018-05-13 10:12:19 +02:00
"outputs": [],
2018-05-09 19:13:08 +02:00
"source": [
2018-05-13 12:44:36 +02:00
"#display(tfidf_vectorizer.get_feature_names()[:10])\n",
"#display(tfidf_vectorizer.get_feature_names()[-10:])"
2018-05-09 19:13:08 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 14,
2018-05-09 19:13:08 +02:00
"metadata": {},
"outputs": [],
"source": [
"#count_df = pd.DataFrame(count_train.A, columns=count_vectorizer.get_feature_names())\n",
"#tfidf_df = pd.DataFrame(count_train.A, columns=tfidf_vectorizer.get_feature_names())\n",
"#diff = set(count_df.columns) - set(tfidf_df.columns)\n",
"#pp(count_df.equals(tfidf_df))"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 16,
2018-05-09 19:13:08 +02:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2018-05-13 12:44:36 +02:00
"'score: 0.8395581273014203'\n",
2018-05-11 12:19:53 +02:00
"Confusion matrix, without normalization\n"
2018-05-09 19:13:08 +02:00
]
},
{
"data": {
2018-05-13 12:44:36 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEmCAYAAABf+4ZQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XecHVXdx/HPNwkBQiChgyEQekCk9yIBAtKbdKQrXURUpAgP+IA06YgoD5CAShEVkSoiIGDoCTWUUNMkhQRIIaT8nj/OWXKzbplJdvfu3nzfec0rO+We+d2Zufc355y5M4oIzMzMWlqnagdgZma1yQnGzMxahROMmZm1CicYMzNrFU4wZmbWKpxgzMysVVQ1wUhaWNLfJH0q6Y/zUM6hkv7ekrFVg6QHJR0xl6+9QNI4Sf9p6bisdczL/q42Sf0kjWhi/oqSJknq3MZxFV5vc++hBWK5QdI5rVV+hxARzQ7AIcALwCRgNPAgsHWR1zZT7mHAc0CXeS2rNQagHxDAX+pNXy9Pf7xgOecBv2vFOFcEpgLLtFLZkyqGACZXjG8DDAC+rLfcgfn1HwD9899HAjMrlnkfuAVYo2J9ffI6Kst6uYn4Ns/xdG9g3mDg5Fba5o8D3632MToXcddt38H1pi+V9+EHBcvpB4yoGP9qP3eUof57aGB+AK8CnSqmXQAMqHLcH+TP+yRgAnA/0Lva27OhodkajKTTgKuAXwDL5i+c64G9mnttASsBb0fEjBYoq7WMBbaQtGTFtCOAt1tqBUrmpTa5IjA+IsbMxbq7NDU/Ij6KiO51Q568XsW0J/O0SyuXi4g7GylyUC6nB9Cf9EF5UdI69ZbrWVHWek3E9wwwAtiv3vtaB1gbuL2p91cNLbC/W6LcbvW2+SGkhG9z+hpwULWDaMAe+XO0PPAxcG2V42lYM5myBylL7t/EMguSEtCoPFwFLFh5hgD8CBhDqv0cleedTzpjmp7XcQz1zvSZfbbVJY8fCbwHfE76MBxaMf2pitdtCTwPfJr/37Ji3uPA/wJP53L+DizV1BkOcANwUp7WGRgJnEtFDQa4GhgOfAa8CGyTp+9c732+XBHHhTmOqcBqVJwVA78G/lRR/iXAo4DqxVj3JT0rlz8gT98TeB2YmMtdq94Z0E+BV4BplKhB5v2xWr1pA4ALmjjbqqzBPNXAMvcBdze0zwvGdBbwz3rTLqWi5kmq6fw7b4+XgX4V85Yg1aRGkc4I76mYtxcwJO/Xd/P+vJBUE/sib/PrCh53Te3vl/nvmmK/ArH/V7nNbKu67fsz4LKK6S8AZ1NRg6m/ryv3MxVn/8Bt+firO6s+vf5+pJnPHc0frz8hHa+TgZtIJ7sP5rL+ASzeyHfGUcDQvNx7wHH1P9/NHOs/Bd6pKG+OGgzwR+A/eZ//C/h6I9trKLB7xbwupJPXDZvbx019pvL4rqQT9brx3Ui1989I30nnVcy7H/h+vfJeAfbJf/cFHgE+Ad4CDqi3njfythwJ/LjZz2YzB+POwAya+LADPweeAZYBls4b6X8rduCMvMwCOcApFQfDecyZUOqPf3WwAIvkDbZmnrd83c6k4ouL9GUxgdT81gU4OI8vWXGgvwusASycxy9u5L31IyWYLYFnKzbyw8B3mTPBfAdYMq/zR/mgW6ih91URx0fA1/NrFmDOL5xupFrSkaRmqHHACk3FWTG+BumDuGMu93RgGNC14gAdAvQGFs7Trgeub/aAaZ0EczTwcUNfEEWG/D5mkJsJSH2LI4C983gvYHzed53ydhkPLF3xobsTWDxvr23z9E1JXxw75tf1AvpW7L/vVsRQ5LhrdH/Xez/HAm8CixWI/b/KbWZb1W3fPqQvn86kmt6bpJOV0gmmkS+9OfYjTXzuKHa8PkNKKr1IJ6svARsACwH/BP6nkfXuBqwKCNiW9P2zYUPvoZFjfXXSCWPd57J+gjkaWJTZJ9pDGtle5wK/r5i3GzC0yPHZzGeqGzAQuLXe98E3clnrkmo4dZ+FA8jfZXl8vbyurqTv2OGkpNwlb99xwNp52dHMPnFevG47NjU0V51eEhgXTTdhHQr8PCLGRMRYUs3ksIr50/P86RHxAOkMZ81m1tuYWcA6khaOiNER8XoDy+wGvBMRt0XEjIi4nfTh2aNimVsi4u2ImArcBazf1Eoj4t/AEpLWBA4Hbm1gmd9FxPi8zstJB1xz73NARLyeXzO9XnlTSNvxCuB3pLOOoh2SBwL3R8Qjudxfkj7UW1Ysc01EDM/bgIg4MSJOLFh+Q34saWIexpV87SjSF3SlcRXl/bipF0fEcNIXVt1xtwNp+9+fx78DPBARD0TErIh4hHTGvquk5YFdgOMjYkI+Tp/IrzsGuDlvx1kRMTIi3mwkjCLHXaP7u46krUlfYntGxGdNxV6m3AaMIJ2h9icd07cVfN28aOxzV+R4vTYiPo6IkcCTpC/JwRHxBfAX0pfhf4mI+yPi3UieINWctikRcwDnAOdI6tpA+TdHxOcRMY10IrmepB4NlPMHYE9J3fL4Icxuvi2yj+u7R9JEZp8AXVYR0+MR8Wou65W8nm3z7HuBNSStnscPA+6MiC+B3UknGLfkY2kw8Cdg/7zsdGBtSYvlz8pLTcQHNH8V2XhgqWba6b8GfFgx/mGe9lUZ9RLUFKA7JUXEZNKBeDwwWtL9kvoWiKcupl4V45VXWhWN5zbgZGA70gE9B0k/ljQ0XxE3kdS8uFQzZQ5vamZEPEuq1ov0gSxqjm0QEbPyuiq3QZPrngu/jIieeWjufdfXi1Qlr7RURXm/LFDGQGYnmMOAOyq+bFcC9q9IWBOBrUm14N7AJxExoYEye5POuosoctw1uc0l9Sbt5yMioq6Pr6nYC5XbhFtJtcqDaZsE09jnrsjx+nHF31MbGG/wMyxpF0nPSPokb7tdaf5zOYd8YjwCOK5e2Z0lXSzpXUmfkWoWNFR+RAwjNZPtkZPMnqSkA8X2cX17R0RPUg3uZOAJScvluDaT9JiksZI+JX1nLpXj+IJUW/9O7q+r3PcrAZvVi+NQYLk8/9uk7fehpCckbdHctmsuwQwitdHv3cQyo3JgdVbM0+bGZFKVr85ylTMj4uGI2JG04d8EbiwQT11MI+cypjq3ASeSzjSmVM6QtA2pWn8AqfmvJ+nMQnWhN1JmY9Pryj2JdCY+Kpdf1BzbQJJIX5aV26DJdbexfUhnpfPiz8AKkrYD9iUlnDrDgdsqElbPiFgkIi7O85aQ1LOBMoeTmlcaUn/7FTnuGt3mkhYG7gGuiogHC8bebLnN+BOp5vVeRHzUwPwpNPF5rGdejqcix2tpkhYkvcdfAsvmz+UDzP5clnE2qa+vcnscQuqj6086oexTt+pGyrid9IW+F/BGTjpQbB83KCJmRsSfSX2CW+fJfyDVVHpHRA9SH3JlTANJiWMHYEpEDKqI44l6cXSPiBPyup6PiL1I3SH3UOCkt8kEExGfktoOfyVpb0ndJC2QzwouzYvdDvxM0tKSlsrL/665FTdiCPDNfC17D+DMuhmSlpW0l6RFSElvEqnJrL4HSFXAQyR1kXQgqY35vrmMCYCIeJ9UzTy7gdmLkvoAxgJdJJ1Laj+v8zHQp8yVQ5LWIDWVfId0Rn66pCab8ircBewmaQdJC5D6hKaR+sfahXz2t7Kka0ltxufPS3m5hns3qbP+w4h4oWL270hnjt/K610o/wZihYiou+z+ekmL5+P7m/l1NwFH5e3YSVKvilrzx8AqFeuY1+PuZuDNiLi03vRGY2+sIEnnSXq8uRXmbbY9qT+xIUOAQ/J6d2Z2M0tD6m+PMlrreO1KOkEbC8yQtAuw09wUFBGPA6+RriCtsygpzvGkxPOLZoq5I6//BGbXXmAu9nEdJXuR+kSGVsT1SUR8IWlTUiKsfC+DSN+dlzNnzfU+0jF8WP4cLCBpE0lrSeqq9HvDHrll4DMa/v6dQ7NfeJH6E04jXXUylpTlTiZlMEhfgi+QrkR4ldT5dkFz5TayrkdI1bdXSB1rlR/OTjmOUaTmlG1JO6p+GeNJbYk/Iu3
2018-05-09 19:13:08 +02:00
"text/plain": [
2018-05-13 12:44:36 +02:00
"<matplotlib.figure.Figure at 0x7fbbb033dc88>"
2018-05-09 19:13:08 +02:00
]
},
"metadata": {},
"output_type": "display_data"
2018-05-11 12:19:53 +02:00
}
],
"source": [
"clf = MultinomialNB()\n",
2018-05-13 10:35:15 +02:00
"clf.fit(tfidf_train_1, y1)\n",
2018-05-13 10:12:19 +02:00
"pred = clf.predict(tfidf_test_1)\n",
2018-05-13 10:35:15 +02:00
"score = metrics.accuracy_score(yt1, pred)\n",
2018-05-13 12:44:36 +02:00
"pp(\"score: \" + str(score))\n",
2018-05-13 10:35:15 +02:00
"cm = metrics.confusion_matrix(yt1, pred, labels=[\"FAKE\", \"REAL\"])\n",
2018-05-11 12:19:53 +02:00
"plot_confusion_matrix(cm, classes=[\"FAKE\", \"REAL\"], title= \"TFIDF_Vecctorizer, Multinomial Naive Bayes\")"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 17,
2018-05-11 12:19:53 +02:00
"metadata": {},
"outputs": [
2018-05-09 19:13:08 +02:00
{
"name": "stdout",
"output_type": "stream",
"text": [
2018-05-13 12:44:36 +02:00
"'score: 0.8869016307206733'\n",
2018-05-09 19:13:08 +02:00
"Confusion matrix, without normalization\n"
]
2018-05-11 12:19:53 +02:00
},
{
"data": {
2018-05-13 12:44:36 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEmCAYAAACj7q2aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xe8FNX9xvHPAyqCBRQUEVBULIkxYo0lKpbYe2KPLSQYYzSWXxJTjWmaakliLDGKGguaGIk1ihpjInbsDQsCIkWBqNjA7++Pcy4s17uzey97797yvHnNi50zszNndubOd06ZGUUEZmZmLdGt3hkwM7OOy0HEzMxazEHEzMxazEHEzMxazEHEzMxazEHEzMxarK5BRFJPSf+QNEfStYuxnMMk/bOWeasHSbdIOrKF3/2ppJmSXq91vqzlOvqxKekVSTsVTG/xMbs4mrPeStuwmPnYRtJzrbHsDiMiKg7AocBDwNvAVOAW4LPVfLfCcg8HHgCWWNxltcYADAcCuL5R+oY5/e4ql/Mj4IpWzOdqwLvAyq24DgEnAE8C7wCTgWuBDVp5HwzJv3WTxwhwMPAKoEbpSwDTgT0XY91HAfe25TFXw9/t7vy7bdgo/fqcPrzK5bwC7JQ/t+px3Iq/xYJtaGLapfn32LwkbWg6NdY1zz8CPszn3LeBZ4DP1/u3bGqoWBKRdDJwNvBzoH8+YZ0H7FPpu1VYHXg+IubVYFmtZQawpaS+JWlHAs/XagVKFqdUuBrwRkRMb8G6l6hy1nOAb5ACyYrAOsDfgT2au84a+zvQB9iuUfqupJPDrW2eo6wZv21rLfd54IiS7/UFtiQd07bQm8BP652JJlwTEctGxLLAicAVkvrXO1MfUyEa9iZFwQMK5ulBCjKv5eFsoEeeNpx0xXoK6apwKnB0nnY68AELo+0IGl3p0OgqlHRl+BLwFvAycFhJ+r0l39sKeBCYk//fqmTa3cBPgP/k5fwT6Fdm2xryfz5wXE7rDkwBfkhJSYR0kp0E/A94GNgmp+/aaDsfK8nHz3I+3iVd/dwNfDlP/yPw15Ll/wIYy8evuHfK3/8oL//SnL438BQwOy/3E42uzL4NPA68T4WSILA2MJ+Sq7Uyx8plpBPUROD7QLeSq6qi/Vp2nwCv5nkbrsi2bGLdFwJ/bpQ2GjirZHxPYHz+Pf4LfLpk2mDgbznvbwC/Bz4BvJe3+21gdhXbeVTehrPycn5KybEJfKtkO97Ox8SlJcu9mPQ3MiV/t3u55VZxJXs36RidXLKcr5OOq8nkkgjpSvynJd8bDkxudKzsRPFx/OWSfN4L/BqYRfob3a1kWasCY0gn7QnAV0qm/YhUsr0iHwNPkC5UvkM6d0wCdm60fQ3rXQu4M/82M4G/AH0ab0OZ3+lS4LfA68B2OW2RkghwNKkk8Bbp/HNMU78X6W/qukbLPwc4t9I+biJfP6JRqS//DlvlzysAN5KOw1n586A87QDg4UbfPRm4oeSc/WvS39Y00vmtZ57WLy9rdt5P/yYf32WPtQoH4q7APApOMsCPgXHAysBKpD/Qn5T8wPPyPEsCuwNzgRXKnFwajw8hn2yAZUgn6HXztAHA+qUHb/68Yv5RD8/fOySP9y05+F4kHaA98/iZZbZtOOkPbivg/py2O3Ab8GUWDSJfBPrmdZ5COiiXLjgg7s47cf38nSVZ9A+jF+lK8ihgG9Ifx6CifJaMr0OqcvpcXu63SH+0S5X8UY0nnTwbDp7zgPPKLP+rwMQKx8plwA3Acnm/PQ+MqLRfK+2TxvOWWffW+dho2JbepMA6LI9vRPoD/AzpIuDI/Bv0yOOPkU7QywBLk6tqaaI6q8J2HkU63o/P+7RnU8vI8w4mXXTtlsevBy7IeViZVM17TLnlFu2Lkt/0y6SA3LCOB0glkWYHkQrHcWkQ+RD4Sv5dj83bqDz9HtJxtjQwjHQC3KFk2e8Bu+RtvIwUhL5HOoa/ArxcZr1DScd6D9I56B7g7Ka2oYnf6VLSyfwEFp5DGgeRPUiBSqQS71xg48a/F6lmZS6wXB7vTgoYW1Tax03ka8Fvnde7B+nE3ien9QU+TzpPLEcKwH/P03qQAkDpheOj5Oow0rE+hnSuXA74B3BGnnYGKagsmYdtaHTh2nioVIXSF5gZxdVNhwE/jojpETGDVMI4vGT6h3n6hxFxM+kqZt0K6y3nI+BTknpGxNSIeKqJefYAXoiIyyNiXkRcBTwL7FUyzyUR8XxEvEu6Yh1WtNKI+C+woqR1SdUDlzUxzxUR8UZe529IO7LSdl4aEU/l73zYaHlzSb/jb0lXZ8dHxOQKy2twEHBTRNyel/tr0gltq5J5zo2ISfk3ICK+FhFfK7O8vqQ/hiZJ6k5qm/hORLwVEa8Av2HR46CSZu2TUhHxH9IV1X456UBSNen4PD4SuCAi7o+I+RExilQC2wLYnHSF/M2IeCci3ouIexdjO1+LiN/lffpumeX0JFXDnRMRt+Qqit2BE3MeppP+0A9uznLLuAw4QtJ6pBPQfc34bktMjIiLImI+MIp0sddf0mBSsP92/o3HA3+ipLoN+HdE3JbPN9eSAsKZ+Ri+GhgiqU/jFUbEhHysv5/PQb/l49WblVwArCZptyaWf1NEvBjJv0iBeZsm5psIPMLC43AHYG5EjKtyHzd2oKTZpHPmGODnETE7r+uNiPhrRMyNiLdItRrb5WnvA9eQLmyRtD7pgudGSSL9PZwUEW/m7/68JB8fkvbZ6vmc/e/I0aWcSkHkDaBfhTrYVUnF+gYTc9qCZTQKQnOBZSus92Mi4h3SyfGrwFRJN+U/jEr5acjTwJLx0h5M1ebnclJ1wPakK4pFSPo/Sc/knmazSVfD/Sosc1LRxIi4n1R8FunEWq1FfoOI+Civq/Q3KFx3I2+QDqxy+pGuWhofBwObnr1JLdknpS5j4QnpcBYN9KsDp0ia3TCQSgKr5v8nVrhQalDNdlbzu14MPBcRvyjJ35Kk47ohfxeQrlabs9ym/I10Mvs66RhubQv2Y74QgrQvVwUaTloNGv9200o+v0u6gJ1fMt6wrEVI6i/paklTJP2PdNFV6W9vEfnE+5M8NF7+bpLGSXoz75vdC5Z/Jan2A1KHpCvz52r2cWOjI6JPRCxDKgkdIemYnKdeki6QNDFv8z1An3yhAymAH5qDxuF5We+TAnMv4OGSfNya0wF+Raq1+KeklySdWpA/oHIQuY90xbZvwTyvkX6gBqvltJZ4h7SBDVYpnZivUj5HOqE9C1xURX4a8jSlhXlqcDnwNeDmkj8OIHXzI1UZHUiqqutDao9RQ9bLLLMwwks6jlSieS0vv1qL/Ab5QBrMor9B4bobGQsMkrRpmekzSVcwjY+DhvUV7tcKqs3n5cCOkrYklTD+UjJtEvCz/AfZMPTKpdRJpCvQpi6UGq+70nZWzG/+o1yH1AZYmr/3Se1ADflbPiLWr3a55eRj9RZS1VJTQaQ5+6ZFecheI5XmlytJq8XfJaQr6SD1FFyedAWu4q806RJSJ439GxIk9QD+SirN989/2zcXLP9aYLikQaQSSUMQqWYfl5VLvbewsEblFFJNx2fyNm/bkOU8/zhSG9Y2pGDWsO9nkgLy+iX56B2p8Z5cwj4lItYktaueLGnHorwVBpGImENqnPuDpH1z9FsyR+Zf5tmuAr4vaSVJ/fL8V1TxuzRlPLCtpNUk9SY1qgELrjb2kbQMaWe8TareauxmYB1Jh0paQtJBwCdJjUUtFhEvk4qL32ti8nKkOusZwBKSfggsXzJ9GqkoXnUPLEnrkOpqv0i6kviWpGqreEYDe0jaUdKSpAPufVJ7VbNFxAukuuyrJA2XtJSkpSUdLOnUfLU4GviZpOUkrU5qyGs4Dsru1yrMIO3nNSvk8RVSo+5VwO0RUVqyuQj4qqTP5J5wy0jaI5/QHiBV1Z2Z05eWtHX+3jRS8Fwqr6PSdhbKVSUnAPuVVklFxFRSFclvJC0vqZuktSSVrZKRNERSSBpSxaq/S2o0fqWJaeOB3SWtKGkVUi+gcpp9HDeIiEmk4++
2018-05-11 12:19:53 +02:00
"text/plain": [
2018-05-13 12:44:36 +02:00
"<matplotlib.figure.Figure at 0x7fbbb02122b0>"
2018-05-11 12:19:53 +02:00
]
},
"metadata": {},
"output_type": "display_data"
2018-05-09 19:13:08 +02:00
}
],
"source": [
2018-05-13 12:44:36 +02:00
"clf_a = MultinomialNB()\n",
"clf_a.fit(count_train_1, y1)\n",
"test_classifier(labels=[\"FAKE\",\"REAL\"], title=\"Count Vectorizer, Multinomial Naive Bayes\", Xt=count_test_1,yt=yt1, clf=clf_a)"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* comparing to PassiveAggresiveClassifier"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 18,
2018-05-11 12:19:53 +02:00
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/jonas/.local/lib/python3.6/site-packages/sklearn/linear_model/stochastic_gradient.py:117: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
" DeprecationWarning)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
2018-05-13 12:44:36 +02:00
"accuracy: 0.933\n",
2018-05-11 12:19:53 +02:00
"Confusion matrix, without normalization\n"
]
},
{
"data": {
2018-05-13 12:44:36 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ8AAAEmCAYAAAC9J50pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xe8HFX9xvHPk0poCV1IAgEJXQgQukpHQCDgj44mQARRxAIWVERQERCVIgiCSJUmSBEpIkVBIdSEXkJPCIQACSUJkOT7++OcaybrTe62u5u7ed73Na87c2Z25kzZ+c45c3ZGEYGZmVkjdWt2BszMbMHj4GNmZg3n4GNmZg3n4GNmZg3n4GNmZg3n4GNmZg3X1OAjqY+kv0qaIunPNcznAEl/r2femkHSzZJGVPnZn0uaJOn1eufLKtcqxySApCckbdXsfDRbs/appK0kjevE+Z8j6ceF4a9KekPS+5KWyv9XqfuCI6LDDtgfeBB4H5gA3Ax8upzPdjDfLwH3Az1qnVdndMBWQADXlqSvl9PvKnM+xwGXdmI+VwSmAct20rzfL3QBfFAY/gxwIfBRyXT75M+/BGyX+w8EZhameRG4AFitsLxBeRnFeY1pJ1/75nmrJL0HMBHYpYZ1PhC4p87bMYBV63hczsrb5j3gGeCgzjq+6rwdBLwAPNnsvMxPHbAxcBMwGXg7nxcPKuzvcQ3KR898Llmvs5fVYclH0pHAacAvgOXyyeh3wLCOPluGlYBnI2JGHebVWd4ENpO0VCFtBPBsvRagpJZS6IrAWxExsYpl95jX+Ih4JSIWbety8nqFtLtz2i+L00XElXOZ5b15Pn2B7UgH+kOS1imZrl9hXuu1M5/rgH7AliXpO5JO9LfMa706U0fbtE5ey9txceD7wHmS1mrAcmv1WWBZYBVJG3XWQhq0D+pC0mbAHcA/gVWBpYCvAjs1ITvLAQsBT9Q6ow73QQdRsC/p6mqveUzTmxScXsvdaUDvYsQGjiJdjU5gdjQ/nnS1/HFexkhKSgjMvgrukYcPJF01vUe6aj6gkH5P4XObAw8AU/L/zQvj7gJ+Bvw7z+fvwNJzWbe2/J8DHJ7TugPjgWMplHyA04FXgXeBh4DP5PQdS9ZzTCEfJ+R8TCMddHcBX87jzwauKcz/ZOB2/vdKv+0E3nYlfGFO3410AE3O812z8JmXSCesR4EPqaDkSTtX8KSSz8/nMv1LzFny+Z8SBXAjcHV7+7yDvJwL/LEk7Srg1MLwLsDovB3+A6xbGDcQ+AvpAuMt4ExgTWA6s0tokwvfhYvztC8DxwDdCuv1b+DUPJ+fF9cV+Bdzlhj36ShvHaz3VpRcCed87Zn7/wy8Tjr+/wWsXZhuZ+BJ0rE/HvhOTl8674e2K++7C+v3Uj7OVsjH2pKF+a0PTAJ65uGDgaeAd4BbgZVK8vlH4E95u59ZMm7lnN/3gH8AZzHn+WB43vZvAT8uObaOA64GLiV9B79Muq1wNPB8/sxVbXknnWAvzemTSeeJ5co9z5C+n78qyf/1wJG5fwXgmrxfXgS+MY/9eQ9wVrn7u7BO7+V9uUdh3KqkIDYl75crc7pIx+fEvH0eA9Ypfn+B1UjHaFvNwx2l33nS+f5XwCvAG6RzY5+S8+X3ScffJfM8jjs4yHcEZjCPEwHwU+A+0tXMMqQv0c8KmZmRp+lJOvCnAksUDpjiwVU6PCiveA9gkbzRVs/jlid/qUoOiiVJB/6X8uf2y8NL5fF35R23GtAnD580r51OCmajCl/eW0kHdzH4fJF0xdKDFGxfBxZqb70K+XgFWDt/pidzBp+FSaWrA0lVW5OAAWUenG0H0fZ5vt8DxgK9CieT0aSTb9uB8zvgd2Wc+Doj+BwMvFG6z8vIyxb5mGhbh76kk+OQwolxIrAJ6aJhRM5P7zw8hvSFXIR0Mvr03PJJCjzXA4vlPD4LjCxMPwM4Iu/LPqXzKN1u88pbGev93/1NOsHuQbq4Wb2wPRdj9oXh6MJnJzD7wmgJYIPcfyLpRNIzd58hX+iU7MM7gEMK8zsFOCf3DyMdZ2vm7XAM8J/CtAvn/bUz8H+kY7pXYfy9pBNbL+DTedpL87i1SCfET+fxv8rrXAw+HwO7523SB/gm6dw0IG+L3wOX5+m/Avw156k7sCGpFFnueeazpItNFbblNFLQ6Ua6AD0253UVUjD7XDv7cmHShc7W5ezvPLxXYTn7kL7ry+dxlwM/yuOKx/Tncp76kQLRmoXPXEj+/tLO9485g8+pwA2k8+xieRueWHK+Pzlv7z7zPI47OMgPAF7vYJrngZ0Lw58DXipkZlrJikwENi0cMJUEn8mkg7ZPSR6KB8WXgPtLxt8LHJj77wKOKYz7GnBLGV/y54DVgSvydpkj+LTz2XfI9aal61XIx0/bSftyYXgT0lXoy8B+FRycPwauKgx3I13lblU4mRzc0UluLsuaW/CZnvfPZGBSYdxLdBx8dgQ+Ltnnkwvdd+aRn+eA/XP/IRTuD5GuTn9WMv0zpKq6zUhXpf8T5ErzSTo5fQSsVUj7Stv+z9O/0sE8SoPPXPNWxj7YilTSbSuljAb2ncu0/fKy++bhV3LeFy+Z7qek4Po/96VK9uGXmX1FLNIJ+LN5+GZyQC4cd1PJpR/SBdqbpO/zQqSr8z3yuBVJJ66FC5+/lNnB51hy4MjDC+d9Ugw+/yrJ91PAtoXh5UkBqgcpQP9PaZPyzzPK27Jt3Q8pbJdN2jkefgBc0M627Z/3zxrlfr/bGT8aGJb7LybVCAwomWYb0gXTpuQSbcn3t8Pgk9f5A+CThXGbAS8W8vkR+aK7o66j+wxvAUt3UHe3Aunk2OblnPbfecSc93SmAotSoYj4gBTlDwMmSPqbpDXKyE9bnvoXhostwsrNzyXA14GtgWtLR0r6jqSncsu9yaSr8KU7mOer8xoZEaNIV0wiVRmUa45tEBGz8rKK22Cey67CryKiX+46Wu9S/Ukn0aKlC/P71Tw+ezGpOgbShcfFhXErAUdJmtzWkUp7K+T/L0d59xuXJpUGSo/zWrbnvPJWjtfytlkyIoZExBUAkrpLOknS85LeJQWOtnWAdFLdGXhZ0j/z/QZIJZixwN8lvSDp6Lks9xrSPdDlSVf/s0hVdG3rdHphfd4mHbtt22kE6aJoRkRMz/MakcetALwdEVMLyypu0xWKw3m6t0ryVroPVgKuLeTnKVIpYznS9/lW4ApJr0n6paSe5Z5nIp1tryDVrEBqlPWnwnJXKNm3P8zLLfUOaRsu3864dkkaLml0Yd7rMHv/fo+0ze/PrRQPzvm9g1StfBYwUdK5khYvd5nZMqSg/1Bh2bfk9DZv5n3boY6Cz72kewK7z2Oa10gbu82KOa0aH5BWrs0niiMj4taI2J60o54GzisjP215Gl9lntpcQiol3VTyBUHSZ0g7fW9SlWI/0lWd2rI+l3nOLb1tvoeTiq+v5fmXa45tIEmkE1txG8xz2Q22B7NPYJW6BNg2n0Q3ZfYJANLJ6IRCEOsXEQtHxOV53IpzubAq3TaTSFfMpcd5LdtzXnmrxf6k6q/tSBdAg3K6ACLigYgYRqomv458URMR70XEURGxCul+4ZGSti2deUS8Q7pPuk9e1hX5RNy2Tl8pWac+EfEfSQNIV99flPR6/knAnsDOkpYmVQcuKan4/R9Y6J9Aqj5LKyP1IVVzz5G9kuFXgZ1K8rNQRIyPiI8j4viIWItUrb4L+SKmzPMMpCquPSWtRCrtXFNY7osly10sInZuZ3tOJZ1n/28uy5hDXtZ5pAvhpfK55nFm79/XI+KQiFiBVML9naRV87gzImJDUhXmasB3y1lmwSRSTdbahfXqG7MbIkEF34N5Bp+ImEIq7p4laXdJC0vqKWknSb/Mk10OHCNpmXwQHUsqLldjNPBZSStK6ksqqgIgaTlJwyQtQgqI75OuGErdBKwmaX9JPSTtQ9rYN1aZJwAi4kVSdc2P2hm9GKnK4E2gh6RjSfXHbd4ABlXSok3SaqSbgF8kXdF/T9KQMj9+FfB5SdtK6km6B/UhqZp
2018-05-11 12:19:53 +02:00
"text/plain": [
2018-05-13 12:44:36 +02:00
"<matplotlib.figure.Figure at 0x7fbbb01b8a90>"
2018-05-11 12:19:53 +02:00
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"linear_clf = PassiveAggressiveClassifier(n_iter=50)\n",
"\n",
2018-05-13 10:35:15 +02:00
"linear_clf.fit(tfidf_train_1, y1)\n",
2018-05-13 10:12:19 +02:00
"pred = linear_clf.predict(tfidf_test_1)\n",
2018-05-13 10:35:15 +02:00
"score = metrics.accuracy_score(yt1, pred)\n",
2018-05-11 12:19:53 +02:00
"print(\"accuracy: %0.3f\" % score)\n",
2018-05-13 10:35:15 +02:00
"cm = metrics.confusion_matrix(yt1, pred, labels=['FAKE', 'REAL'])\n",
2018-05-11 12:19:53 +02:00
"plot_confusion_matrix(cm, classes=['FAKE', 'REAL'], title= \"TFIDF Vectorite, PassiveAggressive Classifier\")"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 19,
2018-05-11 12:19:53 +02:00
"metadata": {},
2018-05-13 10:12:19 +02:00
"outputs": [],
2018-05-11 12:19:53 +02:00
"source": [
2018-05-13 10:12:19 +02:00
"#clf = MultinomialNB(alpha=0.1)\n",
"#last_score = 0\n",
"#for alpha in np.arange(0,1,.1):\n",
"# nb_classifier = MultinomialNB(alpha=alpha)\n",
2018-05-13 10:35:15 +02:00
"# nb_classifier.fit(tfidf_train_1, y1)\n",
2018-05-13 10:12:19 +02:00
"# pred = nb_classifier.predict(tfidf_test_1)\n",
2018-05-13 10:35:15 +02:00
"# score = metrics.accuracy_score(yt1, pred)\n",
2018-05-13 10:12:19 +02:00
"# if score > last_score:\n",
"# clf = nb_classifier\n",
"# print(\"Alpha: {:.2f} Score: {:.5f}\".format(alpha, score))"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* try to get most important features"
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 20,
2018-05-11 12:19:53 +02:00
"metadata": {},
"outputs": [
{
2018-05-13 10:12:19 +02:00
"data": {
"text/plain": [
2018-05-13 12:44:36 +02:00
"[('FAKE', -5.09148236435967, '2016'),\n",
" ('FAKE', -4.711308849297324, 'october'),\n",
" ('FAKE', -3.7170090859114935, 'hillary'),\n",
" ('FAKE', -2.9629451875503636, 'election'),\n",
" ('FAKE', -2.9532733338247192, 'article'),\n",
" ('FAKE', -2.5913388599541607, 'share'),\n",
" ('FAKE', -2.5113445616853625, 'wikileaks'),\n",
" ('FAKE', -2.4154047960961784, 'source'),\n",
" ('FAKE', -2.3661872358294853, 'podesta'),\n",
" ('FAKE', -2.2727185001128776, 'november'),\n",
" ('FAKE', -2.204821774138259, 'advertisement'),\n",
" ('FAKE', -2.1570514850733904, 'corporate'),\n",
" ('FAKE', -2.1316069320786863, 'kelly'),\n",
" ('FAKE', -2.100506081374764, 'establishment'),\n",
" ('FAKE', -2.063743396467079, 'email'),\n",
" ('FAKE', -2.047274951558231, 'snip'),\n",
" ('FAKE', -2.0443830149677353, 'print'),\n",
" ('FAKE', -1.9775756559762447, 'mosul'),\n",
" ('FAKE', -1.8875909018863903, 'oct'),\n",
" ('FAKE', -1.8829089357171849, 'war'),\n",
" ('FAKE', -1.844565871218784, 'corruption'),\n",
" ('FAKE', -1.8422478599882626, '26'),\n",
" ('FAKE', -1.8386058868939217, 'uk'),\n",
" ('FAKE', -1.8285844309040236, 'pipeline'),\n",
" ('FAKE', -1.8173437410790223, 'reuters'),\n",
" ('FAKE', -1.804934691375235, 'donald'),\n",
" ('FAKE', -1.7225692073057894, 'stated'),\n",
" ('FAKE', -1.7216024583216563, 'fbi'),\n",
" ('FAKE', -1.7171991085276201, 'watch'),\n",
" ('FAKE', -1.7084797259301832, 'order')]"
2018-05-13 10:12:19 +02:00
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
2018-05-13 12:44:36 +02:00
"[('FAKE', -5.09148236435967, '2016'),\n",
" ('FAKE', -4.711308849297324, 'october'),\n",
" ('FAKE', -3.7170090859114935, 'hillary'),\n",
" ('FAKE', -2.9629451875503636, 'election'),\n",
" ('FAKE', -2.9532733338247192, 'article'),\n",
" ('FAKE', -2.5913388599541607, 'share'),\n",
" ('FAKE', -2.5113445616853625, 'wikileaks'),\n",
" ('FAKE', -2.4154047960961784, 'source'),\n",
" ('FAKE', -2.3661872358294853, 'podesta'),\n",
" ('FAKE', -2.2727185001128776, 'november'),\n",
" ('FAKE', -2.204821774138259, 'advertisement'),\n",
" ('FAKE', -2.1570514850733904, 'corporate'),\n",
" ('FAKE', -2.1316069320786863, 'kelly'),\n",
" ('FAKE', -2.100506081374764, 'establishment'),\n",
" ('FAKE', -2.063743396467079, 'email'),\n",
" ('FAKE', -2.047274951558231, 'snip'),\n",
" ('FAKE', -2.0443830149677353, 'print'),\n",
" ('FAKE', -1.9775756559762447, 'mosul'),\n",
" ('FAKE', -1.8875909018863903, 'oct'),\n",
" ('FAKE', -1.8829089357171849, 'war'),\n",
" ('FAKE', -1.844565871218784, 'corruption'),\n",
" ('FAKE', -1.8422478599882626, '26'),\n",
" ('FAKE', -1.8386058868939217, 'uk'),\n",
" ('FAKE', -1.8285844309040236, 'pipeline'),\n",
" ('FAKE', -1.8173437410790223, 'reuters'),\n",
" ('FAKE', -1.804934691375235, 'donald'),\n",
" ('FAKE', -1.7225692073057894, 'stated'),\n",
" ('FAKE', -1.7216024583216563, 'fbi'),\n",
" ('FAKE', -1.7171991085276201, 'watch'),\n",
" ('FAKE', -1.7084797259301832, 'order'),\n",
" ('REAL', 4.508982042083319, 'said'),\n",
" ('REAL', 3.3796648925637793, 'says'),\n",
" ('REAL', 2.7428581189713164, 'continue'),\n",
" ('REAL', 2.675308116230245, 'conservative'),\n",
" ('REAL', 2.638551059102554, 'gop'),\n",
" ('REAL', 2.4045606763615432, 'convention'),\n",
" ('REAL', 2.2410618398240536, 'marriage'),\n",
" ('REAL', 2.212540718614501, 'cruz'),\n",
" ('REAL', 2.183898002782542, 'friday'),\n",
" ('REAL', 2.124815973937978, 'rush'),\n",
" ('REAL', 2.0707150774238916, 'candidates'),\n",
" ('REAL', 1.9853629354008258, 'sen'),\n",
" ('REAL', 1.9729043146369485, 'say'),\n",
" ('REAL', 1.9657190715110686, 'religious'),\n",
" ('REAL', 1.9311072591240155, 'tuesday'),\n",
" ('REAL', 1.9138872139225251, 'debate'),\n",
" ('REAL', 1.8162413993482667, 'islamic'),\n",
" ('REAL', 1.8093845232958181, 'fox'),\n",
" ('REAL', 1.784804594337268, 'polarization'),\n",
" ('REAL', 1.7842543921517697, 'attacks'),\n",
" ('REAL', 1.7747155390419898, 'message'),\n",
" ('REAL', 1.7705633058033061, 'department'),\n",
" ('REAL', 1.7491082531650968, 'sanders'),\n",
" ('REAL', 1.7389738921456226, 'trade'),\n",
" ('REAL', 1.7015175820515849, 'conservatives'),\n",
" ('REAL', 1.699272122627768, 'baltimore'),\n",
" ('REAL', 1.6960675135616523, 'told'),\n",
" ('REAL', 1.679538975802049, 'nomination'),\n",
" ('REAL', 1.5914923698104775, 'percent'),\n",
" ('REAL', 1.588312518913584, 'oil')]"
2018-05-13 10:12:19 +02:00
]
},
"metadata": {},
"output_type": "display_data"
2018-05-11 12:19:53 +02:00
}
],
"source": [
"def most_informative_feature_for_binary_classification(vectorizer, classifier, n=100):\n",
" \"\"\"\n",
" See: https://stackoverflow.com/a/26980472\n",
" \n",
" Identify most important features if given a vectorizer and binary classifier. Set n to the number\n",
" of weighted features you would like to show. (Note: current implementation merely prints and does not \n",
" return top classes.)\n",
" \"\"\"\n",
"\n",
" class_labels = classifier.classes_\n",
" feature_names = vectorizer.get_feature_names()\n",
" topn_class1 = sorted(zip(classifier.coef_[0], feature_names))[:n]\n",
" topn_class2 = sorted(zip(classifier.coef_[0], feature_names))[-n:]\n",
2018-05-13 10:12:19 +02:00
" \n",
" l = []\n",
" \n",
2018-05-11 12:19:53 +02:00
" for coef, feat in topn_class1:\n",
2018-05-13 10:12:19 +02:00
" l.append((class_labels[0], coef, feat))\n",
2018-05-11 12:19:53 +02:00
"\n",
2018-05-13 12:44:36 +02:00
" display(l)\n",
2018-05-11 12:19:53 +02:00
"\n",
" for coef, feat in reversed(topn_class2):\n",
2018-05-13 10:12:19 +02:00
" l.append((class_labels[1], coef, feat))\n",
" \n",
2018-05-13 12:44:36 +02:00
" display(l)\n",
2018-05-11 12:19:53 +02:00
"\n",
"\n",
2018-05-13 10:12:19 +02:00
"most_informative_feature_for_binary_classification(tfidf_vectorizer_1, linear_clf, n=30)\n",
2018-05-11 12:19:53 +02:00
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"----\n",
2018-05-13 10:35:15 +02:00
"## configuration 2\n",
2018-05-13 10:12:19 +02:00
"\n",
"* read data"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 21,
2018-05-11 12:19:53 +02:00
"metadata": {},
"outputs": [
{
"data": {
2018-05-13 10:12:19 +02:00
"text/markdown": [
"----\n",
"#### Train Data:"
],
2018-05-11 12:19:53 +02:00
"text/plain": [
2018-05-13 10:12:19 +02:00
"<IPython.core.display.Markdown object>"
2018-05-11 12:19:53 +02:00
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
2018-05-13 10:12:19 +02:00
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>id</th>\n",
" <th>label</th>\n",
" <th>statement</th>\n",
" <th>subjects</th>\n",
" <th>speaker</th>\n",
" <th>job</th>\n",
" <th>state</th>\n",
" <th>party</th>\n",
" <th>#barely_true</th>\n",
" <th>#false</th>\n",
" <th>#half_true</th>\n",
" <th>#mostly_true</th>\n",
" <th>#pants_on_fire</th>\n",
" <th>context</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>2635.json</td>\n",
" <td>false</td>\n",
" <td>Says the Annies List political group supports ...</td>\n",
" <td>abortion</td>\n",
" <td>dwayne-bohac</td>\n",
" <td>State representative</td>\n",
" <td>Texas</td>\n",
" <td>republican</td>\n",
" <td>0.0</td>\n",
" <td>1.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>a mailer</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1123.json</td>\n",
" <td>false</td>\n",
" <td>Health care reform legislation is likely to ma...</td>\n",
" <td>health-care</td>\n",
" <td>blog-posting</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>none</td>\n",
" <td>7.0</td>\n",
" <td>19.0</td>\n",
" <td>3.0</td>\n",
" <td>5.0</td>\n",
" <td>44.0</td>\n",
" <td>a news release</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>12465.json</td>\n",
" <td>true</td>\n",
" <td>The Chicago Bears have had more starting quart...</td>\n",
" <td>education</td>\n",
" <td>robin-vos</td>\n",
" <td>Wisconsin Assembly speaker</td>\n",
" <td>Wisconsin</td>\n",
" <td>republican</td>\n",
" <td>0.0</td>\n",
" <td>3.0</td>\n",
" <td>2.0</td>\n",
" <td>5.0</td>\n",
" <td>1.0</td>\n",
" <td>a an online opinion-piece</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>5947.json</td>\n",
" <td>false</td>\n",
" <td>When Mitt Romney was governor of Massachusetts...</td>\n",
" <td>history,state-budget</td>\n",
" <td>mitt-romney</td>\n",
" <td>Former governor</td>\n",
" <td>Massachusetts</td>\n",
" <td>republican</td>\n",
" <td>34.0</td>\n",
" <td>32.0</td>\n",
" <td>58.0</td>\n",
" <td>33.0</td>\n",
" <td>19.0</td>\n",
" <td>an interview with CBN News</td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>620.json</td>\n",
" <td>true</td>\n",
" <td>McCain opposed a requirement that the governme...</td>\n",
" <td>federal-budget</td>\n",
" <td>barack-obama</td>\n",
" <td>President</td>\n",
" <td>Illinois</td>\n",
" <td>democrat</td>\n",
" <td>70.0</td>\n",
" <td>71.0</td>\n",
" <td>160.0</td>\n",
" <td>163.0</td>\n",
" <td>9.0</td>\n",
" <td>a radio ad</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" id label statement \\\n",
"0 2635.json false Says the Annies List political group supports ... \n",
"3 1123.json false Health care reform legislation is likely to ma... \n",
"5 12465.json true The Chicago Bears have had more starting quart... \n",
"12 5947.json false When Mitt Romney was governor of Massachusetts... \n",
"16 620.json true McCain opposed a requirement that the governme... \n",
"\n",
" subjects speaker job \\\n",
"0 abortion dwayne-bohac State representative \n",
"3 health-care blog-posting NaN \n",
"5 education robin-vos Wisconsin Assembly speaker \n",
"12 history,state-budget mitt-romney Former governor \n",
"16 federal-budget barack-obama President \n",
"\n",
" state party #barely_true #false #half_true #mostly_true \\\n",
"0 Texas republican 0.0 1.0 0.0 0.0 \n",
"3 NaN none 7.0 19.0 3.0 5.0 \n",
"5 Wisconsin republican 0.0 3.0 2.0 5.0 \n",
"12 Massachusetts republican 34.0 32.0 58.0 33.0 \n",
"16 Illinois democrat 70.0 71.0 160.0 163.0 \n",
"\n",
" #pants_on_fire context \n",
"0 0.0 a mailer \n",
"3 44.0 a news release \n",
"5 1.0 a an online opinion-piece \n",
"12 19.0 an interview with CBN News \n",
"16 9.0 a radio ad "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"----\n",
"#### Test Data:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>id</th>\n",
" <th>label</th>\n",
" <th>statement</th>\n",
" <th>subjects</th>\n",
" <th>speaker</th>\n",
" <th>job</th>\n",
" <th>state</th>\n",
" <th>party</th>\n",
" <th>#barely_true</th>\n",
" <th>#false</th>\n",
" <th>#half_true</th>\n",
" <th>#mostly_true</th>\n",
" <th>#pants_on_fire</th>\n",
" <th>context</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>11972.json</td>\n",
" <td>true</td>\n",
" <td>Building a wall on the U.S.-Mexico border will...</td>\n",
" <td>immigration</td>\n",
" <td>rick-perry</td>\n",
" <td>Governor</td>\n",
" <td>Texas</td>\n",
" <td>republican</td>\n",
" <td>30</td>\n",
" <td>30</td>\n",
" <td>42</td>\n",
" <td>23</td>\n",
" <td>18</td>\n",
" <td>Radio interview</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>11685.json</td>\n",
" <td>false</td>\n",
" <td>Wisconsin is on pace to double the number of l...</td>\n",
" <td>jobs</td>\n",
" <td>katrina-shankland</td>\n",
" <td>State representative</td>\n",
" <td>Wisconsin</td>\n",
" <td>democrat</td>\n",
" <td>2</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>a news conference</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>11096.json</td>\n",
" <td>false</td>\n",
" <td>Says John McCain has done nothing to help the ...</td>\n",
" <td>military,veterans,voting-record</td>\n",
" <td>donald-trump</td>\n",
" <td>President-Elect</td>\n",
" <td>New York</td>\n",
" <td>republican</td>\n",
" <td>63</td>\n",
" <td>114</td>\n",
" <td>51</td>\n",
" <td>37</td>\n",
" <td>61</td>\n",
" <td>comments on ABC's This Week.</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>5962.json</td>\n",
" <td>true</td>\n",
" <td>Over the past five years the federal governmen...</td>\n",
" <td>federal-budget,pensions,retirement</td>\n",
" <td>brendan-doherty</td>\n",
" <td>NaN</td>\n",
" <td>Rhode Island</td>\n",
" <td>republican</td>\n",
" <td>1</td>\n",
" <td>2</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>a campaign website</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>7070.json</td>\n",
" <td>true</td>\n",
" <td>Says that Tennessee law requires that schools ...</td>\n",
" <td>county-budget,county-government,education,taxes</td>\n",
" <td>stand-children-tennessee</td>\n",
" <td>Child and education advocacy organization.</td>\n",
" <td>Tennessee</td>\n",
" <td>none</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>in a post on Facebook.</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" id label statement \\\n",
"0 11972.json true Building a wall on the U.S.-Mexico border will... \n",
"1 11685.json false Wisconsin is on pace to double the number of l... \n",
"2 11096.json false Says John McCain has done nothing to help the ... \n",
"5 5962.json true Over the past five years the federal governmen... \n",
"6 7070.json true Says that Tennessee law requires that schools ... \n",
"\n",
" subjects speaker \\\n",
"0 immigration rick-perry \n",
"1 jobs katrina-shankland \n",
"2 military,veterans,voting-record donald-trump \n",
"5 federal-budget,pensions,retirement brendan-doherty \n",
"6 county-budget,county-government,education,taxes stand-children-tennessee \n",
"\n",
" job state party \\\n",
"0 Governor Texas republican \n",
"1 State representative Wisconsin democrat \n",
"2 President-Elect New York republican \n",
"5 NaN Rhode Island republican \n",
"6 Child and education advocacy organization. Tennessee none \n",
"\n",
" #barely_true #false #half_true #mostly_true #pants_on_fire \\\n",
"0 30 30 42 23 18 \n",
"1 2 1 0 0 0 \n",
"2 63 114 51 37 61 \n",
"5 1 2 1 1 0 \n",
"6 0 0 0 0 0 \n",
"\n",
" context \n",
"0 Radio interview \n",
"1 a news conference \n",
"2 comments on ABC's This Week. \n",
"5 a campaign website \n",
"6 in a post on Facebook. "
]
},
"metadata": {},
"output_type": "display_data"
2018-05-11 12:19:53 +02:00
}
],
"source": [
2018-05-13 10:12:19 +02:00
"names = [\n",
" \"id\",\n",
" \"label\",\n",
" \"statement\",\n",
" \"subjects\",\n",
" \"speaker\",\n",
" \"job\",\n",
" \"state\",\n",
" \"party\",\n",
" \"#barely_true\",\n",
" \"#false\",\n",
" \"#half_true\",\n",
" \"#mostly_true\",\n",
" \"#pants_on_fire\",\n",
" \"context\"\n",
"]\n",
"\n",
"df_2_train = pd.read_csv(\"data/train.tsv\", delimiter='\\t', names=names)\n",
"df_2_test = pd.read_csv(\"data/test.tsv\", delimiter='\\t', names=names)\n",
"\n",
"# use only 'False' and 'True' statements\n",
"\n",
"df_2_train = df_2_train[df_2_train['label'].isin([\"false\",\"true\"])]\n",
"df_2_test = df_2_test[df_2_test['label'].isin([\"false\",\"true\"])]\n",
"\n",
2018-05-13 12:44:36 +02:00
"display(Markdown(\"----\\n#### Train Data:\"))\n",
2018-05-13 10:12:19 +02:00
"display(df_2_train.head())\n",
2018-05-13 12:44:36 +02:00
"display(Markdown(\"----\\n#### Test Data:\"))\n",
"display(df_2_test.head())"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2018-05-13 10:12:19 +02:00
"#### tdidf vectorizer on new dataset\n"
2018-05-11 12:19:53 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 22,
2018-05-11 12:19:53 +02:00
"metadata": {},
2018-05-13 10:12:19 +02:00
"outputs": [],
"source": [
2018-05-13 10:35:15 +02:00
"X2 = df_2_train['statement']\n",
"y2 = df_2_train['label']\n",
"Xt2 = df_2_test['statement']\n",
2018-05-13 12:44:36 +02:00
"yt2 = df_2_test['label']\n",
"\n"
2018-05-13 10:12:19 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 23,
2018-05-13 10:12:19 +02:00
"metadata": {},
"outputs": [],
"source": [
2018-05-13 12:44:36 +02:00
"count_vectorizer_2 = TfidfVectorizer(stop_words='english', max_df=0.7)\n",
"count_train_2 = count_vectorizer_2.fit_transform(X2)\n",
"count_test_2 = count_vectorizer_2.transform(Xt2)"
2018-05-13 10:12:19 +02:00
]
},
{
2018-05-13 10:35:15 +02:00
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 24,
2018-05-13 10:12:19 +02:00
"metadata": {},
2018-05-13 12:44:36 +02:00
"outputs": [
{
"data": {
"text/plain": [
"MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
2018-05-13 10:12:19 +02:00
"source": [
2018-05-13 12:44:36 +02:00
"clf_b = MultinomialNB()\n",
"clf_b.fit(count_train_2, y2)"
2018-05-13 10:12:19 +02:00
]
},
{
"cell_type": "code",
2018-05-13 12:44:36 +02:00
"execution_count": 25,
2018-05-13 10:12:19 +02:00
"metadata": {},
2018-05-11 12:19:53 +02:00
"outputs": [
2018-05-13 10:12:19 +02:00
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.6192560175054704'\n",
"Confusion matrix, without normalization\n"
]
},
2018-05-11 12:19:53 +02:00
{
"data": {
2018-05-13 10:35:15 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVAAAAEmCAYAAAA0k8gFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmcVXX9x/HXG1AUIVBxQRZxQ1NTUtSyVEwz3EIrLbMkJXHLsixz+2VpmrarSUZprpFmWZSomUsqCYqGC664gyjgQsq+fH5/nO/onXGWM5c7c8+deT95nAf3fs+553zuvTOf+S7nfI8iAjMza70u1Q7AzKxWOYGamZXJCdTMrExOoGZmZXICNTMrkxOomVmZnEBbQdKakv4uab6kP63Cfg6X9M9KxlYNkm6WNKrM1/5Q0jxJr1Y6rrbS8Psv2vco6XRJv6t2HJ1KRHS4BfgiMBV4B5gN3Ax8vAL7/TJwP9Ct2u+xifiGAwHc2KB8+1R+V879fB+4pg3jHAQsAtav9mdWq99/+q5nttOx9gfuBd4CXgV+B/Sq9mdQhKXD1UAlfQv4JXAesAHZL+tYYGQFdr8x8HRELK/AvtrKXOCjktYtKRsFPF2pAyizKj87g4DXI2JOGcfutgrHXVXt8v1X4POttN7AD4GNgA8C/YGfVDWioqh2Bq/wX8reZLXOQ5rZpjtZgn0lLb8Euqd1w4GZwMnAHLLa65Fp3Q+ApcCydIzRNKipAYPJanrd0vOvAM8BbwPPA4eXlN9b8rpdgQeA+en/XUvW3QWcA0xK+/kn0LeJ91YX/6XACamsKzAL+B4lNVDgQuBl4H/Ag8BuqXxEg/f5cEkc56Y4FgGbp7KvpvW/Bv5csv8LgNsBNYhx7/T6lWn/V6TyTwPTyWo5dwEfLHnNC8B3gUeAJeSoAQLbALcBbwCvAae3wfff8HvcB3gqfY9jgX+XfD7fp/mflcY+3yOBJ9L3/hxwTNp2rQaf4Ttkya3hMVr6TL+dPtP5wHXAGjl/zz4DPFrt3/ciLFUPoKJvJvvlX97cLxhwNjAZWB9YD/gPcE5aNzy9/mxgNWA/YCGwdlrf8Ae0yV+K9EP+P2DLtK4fsE16/O4vHrAO8CZZ87AbcFh6vm5afxfwLDAEWDM9P7+J9zacLAHsCkxJZfsBtwJfpX4C/RKwbjrmyWRNszUae18lcbxElpi6pc/nLt5LED3IarlfAXYD5gEDmouz5PkQYAHwybTfU4AZwOpp/QvANGAgsGYqGwuMbWL/vciS38nAGun5Lm3w/Zd+j33T9/2Z9Pl8gyzZtiaBNvx89wc2AwTskWLZobHPsOExcn6m95Ml3nXIEvWxOX/Pfgn8sdq/70VYitRMqIR1gXnRfBPrcODsiJgTEXPJahZfLlm/LK1fFhETyf66b1lmPCuBbSWtGRGzI2J6I9vsDzwTEVdHxPKIGA88CRxYss3vI+LpiFgEXA8Mbe6gEfEfYB1JWwJHAFc1ss01EfF6OubPyGpmLb3PKyJienrNsgb7W0j2Of4cuAY4MSJmtrC/Op8HboqI29J+f0r2x2LXkm0uioiX02dARBwfEcc3sb8DgFcj4mcRsTgi3o6IKWldW33/+wHTI+Iv6efvIrI/Sq1R7/ONiJsi4tnI/Jus9bFbzn3l/UxfiYg3gL/Tws8VgKRPknUJfa81b6yj6mgJ9HWgbwv9ZBsBL5Y8fzGVvbuPBgl4IdCztYFExAKyH+JjgdmSbpK0VY546mLqX/K89BcxbzxXA18D9gRubLhS0rclPZFGlN8i6/7o28I+X25uZUpSz5HVmK7PEWOdep9BRKxMxyr9DJo9dgMDyWrtLR6Lyn3/G5XGGFlVLe8fkDr13qOkfSVNlvRG+o72o+XvqDSelj7TVv1cSfoI8AfgcxFRsT71WtbREuh9ZH1kBzWzzStkgwF1BqWyciwga7rW2bB0ZUTcGhGfJGu+Pwn8Nkc8dTHNKjOmOlcDxwMTU+3wXZJ2I2vSHUrWPO1D1g+mutCb2GezU3dJOoGsJvtK2n9e9T4DSSJLgqWfQWumDXsZ2DTPsVi177/UbGBA3ZP0HgaUrG/2ZyV59z1K6g78mazmuEH6jibS8ndUJ89nmpukDwMTgKMi4vZy9tERdagEGhHzyZoWl0g6SFIPSaulv+Q/TpuNB86UtJ6kvmn7a8o85DRgd0mDJPUGTqtbIWkDSSMlrUWW1N8ha9I3NBEYIumLkrpJ+jywNfCPMmMCICKeJ+s3O6OR1b3I+vrmAt0kfQ/4QMn614DBrRkJljSEbKT2S2RN4lMktdgkTK4H9pe0l6TVyPoul5D1T5bjH0A/SSdJ6i6pl6Rd0rpKfv+lbgI+lH7uugEnUD9JNvmz0oTVyf4YzQWWS9qXbJCqzmvAumlfjanYZyppW+AWsm6Zv7f29R1Zh0qgAKk/71vAmWQ/fC+TNWX/mjb5Idk5oo8AjwIPpbJyjnUb2ejlI2Qj2aVJr0uK4xWykeA9gOMa2cfrZH12J5N1QZwCHBAR88qJqcG+742IxmpXt5L9QjxN1sxbTP3mY91FAq9Leqil46SEcQ1wQUQ8HBHPAKcDV6eaVEtxPkWWeC8mG3w6EDgwIpY2c8xLJV3axP7eJhs8OZCsmfoMWVcGVPD7b3DMecAhwI/Jvset03GWpPXN/aw09R6+TpYI3yQ7t3lCyfonyf4YPCfpLUkbNXh9qz/TZpxMNuB2maR30tJYf36no6yrxswqKdXeZ5KdunZnteOxttHhaqBm1SLpU5L6pFr36WT9lZOrHJa1ISdQs8r5KNnof12T+aC6066sY3IT3sysTK6BmpmVqZoTM7SJPuusG/36D6p2GNZKby1Z1vJGVjivPjN9XkSsV6n9df3AxhHL8/V6xKK5t0bEiEoduxwdLoH26z+I39/oQc9a848ZrZ6YyQrg3H23bHgV3SqJ5YvovuWhubZdPO2SvFdltZkOl0DNrJYJCjWTX/OcQM2sOAR06VrtKHJzAjWzYpFa3qYgnEDNrEDchDczK59roGZmZRCugZqZlUeugZqZlc2j8GZm5fAgkplZeYSb8GZmZXMN1MysHG7Cm5mVr4ub8GZmredr4c3MyuUmvJlZ+TwKb2ZWphqqgdZOpGbW8Un5lxZ3pcslzZH0WEnZUEmTJU2TNFXSzqlcki6SNEPSI5J2yBOuE6iZFYu65FtadgXQ8J5JPwZ+EBFDge+l5wD7AlukZQzw6zwHcAI1swJRNgqfZ2lBRNwNvNGwGPhAetwbeCU9HglcFZnJQB9J/Vo6hvtAzaxY8g8i9ZU0teT5uIgY18JrTgJulfRTsgrkrqm8P/ByyXYzU9ns5nbmBGpmxdG6+UDnRcSwVh7hOOCbEfFnSYcClwF7t3If73IT3swKRJXsA23MKOAv6fGfgJ3T41nAwJLtBqSyZjmBmlmxVGgUvgmvAHukx58AnkmPJwBHpNH4jwDzI6LZ5ju4CW9mRVOh80AljQeGk/WVzgTOAo4GLpTUDVhMNuIOMBHYD5gBLASOzHMMJ1AzKw6pYtfCR8RhTazasZFtAzihtcdwAjWzYvGlnGZm5ZETqJlZ62V39HACNTNrPaWlRjiBmlmByDVQM7NydelSO6enO4GaWaG4BmpmVg73gZqZlUfuAzUzK58TqJlZmZxAzczKIVAXJ1Azs7K4BmpmVgYPIpmZrQInUDOzctVO/nQCNbMCkWugZmZl87XwZmZl8CCSmdmqqJ386QRaVON/P5a/X381Emw2ZGvOuOASHn3ofi4+//9YvmwpW247lNPPu5hu3fwVVtPB227AluutxYKlK7h40osAbLNBTz6x+bqs13N1Lr3vJV753xIAugpGbrMBG/VegwiY+OQcnn9jUTXDL54a6wOtnc6GTmTOq6/wp6t+w+U33sG1E+9jxcqV/HPCDZxzynGc88vLuHbifWy40UAm3ji+2qF2ev+d9T+ufHBWvbI57yxl/LRXePHN+slx2MDeAPxq0otcMXUmI7Zcr5YqW+1GUq6lCJxAC2rF8uUsWbyY5cuXs3jRQtbs0YPVVludQZt
2018-05-11 12:19:53 +02:00
"text/plain": [
2018-05-13 12:44:36 +02:00
"<matplotlib.figure.Figure at 0x7fbbaf9c4908>"
2018-05-11 12:19:53 +02:00
]
},
"metadata": {},
2018-05-13 10:12:19 +02:00
"output_type": "display_data"
2018-05-11 12:19:53 +02:00
}
],
"source": [
2018-05-13 12:44:36 +02:00
"test_classifier(labels=[\"true\", \"false\"], title=\"configuration 2\", Xt=count_test_2, yt=yt2, clf=clf_b)"
2018-05-13 10:35:15 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----\n",
"## configuration 3"
2018-05-09 19:13:08 +02:00
]
},
2018-05-13 12:44:36 +02:00
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.45295404814004375'\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAEmCAYAAACQ+XDWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xe8HFX9//HXOwkkBCIJBAKBFDoi0pv4RSO9B1B6VRRRlK+iImJFRbDzRUXEHxiKhiKCdESKKFIMEHpNQkkIKQRiIJBA8vn9cc4Ne5db9iZ7M3dn38889pG7Z2bnfGZm97NnzpydUURgZmaNrVfRAZiZ2ZJzMjczKwEnczOzEnAyNzMrASdzM7MScDI3MyuBQpO5pOUkXStptqQrlmA5h0v6Wz1jK4KkGyUdvZiv/aGkmZJerndc3aV6//e0/SjpVEn/r+g4liZJYyT9sMZ5n5O082LUMUTSE5L6dj3CLtf1PUmXdHc9PUFNyVzSYZLGSXpd0tScdP6nDvV/AhgCrBwRBy7uQiLijxGxax3iaUXSKEkh6aqq8k1z+R01LqemN1RE7BERFy5GnMOBrwAbRcRqXX19gVrt/+7aj7XI+3pyZVlE/CgiPt0NdX1M0iOSXpP0iqSrJK1R73p6qoiYBtwOHFd0LJW68kXWnfVIWlXSWEkv5YbOXZK27Wy5nSZzSScBZwE/In3whgPnAKNrD79dI4CnI+KdOiyru8wAPiRp5Yqyo4Gn61WBkiU5ShoOvBIR0xej7j5LUO+SWir7vw7bt94eB3aLiIHAUOAZ4LfFhrTU/RH4bNFB9FArAP8BtgRWAi4Erpe0Qoevioh2H8CKwOvAgR3M05eU7F/Kj7OAvnnaKGAyqdU4HZgKfDJPOw2YD7yd6zgW+B5wScWyRwIB9MnPjwEmAnOAScDhFeX/qnjd9nljzM7/b18x7Q7gB8BdeTl/Awa3s24t8Z8LnJDLegNTgO8Ad1TM+3/Ai8B/gfuBHXL57lXr+VBFHKfnON4E1s1ln87TfwtcWbH8HwO3AqqKcef8+oV5+WNy+b7AY8Brebnvr3jNc8DXgYeBeS3bt5P3wgeAW4BZwDTg1G7Y/9X7cVfgqbwfzwH+UbF9vkfH75W2tu8ngSfyfp8IfDbPu3zVNnydlGSr6+hsm341b9PZwGVAvxq2a1/gDODxzuatqutrua43gPNJDa0b87r9HRhUY9ybAw/k110GXAr8sGL63sD4/Np/A5tUxbFzOzHuBTxI+jy8CHyvanofYC4wotb1zq9TJ9PXyu+TOaT366+r9uEVwMt5H90JfCCXH5ffi/Pz/r82l58CTMjLexzYv2JZ6+a6ZgMzgcsqpm3Iu5+Xp4CDOqqnhvX+L7Blh/N0soDdgXfo4MMOfB+4B1gVWCXv8B9UfJjfyfMsA+yZd+Cgdj6Q1c9Hkj+gpA/cf4EN8rTVK3bEMeQkQPomexU4Mr/u0Px85YoP+QRgfWC5/PzMdtZtFCkZbQ/cm8v2BG4GPk3rZH4EsHKu8yv5DdOvrfWqiOMFUpLsk7fPHbybrPqTWv/HADvkN8uaHcVZ8Xx90od8l7zck4FngWUrPoTjgWHAcrnsHOCcdpY/gJSIvwL0y8+37Yb9X7kfB+f9fUDePv9L+hB0JZlXb9+9gHUAAR/NsWzR1jasrqPGbXof6UtgJdKXxvEdfG6GkxLkwrxex1RMOwW4roPXPpe3+RBgDdIX5QOkxNwPuA34bmdx58fzwJfztE/kWH6YX7t5Xva2pEbM0bnuvhVxtJfMRwEfJB39b0JqAOxXNc/DwL61JLM8/0BgHLB2B/PcDfyC9CX5EVISrnyffIr0/m1phIyvmDaGii+yXHZg3qe9gIPztlw9TxsLfDNP6wf8Ty5fnvQF9knSe29z0ud3o/bq6WS9NwPeAlbsaL7ODj1XBmZGx4fBhwPfj4jpETGD1OI6smL623n62xFxA+nbaINO6m3PQmBjSctFxNSIeKyNefYCnomIiyPinYgYCzwJ7FMxzx8i4umIeBO4nLSx2hUR/wZWkrQBcBRwURvzXBIRr+Q6f056s3S2nmMi4rH8mrerljeXtB1/AVwCfDEiJre1kDYcDFwfEbfk5f6M9MW1fcU8Z0fEi3kbEBGfj4jPt7O8vYGXI+LnEfFWRMyJiHvztO7a/3sCj0XEX/L772zSF2RXtNq+EXF9REyI5B+ko7IdalxWrdv0pYiYBVxLB++riHghUjfLYOBbpPdoy7QzI2LvTuL5VURMi4gpwD9JjY0HI+It4CpSAuks7u1ISfysvH3+TDqSbXEc8LuIuDciFkQ6nzMvv65DEXFHRDwSEQsj4mFS4vto1WxzSAm6JhHxGvA74DZJa1dPz+eOtga+HRHzIuJO0n6oXMYF+f07j/RlvamkFTuo84q8TxdGxGWkLrFt8uS3SV2FQ/Pn4l+5fG/guYj4Q37vPQhcSfpi6BJJ7wMuBk6LiNkdzdtZMn8FGNxJv+pQ0rd7i+dz2aJlVH0ZzCX1CXVJRLxBemMeD0yVdL2kDWuIpyWmyhNMlUmh1nguBr4AfIz0YWlF0lfzGfrZkl4jdVEN7mSZL3Y0MSfMiaSW5OU1xNii1TaIiIW5rspt0GHdVYaRjmY6rYv67f+hlTFGaqLU+mXWotU6StpD0j2SZuV9tCed76PKeDrbpl1+X+XEfyHw1y6ev5hW8febbTxvqbujuIcCU/K2bVG5L0cAX8knal/L22wYrfdvmyRtK+l2STMkzSZ9bqu39QDS0Un1a/vlAQbveQDn5bh+2Ua1Q4FXc654z/pI6i3pTEkTJP2XdGRBG3FVxnKUpPEV679xxfwnkz6b90l6TNKncvkIYNuq7XY40KXBCZKWI30Z3RMRZ3Q2f2fJ/G7SN/F+HczzEin4FsNz2eJ4g9S90KLVykfEzRGxC6mL5Ung9zXE0xLTlMWMqcXFwOeBG3KreRFJO5B27EGkLoSBpH40tYTezjI7vGSlpBNILfyX8vJr1WobSBLpQ1i5DbpyucwXgfe0hNqqiyXb/5WmAmu2PMnrsGbF9A7fK9midczD4K4ktUyH5H10A53voxa1bNPF1YfUTfW+OiyrWkdxTwXWyGUthlf8/SJwekQMrHj0z0e7nfkTcA0wLCJWJJ13WlRP/uJaF3io+oW5lau2HqSjhZauoWpTgUGSlm9nfQ4jDdzYmdTYGtkSTkvVlQuTNIKUY75A6qYdCDzaMn9EvBwRn4mIoaSTuedIWpe03f5Rtd1WiIjPtVVPW/L79WpSA6amE8UdJvPcrP8O8BtJ+0nqL2mZ3ML5SZ5tLPAtSatIGpznX9xxneOBj0gang99vtEyIY9NHZ131DzS4frCNpZxA7B+Hk7ZR9LBwEbAdYsZEwARMYl0mPjNNiYPIPUNzwD6SPoOrT+Y04CRXRlRIWl94IekvvgjgZMlddgdVOFyYC9JO0lahtTXPY/Un704rgNWl/QlSX0lDagYKlXP/V/peuCD+X3XBziB1gm73fdKO5YlfTHOAN6RtAfpBGuLacDKHRxy122bSjpA0gaSeklahdSV9mBupddbR3HfTXrfnpg/1wfwbhcCpER2fG5lS9LykvaSNKCGegcAsyLiLUnbkBJppW1IXRHVR9HtkjSQlNh2jIiJ1dPzssYBp0laVmn4dGX36gDSur9Cagj8qGoR02jdaFmelHhn5Po/SWqZt8RzoKSWBsared6FpM/L+pKOzNt1GUlbS3p/O/VUr+cywJ9JR1hH56OpTnWaXHL/70mkfr0ZpG+dL5C+NSAlnHGkkxmPkE7ELNZYzYi4hXRG/WHSiJDKBNwrx/ES6QzxR4HPtbGMV0h9Vl8h7bSTgb0jYubixFS17H9FRFutzpuBm0gnLJ8nnayoPMRv+UHUK5Ie6KyenLwuAX4cEQ9FxDPAqcDFquGHFhHxFOlL4FekEy/7APtExPwO6jxX0rntLG8O6QTaPqSuhGdI3U1Qx/1fVedMUh/jT0j7caNcz7w8vaP3SnvrcCIpub1KSi7XVEx/kvTFNDEfGg+ten2Xt2kH1iC9X+aQttl
"text/plain": [
"<matplotlib.figure.Figure at 0x7fbbafe56f60>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"yt2_c3 = yt2.copy()\n",
"yt2_c3[yt2_c3 == \"true\"] = \"REAL\"\n",
"yt2_c3[yt2_c3 == \"false\"] = \"FAKE\"\n",
"\n",
"test_classifier(labels=[\"FAKE\", \"REAL\"], \n",
" title=\"configuration 3: model a) → dataset 2\",\n",
" Xt=count_vectorizer_1.transform(Xt2),\n",
" yt=yt2_c3, clf=clf_a)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"'score: 0.4807995791688585'\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEmCAYAAAByJWuvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xm8HfP9x/HX+yaRBFlICELEElsRS+z7UmusP1G1RaVSS7WKn9pKtRTdtKqoNiWWWmqplKB+iqKWBLFvsUQ2kUViDRKf3x/zvXFy3eXcOPfOvXPez/uYx50z62dmzvmc7/nOzHcUEZiZWftXk3cAZmZWGU7oZmYF4YRuZlYQTuhmZgXhhG5mVhBO6GZmBZFrQpfUVdI/Jc2R9PevsZxDJP2rkrHlQdJdkoYu4rznSpoh6Z1Kx9VS6h7/tnYcJZ0u6S95x9GaJF0l6dwyp31L0s7NXY6kzpJelrTM14m1HJKOkPRwS6+nrSgroUs6WNJYSR9KmpoSz9YVWP8BQB+gV0QMWdSFRMR1EbFLBeJZiKTtJYWk2+oMH5iGP1Dmcn4q6dqmpouI3SNi5CLE2Q84CVgnIpZr7vw5Wuj4t9RxLEc61pNKh0XELyLiuy2wrh0kPSdptqSZkm6T1LfS62mrIuJT4K/AqXnHUqrcz2lrrEfS91PO/VTSVeUuu8mELulE4HfAL8g+fP2AS4F9yl1JI1YGXo2IeRVYVkuZDmwhqVfJsKHAq5VagTJf59dSP2BmRLy7COvu+DXW+3W1yvGvwP6ttBeBXSOiJ7AC8BpwWb4htbq/AUMldc47kDZqCnAu2Rdf+SKiwQ7oAXwIDGlkms5kCX9K6n4HdE7jtgcmkZUe3wWmAt9J484BPgM+T+sYBvwUuLZk2f2BADqm10cAbwAfAG8Ch5QMf7hkvi2BMcCc9H/LknEPAD8HHknL+RfQu4Ftq43/cuC4NKwDMBk4C3igZNrfAxOB94EngW3S8N3qbOczJXGcl+L4BFg9DftuGn8ZcEvJ8i8E7gNUJ8ad0/xfpOVflYbvDbwAzE7LXbtknreAHwPPAp/W7t8m3gvfAO4FZgHTgNNb4PjXPY67AK+k43gp8GDJ/vkpjb9X6tu/3wFeSsf9DeB7adol6uzDD8kSbd11NLVPT077dA5wI9CljP3aGTgfeLGpaeus63/Tuj4CRpAVtu5K2/Z/wFJlxr0h8FSa70bgBuDckvGDgXFp3v8C69eJY+cGYryK7HNzb1r2g8DKdaZ5Ddiu3O1O86iJ8b2AUWSfwyfIPuul76nmfk7rfc+kcb2BO9K+mQU8BNSkcSsAt5AVCN8EftDYehrZnnNJn+my9k8TC9sNmEcjH3jgZ8BjwLLAMumg/7zkAz0vTdMJ2AP4uPbNxlc/MHVf9yd9SMk+dO8Da6ZxywPfSP1H1B40YGngPeCwNN+30+teJR/014E1gK7p9QUNbNv2ZAlpS+DxNGwP4B7guyyc0A9Nb6aOZAnsHdIHuu52lcTxNlmi7Jj2zwN8mbAWJ/sVcASwDTADWLGxOEter0H2Qf9mWu4pwHhgsZIP4jhgJaBrGnYpcGkDy+9GloxPArqk15u1wPEvPY690/HeP+2fH5J9CJqT0Ovu3z2B1QAB26VYNqpvH9ZdR5n79AmyD/LSZEng6EY+N/3IEsEXabuOKBl3KnBHI/O+lfZ5H6Av2ZflU2TJuQvwb+DspuJO3QTgR2ncASmWc9O8G6Zlb0ZWkBma1t25JI7GEvoHwLZkX1q/pySxpmlGkRJdWckqO47/BQY1Ms0NwE1k+WJdssJXaUJv7ue0sffM+WRfWp1St02arobsy+KstI9XJfsy2LWh9TSyPc1K6E39DO0FzIjGfxIfAvwsIt6NiOlkJa/DSsZ/nsZ/HhGjyb6V1mxivQ35AlhXUteImBoRL9QzzZ7AaxFxTUTMi4jrgZeBvUqmuTIiXo2IT8gO/gaNrTQi/gssLWlN4HDg6nqmuTYiZqZ1/obsTdzUdl4VES+keT6vs7yPyfbjb4FrgeMjYlJ9C6nHt4A7I+LetNxfk315bVkyzcURMTHtAyLi2Ig4toHlDQbeiYjfRMTciPggIh5P41rq+O8BvBARt6b338VkH77mWGj/RsSdEfF6ZB4k+3W2TZnLKnefTomIWcA/aeR9FRFvR1bl0hs4k+w9WjvugogY3EQ8f4iIaRExmaxk+HhEPB0Rc4HbyJJxU3FvTpaIfpf2z81kv2hrDQf+FBGPR8T8yM7vfJrmK8edEfGfyOrMzyCrulypZPwHQM8yl0V6H5wH3CFpUN3xkjoA/wOcFREfRcTzwMg6y2jW57SJ98znZAXLldP+eyiyLLwJsExE/CwiPouIN4A/AweVu62LqqmEPhPo3UQ96wpk3/K1JqRhC5ZR5wvhY2DJZkUJRMRHZG/Oo4Gpku6UtFYZ8dTGVHrSqTQxlBvPNcD3gR3IPjALkXSypJfSFRuzyaqrejexzImNjUxJ8w2yb/2byoix1kL7ICK+SOsq3QeNrruOlch+1TS5Lip3/FcojTF9UMr9Qqu10DZK2l3SY5JmpWO0B00fo9J4mtqnzX5fpeQ/Eri9meczppX0f1LP69p1Nxb3CsDktG9rlR7LlYGT0snb2WmfrcTCx7cxpcfvQ7JqidJ5u5H9SvkKSe+kCw8W6siqOPqQVTPVtQxZybv0uC+UC5r7OW3iPfMrsl87/5L0hqTak7wrAyvU2W+np7hbVFMJ/VGyb+R9G5lmCtkG1OqXhi2Kj8iqGmotdMVGRNwTEd8k+1Z8mexbr6l4amOavIgx1boGOBYYnUrPC0jahuyn7IFk1Qk9yepRVRt6A8tstKlLSceRlSCmpOWXa6F9IElkH8TSfdCcZjYnkv1sbHJdfL3jX2oqsGLti7QNK5aMb/S9kizYxnTy7RayEmqfdIxG0/QxqlXOPl1UHcmqrLpXYFl1NRb3VKBvGlarX0n/ROC8iOhZ0i2efvWWY0FpXNKSZFVRpe+NtYFn6psxIpaLCNXtyH4tTiM751LXdLIqvtJfAQu2p7mf06beM+mX6kkRsSrZeYoTJe1Ett/erLPfukXEHvWtp5IaTegRMYesHuiPkvaVtLikTulb65dpsuuBMyUtI6l3mn5RL/0ZB2wrqZ+kHsBptSMk9ZG0j6QlyL5kPiSrgqlrNLBGutSyo6RvAeuQfbMvsoh4k6wO7Yx6RncjeyNNBzpKOouFP5zTgP7NudJC0hpk9WeHklVhnCKp0aqhEjcBe0raSVInsrrCT8nqHxfFHcDykk5Qdg1xN0mbpXGVPP6l7gTWS++7jsBxLJy0G3yvNGAxsi/H6cA8SbuTnXStNQ3olZZVn4rtU0n7S1pTUo2ya7F/CzydSuuV1ljcj5K9b3+QPtf7A5uWzPtn4GhJm6UrhZaQtKekbmWuew9JW0tajOzk5GMRMRFA2WWaS5OdCyhLeh+cAQyOiLF1x0fEfOBW4KcpV61DVu9fq7mf00bfM5IGS1o9fSHOAeaT5aQngA8k/VjZvRYdJK0raZMG1lPvtkrqQnbuooOkLuX8gmsywaR6phPJ6vmmk337fB/4R5rkXGAs2Rn358hOzpR1Y0I967qX7Ez7s2QnFUqTcE2KYwrZT7ftgGPqWcZMsm/xk8iqjE4hewPMWJSY6iz74Yior/R5D3A32UnMCcBcFv7ZV3vT1ExJTzW1nnTgrgUujIhnIuI1sp9s16iMy7wi4hWyL4I/kJ1M3QvYKyI+a2Sdl0u6vIHlfUB2Um0vsmqF18iqnqCCx7/OOmcAQ4Bfkh3HddJ6Pk3jG3uvNLQNPyBLcO8BB5OdlKsd/zLZl9Mb6WfyCnXmb/Y+bURfsvfLB2T77Atgv9qRym5oumsRlvsVjcWdYt+f7GT0LLIqzVtL5h0LHAVcQrbPxqdpy/U34Oy07I1THLUOBkam+vVyt2UesFV9ybzE98mqm94hOzF7Zcm4Zn1Om3rPAAPIrij6kOzL8dKIuD99sQwmO4fyJtl+/wtZ9c5X1tPAdpxJVnV2Ktl++yQNa5QWrj4za5tSaWYS2aW
"text/plain": [
"<matplotlib.figure.Figure at 0x7fbbafdd90f0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"yt1_c3 = yt1.copy()\n",
"yt1_c3[yt1_c3 == \"REAL\"] = \"true\"\n",
"yt1_c3[yt1_c3 == \"FAKE\"] = \"false\"\n",
"\n",
"test_classifier(labels=[\"false\", \"true\"], \n",
" title=\"configuration 3: model b) → dataset 1\",\n",
" Xt=count_vectorizer_2.transform(Xt1),\n",
" yt=yt1_c3, clf=clf_b)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"----\n",
"## configuration 4)"
]
},
2018-05-09 19:13:08 +02:00
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
2018-05-09 10:47:51 +02:00
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}