nlp-lab/Carsten_Solutions/Task 2 - Carsten Draschner.ipynb
2018-05-15 10:28:16 +02:00

1549 lines
218 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"import all usefull tool"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Populating the interactive namespace from numpy and matplotlib\n"
]
}
],
"source": [
"%pylab inline\n",
"\n",
"import pandas as pd\n",
"import numpy as np\n",
"import itertools\n",
"from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer, HashingVectorizer\n",
"from sklearn.model_selection import train_test_split\n",
"from sklearn.linear_model import PassiveAggressiveClassifier\n",
"from sklearn.naive_bayes import MultinomialNB\n",
"from sklearn.neural_network import MLPClassifier\n",
"from sklearn import metrics\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Read in Datasets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load Dataset 1 - fake_or_real_news.csv\n",
"Read in File fake_or_real_news.csv"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"df = pd.read_csv('/Users/Carsten/GitRepos/NLP-LAB/Carsten_Solutions/sets/fact checking/fake_or_real_news.csv')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"ignores first column"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df = df.set_index('Unnamed: 0')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"quick view at the data"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>title</th>\n",
" <th>text</th>\n",
" <th>label</th>\n",
" </tr>\n",
" <tr>\n",
" <th>Unnamed: 0</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>8476</th>\n",
" <td>You Can Smell Hillarys Fear</td>\n",
" <td>Daniel Greenfield, a Shillman Journalism Fello...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10294</th>\n",
" <td>Watch The Exact Moment Paul Ryan Committed Pol...</td>\n",
" <td>Google Pinterest Digg Linkedin Reddit Stumbleu...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3608</th>\n",
" <td>Kerry to go to Paris in gesture of sympathy</td>\n",
" <td>U.S. Secretary of State John F. Kerry said Mon...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10142</th>\n",
" <td>Bernie supporters on Twitter erupt in anger ag...</td>\n",
" <td>— Kaydee King (@KaydeeKing) November 9, 2016 T...</td>\n",
" <td>FAKE</td>\n",
" </tr>\n",
" <tr>\n",
" <th>875</th>\n",
" <td>The Battle of New York: Why This Primary Matters</td>\n",
" <td>It's primary day in New York and front-runners...</td>\n",
" <td>REAL</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" title \\\n",
"Unnamed: 0 \n",
"8476 You Can Smell Hillarys Fear \n",
"10294 Watch The Exact Moment Paul Ryan Committed Pol... \n",
"3608 Kerry to go to Paris in gesture of sympathy \n",
"10142 Bernie supporters on Twitter erupt in anger ag... \n",
"875 The Battle of New York: Why This Primary Matters \n",
"\n",
" text label \n",
"Unnamed: 0 \n",
"8476 Daniel Greenfield, a Shillman Journalism Fello... FAKE \n",
"10294 Google Pinterest Digg Linkedin Reddit Stumbleu... FAKE \n",
"3608 U.S. Secretary of State John F. Kerry said Mon... REAL \n",
"10142 — Kaydee King (@KaydeeKing) November 9, 2016 T... FAKE \n",
"875 It's primary day in New York and front-runners... REAL "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"store label column with the classification of each text"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"#store label before dropping it\n",
"bin_y = df.label\n",
"#y.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"cut of label column to get an unlabled array"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"df = df.drop('label', axis=1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"from skikit learn the function: train_test_split\n",
"* in the dataframe get text column by df['text']\n",
"* use stored y label df\n",
"* use seed 4222\n",
"* determine split size: in this case 0.25\n",
"* **take care that these config is the same to get compareable results**"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"bin_X_train, bin_X_test, bin_y_train, bin_y_test = train_test_split(df['text'], bin_y, test_size=0.25, random_state=4222)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load Dataset 2 - liar_dataset.zip\n",
"* Read in File liar_dataset.zip\n",
"* cause of three file read in each file on its own and assign it to train test vaidation \n",
"\n",
"#### So first train set\n",
"\n",
"* only use the text part and ignore the others\n",
"* it would also be possible to for example to concatiate the other information ito the text part but youll lose the robustness againstother datasets without these coulums and you'll mix up information"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"#training data file\n",
"df = pd.read_csv('/Users/Carsten/GitRepos/NLP-LAB/Carsten_Solutions/sets/fact checking/train.tsv', delimiter=\"\\t\", header=None, usecols=[1,2], names=['y', 'claim'])"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>y</th>\n",
" <th>claim</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>false</td>\n",
" <td>Says the Annies List political group supports ...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>half-true</td>\n",
" <td>When did the decline of coal start? It started...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>mostly-true</td>\n",
" <td>Hillary Clinton agrees with John McCain \"by vo...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>false</td>\n",
" <td>Health care reform legislation is likely to ma...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>half-true</td>\n",
" <td>The economic turnaround started at the end of ...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" y claim\n",
"0 false Says the Annies List political group supports ...\n",
"1 half-true When did the decline of coal start? It started...\n",
"2 mostly-true Hillary Clinton agrees with John McCain \"by vo...\n",
"3 false Health care reform legislation is likely to ma...\n",
"4 half-true The economic turnaround started at the end of ..."
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"mul_X_train = df.claim\n",
"mul_y_train = df.y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### second dataset: test set"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"#test data file\n",
"df = pd.read_csv('/Users/Carsten/GitRepos/NLP-LAB/Carsten_Solutions/sets/fact checking/test.tsv', delimiter=\"\\t\", header=None, usecols=[1,2], names=['y', 'claim'])"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>y</th>\n",
" <th>claim</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>true</td>\n",
" <td>Building a wall on the U.S.-Mexico border will...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>false</td>\n",
" <td>Wisconsin is on pace to double the number of l...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>false</td>\n",
" <td>Says John McCain has done nothing to help the ...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>half-true</td>\n",
" <td>Suzanne Bonamici supports a plan that will cut...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>pants-fire</td>\n",
" <td>When asked by a reporter whether hes at the ce...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" y claim\n",
"0 true Building a wall on the U.S.-Mexico border will...\n",
"1 false Wisconsin is on pace to double the number of l...\n",
"2 false Says John McCain has done nothing to help the ...\n",
"3 half-true Suzanne Bonamici supports a plan that will cut...\n",
"4 pants-fire When asked by a reporter whether hes at the ce..."
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"mul_X_test = df.claim\n",
"mul_y_test = df.y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### as third dataset: validation set"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"#test data file\n",
"df = pd.read_csv('/Users/Carsten/GitRepos/NLP-LAB/Carsten_Solutions/sets/fact checking/valid.tsv', delimiter=\"\\t\", header=None, usecols=[1,2], names=['y', 'claim'])"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"mul_X_valid = df.claim\n",
"mul_y_valid = df.y"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.series.Series'>\n"
]
}
],
"source": [
"print(type(mul_X_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Generate Dataset 3\n",
"* using code from Diego by copy paste with some small modifications\n",
"* thanks for distributing this code @diego ;)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"-- fake news\n",
"Index(['y', 'claim'], dtype='object')\n",
"3171\n",
"3164\n",
"6335\n",
"-- liar liar\n",
"Index(['y', 'claim'], dtype='object')\n",
"{'true', 'barely-true', 'half-true', 'pants-fire', 'mostly-true', 'false'} 10240\n",
"1676\n",
"1995\n",
"{'true', 'false'} 3671\n",
"false 5159\n",
"true 4847\n",
"Name: y, dtype: int64\n",
"done\n"
]
}
],
"source": [
"'''import random\n",
"import sys\n",
"import pandas as pd\n",
"import numpy as np\n",
"from sklearn.cross_validation import train_test_split\n",
"\n",
"ds1 = sys.argv[1]\n",
"ds2 = sys.argv[2]'''\n",
"\n",
"try:\n",
" print('-- fake news')\n",
" df1 = pd.read_csv('/Users/Carsten/GitRepos/NLP-LAB/Carsten_Solutions/sets/fact checking/fake_or_real_news.csv', sep=',', usecols=['title','text','label'])\n",
" df1['claim'] = df1[['title', 'text']].apply(lambda x: '. '.join(x), axis=1)\n",
" del df1['title']\n",
" del df1['text']\n",
" df1.rename(index=str, columns={'label': 'y'}, inplace=True)\n",
" print(df1.keys())\n",
" print(len(df1[df1['y']=='REAL']))\n",
" print(len(df1[df1['y']=='FAKE']))\n",
" df1['y'] = np.where(df1['y'] == 'FAKE', 'false', 'true')\n",
" print(len(df1))\n",
"\n",
" print('-- liar liar')\n",
" df2 = pd.read_csv('/Users/Carsten/GitRepos/NLP-LAB/Carsten_Solutions/sets/fact checking/train.tsv', sep='\\t', header=None, usecols=[1,2], names=['y', 'claim'])\n",
" print(df2.keys())\n",
" print(set(df2.y), len(df2))\n",
" print(len(df2[df2['y'] == 'true']))\n",
" print(len(df2[df2['y'] == 'false']))\n",
" df2=df2[(df2['y'] == 'true') | (df2['y'] == 'false')]\n",
" print(set(df2.y), len(df2))\n",
"\n",
" df3=pd.concat([df1, df2], ignore_index=True)\n",
"\n",
" print(df3['y'].value_counts())\n",
" print('done')\n",
" concat_X_train, concat_X_test, concat_y_train, concat_y_test = train_test_split(df3['claim'], df3['y'], test_size=0.25, random_state=4222)\n",
" \n",
" \n",
"except Exception as e:\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Vectorizer Classifiers\n",
"* tfids removes words in pregenerating the vectors by evaluating if this word appears more than 70% often in all articles (tfidf)\n",
"* an immense naive approach would be to store a set of the over all occuring word in all the texts and for each text determining how often this word occurs.\n",
"* also testing some min df thresholds as lower bound for regarded occurences of words over all texts ... not easy to predict if this is a good idea"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generate Vectorizer on Binary Classes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"generate two different vectorizers"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"#building the vectorspace\n",
"bin_count_vectorizer = CountVectorizer(stop_words='english')\n",
"bin_count_train = bin_count_vectorizer.fit_transform(bin_X_train)\n",
"#transform the other sets into same feature vector as trained on train data \n",
"bin_count_test = bin_count_vectorizer.transform(bin_X_test)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"#building the vectorspace\n",
"bin_tfidf_vectorizer = TfidfVectorizer(stop_words='english', max_df=0.7)#, min_df=0.0005)\n",
"bin_tfidf_train = bin_tfidf_vectorizer.fit_transform(bin_X_train)\n",
"#transform the other sets into same feature vector as trained on train data \n",
"bin_tfidf_test = bin_tfidf_vectorizer.transform(bin_X_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"a short look on the last 10 tokens for the vectors"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['zoomed',\n",
" 'zooming',\n",
" 'zor',\n",
" 'zucker',\n",
" 'zuckerberg',\n",
" 'zuesse',\n",
" 'zurich',\n",
" 'zwick',\n",
" 'état',\n",
" 'œthe']"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"bin_tfidf_vectorizer.get_feature_names()[-10:]"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['تنجح', 'حلب', 'عن', 'لم', 'ما', 'محاولات', 'من', 'هذا', 'والمرضى', 'ยงade']"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"bin_count_vectorizer.get_feature_names()[-10:]#[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generate Vectorizer on Multilabel Classes\n",
"* same again\n",
"* only additional transform for validation set"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"mul_count_vectorizer = CountVectorizer(stop_words='english')\n",
"mul_count_train = mul_count_vectorizer.fit_transform(mul_X_train)\n",
"mul_count_test = mul_count_vectorizer.transform(mul_X_test)\n",
"mul_count_valid = mul_count_vectorizer.transform(mul_X_valid)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"mul_tfidf_vectorizer = TfidfVectorizer(stop_words='english', max_df=0.7)#, min_df=0.0005)\n",
"mul_tfidf_train = mul_tfidf_vectorizer.fit_transform(mul_X_train)\n",
"mul_tfidf_test = mul_tfidf_vectorizer.transform(mul_X_test)\n",
"mul_tfidf_valid = mul_tfidf_vectorizer.transform(mul_X_valid)"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['zip',\n",
" 'zippo',\n",
" 'zombie',\n",
" 'zombies',\n",
" 'zone',\n",
" 'zones',\n",
" 'zoning',\n",
" 'zoo',\n",
" 'zuckerberg',\n",
" 'zuckerbergs']"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mul_tfidf_vectorizer.get_feature_names()[-10:]"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"['zip',\n",
" 'zippo',\n",
" 'zombie',\n",
" 'zombies',\n",
" 'zone',\n",
" 'zones',\n",
" 'zoning',\n",
" 'zoo',\n",
" 'zuckerberg',\n",
" 'zuckerbergs']"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mul_count_vectorizer.get_feature_names()[-10:]#[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generate Vectors on merged Sets\n",
"* again the same ..."
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"concat_count_vectorizer = CountVectorizer(stop_words='english')\n",
"concat_count_train = concat_count_vectorizer.fit_transform(concat_X_train)\n",
"concat_count_test = concat_count_vectorizer.transform(concat_X_test)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"concat_tfidf_vectorizer = TfidfVectorizer(stop_words='english', max_df=0.7)#, min_df=0.0005)\n",
"concat_tfidf_train = concat_tfidf_vectorizer.fit_transform(concat_X_train)\n",
"concat_tfidf_test = concat_tfidf_vectorizer.transform(concat_X_test)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['القادمون',\n",
" 'ایران',\n",
" 'جنگ',\n",
" 'سال',\n",
" 'عربي',\n",
" 'علیه',\n",
" 'مطالعاتی',\n",
" 'مورد',\n",
" 'کدآمایی',\n",
" 'ยงade']"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"concat_tfidf_vectorizer.get_feature_names()[-10:]"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['القادمون',\n",
" 'ایران',\n",
" 'جنگ',\n",
" 'سال',\n",
" 'عربي',\n",
" 'علیه',\n",
" 'مطالعاتی',\n",
" 'مورد',\n",
" 'کدآمایی',\n",
" 'ยงade']"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"concat_count_vectorizer.get_feature_names()[-10:]#[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Confusion Matrix Code\n",
"* copy paste by distributed notebook\n",
"* thx for providing"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def plot_confusion_matrix(cm, classes,\n",
" normalize=False,\n",
" title='Confusion matrix',\n",
" cmap=plt.cm.Blues):\n",
" \"\"\"\n",
" See full source and example: \n",
" http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html\n",
" \n",
" This function prints and plots the confusion matrix.\n",
" Normalization can be applied by setting `normalize=True`.\n",
" \"\"\"\n",
" #added after jonas hint\n",
" fig_1,ax_1 = plt.subplots()\n",
" \n",
" plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
" plt.title(title)\n",
" plt.colorbar()\n",
" tick_marks = np.arange(len(classes))\n",
" plt.xticks(tick_marks, classes, rotation=45)\n",
" plt.yticks(tick_marks, classes)\n",
"\n",
" if normalize:\n",
" cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
" print(\"Normalized confusion matrix\")\n",
" else:\n",
" print('Confusion matrix, without normalization')\n",
"\n",
" thresh = cm.max() / 2.\n",
" for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
" plt.text(j, i, cm[i, j],\n",
" horizontalalignment=\"center\",\n",
" color=\"white\" if cm[i, j] > thresh else \"black\")\n",
"\n",
" plt.tight_layout()\n",
" plt.ylabel('True label')\n",
" plt.xlabel('Predicted label')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Configurations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuration 1\n",
"* model a - train - [performance measures][0:4]\n",
"* model a - test - [performance measures][0:4]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* model a = MultinomialNB\n",
" * with tfidf vectorizer* dataset 1\n",
" * in contrast to the notebook with count vect.\n",
" * with dataset 1: fake_or_real_news.csv\n",
" * ** Take care with seeds for example split train test data function**\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.914\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcFNXZ9vHfNYOgCCqCuOCCC2qM\nUdzQaIwaFQEX0LgbtxjQRLM+MSFGozEazeZuNBp3n7g8rzFRXBD3JW6IuEUUVFQUWRURAWG43z+q\nBpuBmW6gerp6+vrmU5+ZPnW66m7G3HPmrlOnFBGYmVl26iodgJlZW+PEamaWMSdWM7OMObGamWXM\nidXMLGNOrGZmGXNitWZJWknS3ZJmSPq/5TjOUZIeyDK2SpG0q6Q3Kh2H5Zs8j7X6SToS+BmwOTAT\nGA2cGxFPLudxjwZ+COwcEfOXO9CckxRAr4gYV+lYrLp5xFrlJP0MuAj4PbAmsD7wV2BgBoffAHiz\nFpJqKSS1q3QMViUiwluVbsCqwGfAIS306UCSeD9Mt4uADum+3YEJwP8Ak4GJwPHpvt8CXwDz0nOc\nAJwF3Fxw7J5AAO3S18cBb5OMmt8Bjipof7LgfTsDzwMz0q87F+x7FPgd8FR6nAeAbs18tsb4f1EQ\n/yBgAPAmMB04raB/H+Bp4JO072VA+3Tf4+lnmZV+3sMKjv9L4CPgpsa29D0bp+fYNn29DjAV2L3S\n/214q+zmEWt1+zqwInBnC31+DewE9Aa2JkkupxfsX4skQfcgSZ6XS+oSEWeSjIJvi4hOEXFNS4FI\nWhm4BOgfEZ1JkufoJfRbHbgn7dsVuAC4R1LXgm5HAscD3YH2wM9bOPVaJP8GPYDfAFcD3wG2A3YF\nfiNpo7RvA/BToBvJv92ewA8AIuKbaZ+t0897W8HxVycZvQ8pPHFEvEWSdP9XUkfgOuD6iHi0hXit\nBjixVreuwNRo+U/1o4CzI2JyREwhGYkeXbB/Xrp/XkTcSzJa22wZ41kAbClppYiYGBGvLaHPvsDY\niLgpIuZHxC3AGGD/gj7XRcSbETEbuJ3kl0Jz5pHUk+cBt5IkzYsjYmZ6/teArQAi4oWIeCY973jg\nb8BuJXymMyNibhrPIiLiamAs8CywNskvMqtxTqzVbRrQrUjtbx3g3YLX76ZtC4/RJDF/DnRa2kAi\nYhbJn88nARMl3SNp8xLiaYypR8Hrj5YinmkR0ZB+35j4JhXsn934fkmbShom6SNJn5KMyLu1cGyA\nKRExp0ifq4EtgUsjYm6RvlYDnFir29PAHJK6YnM+JPkzttH6aduymAV0LHi9VuHOiBgeEXuTjNzG\nkCScYvE0xvTBMsa0NK4giatXRKwCnAaoyHtanDYjqRNJ3foa4Ky01GE1zom1ikXEDJK64uWSBknq\nKGkFSf0l/THtdgtwuqQ1JHVL+9+8jKccDXxT0vqSVgV+1bhD0pqSDkhrrXNJSgoNSzjGvcCmko6U\n1E7SYcAWwLBljGlpdAY+BT5LR9Pfb7J/ErDRYu9q2cXACxHxPZLa8ZXLHaVVPSfWKhcRF5DMYT0d\nmAK8D5wC/Cvtcg4wEngZeAUYlbYty7lGALelx3qBRZNhHcnsgg9JrpTvRnphqMkxpgH7pX2nkVzR\n3y8ipi5LTEvp5yQXxmaSjKZva7L/LOAGSZ9IOrTYwSQNBPqRlD8g+TlsK+mozCK2quQbBMzMMuYR\nq5lZxpxYzcwy5sRqZpYxJ1Yzs4y12UUl1KFz1HUqNvfb8mSL9bpUOgRbSh+8/x7Tp00tNhd4qdSv\nskHE/MVucltMzJ4yPCL6ZXnurLTZxFrXqRud9jm70mHYUvj3xQdXOgRbSgP33iXzY8b82XTYrOhs\nN+aMvjy3I6c2m1jNrFoJVN1VSidWM8sXAXX1lY5iuTixmln+KNOybatzYjWznHEpwMwsex6xmpll\nSHjEamaWLXnEamaWOc8KMDPLki9emZllS7gUYGaWOY9Yzcyy5FKAmVn26lwKMDPLjtcKMDPLmksB\nZmbZ86wAM7OMecRqZpYh+ZZWM7PsecRqZpYleVaAmVnmXAowM8uQ12M1M8ua57GamWXPpQAzs4z5\n4pWZWYbkUoCZWfZcCjAzy5acWM3MspM8maW6E2t1FzLMrO1RiVuxw0jrSXpE0uuSXpP047R9dUkj\nJI1Nv3ZJ2yXpEknjJL0saduCYx2b9h8r6dhi53ZiNbOcEXV1dUW3EswH/icivgLsBJwsaQtgKPBQ\nRPQCHkpfA/QHeqXbEOAKSBIxcCawI9AHOLMxGTfHidXMckdS0a2YiJgYEaPS72cCrwM9gIHADWm3\nG4BB6fcDgRsj8QywmqS1gX2AERExPSI+BkYA/Vo6t2usZpY7JdZYu0kaWfD6qoi4qpnj9QS2AZ4F\n1oyIiZAkX0nd0249gPcL3jYhbWuuvVlOrGaWLyXWUIGpEbF90cNJnYA7gJ9ExKctJO0l7YgW2pvl\nUoCZ5YooXgYoddaApBVIkur/RsQ/0+ZJ6Z/4pF8np+0TgPUK3r4u8GEL7c1yYjWz3MkisSrpdA3w\nekRcULDrLqDxyv6xwL8L2o9JZwfsBMxISwbDgb6SuqQXrfqmbc1yKcDMcqfEq/7F7AIcDbwiaXTa\ndhpwPnC7pBOA94BD0n33AgOAccDnwPEAETFd0u+A59N+Z0fE9JZO7MRqZvlSeo21RRHxZAtH2nMJ\n/QM4uZljXQtcW+q5nVjNLHeq/c4rJ1Yzy5XGi1fVzInVzHLHidXMLGvVnVedWM0sZ5TZrICKcWI1\ns9xxKcDMLEO+eGVmVg7VnVedWPPgsiFfp9826zLl0zl8/Zd3A7Dl+l248IQdWblDO96bOovBlz/J\nzNnzWKG+jou+tyPbbNiVBREMvXEkT74+CYAzDu3N4btuxGort6fHd2+t5EeqKXPnzOHwgXvzxdwv\naGiYT7/9BvGTX57BqT8cwnNPP0HnzqsA8MdLrmKLr2298H0vvziSb/ffnUuuvon++x9YqfDzR9Vf\nCqjuCnEb8Y/H3+Lbf3hokbZLB+/EWbeMYuehwxj2/Hv8aL8tADj2W5sAsPPQYQw67yHO/c52C5+7\ndt+oCXzrjPtaNXaD9h06cPMd93HPo89y98PP8PgjI3hx5HMADD3z9wx75FmGPfLsIkm1oaGBP/zu\nDHbdY69KhZ1rWS3CUilOrDnwnzGT+fizuYu0bbL2Kjw1Jll055FXJnLADusDsHmP1Xjs1Y8AmPrp\nHGbM+oJtNuoKwMhxU5n0yexWjNwgSQIrd+oEwPx585g/b17Rh4ze+Pcr6LfvQLp2695yxxqlOhXd\n8syJNaden/AJA7ZbF4BBO21Aj64rA/Dqex+z7/brUV8nNlijE1tv2JV1V1+5kqEayQh0vz12pM8W\nG7DLbnvSe7s+APzl92cxYLc+nHPGL5g7N/nl+dHED3jg3rs48rjBlQw51zxibYakBkmjC7aeBfsu\nlvSBpLqCtuMkXZZ+XyfpBknXpkt4jZf0SsGxLilX3Hlx8lVPM3jvzXjs3AF0WnEF5s1fAMBNj47j\ng2mf8+g5Azjv6O15buwU5i9YUOForb6+nmGPPMtTL43lpRdH8sbrr3Hq6b9lxH9Gc+cDT/DJxx9z\n1aV/AeCc03/BL844h/r6+gpHnU+lJNW8J9ZyXryaHRG9mzamyfRAkkcdfBN4tMl+AVcCKwDHR0Sk\n/4h7RMTUMsabK2M//JQDz0/qrhuv1Zl9tkmeBNGwIDjt5i+fRvHAWfvw1kczKxKjLW6VVVdjp513\n5fGHRzD45J8A0KFDBw4+4mj+/teLAHjlpVH8+MRjAPh42jQefWg49fX19B1wQMXizpu8J85iKlEK\n2AN4leQJiEcsYf/FQFfgmIio2aFYt1VWBECCUw/8Gtc++CYAK7Wvp2OH5PfhHluuzfyG4I0PZlQs\nToNpU6fw6YxPAJgzezZPPf4IG/falMmTJgIQEYy472423fyrADw28nUef2EMj78whn77H8jZf7jI\nSbUJj1ibt1LB4rLvRETjfJIjgFtIVu3+vaQVImJeuu9Ikicp7h4R85sc7xFJDen3N0TEhU1PKGkI\nyWNrUceuGX6U8rrmlG/wja+sSdfOK/LfSw/ivDteZuUV2zF4780AuPv597j5sbcAWGOVFfnn0D1Z\nEDDx48858YqnFh7n7CO25eCde9KxfTv+e+lB3PjoOM6/4+WKfKZaMmXSR5z6w8E0NCxgQSxg3wMO\n4lt9B3DUQf2ZPm0qEcEWX92K3/2pzVewMpP3i1PFKFnbtQwHlj6LiE5N2toD44HNImKmpH8C10TE\nPZKOA74DbA4cFhFPFbxvPLD90pQC6rtuGJ32OXv5P4i1mhcvPrjSIdhSGrj3LrwyelSmWbDDWr1i\n3aOK/xJ6+4IBL5TyMMFKaO1SQD9gVZJHJYwHvsGi5YAxwKHAbZK+2sqxmVkOiKQEVmzLs9ZOrEcA\n34uInhHRE9iQ5CFdHRs7RMR/gJOAeySt38rxmVnFeVZAydLkuQ9wYmNbRMyS9CSwf2HfiBgmaQ3g\nfkm7ps2FNdaXI+KY1ojbzFpfzvNmUWVLrE3rqxHxObD6EvodVPDy+oL264Dr0pc9s4/QzPIq7yPS\nYrwIi5nligT19U6sZmaZqvIBqxOrmeWPSwFmZlmqgulUxTixmlmuJPNYqzuzOrGaWc7kf55qMU6s\nZpY7dVW+VoATq5nli2usZmbZco3VzKwMqjyvOrGaWf54xGpmlrEqz6tOrGaWL5JnBZiZZczzWM3M\nMlfledWJ1czyxyNWM7MstYEbBFr7mVdmZi0SUFdXV3Qr6VjStZImS3q1oO0sSR9IGp1uAwr2/UrS\nOElvSNqnoL1f2jZO0tBi53ViNbPcyfAprdeTPB26qQsjone63ZucU1sAhwNfTd/zV0n1kuqBy4H+\nwBbAEWnfZrkUYGa5k1WNNSIel9SzxO4DgVsjYi7wjqRxQJ9037iIeDuN7da073+bO5BHrGaWLyWM\nVtO8203SyIJtyFKc5RRJL6elgi5pWw/g/YI+E9K25tqb5cRqZrmidB5rsQ2YGhHbF2xXlXiKK4CN\ngd7AROAvC0+9uGihvVkuBZhZ7pRzVkBETPryPLoaGJa+nACsV9B1XeDD9Pvm2pfII1Yzy536OhXd\nlpWktQteHgg0zhi4CzhcUgdJGwK9gOeA54FekjaU1J7kAtddLZ3DI1Yzy5WkhprNkFXSLcDuJPXY\nCcCZwO6SepP8OT8eOBEgIl6TdDvJRan5wMkR0ZAe5xRgOFAPXBsRr7V03mYTq6RVWnpjRHxa0icz\nM1tKWa3BEhFHLKH5mhb6nwucu4T2e4F7Sz1vSyPW11i8cNv4OoD1Sz2JmdnSaLO3tEbEes3tMzMr\npyrPq6VdvJJ0uKTT0u/XlbRdecMys1ol0ilXRf6XZ0UTq6TLgD2Ao9Omz4EryxmUmdUwFZ8RsDyz\nAlpDKbMCdo6IbSW9CBAR09MpB2ZmZVHtpYBSEus8SXWkdxpI6gosKGtUZlazBNRVeWYtpcZ6OXAH\nsIak3wJPAn8oa1RmVtMyXN2qIoqOWCPiRkkvAHulTYdExKstvcfMbHm02elWTdQD80jKAb4N1szK\nphpGpMWUMivg18AtwDokiw/8Q9Kvyh2YmdWueqnolmeljFi/A2wXEZ8DSDoXeAE4r5yBmVntqoVS\nwLtN+rUD3i5POGZW65JZAZWOYvm0tAjLhSQ11c+B1yQNT1/3JZkZYGaWvS8Xsq5aLY1YG6/8vwbc\nU9D+TPnCMTOr/otXLS3C0uzSWmZm5dSWR6wASNqYZH3CLYAVG9sjYtMyxmVmNUqQ+7UAiillTur1\nwHUkn7c/cDtwaxljMrMapxK2PCslsXaMiOEAEfFWRJxOstqVmVnmpGStgGJbnpUy3WqukoLHW5JO\nAj4Aupc3LDOrZTnPm0WVklh/CnQCfkRSa10V+G45gzKz2tbmL15FxLPptzP5crFrM7OyEPlfyLqY\nlm4QuJN0DdYliYiDyhKRmdW2NrAIS0sj1staLYoy6N2zK0/d6AF2NemywymVDsGW0tw3J5TluG22\nFBARD7VmIGZmjap9bdJS12M1M2sVog2PWM3MKqXKr12VnlgldYiIueUMxsxMqoFbWiX1kfQKMDZ9\nvbWkS8semZnVrDoV3/KslBrxJcB+wDSAiHgJ39JqZmXU5p/SCtRFxLtNiskNZYrHzGpc8gSBnGfO\nIkpJrO9L6gOEpHrgh8Cb5Q3LzGpZLUy3+j5JOWB9YBLwYNpmZlYWVT5gLWmtgMnA4a0Qi5kZUhte\nK6CRpKtZwpoBETGkLBGZWc2r8rxaUingwYLvVwQOBN4vTzhmVutq4uJVRNxW+FrSTcCIskVkZjWv\nyvPqMt3SuiGwQdaBmJkBUAU3ABRTSo31Y76ssdYB04Gh5QzKzGqbcv+4wJa1mFjTZ11tTfKcK4AF\nEdHs4tdmZstLQLsqn8jaYvhpEr0zIhrSzUnVzMpOUtGtxONcK2mypFcL2laXNELS2PRrl7Rdki6R\nNE7Sy5K2LXjPsWn/sZKOLXbeUn4vPFd4AjOzckpmBWS2CMv1QL8mbUOBhyKiF/AQX5Y2+wO90m0I\ncAUkiRg4E9gR6AOc2ZiMm9NsYpXUWCb4BklyfUPSKEkvShpV8scyM1saJSzAUuqsgYh4nOS6UKGB\nwA3p9zcAgwrab4zEM8BqktYG9gFGRMT0iPiYZFZU02S9iJZqrM8B2xac1MysVZQ4j7WbpJEFr6+K\niKtKeN+aETERICImSuqetvdg0Tn6E9K25tqb1VJiVXrit0oI1MwsE42lgBJMjYjtMz51U9FCe7Na\nSqxrSPpZczsj4oKWDmxmtmxEfXnvEJgkae10tLo2MDltnwCsV9BvXeDDtH33Ju2PtnSCli5e1QOd\ngM7NbGZmmUseJljWha7vAhqv7B8L/Lug/Zh0dsBOwIy0ZDAc6CupS3rRqm/a1qyWRqwTI+Ls5Qrf\nzGxpZXjnlaRbSEab3SRNILm6fz5wu6QTgPeAQ9Lu9wIDgHHA58DxABExXdLvgOfTfmdHRNMLYoso\nWmM1M2ttWS3CEhFHNLNrzyX0DeDkZo5zLXBtqedtKbEudmIzs3JrLAVUs2YTa7GhrplZubT5ha7N\nzFqTqI1nXpmZtR5R8loAeeXEama5U91p1YnVzHKmJh7NYmbW2qo7rTqxmlnuiDrPCjAzy45nBZiZ\nlYFnBZiZZay606oTq5nljeexmpllyzVWM7My8DxWM7OMVXledWI1s3xJSgHVnVmdWM0sdzxiNTPL\nlJBHrGZm2fKI1cwsQxLlfvx12TmxmlnuVHledWI1s/xxjdXKarNNetK5U2fq6+tp164dTz07cuG+\nCy/4M6f98lTenziFbt26VTDK2rLumqvx998dw5pdV2FBBNfe8RSX3/Iov/nBvuy321YsiGDK9JkM\nOfNmJk6Zwa7b9eL/LhzC+A+nAfDvh0dz3lX3N3ucWpcsdF3pKJaPE2sVuP/BRxZLnO+//z4PPziC\n9dZfv0JR1a75DQsYesE/GT1mAp06duA///glDz07hgtveIiz/3oPAD84Yjd+NaQ/Pzr3VgCeevEt\nvv3jK0s6zpi3P2r1z5Q31T5irfZbcmvWL37+U849749Vv1hFNfpo6qeMHjMBgM8+n8uYdz5inTVW\nY+asOQv7dFypAxGxTMexpMZabMszj1hzThL79++LJE4YfCInDB7CsLvvYp11erDV1ltXOryat/7a\nq9N7s3V5/tXxAJx18v4ctV8fZnw2m35DLlnYb8etNuTZ24YyccoMfnXBnbzeZFTa9Di1THhWQLMk\nNQCvpOd4Bzg6Ij6R1BN4HXijoPsFEXFj+r5tgFFAv4gYXnC8zyKiU7nizauHH3uKddZZh8mTJ7Nf\nv73ZbPPN+cN55zLsvgcqHVrNW3ml9tzy5+9x6p/vWDhaPevyuznr8rv5+Xf7ctJh3+ScK+9l9Jj3\n2WzAGcya/QX7fGMLbr9wCF8beHaLx6lt1X+DQDlLAbMjondEbAlMB04u2PdWuq9xu7Fg3xHAk+nX\nmrfOOusA0L17dw4YdCBPPP4Y745/hz7bbc1mm/TkgwkT+HqfbfnoI9flWlO7dnXc8ufB3HbfSP79\n8EuL7b/9vucZtGdvAGbOmsOs2V8AMPzJ/7JCu3q6rrZyScepSSWUAfI+oG2tGuvTQI9inZQUDA8G\njgP6SlqxzHHl2qxZs5g5c+bC7x8c8QDbbb8D7304mTfGjeeNcePpse66PP3cKNZaa60KR1tbrjzz\nKN545yMuufnhhW0br7/Gwu/33W0r3hw/CYA1u3Ze2L79VzegTmLaJ7OaPY4l5YBiW56VvcYqqR7Y\nE7imoHljSaMLXv8wIp4AdgHeiYi3JD0KDAD+uRTnGgIMAdrE1fLJkyZx2MEHAjC/YT6HHX4kfffp\nV+GobOfeG3HUfjvyypsf8MytQwE487K7OG7QzvTaoDsLFgTvTZy+cEbAgXttw+BDdmV+QwNz5szj\nmF9d1+Jxhj/538p8sJxIplvlPXW2TMWuXC7zgb+ssfYEXgD6RkRDWmMdlpYImr7ncmB0RFwt6QCS\nuuwh6b6lqrFut932UTjn0/Kvyw6nVDoEW0pz37idBZ9PzjQLfuVr28R1/3qkaL+vb9LlhYjYPstz\nZ6XsNVZgA6A9i9ZYF5OObL8N/EbSeOBSoL+kzi29z8zaHpXwvzwre401ImYAPwJ+LmmFFrruBbwU\nEetFRM+I2AC4AxhU7hjNLF988aoEEfEi8BJweNq0saTRBduPSGYB3NnkrXcAR6bfd5Q0oWD7WWvE\nbmatzxevmtG0HhoR+xe8XKnEY9wF3JV+77vEzGpF3jNnEb7zysxyJRmRVndmdWI1s3yRV7cyM8te\nlSdW1y3NLGdKmWxVWuaVNF7SK+lF8pFp2+qSRkgam37tkrZL0iWSxkl6WdK2y/oJnFjNLHcynm61\nR7omSePNBEOBhyKiF/BQ+hqgP9Ar3YYAVyxr/E6sZpYrpUy1Ws5KwUDghvT7G/hyrvxA4MZIPAOs\nJmntZTmBE6uZ5U9pmbWbpJEF25AlHCmAByS9ULB/zYiYCJB+7Z629wDeL3jvBEpYPGpJfPHKzHKn\nxEVYppawVsAuEfGhpO7ACEljWui7pJMu02IqHrGaWe5kVQqIiA/Tr5NJ7uzsA0xq/BM//To57T4B\nWK/g7esCHy5L/E6sZpYvGRVZJa3cuIiTpJWBvsCrJHdzHpt2Oxb4d/r9XcAx6eyAnYAZjSWDpeVS\ngJnlTkZ3Xq0J3Jk+cLMd8I+IuF/S88Dtkk4A3gMOSfvfS7IG9Djgc+D4ZT2xE6uZ5YrIZvWqiHgb\nWOyJmxExjWTx/abtQZHlTUvlxGpmuVPlN145sZpZ/ijvC64W4cRqZrlT5XnVidXM8qfK86oTq5nl\nUJVnVidWM8sVL3RtZpY1L3RtZlYGTqxmZlkqfSHrvHJiNbPc8XQrM7MMZbCQdcU5sZpZ/lR5ZnVi\nNbPcKXGh69xyYjWz3KnutOrEamZ5s/RPYc0dJ1Yzy6HqzqxOrGaWK1ktdF1JTqxmljtVnledWM0s\nfzwrwMwsa9WdV51YzSx/qjyvOrGaWb7I063MzLLn1a3MzLJW3XnVidXM8sdPEDAzy5QXujYzy1Rb\nuPOqrtIBmJm1NR6xmlnuVPuI1YnVzHLHNVYzswxJnhVgZpY9J1Yzs2y5FGBmljFfvDIzy1iV51Un\nVjPLH1X5kNWJ1cxypS3ceaWIqHQMZSFpCvBupeMog27A1EoHYUulLf/MNoiINbI8oKT7Sf7Nipka\nEf2yPHdW2mxibaskjYyI7Ssdh5XOP7Pa47UCzMwy5sRqZpYxJ9bqc1WlA7Cl5p9ZjXGN1cwsYx6x\nmpllzInVzCxjTqxVTlLXSsdgZotyYq1ikvoCF0nqomq/B7BG+OdUG5xYq1SaVP8EXBMRH+Pbk6tF\nVwBJ/v9eG+YfbhWS1I8kqZ4YEY9KWg84TVIptwFaBSjRHXhX0gERscDJte3yD7Y67Qh0jIhnJK0B\n3AlMjoi2ej961YvEZOB44DpJAxqTq6T6Ssdn2fKfj1VE0i7AbhHxW0kbSXqa5Jfj3yLi6oJ+60XE\n+xUL1JoVEbdL+gK4VdIREXFP48hV0v5JlxhW2ShteXnEWgUK/mTsC6wKEBHHAo8DXZok1aOASyR1\nbvVAbTGS+kk6Q9LXG9si4l8kI9dbJe2XjlxPBK4ExlQqVsuOR6zVYVXgY2AOsPDPxoj4paQ1JD0S\nEXtI+jbwU+CYiJhZoVhtUbsBJwH9JL0GXAa8ExF3pDMErpc0DOgDDIiIcRWM1TLiEWvOSdoQOE/S\nRsAkoHPavhJARHwXeFvSROA0kqT630rFa4u5C3gQ+DbwOXA4cJOkjSLi/wGHAgcAR0bES5UL07Lk\nEWv+rQhMBk4E1gAmpO0dJM1JL4qcIOnnwL1OqpUnaXNgbkS8ExFPS+oA/CQifiLpSGAo0EnSBOBi\nYK2I+KKSMVu2vAhLFZC0JdAPOAVYn2QUtA3wITAPmAkMioh5FQvSAJA0ADgDOLrxz3pJvYDBwBsk\nf1V8j+RntzPwaES8U6FwrUw8Ys0hSbuT/Gwej4gvIuJVSfOAjsBXgOuBV4CVgVVIplo5qVaYpH1I\nkupZETFOUicgSB7LsgFwMtA/Ih5P+78ZHtm0SR6x5oykVYF7gA2Bi4CGiLgg3bcxcBiwNnBTRDxX\nsUBtEZK+BrwE7BURD6c/q78BP4uIlyVtRfIL8eCIeLuCoVor8MWrnImIGcAw4AtgLDBA0vWSBpHU\nWi8nmSFwqKQVfe95ZRX8+48nuVHjUEk9SRa3Hp4m1bqIeBl4AtjDNwS0fU6sOSFprYL/k/4FuA+Y\nGRF7Ae2BC0jmre6Wfv19RMzxn5IV1x4gnd52FNAJeAv4V0T8KU2qCyT1JikJ3B8RDZUL11qDE2sO\nSNqX5IJUt/RmAJGMTrdJp1ntRDKh/CLgIODFiJheqXgtkS6Ec6uksyQdFBFzSGZv/AP4OkCaVE8A\nLgGujogPKhextRbXWCssXVDl18C5EXG/pPYR8UW6sMoLJCOgQxtvc5TUMSI+r2DIxsKf22+BG4Hu\nwDrAHyNibHrX219JLlw9QHKKMmwIAAAEcklEQVSDwEkR8Wql4rXW5cRaQZJWJ/nz8KCI+Fd6weM3\nwKkRMVnSEGCriDilMeFWNGADFvm5DYyIuyWtC5wLXBERz6R92gO3kdyGvIPnF9cWlwIqKP1zfn/g\nN+lV46tI/syfnHZ5CdhT0qZOqvlR8HM7X9IqETGB5OaN8yVdJOl/SKbCnQBs4qRaezyPtcLS1Y0a\ngNHAaRFxkaT6iGiIiGcl/aPSMdri0p/bAuAFSfeTXMS6HFid5AaAr5BMtXItvAa5FJATkvYGLgV2\njIgZkjpExNxKx2Utk7QXSR117YiYlLbVAat7fdza5VJATkTECJKVqZ6TtLqTanWIiAeBfYGHJa2Z\nti1wUq1tLgXkSETcl170eFDS9qQLz1c6LmtZwc/tPknbR8SCSsdkleVSQA5J6hQRn1U6Dls6/rlZ\nIydWM7OMucZqZpYxJ1Yzs4w5sZqZZcyJ1cwsY06sNUpSg6TRkl6V9H+SOi7HsXZPnzSKpAMkDW2h\n72qSfrAM5zgrfa5XSe1N+lwv6eClOFdPSV4wxZaZE2vtmh0RvSNiS5JFtU8q3KnEUv/3ERF3RcT5\nLXRZDVjqxGpWTZxYDZKV7TdJR2qvS/orMApYT1JfSU9LGpWObDtBsmyepDGSniRZI5a0/ThJl6Xf\nrynpTkkvpdvOwPnAxulo+U9pv1MlPS/pZUm/LTjWryW9IelBYLNiH0LS4PQ4L0m6o8kofC9JT0h6\nU9J+af96SX8qOPeJy/sPaQZOrDVPUjugP8nDCSFJYDdGxDbALOB0kuc4bQuMBH4maUXgapIVnnYF\n1mrm8JcAj0XE1sC2wGskj35+Kx0tn5ouFt0L6AP0BraT9E1J2wGHkzyN9iBghxI+zj8jYof0fK+T\nrC7VqCfJ0xf2Ba5MP8MJwIyI2CE9/mBJG5ZwHrMW+ZbW2rWSpNHp908A15As1vxu45qiJE8u2AJ4\nKn1qTHvgaWBz4J2IGAsg6WZgyBLO8S3gGID0cSQzJHVp0qdvur2Yvu5Ekmg7A3c2Luot6a4SPtOW\nks4hKTd0AoYX7Ls9vdV0rKS308/QF9iqoP66anruN0s4l1mznFhr1+yI6F3YkCbPWYVNwIiIOKJJ\nv94kq+NnQcB5EfG3Juf4yTKc43pgUES8JOk4YPeCfU2PFem5fxgRhQkYJQ8DNFtmLgVYS54BdpG0\nCSSPhZG0KTAG2DB94gHAEc28/yHg++l76yWtAswkGY02Gg58t6B220NSd5IHJh4oaaX0USf7lxBv\nZ2CipBVIHuxX6BBJdWnMGwFvpOf+ftofSZtKWrmE85i1yCNWa1ZETElHfrdI6pA2nx4Rbyp5bMw9\nkqYCTwJbLuEQPwauUvIwvQbg+xHxtKSn0ulM96V11q8AT6cj5s+A70TEKEm3kSwA/i5JuaKYM4Bn\n0/6vsGgCfwN4DFiT5PlTcyT9naT2OkrJyacAg0r71zFrnhdhMTPLmEsBZmYZc2I1M8uYE6uZWcac\nWM3MMubEamaWMSdWM7OMObGamWXs/wP8oyQamEghrgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x10f1a75f8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(bin_tfidf_train, bin_y_train)\n",
"pred = clf.predict(bin_tfidf_train)\n",
"score = metrics.accuracy_score(bin_y_train, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(bin_y_train, pred, labels=['FAKE', 'REAL'])\n",
"plot_confusion_matrix(cm, classes=['FAKE', 'REAL'])"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.872\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVAAAAEmCAYAAAA0k8gFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XecXFX9//HXe9MoAUJIgyQQSij+\nQAIJCKhIEwgCQaUjNRJBkC+CBRHbV1DUr3SkiRBQKV9pMSDFACJ8aQmEJiUJBFgSSEIJJZSQfH5/\n3LNh2OzOzE5md+7svp8+7mNnzj1z7md25ZNz7rn3XEUEZmbWdg21DsDMrF45gZqZVcgJ1MysQk6g\nZmYVcgI1M6uQE6iZWYWcQK1VkpaX9HdJ8yX97zK0c5Ck26sZW61I+qKkZ2sdh+WDfB1o/ZN0IHAC\nsCHwDjAVOC0i7l3Gdg8GvgNsExEfL3OgOScpgOERMb3WsVh9cA+0zkk6ATgL+BUwEFgT+AMwpgrN\nrwU81xWSZzkkda91DJYzEeGtTjdgFeBdYJ8idXqRJdhZaTsL6JX2bQc0AicCc4DZwOFp3y+Aj4CF\n6RhjgZ8Dfy5oexgQQPf0/jDgebJe8AvAQQXl9xZ8bhvgYWB++rlNwb67gV8C96V2bgf6tfLdmuL/\nQUH8ewG7Ac8BbwAnF9TfErgfeCvVPQ/omfbdk77Le+n77lfQ/g+BV4Erm8rSZ9ZNx9g8vV8DmAds\nV+v/b3jrmM090Pq2NbAccEOROj8GtgJGAJuSJZFTCvYPIkvEg8mS5PmSVo2In5H1aq+JiN4RcWmx\nQCStCJwDjI6IlciS5NQW6vUFbk51VwPOAG6WtFpBtQOBw4EBQE/ge0UOPYjsdzAY+ClwCfANYCTw\nReCnktZJdRcB3wX6kf3udgS+DRAR26Y6m6bve01B+33JeuPjCg8cETPIkutfJK0AXAZcHhF3F4nX\nOhEn0Pq2GjAvig+xDwL+OyLmRMRcsp7lwQX7F6b9CyPiFrLe1wYVxrMY2FjS8hExOyKeaqHOV4Bp\nEXFlRHwcEVcBzwB7FNS5LCKei4j3gWvJkn9rFpKd710IXE2WHM+OiHfS8Z8CPgsQEVMi4oF03JnA\nRcCXyvhOP4uID1M8nxIRlwDTgAeB1cn+wbIuwgm0vr0O9Ctxbm4N4MWC9y+msiVtNEvAC4DebQ0k\nIt4jG/YeBcyWdLOkDcuIpymmwQXvX21DPK9HxKL0uinBvVaw//2mz0taX9JESa9Kepush92vSNsA\ncyPigxJ1LgE2Bs6NiA9L1LVOxAm0vt0PfEB23q81s8iGn03WTGWVeA9YoeD9oMKdEXFbRHyZrCf2\nDFliKRVPU0yvVBhTW1xAFtfwiFgZOBlQic8UvUxFUm+y88qXAj9Ppyisi3ACrWMRMZ/svN/5kvaS\ntIKkHpJGS/ptqnYVcIqk/pL6pfp/rvCQU4FtJa0paRXgR007JA2UtGc6F/oh2amARS20cQuwvqQD\nJXWXtB/wGWBihTG1xUrA28C7qXd8dLP9rwHrLPWp4s4GpkTEN8nO7V64zFFa3XACrXMRcQbZNaCn\nAHOBl4FjgRtTlVOBycDjwBPAI6mskmPdAVyT2prCp5NeA9ls/iyymekvkSZomrXxOrB7qvs62Qz6\n7hExr5KY2uh7ZBNU75D1jq9ptv/nwHhJb0nat1RjksYAu5KdtoDs77C5pIOqFrHlmi+kNzOrkHug\nZmYVcgI1M6uQE6iZWYWcQM3MKtRpF0dQzxVDy/uSvHqy0bD+tQ7B2mjWyy/x5hvzSl1L2ybdVl4r\n4uOlbvpaSrw/97aI2LWax26rzptAl+9Lr61PqHUY1gbXXnpkrUOwNtp3t21LV2qj+Ph9em1Q8ioy\nPph6fqm7yNpdp02gZlavBKqPs4tOoGaWLwIautU6irLUR5o3s65FKr2VbEIbSJpasL0t6XhJfSXd\nIWla+rlqqi9J50iaLulxSZuXOoYTqJnlTBrCl9pKiIhnI2JERIwgWx92AdnauScBkyJiODApvQcY\nDQxP2ziyxWeKcgI1s/ypQg+0mR2BGRHxItnjbsan8vF8sprZGOCKyDwA9JG0erFGfQ7UzPJFlDuJ\n1E/S5IL3F0fExa3U3Z9sZTKAgRExGyAiZksakMoHky3G06Qxlc1uLQAnUDPLmbJ7mPMiYlTJ1qSe\nwJ4ULL/Y+oGXUnS1JSdQM8uf6s7CjwYeiYimJxW8Jmn11PtcneyBhJD1OIcWfG4IJRYf9zlQM8uZ\n6kwiFTiAT4bvABOAQ9PrQ4GbCsoPSbPxWwHzm4b6rXEP1MzyRVQySdRyU9nTUr8MfKug+HTgWklj\ngZeAfVL5LWSPxJ5ONmN/eKn2nUDNLH+qdCdSRCwge3ptYdnrZLPyzesGcExb2ncCNbOc8a2cZmaV\na6jqAk/txgnUzPKlju6FdwI1s5zxEN7MrHJVmoVvb06gZpY/7oGamVWgssVCasIJ1Mzyxz1QM7NK\nyLPwZmYV8xDezKwC5a8HWnNOoGaWM74O1Mysch7Cm5lVyJNIZmYVkIfwZmaV8xDezKwycgI1M2u7\n7IkeTqBmZm0nWn7AcA45gZpZzoiGBk8imZlVxEN4M7MKOYGamVWijs6B1seJBjPrMoSQSm9ltSX1\nkfQ3Sc9IelrS1pL6SrpD0rT0c9VUV5LOkTRd0uOSNi/VvhOomeVOtRIocDZwa0RsCGwKPA2cBEyK\niOHApPQeYDQwPG3jgAtKNe4Eama509DQUHIrRdLKwLbApQAR8VFEvAWMAcanauOBvdLrMcAVkXkA\n6CNp9aJxVvb1zMzaicrcSlsHmAtcJulRSX+UtCIwMCJmA6SfA1L9wcDLBZ9vTGWtcgI1s9wpcwjf\nT9Lkgm1cs2a6A5sDF0TEZsB7fDJcb/GwLZRFsTg9C29mudI0iVSGeRExqsj+RqAxIh5M7/9GlkBf\nk7R6RMxOQ/Q5BfWHFnx+CDCrWADugZpZ7lRjEikiXgVelrRBKtoR+A8wATg0lR0K3JReTwAOSbPx\nWwHzm4b6rXEP1Mzyp3rXgX4H+IuknsDzwOFkHcdrJY0FXgL2SXVvAXYDpgMLUt2inEDNLF9E1e6F\nj4ipQEvD/B1bqBvAMW1p3wnUzHLHt3KamVWgDZNINecEamb5Ux/50wk0b1ZZsRcXHP9lPjNsNSKC\no868g51GrsURu27C3PkLAPjZ5fdx28Mz6dG9gfOO24nNhw9kcQTfu/Bu/v14Y42/QddzyolHc88/\nb6Vvv/7cOOmhJeV/+dOFXHX5RXTr3p1td9iFE085FYBLzvsfrr/qSrp1a+BH//07Pr/dTrUKPZ/k\nIbxV6H+O2o7bp8zkwNMm0qN7Ayv06sFOI9fi3Bse4azrpnyq7hGjNwFgi6OvpP8qy3PjqV/lC8f9\nlSh66a9V2177HMSBh32Lk4//5Druh+67h7tuv5nr73iAnr168fq8uQDMeO4Z/nHTddx050PMeW02\n3zxgT26+51G6dauPx/h2lHpJoL4ONEdWWqEnX9hkMJff+iQACz9ezPz3Pmy1/oZr9uWuqS8BMHf+\n+8x/90NGDh/YIbHaJ0Zt9QVW6bPqp8quufKPjD3mBHr26gXAav36A3Dn7RMZPebr9OzViyFrDmPN\nYevwxNTJHR5z3qlBJbc8cALNkbUHrcK8+e9z8Yk7c/95B/GH43dihV7ZIOGoPTfloQu+wYXf/TJ9\nemf/UT7x/Dz22HpdujWItQauzGbDBzCk/0q1/AqWzHx+OlMe/D8O2H17Dvv6rjwxNRs9zJk9m0Gr\nD1lSb+CgNZgzu+i12l1SFVdjalftlkAlLZI0tWAbVrDvbEmvSGooKDtM0nnpdYOk8ZL+lO4KmCnp\niYK2zmmvuGupe7cGRqw3gEsmPs7Wx/6FBR98zPf224JLJj7OZw6/jM99+8+8+sZ7nH7ktgCMv+1J\nXpn7LvedeyC/O2o7HvjPbD5etLjG38IAFi36mLfnv8Vf/34nJ55yKt87+lAigmjh1uq8JIO8KCd5\n5uV31p7nQN+PiBHNC1PS/CrZqifbAnc32y/gQqAHcHhERPplbR8R89ox3pp7Zd47vDLvHR5+9lUA\nbvj3NE7cbxRz3lqwpM6fbn2S638xBoBFi4MfXPyvJfvuOmM/ps96q2ODthYNHDSYnUbviSQ22WwU\namjgzTfmMXD1NXh19icTfa+9Oov+gwbVMNJ8ykuCLKUWQ/jtgSfJFis9oIX9ZwOrAYdERJfqTr32\n5gIa577L8CHZ+bTtNhvKMy+9waC+Ky6pM2abdfnPzNcBWL5X9yVD/B02W5OPFy3mmZfe6PjAbSk7\n7Lo7D92X/eM28/lpLPzoI1bt24/tv/wV/nHTdXz04Yc0vjSTl16YwSYjiq2H0TW5BwrLS5qaXr8Q\nEV9Nrw8AriK7gf9XknpExMK070CyFaO3i4iPm7V3l6RF6fX4iDiz+QHTclbZVOhyqzbfXRdO+MNd\nXPaD0fTs0cDM2fMZd8bt/P7o7fnsOv0Jghdfe5vvnDMJgP59VuDvp32VxYuDWa+/x9jf3Vrj6Lum\n7x9zOA/f/2/eeuN1dhy1Ad8+8WS+tt/BnHLit9lrxy3p0aMnvzrrIiSx3gYbscseX2PPHbage7du\n/PjU33sGvgV5mSQqRdFO17xIejciejcr6wnMBDaIiHckXQ9cGhE3SzoM+AawIbBfRNxX8LmZwKi2\nDOEbVhkavbY+Ydm/iHWYyZceWesQrI323W1bnnrskapmu16DhseQg0pPczx/xm5TSixn1+46egi/\nK7AK8ERKil/g08P4Z4B9gWsk/b8Ojs3MckCAVHrLg45OoAcA34yIYRExDFgb2FnSCk0VIuL/gKOA\nmyWt2cHxmVnNeRZ+KSlJ7gJ8q6ksIt6TdC+wR2HdiJgoqT9wq6QvpuLCc6CPR8QhHRG3mXW8nOTH\nktotgTY//xkRC4C+LdT7WsHbywvKLwMuS2+HVT9CM8urvPQwS/G98GaWKxJ06+YEamZWkTrpgDqB\nmln+eAhvZlaJHF2mVIoTqJnlSnYdaH1kUCdQM8uZ/FznWYoTqJnlTkOd3AvvBGpm+VJH50C9Ir2Z\n5UrTOdBq3MrZbDH2yamsr6Q7JE1LP1dN5ZJ0jqTpkh6XtHmp9p1AzSx3qryYyPYRMaJg5aaTgEkR\nMRyYlN4DjAaGp20c2ZrFRTmBmlnutPNiImOA8en1eGCvgvIrIvMA0EfS6sUacgI1s9wpswfaT9Lk\ngm1cC00FcLukKQX7B0bEbID0c0AqH0z2qKEmjamsVZ5EMrNckcqehZ9XxoLKn4+IWZIGAHdIeqbY\noVsoK7rivHugZpYz1VsPNCJmpZ9zgBuALYHXmobm6eecVL0RGFrw8SHArGLtO4GaWe5UYxJJ0oqS\nVmp6DexM9kDLCcChqdqhZM9nI5UfkmbjtwLmNw31W+MhvJnlTpXuRBoI3JDa6g78NSJulfQwcK2k\nscBLwD6p/i3AbsB0YAFweKkDOIGaWb5U6UL6iHge2LSF8teBHVsoD+CYthzDCdTMckVAQ0N9nF10\nAjWz3KmXWzmdQM0sd7wak5lZJepoMREnUDPLFXk9UDOzytVJ/nQCNbP86eYFlc3M2i6706jOE6ik\nlYt9MCLern44ZmZQJx3Qoj3Qp8hWIin8Kk3vA1izHeMysy6s7nugETG0tX1mZu2pTvJneasxSdpf\n0snp9RBJI9s3LDPrqkS6lKnE//KgZAKVdB6wPXBwKloAXNieQZlZFybRraH0lgflzMJvExGbS3oU\nICLekNSzneMysy6sXobw5STQhZIaSEvbS1oNWNyuUZlZlyWgoU4yaDnnQM8HrgP6S/oFcC/wm3aN\nysy6tCo/1rjdlOyBRsQVkqYAO6WifSLiyfYNy8y6srq/jKmZbsBCsmF8fax0amZ1KU89zFLKmYX/\nMXAVsAbZU+r+KulH7R2YmXVd3aSSWx6U0wP9BjAyIhYASDoNmAL8uj0DM7OuqzMN4V9sVq878Hz7\nhGNmXV02C1/rKMpTbDGRM8nOeS4AnpJ0W3q/M9lMvJlZ9alzLKjcNNP+FHBzQfkD7ReOmVn9TCIV\nW0zk0o4MxMysSTV7oJK6AZOBVyJid0lrA1cDfYFHgIMj4iNJvYArgJHA68B+ETGzWNvlzMKvK+lq\nSY9Leq5pW8bvZGbWIkG174X/L+Dpgve/Ac6MiOHAm8DYVD4WeDMi1gPOpIwbhsq5pvNy4DKy7zUa\nuJYse5uZtQuVsZXVjjQE+Arwx/RewA7A31KV8cBe6fWY9J60f0eV6AqXk0BXiIjbACJiRkScQrY6\nk5lZ1UnZvfClNqCfpMkF27gWmjsL+AGfrN+xGvBWRHyc3jcCg9PrwcDLAGn//FS/VeVcxvRhysIz\nJB0FvAIMKONzZmYVKfMU6LyIGNV6G9odmBMRUyRt11TcQtUoY1+Lykmg3wV6A8cBpwGrAEeU8Tkz\ns4pUaRLp88CeknYDlgNWJuuR9pHUPfUyhwCzUv1GYCjQKKk7Wa57o9gBSg7hI+LBiHgnIl6KiIMj\nYs+IuK/y72Rm1jpRnQWVI+JHETEkIoYB+wN3RsRBwF3A3qnaocBN6fWE9J60/86IqKwHKukGinRf\nI+JrJb+BmVlbtf9iIj8ErpZ0KvAo0HTJ5qXAlZKmk/U89y/VULEh/HnLGmUtbbbeQO6b+N1ah2Ft\nsOoWx9Y6BGujD6c1tku71b4TKSLuBu5Or58HtmyhzgfAPm1pt9iF9JPaFKGZWZXUy5qZ5a4HambW\nIUTnWo3JzKxD1f1qTM1J6hURH7ZnMGZmErl5bHEp5dwLv6WkJ4Bp6f2mks5t98jMrMtqUOktD8o5\nV3sOsDvZ6iRExGP4Vk4za0ed5qmcQENEvNjspO6idorHzLq4enoufDkJ9GVJWwKR1tX7DuDl7Mys\n3XSmy5iOJhvGrwm8BvwzlZmZtYs66YCWTqARMYcybmkyM6sGqc0LJtdMyQQq6RJauCc+Ilpae8/M\nbJnVSf4sawj/z4LXywFfJS06amZWbZ1qEikiril8L+lK4I52i8jMurw6yZ8V3cq5NrBWtQMxMwMg\nRxfKl1LOOdA3+eQcaAPZOnkntWdQZta1qezHxtVW0QSanoW0KdlzkAAWl1qh2cxsWQjoXicXghYN\nMyXLGyJiUdqcPM2s3UkqueVBOXn+IUmbt3skZmY0zcLXx2IixZ6J1PTUui8AR0qaAbxH9v0iIpxU\nzaz6crRYSCnFzoE+BGwO7NVBsZiZAZ3jOlABRMSMDorFzGzJEL4eFEug/SWd0NrOiDijHeIxsy5P\ndOsEPdBuQG+okwuyzKxTyB4qV+soylMsgc6OiP/usEjMzKBqdyJJWg64B+hFluv+FhE/k7Q2cDXQ\nF3gEODgiPpLUC7gCGEn2BI79ImJmsWMUu4ypTv4NMLPOpkEquZXhQ2CHiNgUGAHsKmkr4DfAmREx\nHHgTGJvqjwXejIj1gDNTveJxFtm3YzkRmplVU9MQflmfiRSZd9PbHmkLYAfgb6l8PJ9caTQmvSft\n31ElrthvNYFGxBulQzQzq75uDSq5Af0kTS7YllqjWFI3SVOBOWSryM0A3krXuAM0AoPT68GkpTrT\n/vnAasXirGQ1JjOzdiPKfibSvIgYVaxCRCwCRkjqA9wAbNRStYJDt7avRXVyy76ZdRmq/r3wEfEW\ncDewFdBHUlPncQgwK71uBIZCdicmsArZ6nOtcgI1s9xRGVvJNqT+qeeJpOWBnYCngbuAvVO1Q4Gb\n0usJ6T1p/52lFlDyEN7McqWKj/RYHRifHsfeAFwbERMl/Qe4WtKpwKPApan+pcCVkqaT9TxLPkzT\nCdTMcqca6TMiHgc2a6H8eWDLFso/APZpyzGcQM0sZ0RDndwM7wRqZrnShln4mnMCNbPcycuK86U4\ngZpZ7tRH+nQCNbO8kXugZmYV8TlQM7Nl0Bke6WFmVhN1kj+dQM0sX7IhfH1kUCdQM8sd90DNzCoi\n5B6omVll3AM1M6uARKd4rLGZWU3USf50AjWz/PE5UKuKRYsW8fnPjWKNwYO5/qaJHHXkWB6ZMpmI\nYL311+eSSy+nd+/etQ6zSxu+1gCu/M0RS96vPXg1fnnBzawxoA+7bbsxHy1cxAuN8xj3sz8z/933\n2eFzG/LL4/akZ4/ufLTwY04+60b+9fBzNfwG+ZItqFzrKMpTL3dMdVnnnXM2G2z0yXOwfvv7M3no\nkcd4+NHHGTp0TS74w3k1jM4Apr04h632P52t9j+dbQ78DQs+WMiEux5j0gPPMHKfX7Hlfr9m2otz\n+P4ROwPw+lvvsvfxF7HFvr/iyJ9eyZ9OPaTG3yB/VMb/8sAJNMcaGxu59R83c/gR31xStvLKKwMQ\nEXzw/vt1s+hCV7H9lhvwQuNcXpr9JpMeeIZFixYD8NATLzB4YB8AHnu2kdlz5wPwnxmz6dWzBz17\neDBYqBrPhe8ITqA59v0Tj+e0X/+WhoZP/5nGjT2cYUMG8eyzz/DtY75To+isJfvsMpJrb52yVPkh\nY7bmtvv+s1T5V3cawWPPvsxHCz9eal9XJbJZ+FJbHrRbApW0SNJUSU9K+nvB0/GGSXo/7WvaDin4\n3GaSQtIuzdp7t71izaNbbp7IgP4D2HzkyKX2XXzpZTz/0iw23HAj/nbtNTWIzlrSo3s3vvKlTbj+\njkc/Vf6DsbuwaNFirr7l4U+Vb7TOIE49bgzHnnp1R4ZZB8oZwHfyBAq8HxEjImJjsifcHVOwb0ba\n17RdUbDvAODe9LPLuv//7mPixAlssN4wDjlof+6+604OP+QbS/Z369aNvffdjxtvuK6GUVqhXb7w\nGaY+8zJz3nhnSdlBe3yO3bbdmMN+fPmn6g4e0IdrzhjHN39yJS80zuvgSHOujOF7TjqgHTaEvx8Y\nXKqSshN6ewOHATtLWq6d48qtX572a2bMbOTZ6TO54i9Xs932O/Cn8VcyY/p0IDsHevPEv7P+BhvW\nOFJrsu+uoz41fP/yNhtx4mE7sffxF/H+BwuXlK/Se3muP/cofnruBO5/7PlahJp71XgufEdo9zPX\n6ZnMO/LJs5cB1pU0teD9dyLi38DngRciYoaku4HdgOvbcKxxwDiAoWuuuayh505E8M0jDuWdt98m\nCDbZZFPOOf+CWodlwPLL9WCHz23IsadetaTszB/uS6+e3Zl4wbEAPPTETI477WqO2n9b1h3an5OO\n3JWTjtwVgD2OPo+5b3aps1StquJz4dudIqJ9GpYWAU8Aw4ApwM4RsUjSMGBiGto3/8z5wNSIuETS\nnsDBEbFP2vduRJR9wePIkaPivgcnL/sXsQ6z6hbH1joEa6MPn72WxQvmVDXbbbTJZnHZjXeVrLf1\neqtOiYhRre2XNBS4AhgELAYujoizJfUFriHLTTOBfSPizTQCPpus47YAOCwiHikWQ7ufAwXWAnry\n6XOgS0k91a8DP5U0EzgXGC1ppXaM0cxyqEqTSB8DJ0bERsBWwDGSPgOcBEyKiOHApPQeYDQwPG3j\ngJLDu3Y/BxoR84HjgO9J6lGk6k7AYxExNCKGRcRawHXAXu0do5nlSzUmkSJidlMPMiLeAZ4mm4sZ\nA4xP1cbzSY4ZA1wRmQeAPpJWL3aMDplEiohHgceA/VPRus0uYzqObNb9hmYfvQ44ML1eQVJjwXZC\nR8RuZh2vzEmkfpImF2zjWm0vO3W4GfAgMDAiZkOWZIEBqdpg4OWCjzVSYvK73SaRmp+vjIg9Ct4u\nX2YbE4AJ6bUv+jfrKso7qzqv2DnQJU1Jvck6Y8dHxNtF7t5raUfRSSInJTPLlayHWZ0L6dNpw+uA\nv0RE0xU9rzUNzdPPOam8ERha8PEhwKxi7TuBmlm+KFuNqdRWspmsq3kp8HREnFGwawJwaHp9KHBT\nQfkhymwFzG8a6rfGKxiYWf5U58KozwMHA08UXHd+MnA6cK2kscBLwD5p3y1klzBNJ7uM6fBSB3AC\nNbOcqc697hFxL62n4h1bqB+UuNyyOSdQM8udOrkRyQnUzPIlT/e6l+IEamb5UycZ1AnUzHKnXhYT\ncQI1s9ypj/TpBGpmeVNHJ0GdQM0sd/LyyI5SnEDNLFeEL2MyM6tYneRPJ1Azy58iKyblihOomeVO\nneRPJ1Azy586yZ9OoGaWQ3WSQZ1AzSxXmhZUrgdOoGaWL2UumJwHTqBmlj9OoGZmlajOgsodwQnU\nzHLHlzGZmVWgjtYScQI1sxyqkwzqBGpmueMFlc3MKlQf6dMJ1MzyRvUzidRQ6wDMzJamMrYyWpH+\nJGmOpCcLyvpKukPStPRz1VQuSedImi7pcUmbl2rfCdTMcqVpQeVSW5kuB3ZtVnYSMCkihgOT0nuA\n0cDwtI0DLijVuBOomeVOdfqfEBH3AG80Kx4DjE+vxwN7FZRfEZkHgD6SVi/Wvs+BmlnulDkL30/S\n5IL3F0fExWV8bmBEzAaIiNmSBqTywcDLBfUaU9ns1hpyAjWz/CmvizkvIka181Gj2Ac8hDez3KnW\nEL4VrzUNzdPPOam8ERhaUG8IMKtYQ06gZpYr5UwgLeNlThOAQ9PrQ4GbCsoPSbPxWwHzm4b6rfEQ\n3sxyp1qrMUm6CtiO7HxpI/Az4HTgWkljgZeAfVL1W4DdgOnAAuDwUu07gZpZ/lTpQvqIOKCVXTu2\nUDeAY9rSvhOomeWOV6Q3M6uIF1Q2M6tI051I9cCz8GZmFXIP1Mxyp156oE6gZpY7PgdqZlYB+bnw\nZmbLwAnUzKwyHsKbmVXIk0hmZhWqk/zpBGpm+aM66YI6gZpZrtTTnUjKFiDpfCTNBV6sdRztoB8w\nr9ZBWJt05r/ZWhHRv5oNSrqV7HdWyryIaP7AuA7VaRNoZyVpcpUfY2DtzH+zzsv3wpuZVcgJ1Mys\nQk6g9aecx7Zavvhv1kn5HKiZWYXcAzUzq5ATqJlZhZxA65yk1Wodg1lX5QRaxyTtDJwlaVXVy71v\nXZz/Tp2LE2idSsnzd8ClEfEmvi23XqwGIMn/7XUC/iPWIUm7kiXPb0XE3ZKGAidLKuf2N6sBZQYA\nL0raMyIWO4nWP/8B69PngBUi4gFJ/YEbgDkR0Vnvt657kZkDHA5cJmm3piQqqVut47PKeNhXRyR9\nHvhSRPxC0jqS7if7R/CiiLh22fjKAAAHZUlEQVSkoN7QiHi5ZoFaqyLiWkkfAVdLOiAibm7qiUra\nI6sSE2sbpZXLPdA6UDDU2xlYBSAiDgXuAVZtljwPAs6RtFKHB2pLkbSrpJ9I2rqpLCJuJOuJXi1p\n99QT/RZwIfBMrWK1tnMPtD6sArwJfAAsGe5FxA8l9Zd0V0RsL+nrwHeBQyLinRrFap/2JeAoYFdJ\nTwHnAS9ExHVpRv5ySROBLYHdImJ6DWO1NnIPNOckrQ38WtI6wGvASql8eYCIOAJ4XtJs4GSy5Pmf\nWsVrS5kA/BP4OrAA2B+4UtI6EfE3YF9gT+DAiHisdmFaJdwDzb/lgDnAt4D+QGMq7yXpgzQ5MVbS\n94BbnDxrT9KGwIcR8UJE3C+pF3B8RBwv6UDgJKC3pEbgbGBQRHxUy5itMl5MpA5I2hjYFTgWWJOs\nV7MZMAtYCLwD7BURC2sWpAEgaTfgJ8DBTcNxScOBI4FnyUYJ3yT7220D3B0RL9QoXFtG7oHmkKTt\nyP4290TERxHxpKSFwArARsDlwBPAisDKZJcwOXnWmKRdyJLnzyNiuqTeQJA9zmMt4BhgdETck+o/\nF+7B1DX3QHNG0irAzcDawFnAoog4I+1bF9gPWB24MiIeqlmg9imSNgEeA3aKiDvT3+oi4ISIeFzS\nZ8n+4ds7Ip6vYahWRZ5EypmImA9MBD4CpgG7Sbpc0l5k50LPJ5uR31fScr63urYKfv8zyW5o2FfS\nMLJFlG9LybMhIh4H/g1s7wvnOw8n0JyQNKjgP8bfA/8A3omInYCewBlk131+Kf38VUR84CFgzfUE\nSJeNHQT0BmYAN0bE71LyXCxpBNlQ/taIWFS7cK2anEBzQNJXyCaG+qWL5kXW29wsXb60FdmF12cB\nXwMejYg3ahWvZdKCLldL+rmkr0XEB2RXS/wV2BogJc+xwDnAJRHxSu0itmrzOdAaSwuD/Bg4LSJu\nldQzIj5KC4RMIevR7Nt0e5+kFSJiQQ1DNpb83X4BXAEMANYAfhsR09JdYH8gm0C6nexC+qMi4sla\nxWvtwwm0hiT1JRvWfS0ibkwTDz8Fvh8RcySNAz4bEcc2JdaaBmzAp/5uYyLi75KGAKcBF0TEA6lO\nT+Aasttvt/D1uZ2Th/A1lIbhewA/TbO0F5MNz+ekKo8BO0pa38kzPwr+bqdLWjkiGslucjhd0lmS\nTiS7xGwssJ6TZ+fl60BrLK3GswiYCpwcEWdJ6hYRiyLiQUl/rXWMtrT0d1sMTJF0K9lk0vlAX7IL\n5Tciu4TJ56o7MQ/hc0LSl4Fzgc9FxHxJvSLiw1rHZcVJ2onsPOfqEfFaKmsA+np91s7PQ/iciIg7\nyFZSekhSXyfP+hAR/wS+AtwpaWAqW+zk2TV4CJ8jEfGPNPnwT0mjSAuZ1zouK67g7/YPSaMiYnGt\nY7KO4SF8DknqHRHv1joOaxv/3boeJ1Azswr5HKiZWYWcQM3MKuQEamZWISdQM7MKOYF2UZIWSZoq\n6UlJ/ytphWVoa7v0ZEkk7SnppCJ1+0j6dgXH+Hl67lNZ5c3qXC5p7zYca5gkL/xhJTmBdl3vR8SI\niNiYbPHmowp3KtPm/39ExISIOL1IlT5AmxOoWR45gRpkK6Wvl3peT0v6A/AIMFTSzpLul/RI6qn2\nhmw5N0nPSLqXbI1SUvlhks5LrwdKukHSY2nbBjgdWDf1fn+X6n1f0sOSHpf0i4K2fizpWUn/BDYo\n9SUkHZnaeUzSdc161TtJ+rek5yTtnup3k/S7gmN/a1l/kda1OIF2cZK6A6PJHlIHWaK6IiI2A94D\nTiF7zs/mwGTgBEnLAZeQrUj0RWBQK82fA/wrIjYFNgeeInuk74zU+/1+WpR4OLAlMAIYKWlbSSPJ\nnqG+GVmC3qKMr3N9RGyRjvc02WpITYaRreb/FeDC9B3GAvMjYovU/pGS1i7jOGaAb+XsypaXNDW9\n/jdwKdmiwC82rWlJthL+Z4D70tNGegL3AxsCL0TENABJfwbGtXCMHYBDANJjLOZLWrVZnZ3T9mh6\n35ssoa4E3NC0eLSkCWV8p40lnUp2mqA3cFvBvmvTLZbTJD2fvsPOwGcLzo+uko79XBnHMnMC7cLe\nj4gRhQUpSb5XWATcEREHNKs3gmy19WoQ8OuIuKjZMY6v4BiXA3tFxGOSDgO2K9jXvK1Ix/5ORBQm\nWpQ9FM6sJA/hrZgHgM9LWg+yx4lIWh94Blg7raAPcEArn58EHJ0+203SysA7ZL3LJrcBRxScWx0s\naQDZg/O+Kmn59IiMPcqIdyVgtqQeZA94K7SPpIYU8zrAs+nYR6f6SFpf0oplHMcMcA/UioiIuakn\nd5WkXqn4lIh4TtnjRm6WNA+4F9i4hSb+C7hY2UPVFgFHR8T9ku5Llwn9I50H3Qi4P/WA3wW+ERGP\nSLqGbKHpF8lOM5TyE+DBVP8JPp2onwX+BQwkez7RB5L+SHZu9BFlB58L7FXeb8fMi4mYmVXMQ3gz\nswo5gZqZVcgJ1MysQk6gZmYVcgI1M6uQE6iZWYWcQM3MKvT/AXvf3EySXiFQAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11acc6e80>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(bin_tfidf_train, bin_y_train)\n",
"pred = clf.predict(bin_tfidf_test)\n",
"score = metrics.accuracy_score(bin_y_test, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(bin_y_test, pred, labels=['FAKE', 'REAL'])\n",
"plot_confusion_matrix(cm, classes=['FAKE', 'REAL'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuration 2\n",
"* model b - train - [performance measures]\n",
"* model b - validation - [performance measures]\n",
"* model b - test - [performance measures]"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.602\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAV4AAAEmCAYAAAAqWvi2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXd8VMXXh5+TBKL0DqG3UKTX0KU3\n4QcISFMsKIJd6UVA6UhRaSqKCiLiKyq99y4JXToCEnqHUJNw3j/uTdyEJBtCdpPVefzMJ/eeO3fm\nu7t4dvbM3DmiqhgMBoPBfXgltQCDwWD4r2Ecr8FgMLgZ43gNBoPBzRjHazAYDG7GOF6DwWBwM8bx\nGgwGg5sxjtfgVkTkSRFZICLXReT/HqOdTiKyPDG1JRUiUlNEDiW1DoP7ELOO1xATItIR+AAoBtwE\ndgHDVXXjY7b7AvA2UE1Vwx5baDJHRBTwV9WjSa3FkHwwI17DQ4jIB8CnwAggO5AXmAK0SITm8wGH\n/wtONz6IiE9SazAkAapqiimRBUgPhABt46jji+WYz9jlU8DXvlYbCAZ6ABeAs8DL9rWPgPtAqN1H\nF2AI8IND2/kBBXzs85eAv7BG3ceBTg72jQ73VQO2A9ftv9Ucrq0FhgKb7HaWA1lieW0R+ns76G8J\nNAUOA1eA/g71KwNbgGt23UlASvvaevu13LJfbzuH9vsA54CZETb7nkJ2H+Xt85zAJaB2Uv/bMCXx\nihnxGqJTFXgC+C2OOgOAKkBZoAyW8xnocD0HlgPPheVcJ4tIRlUdjDWKnqOqaVT1m7iEiEhq4HOg\niaqmxXKuu2KolwlYZNfNDIwHFolIZodqHYGXgWxASqBnHF3nwHoPcgGDgGnA80AFoCYwSEQK2nXD\ngfeBLFjvXT3gDQBVrWXXKWO/3jkO7WfCGv13dexYVY9hOeVZIpIK+Bb4TlXXxqHX4GEYx2uITmbg\nksYdCugEfKyqF1T1ItZI9gWH66H29VBVXYw12iuaQD0PgJIi8qSqnlXVP2Oo8wxwRFVnqmqYqs4G\nDgLNHep8q6qHVfUO8DPWl0ZshGLFs0OBn7Cc6meqetPu/0+gNICqBqnqVrvfE8CXwNPxeE2DVfWe\nrScKqjoNOAJsA/ywvugM/yKM4zVE5zKQxUnsMSdw0uH8pG2LbCOa474NpHlUIap6C+vneTfgrIgs\nEpFi8dAToSmXw/m5R9BzWVXD7eMIx3je4fqdiPtFpIiILBSRcyJyA2tEnyWOtgEuqupdJ3WmASWB\niap6z0ldg4dhHK8hOluAu1hxzdg4g/UzOYK8ti0h3AJSOZzncLyoqstUtQHWyO8glkNypidC0+kE\nanoUpmLp8lfVdEB/QJzcE+dSIhFJgxU3/wYYYodSDP8ijOM1REFVr2PFNSeLSEsRSSUiKUSkiYiM\nsavNBgaKSFYRyWLX/yGBXe4CaolIXhFJD/SLuCAi2UXkf3as9x5WyCI8hjYWA0VEpKOI+IhIO+Ap\nYGECNT0KaYEbQIg9Gu8e7fp5oOBDd8XNZ0CQqr6KFbv+4rFVGpIVxvEaHkJVx2Ot4R0IXAROAW8B\nv9tVhgGBwB5gL7DDtiWkrxXAHLutIKI6Sy+s1RFnsGb6n8aeuIrWxmWgmV33MtaKhGaqeikhmh6R\nnlgTdzexRuNzol0fAnwvItdE5DlnjYlIC6AxVngFrM+hvIh0SjTFhiTHPEBhMBgMbsaMeA0Gg8HN\nGMdrMBgMbsY4XoPBYHAzxvEaDAaDmzEbdLgI7yfTq0/6bEktI0ZK5Eqf1BJi5V7og6SWECc+3s6W\n6CYd3l7JU9vfJ09w6dKlRBPnnS6fathDD/w9hN65uExVGydWv4mJcbwuwid9NnJ3+iypZcTIxpFN\nklpCrBw9H5LUEuIkc5qUSS0hVjIlU23Vq1RK1PY07A6+RZ2uzOPursnOniBMMozjNRgMnoUIeHkn\ntYrHwjheg8HgeYhnT08Zx2swGDwPSZ7x7PhiHK/BYPAwTKjBYDAY3ItgQg0Gg8HgXsTjQw2e/bVh\nMBj+m4iX8+KsCZHpInJBRPY52OaIyC67nBCRXbY9v4jccbj2hcM9FURkr4gcFZHPRZx/K5gRr8Fg\n8DASLcb7HVZy0hkRBlVtF9mLyDis5KkRHFPVmFJGTcXKnbcVa2/oxsCSuDo2I16DweBZCFaowVlx\ngqqux9rn+eEurFHrc1ib/scuRcQPSKeqW9TaY3cGcWdvAYzjNRgMnkj8Qg1ZRCTQoXR11qwDNYHz\nqnrEwVZARHaKyDoRqWnbcgHBDnWCiZrrL0ZMqMFgMHgYAt7xCjVcUtWKCeykA1FHu2eBvKp6WUQq\nAL+LSAlizq/nNLuEGfEmEaPaleKPIfVY0rNmpO39xv4s6lGDBR/U4LuulciWzheANE/48NUrFVjY\nowZLetWkdaXcABTPmZb/e7sqS3rVZFGPGjxT1s+lmu/evUut6gEEVCxLxbIlGfbx4CjXe7z3Ntky\npXWpBkfu3b1Lx+a1aduoGq3qVWbKuOEAqCoTx3xM86fL0bJuRWZNnxp5z/YtG3iucXVa1avMK21d\nt2dFj7e6UqZIHupVKx9p+3Pvbpo3qEXDWpVpWrcaO4O2A3Dt2lW6vPAc9WtU5Jn6NTi4P6YM9q4h\n+NQpGjeoS7lST1GhTEkmT4y6v8in48eSKqUXly65I4tSPIlYTvaYk2uxNm9l2H4WhzROqnrPTjGF\nqgYBx4AiWCPc3A635yYeiV/NiDeJmLs9mJkbTzK2Q5lI27Q1x5mw1Ppl82KNfLzdwJ8P5+7jher5\nOHo+hK7Tg8iUOiUr+tZi/o7T3Ln/gF6zd3Pi0m2ypfNl3vvVWX/wIjfvhsXW7WPh6+vL4mWrSJMm\nDaGhodSvU5OGjZpQOaAKO4ICuXb9uvNGEpGUvr58/dNCUqW29LzUuiE16jTgr6OHOXcmmHlrgvDy\n8uLypYsA3Lh+jREDPmDKzF/xy5Un0u4K2nZ8gZde68573btE2oYP7s/7vQdQt0EjVq1YyvAh/fll\nwQomjh9DiZKl+Wbmzxw9fIgBvd9lzu9LXabNEW8fH0aOGUu5cuW5efMm1QMqUrdeA4o/9RTBp06x\netVK8uTN6xYtj4Rrl5PVBw6qamQIQUSyAldUNVxECgL+wF+qekVEbopIFWAb0BmY6KwDM+JNIrb/\ndZVrt0Oj2ELu/eMwn0zpg9q/WFQhta/1HZnK15vrt0MJe6CcuHSLE5duA3Dhxj0uh9x36e5ZIkKa\nNGkACA0NJTQ0FBEhPDycAf16M2zEaJf1HZueVKktPWFhoYSFhYEIP8/8mtff64OXl/XPO3OWrAAs\nmfd/1GvSHL9ceaLYXUGVajXJkDHjQ3pDbt4A4OaN62TPYf1COXLoADWergNA4SJFCf77JBcvnHeZ\nNkf8/PwoV84aladNm5aixYpz5sxpAHr3/IBhI0YTj9VRbsZe1eCsOGtFZDawBSgqIsEiEvEt2Z6H\nJ9VqAXtEZDfwC9BNVSMm5roDXwNHsUbCca5oADPiTXb0aFKEVhVzcfNOGJ2mbgNg5qYTfPVKRbYM\nrktqXx/embmT6DlKS+dJTwpvL05evu1SfeHh4VSvUpG/jh2la7c3qFQ5gMkTP6PpM83x83NtqCM2\nPR2eqcXfJ/6iXefXKF2uEsEnj7Nswa+sXrqQjJkz0+ejMeQrUJiTfx0lLCyULs815VZICJ1e6Ubz\nNh3dpnXIiLF0atOMoYP68kCVeUvXAPBUyVIsWTCPylWqszNoO8Gn/ubsmdNkzZbdbdoATp44we7d\nO6lUOYCFC+aTM1dOSpcp4/zGpCARnlxT1Q6x2F+KwTYXmBtL/UCg5KP0/a8f8YrIOyJyQERmxXK9\ntogsjOlaUjBuyWFqDF3DvB1neKFGPgBqFs3K/tM3qPrRapqP28iQViVI4/vPd2bWtL6M61iGPj/t\necghJzbe3t5s3b6Tw3+dIihwOxs3rOe3X3+h+5tvu7bjOPT8vHQTy7cdYN/uII4c2s/9+/dJ6fsE\nsxet49kOLzG455sAhIWHsX/vLiZ+939M/eE3vvp8DCf+OuKkh8RjxrdfMXj4J2zfd4whw8bQ8x0r\ng/ub7/bi+rWrNKxVmW+nTaFk6bL4+Lh3TBQSEkKHdm0YM3YCPj4+jBk1gg8Hf+xWDfEmPkvJkt0o\nPSr/escLvAE0VdVOSS3kUZi/8zSNS+UAoE2l3Czbew6Ak5dvE3zlNgWzpQYgja8PX79akfFLDrPr\n72tu05chQwZq1nqa9evWcOzYUUo95U/xIgW4ffs2pYr7u01HBOnSZ6BSlRpsXruS7H45qd/kfwDU\na9ycIwetyarsOXJR/en6pEqVmoyZMlM+oDqH9++Lq9lE5ZfZP9C0ubXEs1nL1uwKCgQgbbp0jJ88\njeXr/+CzqdO5fOkiefLmd5uu0NBQOrZrQ/sOHWnZ6ln+OnaMkyeOE1CxLMX8C3A6OJhqARU4d+6c\n2zQ5JRFCDUnJv9rx2o/1FQTmi0gfEdlsr8PbLCJFY6j/tMMjgTtFJK1t7yUi20Vkj4h85Cq9+bOk\nijyuXyI7xy5Y2RjOXLtDNX9rM/3MaVJSIFsaTl25TQpvYerL5fkt8DRL9rj+f4qLFy9y7Zrl3O/c\nucOa1asoV64Cx/8+y4HDxzlw+DipUqVi7wH3jCKvXL7EjeuWnrt377B141ryF/KnTsNm/LF5PQCB\nWzeSr0AhAOo0fIYdf2whLCyMO3dus3dnIAX8H/pn4DKy5/BjyyZL16b1ayhQqDAA169f4/79+wD8\nOGM6AdVqkDZdOrdoUlW6d32VosWK8c57HwBQslQpTp4+z8Ejxzl45Di5cudm87YgcuTI4RZNzhGX\nrmpwB//qGK+qdhORxkAd4D4wTlXDRKQ+MAJoHe2WnsCbqrpJRNIAd0WkIdYMZmWshSzzRaSW/dRL\nFOwF2l0BfNLGPXHz6fNlCSiUiYypU7Lxwzp8tuwItYtno2DW1DxQ5fTVO3z4izUam7TiKGPal2Zx\nz5oIMGbhQa7eCqVF+ZxUKpiJDKlSRi4x6/3Tbg6cuZnAdyxuzp07S9cuLxEeHs6DBw9o3aYtTZ5p\n5pK+4sOlC+cY+EE3Hth6GjZrxdP1m1CuUlX6v/sqP3w9mVSpUzN4zCQACvoXpXrt+rRtWBXx8uLZ\n9p3xL/qUS7S9+eoLbNm0gSuXL1GxRCF69B3ImM+mMLhfT8LCwvD1fYLREyYDcPTQQd59owve3t74\nFy3O2M+/cNJ64rFl8yZ+nDWTkiVLEVCxHAAfDR1O4yZN3aYhQSTzUIIzRF0dFExiROQEUBF4Evgc\ny4kqkEJVi4lIbaCnqjYTkb5AK2AW8KuqBovIWKANEPE7Pg0wUlW/iatf3xz+mlxzru01OdcSjMm5\n9uhUr1KJHUGBieYpvTLkVd8avZ3Wu7vo7aDHeIDCpfyrR7zRGAqsUdVWIpIfWBu9gqqOEpFFQFNg\nqz0yFixH+6UbtRoMhljx/I3Qk3cgJHFJD5y2j1+KqYKIFFLVvao6GggEigHLgFfs0AMikktEkmfe\ndoPhv4KJ8XoMY4DvReQDYHUsdd4TkTpAOLAfWKKq90SkOLDFXkgeAjwPXHCDZoPBEBMeHuP91zte\nVc1vH17CerY6gg/t62uxww6qGuNiVFX9DEieAVuD4b+GSe9uMBgM7if5Pcb8aBjHazAYPAprH3Tj\neA0Gg8F9iCBexvEaDAaDWzEjXoPBYHAzxvEaDAaDOxFMqMFgMBjciSBmxGswGAzuxjheg8FgcDOe\n7niT9wPNBoPBEB07xuusOG1GZLqIXBCRfQ62ISJy2mFf7qYO1/qJyFEROSQijRzsjW3bUXuHQ6cY\nx2swGDwOEXFa4sF3QOMY7BNUtaxdFtv9PYWVBLOEfc8UEfEWEW9gMtAEeAroYNeNExNqMBgMHkVi\nTa6p6np7i9j40AL4SVXvAcdF5ChWcgSAo6r6F4CI/GTX3R9XY2bEazAYPI54hhqyiEigQ+kaz+bf\nstN8TReRjLYtF3DKoU6wbYvNHidmxOsiivmlY/HAekktI0bm7D7lvFISUSh9mqSWECdpn0yR1BJi\nJblOOCW6Kon3a72UgAwUU7GSJqj9dxzwCjG/DCXmwavTtD7G8RoMBo/DVV8yqnreoY9pwEL7NBjI\n41A1N3DGPo7NHism1GAwGDwKQfDy8nJaEtS2iJ/DaSsgYsXDfKC9iPiKSAGs3I1/ANsBfxEpICIp\nsSbg5jvrx4x4DQaD55EIA14RmQ3UxooFBwODgdoiUhYrXHACeB1AVf8UkZ+xJs3CsLKRh9vtvIWV\nIswbmK6qfzrr2zheg8HgWcQ/xhsnqtohBnOs2cNVdTgwPAb7YmDxo/RtHK/BYPA4EhpKSC4Yx2sw\nGDyP5LmAI94Yx2swGDyO5Lp0Lr4Yx2swGDyKR3gkONliHK/BYPA4TIzXYDAY3I1nD3iN4zUYDJ6H\nCTUYDAaDGxEBLw/PuebZgZJ/CT3e6kqZInmoV618pO3Pvbtp3qAWDWtVpmndauwM2g7AtWtX6fLC\nc9SvUZFn6tfg4H6nD8k8FmdPHmNwpyaR5Y06JVg++xtCrl9j7Fud6Nv6aca+1YlbN64DoKrMGjuY\nvs/WYlDHRpw8uNel+gDCw8Pp0rI2fV+31sPv2LKeV1vV4aVm1RnR5w3CwsIAuHn9GgPefIGXm9fk\n9Tb1+evwAZdpOnP6FB1bNqJBtbI0qlGeb7+cFOX6tMkTKJj1Sa5cvhTFvntnIIWzp2bx/F9dps2R\nU6dO0ah+HcqWKk75MiWY9PlnAPTr04syJYtRqVxpnmvTimvXrrlFT/xwvhdvch8RG8ebDGjb8QV+\n+L+oj3cPH9yf93sPYPn6P+jRbxDDh/QHYOL4MZQoWZqVGwP5bMo3DO7fw6Xa/PIV4qNZS/ho1hIG\nz1hISt8nKV+7EYu/n0LxStUZNXcdxStVZ/H3UwDYu3kN508dZ+TcdbzYbyQzRg90qT6AX2Z8Sb5C\nRQB48OABI/q+yeDx0/hu4Say58zDst9+AuCHLybgX7wU3y7YQP/RU5g4vJ/LNPl4+9D/o1Gs2LyL\nuUvXMXP6lxw5ZDn6M6dPsXHtanLmzhPlnvDwcMZ8PJCadRq4TNdDOn18GDVmHLv2HmDdxq18+cVk\nDuzfT736DQjatY/tO/fg71+ET0aPdJum+CDivCRnjONNBlSpVpMMGTNGsYkIITdvAHDzxnWy57D2\n7jhy6AA1nq4DQOEiRQn++yQXL5zHHezfvolsufOSxS83O9evoPozrQGo/kxrdqxbDsDO9Suo1rQ1\nIkKhUuW5ffMG1y65Tt+Fc6fZunY5zdo8D8CNa1dImdKXPAUKA1Cxem3WLV8AwIljhyhfpRYA+QoV\n4dzpU1y5dMElurLl8KNkmXIApEmTlsJFinHurLVp1bCBvek7ePhDo7Lvp02hUbOWZMmS1SWaYsLP\nz49y5a1fWmnTpqVYseKcOXOa+g0a4uNjRSIrB1ThdHCw2zQ5xQ41OCvJGeN4kylDRoxl2OB+VCpZ\niKGD+tFv0FAAnipZiiUL5gGwM2g7waf+5uyZ027R9MeK+QQ0/B8AN65cIkOW7ABkyJKdm1etn8xX\nL5wjU/ackfdkypaDqy78Ypg0YgDdeg1B7OVF6TNmJiwslIN7dwKwbul8Lpyz3p9CxUqwfoW1y9+B\nPUGcP3OKi+ec7uD32AT/fZI/9+6ibIVKrFy6kBx+OSlesnSUOufOnmb54vl0euk1l+uJjZMnTrBr\n104qVQ6IYp/x3XQaNW6SRKoeRjCO1ykikt8xmVwitvuSiExyXjOyfgYReSOxdbiKGd9+xeDhn7B9\n3zGGDBtDz3e6AfDmu724fu0qDWtV5ttpUyhZumzkyMSVhIXeZ9f6lVSs94yTmg/vAe2qeNvmNcvI\nkCkLRUuWjdLXoPFfM2nkQF5vU59UqdPg7W29P526vsvNG9fo0uJp5s6cRuHipfB28Xt3KySEN17u\nwIfDPsHH24fJE0bzXt9BD9UbOqAXfQYNw9vb26V6YiMkJIQOz7Xmk3Gfki5dukj76JHD8fbxoX3H\nTkmiKzY8PdSQrFc1iIiPqoYlUnMZgDeAKTH04x2xxVty4ZfZP/DxyHEANGvZml7vdgcgbbp0jJ88\nDbAmsqqWLUqevPldrmfv5rXkK1aS9Jmtn8HpMmXh2qXzZMiSnWuXzpM2YxYAMmbz48r5f0aRVy6c\nI0PWbC7RtG/HNjavXsq29Su5f+8et0JuMqzn6wwc+yWTflwEwPaNazh14hgAqdOko99I67taVWlf\nrxx+ufO6RBtAaGgob7zcgf+1aUfjZi05uH8fwX+f5JnaVqquc2dO07xeVX5ftoG9u3fwTtfOAFy9\nfJm1q5bh4+NDw6b/c5k+R50dnmtNuw6daNnq2Uj7DzO+Z/GihSxZvip5TVaZVQ3xxkdEvrfzGP0i\nIqlEZJCIbBeRfSLyldifrIisFZERIrIOeFdEsorIXLvudhGp7tiwiKQVkeMiksI+TyciJyLOHRgF\nFLJTNn8iIrVFZI2I/AjsjT4yF5GeIjLEPi4kIktFJEhENohIMRe+VwBkz+HHlk3rAdi0fg0FClkx\ny+vXr3H//n0AfpwxnYBqNUjrMEJxFduWz6dyw3+cQLla9dm0aK6lb9FcytWyJoTK1qzP5sVzUVWO\n7d1BqjRpI0MSiU3XHoP4Zf0+5qzexaDx0yhfpSYDx37J1csXAbh//x4/TvuMFu1fAqxYeaj93i38\nv5mUrliV1Glc896pKn3f60ahIkV5tfu7ABR7qiTbD/zNhh2H2LDjEDly5mLBqi1kzZ6D9UEHI+1N\nmrfio9GfusXpqirdXutC0WLFeff9DyLty5ctZdzY0fzy23xSpUrlch2PgpBoWYaTDHeNeIsCXVR1\nk4hMxxp5TlLVjwFEZCbQDFhg18+gqk/b137ESre8UUTyYm04XDyiYVW9KSJrgWeA37F2gJ+rqqHR\nNPQFSqpqWbvd2lhZQkuq6nEn2Ua/Arqp6hERCcAaNdeNXslOptcVIFe0Geu4ePPVF9iyaQNXLl+i\nYolC9Og7kDGfTWFwv56EhYXh6/sEoydMBuDooYO8+0YXvL298S9anLGffxHvfhLKvbt3+HPbBjr3\nGxFpa9r5Dab2f4MN8+eQOXtOuo+cCkDp6nXZs3kNfZ+tRconnuSVD8e6XF90fvp6EpvXLkMfPKBF\nh1coX9WaUDt57DAj+ryBt5cX+QoXpc/wz12mIXDbZn77+UeKPlWSZ2pbMdOeAz6iToOYsoknHZs3\nbeLHWTMpWbIUARWskM1Hw0bQ4/13uHfvHs0aW1+olQOqMHGK6/+txY/k71idIapO87I9XgeWQ1uv\nqnnt87rAO8BMoDeQCsgETFTVUbYTHayq6+z6F4iawygrUAxoDVRU1bfsUXBvVW0hIluA11Q1SlzZ\n1rFQVUva57XtfurEcr0nkAYYC1wEDjk056uqxYmDMuUq6OLVm+P1Hrmb5UfdswoiIST3ZJd5MiWv\n0Z8jfhmeSGoJMVI9oCJBQYGJ5ilT5SyqRV+f6rTeriH1ghKQ7NItuGvEG927K9aosaKqnrJ/0jv+\nq7nlcOwFVFXVO44NOH7j2SPp/CLyNOCtqvtEJA//jKC/AJbGoMuxnzCihl4i9HgB1yJGygaDIYlJ\npMkz+9d3M+CCw4DrE6A5cB84BrysqtfsgdkB/hmAbVXVbvY9FYDvgCexMlG8q05GtO6K8eYVkar2\ncQdgo318SUTSAG3iuHc58FbEiZ0PKSZmALOBbwFU9ZSqlrXLF8BNIG0c/ZwHsolIZhHxxfpAUNUb\nwHERaWv3LyJSJo52DAaDC0nEGO93QPTYzwqs8GNp4DDg+JTNMQef0s3BPhUrxOhvF6fxJHc53gPA\niyKyByusMBWYBuzFistuj+Ped4CK9sTcfqBbLPVmARmxnO9DqOplYJM9mfdJDNdDgY+BbVgpnQ86\nXO4EdBGR3cCfQIs49BoMBheTGMvJVHU9cCWabbnDSqqtWOna49AhfkA6Vd1ij3JnAC2d9e3yUIOq\nngCeiuHSQLtEr1872vkloF0M9b7D+saKoAbwi6rG+lC5qnaMZlob7frnwEMzLqp6nHh8ixkMBvcQ\nz+VkWUQk0OH8K1X96hG6eQWY43BeQER2AjeAgaq6AcgFOD7WF2zb4iRZr+ONLyIyEWgCNE1qLQaD\nwcXEP8vwpYROronIAKx5n1m26SyQV1Uv2zHd30WkBDHvDOx0xcK/wvGq6ttJrcFgMLgHK8brwvZF\nXsSa46kXMUmmqveAe/ZxkIgcA4pgjXAdwxG5iboKK0bMXg0Gg8HDcL5PQ0KfbBORxkAf4H+qetvB\nnlVEvO3jgliTaH+p6lngpohUsR8C6wzMc9bPv2LEazAY/lskxgMUIjIbqI0VCw4GBmOtYvAFVth9\nRCwbqwV8LCJhQDjWA1URE3Pd+Wc52RK7xIlxvAaDwbNIpHW8qtohBvM3sdSdC8yN5VogUPJR+jaO\n12AweBTWtpCeHSU1jtdgMHgcHr5Vg3G8BoPB8/D0TXKM4zUYDB6FSPLPMOEM43gNBoPH4eED3tgd\nr4jEuUO0vXmMwWAwuB0vD/e8cY14/8R69M3xFUacK+C6nCkGg8EQBx7ud2N3vKoa/xQKBoPB4CZE\nwPu/EOMVkfZAQVUdISK5geyqGuRaaZ6NlwipfZNnCL1DueT7YyVjpbecV0pCTq6fkNQSDHj+qgan\nq5DFSqFeB3jBNt3GyuhgMBgMScJ/Ib17NVUtb+9DiapeEZGULtZlMBgMMSKAd3L3rE6Ij+MNFREv\n7D0mRSQz8MClqgwGgyE2PCB9uzPi88DzZKzNIbKKyEdY+dJGu1SVwWAwxMG/PtSgqjNEJAiob5va\nRk+dbjAYDO5C+I+sagC8gVCscINnbwtkMBg8nn99qMHOPTQbyImV1uJHEekX910Gg8HgGuITZkju\nfjk+I97ngQoRaTBEZDgQBIx0pTCDwWCIjf/CqoaT0er5AH+5Ro7BYDA4518bahCRCSIyHuuBiT9F\n5GsRmQbsBa65S6DBYDA4IoCXOC9O2xGZLiIXRGSfgy2TiKwQkSP234y2XUTkcxE5KiJ7RKS8wz0v\n2vWP2BmKnRLXiDdCzJ/AIgefzwWPAAAgAElEQVT71vg0bDAYDC4h8dbxfgdMAmY42PoCq1R1lIj0\ntc/7AE2wMgv7AwHAVCBARDJhJcmsiLX4IEhE5qvq1bg6jmuTnBiTvhkMBkNSkxgboavqehHJH83c\nAivzMMD3wFosx9sCmKGqCmwVkQwi4mfXXRGRcVhEVgCNsRYkxK7fmTgRKSQiP9nD68MRJZ6vzZAA\nvpj8OVUrlqFqxdJMnfQZAFevXKFVs0ZUKF2MVs0ace1qnF+oLuH1V18hb85sVCj7T0LV3bt2Uat6\nFQIqlKV6QEW2//GHSzV8MbgTJ1eNJPD/+kfaShfJxbrve7D1p75snNWbiiXyAVCzgj/n1n/C1p/6\nsvWnvvTr2hiA3NkzsPSrd9g5dyBBvwzgzQ61E13nu2+8xlMFc1EroGykbdTQwdSuWp661SvyXIum\nnDt7BoBrV6/yUsc21K5anka1q3Fgv3uXycf0uQ77eAgF8+UioEJZAiqUZemSxW7VFBePEGrIIiKB\nDqVrPJrPrqpnAey/2Wx7LuCUQ71g2xabPU7isyb3O+BbrNfbBPgZ+Cke9xkSwP4/9/H9t9+wav0W\nNmzdwbIlizh29AgTxo2mVu26BO05SK3adZkwzv0PD77w4kvMW7g0im1Av94M+HAw24J28eGQjxnQ\nr7dLNcxcsJUWb06OYhv+XkuGf7WEKu1HMXTqQoa/1zLy2qadx6jSfhRV2o9i5FeW9rDwB/Qd/yvl\nWg/j6c5jeb1dLYoVzJGoOtt36sxPvy6MYnvz3R6s3bKD1ZsCadC4KeNGDwfgs3GjKVmqDGu37GDS\nV9MZ2KdHompxRkyfK8Db777PtqBdbAvaReMmTd2qyRlihxviKsAlVa3oUL56nC5jsEXfr9zRHifx\ncbypVHUZgKoeU9WBWLuVGVzA4UMHqVQ5gFSpUuHj40P1mrVYOP93lixaQIdOnQHo0KkzixfOd7u2\nGjVrkSlTpig2EeHGDSsZyfXr1/HLmdOlGjbtOMaV67ej2FQhXeonAEif5knOXrweZxvnLt1g18Fg\nAEJu3+Pg8XPkzJohUXVWrV6TDBkzRrGlTfdPUpfbt29FxikPHzxAzdp1AfAvUoxTJ09y4cL5RNUT\nFzF9rskZEWs5mbOSQM7bIQTsvxdsezDguEd5buBMHPY4iY/jvSfWv5BjItJNRJrzz/DbkMgUf6oE\nmzdt4Mrly9y+fZsVy5Zw+nQwFy6cJ4efHwA5/Py4ePGCk5bcwyfjPqV/314ULpCHfn168vEw9y/v\n7jX2F0a815IjS4Yy8v1WDJo4L/JaQOkCbJvTl98ndad4DKPavH6ZKFs0N9v3nXCL1hEff0i54gWZ\n+/Nseg8YDMBTpUqxaP7vAOwI3E7wqZOcPX3aLXri4ospk6hUrjSvv/oKV5MgtBUXLnyAYj4QsTLh\nRWCeg72zvbqhCnDdDkUsAxqKSEZ7BURD2xYn8XG87wNpgHeA6sBrwCuP8koeFRHJ77jEIx71h4hI\nT/u4mIjsEpGdIlIoWr3aIlItsfUmJkWLFefdD3rRqnlj2rRsSolSZfDx9k5qWbHy1ZdTGTN2AkeP\nn2LM2Al079rF7Rq6tq1J73G/4t/kQ3qPncvUwZ0A2HXwFEWbfkhAu1FM/WkdP0+IGuJL/WRKZo99\nlV5j53Lz1l23aO0/aCg7D/xF6+c6MP3LKQC8835vrl+7St3qFfnmy8mUKl0WH5+k/cxfe707+w8d\nY1vQLnL4+dG3l3vDH86IZ6jBWRuzgS1AUREJFpEuwCiggYgcARrY5wCLsZ5fOApMA94Aa5tcYCiw\n3S4fR0y0xYVTx6uq21T1pqr+raovqOr/VHWT01eVdLQE5qlqOVU9Fu1abSBGxysiySZdxAsvvsK6\nzdtZvHwtGTNmpGBhf7Jly865s2cBOHf2LFmzJo8fHbNmfk/LVs8C0LpNWwK3u3ZyLSY6NQvg91W7\nAJi7Ymfk5NrNW3e5dec+AMs27ieFjzeZM6QGwMfHi9ljX2POkkDmrd7tds3Ptm3Pwvm/AVYI4rOp\nX7N6UyCTvvqWy5cvkTdfAbdrciR79ux4e3vj5eXFK11eIzDQ/Z9rbAiCt5fz4gxV7aCqfqqaQlVz\nq+o3qnpZVeupqr/994pdV1X1TVUtpKqlVDXQoZ3pqlrYLt/G5zXE9QDFbyLya2wlPo0/Jt4iMk1E\n/hSR5SLypIi8JiLbRWS3iMwVkVTRNDcF3gNeFZE10a7lB7oB79sj4poi8p2IjLfrjnYcOdv37ItY\nbiIiz4vIH/a9X4qIy4YkFy9YYYRTp/5m4fzfadO2PY2bNmP2LGu54exZM2jyTHNXdf9I+OXMyYb1\n6wBYu2Y1hQv7u13D2YvXqVnB6rd25SIc/fsiANkzp42sU7FEPrxEuHztFmCtjjh0/Byf/7DabTr/\nOnok8njZ4oX4FykKwPVr17h/3/qC+OH76VSpViNKPDgpOGt/yQPM+/03nipRMo7abuZfvlfDJLep\niBl/oIOqviYiPwOtgV9VdRqAiAwDugATI25Q1cUi8gUQoqpjHRtT1RPRr9k/LYoA9VU1XESGxCRE\nRIoD7YDqqhoqIlOATkRdeI29XKUrQO48Cc9r1rlTW65euYKPTwo+Gf85GTJm5P0efXj5hfb8MONb\ncufOw3c/zElw+wnW9XwHNqxby6VLlyiUPzcfDvqIyVOn0euDdwkLC8P3iSeYNPVxJo6d8/3Il6hZ\nwZ8sGdJwdOlQhn6xmDeH/sgnvdrg4+PFvXthvDXMWkLZqn45Xmtbk7DwcO7eDaVzP2swUq1sQTo1\nC2Dv4dNs/akvAIMnzWfZxv2JpvP1l59n88b1XLl8ibLFCtCr/yBWLV/C0SOH8fLyIneevHzyqbU6\n4/Chg7z9+it4e3tRpFhxJkxy7XsYnZg+1/Xr1rJn9y5EhHz58zNxypdu1eSMRHqAIskQaz1w8sIe\nZa5QVX/7vA+QAtgADAMyYMWdl6lqN9thhqjqWMfjGNqNck1EvgPWqOr3sVzfBzSzS3/+meF8Epit\nqkNiew3lylfUNRu3JfQtcClPpEy+MWOT7DLhpHsyRVJLiJHqARUJCgpMNE+ZvXBJbTf2F6f1JrYq\nHqSqFROr38Qk2cQ1Y+Cew3E4lrP7DmipqrtF5CX+ecIkRkTkTazJQIDYFiLecjgOI2r45YmIpoDv\nVdVsh2kwJAM8fB90j9vUPC1wVkRSYP3UjxNVnayqZe1yBrhptxEbJ4DyAPYmGBEzHKuANiKSzb6W\nSUTyJfxlGAyGxyExNslJSuLteEXE15VC4smHwDZgBXAwAfcvAFpFTK7FcH0ukElEdgHdgcMAqrof\nGAgsF5E9dv9+CejfYDA8Jtbk2eMvJ0tKnIYaRKQy8A2QHsgrImWAV1X1bVeJUtUTQEmHc8d47dQY\n6g+J6TiGeoeB0g6mDdGu38FaAB3TvXMA989oGQyGh/D2tN/q0YiP/M+xJpcuA6jqbswjwwaDIYmw\nNskRpyU5E5/JNS9VPRlt6B7uIj0Gg8HgFA8f8MbL8Z6yww1qPzTwNnbs02AwGNyNSPyeTEvOxMfx\ndscKN+QFzgMrbZvBYDAkCck8kuAUp45XVS8A7d2gxWAwGOKFhw9447WqYRoxbOyrqvHZzd1gMBgS\nFYH/RKhhpcPxE0Aroqa6MBgMBvfhAQ9IOCM+oYYoa1dFZCbWAwQGg8GQJEiMGXc8h4Ts1VAAMI/L\nGgyGJEEAHw9fTxafGO9V/onxegFXsHLNGwwGQ5KQ3B8JdkacjtfOtVYGiEgA9UCT4z6SBoPhP0NE\nendPJs4Bu+1kf1PVcLsYp2swGJIWIVFS/4hIUXvDrIhyQ0TeszPRnHawN3W4p5+IHBWRQyLSKKEv\nIT4x3j9EpLyq7khoJwaDwZBYJNaIV1UPAWUB7KdyTwO/AS8DE6InUxCRp7CeaSgB5ARWikgRVX3k\nLRTiyrkW4ZRrYDnfQyKyw87ea5ywwWBIMlyQc60ecExVT8ZRpwXwk6reU9XjWBmHKydEf1wj3j+w\nNgVvmZCG/+soSviD5BmZSc4Ro8CFo5xXSkLK9lqQ1BJi5djEVkktIUYS/1+b4BW/5WRZRCTQ4fwr\nVY0toV17YLbD+Vsi0hkIBHqo6lUgF7DVoU6wbXtk4nK8AhBDinSDwWBIMkTivR/vpfjkXBORlMD/\ngIjUXlOBoVjfGUOBccArEKO3T9D3SlyON6uIfBDbRVUdn5AODQaD4XFJ5P12mwA7VPU8QMRfiNwy\nYaF9GgzkcbgvN3AmIR3G9b3hjZXJN20sxWAwGNyOkOgx3g44hBlExDGtVytgn308H2gvIr4iUgDw\nxwrJPjJxjXjPqurHCWnUYDAYXElibZIjIqmABsDrDuYxIlIWK4xwIuKaqv4pIj8D+7Eykr+ZkBUN\nEI8Yr8FgMCQnhMTLQKGqt4HM0WwvxFF/ODD8cfuNy/HWe9zGDQaDIdGRf/Ejw6p6xZ1CDAaDIT4I\n4P1vdbwGg8GQXPFst2scr8Fg8EA8fMBrHK/BYPAsBDGhBoPBYHA3nj655uH7uP87eKf7qxQrkJMa\nlctG2gYP6EOV8iWpVaUcnTu04fq1a5HXPh07mkplihFQrgSrVy53m87gU6do3KAu5Uo9RYUyJZk8\n8TMAPhr8IZXLlyGgYjmaN23EmTMJepjnkbl39y7tn6nNsw2q0qJuJSaNtVb5fNjjDZ5tUJVW9avw\nftfnuX0rJMp9yxf+Tsncadm3O3H3ehr/Qnn2jGnK6g8fXhDUrYE/Z754lkypUwJQtUgWDk5ozooB\ndVkxoC7vNy0WWfe1eoVZM6g+qz+sx5QulfB1YbqF2D7TFzq2J6BiOQIqlqOYfwECKpZzmYaEIPEo\nyRnjeJMB7Tu9yJzfFkax1a5bn41/7GL91p0UKuzPp+NGA3Do4H5+mzuHjX/s5uffFtL7g7cJD0/Q\nGu5HxtvHh5FjxrJz737WbtzCl1OncGD/ft7v0Ys/duxmW+BOmjR9hpHD3fPcTUpfX6b/vJBfV2zh\nl2Wb2bR2JbuD/qDPkFH8umILv63cil+u3Pz47ZeR99wKucmsb6dSupzTR/gfmTlbTtJp4uaH7Dkz\nPkmtYtkIvnw7in3bkUs0GL6aBsNXM2HxQQByZHiCLnUK0WTkauoOXYWXl9CiUu5E1xpBbJ/pzB9/\nYlvgTrYF7qRlq2dp0TIZbcBjLydzVpIzxvEmA6rVqEnGjJmi2OrUa4CPjxUJqlgpgDNnggFYsnAB\nrVq3w9fXl3z5C1CgYCF2BCboqcVHxs/Pj3LlygOQNm1aihYrzpkzp0mXLl1knVu3brntH72IkCp1\nGgDCwkIJCwtFREiT1tKjqty9ezeKnomfDOPl7u+R0veJRNez7ehlrt6+/5B9SNvSDPt1HxrP/VR8\nvIQnUnjj7SU8mcKb89fuJrbUSGL7TCNQVeb+8n88166DyzQ8KhHLyZyV5IxxvB7ArJnfUa9BYwDO\nnj1Nztz/jIBy5szF2bPu+WnvyMkTJ9i9eyeVKgcAMPjDAfgXzMuc2T/y4WD3PWkeHh5O64bVqFWm\nIFVr1qF0+UoADPygG0+XK8Txo4fp+Eo3AA7s2825M8HUrt/Ebfoalvbj3LU77D99/aFrFQpmYsXA\nuvzwVjWK+Fnbn5y7dpepK4+wfUQTdo1uys27oaw7cMEtWqN/pgCbNm4gW7bsFPb3d4uG+GJCDW5C\nRPKLSEeH89oisjCue2Joo3/iK3Mt4z8ZiY+PD23bWS89pr103f2zKiQkhA7t2jBm7ITI0e5HQ4dz\n5K+/adehI19MmeQ2Ld7e3sxdvplV2w+yd1cQRw7uB2DY+C9YE3SEgv5FWTp/Lg8ePGD0kL70GjTC\nbdqeTOHNO02K8sn8/Q9d2/v3NSoPWEqDYauZvvYY07tXBSB9qhQ0Ku1HwMCllOuzmFQpfXi2cp6H\n7k9sYvpMAX6eM5vn2rV3ef+Pigs2QncrHuN4gfxAR2eVnBCj4xWLZPde/DRrBsuXLOKLb2ZEOtec\nOXNzJjg4ss6ZM6fJkcMvtiYSndDQUDq2a0P7Dh1p2erZh663a9+Reb/96jY9EaRLn4FKVWuyce2K\nSJu3tzeNm7dmxeJ53Aq5ydFD+3m5bVMaVinBnp3befuVdok+weZIvqypyZs5FSs/rMe24Y3wy/Ak\nywbUJWs6X0LuhnH7nhWbX73vPCm8hUypU1KzWDZOXb7NlZD7hD1QFu88Q8VCmZ309HjE9pmGhYUx\n//ffaN22nUv7f1RMqCEO7BHqQRH5WkT2icgsEakvIptE5IiIVBaRTCLyu4jsEZGtIlLavvdph0Rz\nO0UkLTAKqGnb3nfox8tuL6vD+VERyRJNzyjgSfv+Wba+AyIyBdgB5BGREIf6bUTkO/s4q4jMFZHt\ndqnuqvctglUrlvH5hLH8MOc3UqVKFWlv/Ewzfps7h3v37nHyxHH+OnaU8hUTlH3kkVFVund9laLF\nivHOe/9s1Xz0yJHI40UL51OkaLGYbk90rly+yI3r1mqPu3fusHXjGgoU8ufv48ci9a5duZgChYuQ\nNl16Nu49yfKtf7J865+ULleJidPnULJMeZfpO3jmBqV7LyZgwDICBizj7LU7NBq+mos37pE1nW9k\nvbL5M+IlwpVb9zl95TblC2TiyRTeANQolpWjZ2+4TGNsnynA6lUrKVK0GLlzu25yL2FIvP5Lzrh6\nHW9hoC3QFdiONWKtgbXbe3/gFLBTVVuKSF1gBlbyuZ5YW65tEpE0wF2gL9BTVZuBFWoAUNUHIvID\n0An4FKgP7FbVS45CVLWviLylqhHJ7fIDRYGXVfUN2xbb6/gMK/ndRhHJCywDikevJCJd7ddK7jx5\n4/0mvfby82zasI4rly9Rqmh++vQfxGfjx3Dv3j3atLBiuxUqBTDusykUK16CFs+2pXql0nh7+zB6\n3Od4e3vHu6/HYcvmTfw4ayYlS5aKXF700dDhfP/tdI4cPoSXlxd58ubj88lT3aLn4vnzDHj/dcLD\nw1F9QKNmz1KrXmM6P9uQWzdvoihFi5fiw5ET3KJnSpdKVC2SlUxpUhI4sgnjFuxn9uaYU3g1K5+L\nzrUKEvbgAXfvP6D719YE6c4TV1m04zTLBtQlLPwB+05d54eNJ1ymObbPtHGTpvzy8xzaJsMwAyT/\nUIIzxFX5t2zHtkJV/e3zGcAyVZ0lIgWBX7H2u2ytqn/ZdU4BJYHuWBsQzwJ+VdVg29FGd7w9VbWZ\niOQB5qlqeRH5CfhBVR+K/4pIiKqmcdC3RlULxHK9DdBMVV8SkQtE3Wk+K1BMVW/G9vrLlq+gq9Zv\ne6T3zF2k8nWPo04If124ldQS4qTBUPetm35UkmvOtepVKrEjKDDRXGWRkmV14s8rnNZrXCJbUHxS\n/yQFrh7x3nM4fuBw/sDuOyyGe1RVR4nIIqApsFVE6sfViaqeEpHz9qg5AOhkp2sOsqvMV9VBMdwa\n/f9yx28hx/VGXkBVVb0Tlw6DweAePH3Em9QTSuuxQgQRI9hLqnpDRAqp6l5VHY2V5bMYcJO4Uw59\nDfwA/Kyq4XYpa5cIpxsqIiniaOO8iBS3J9ochw/LgbciTuzd6Q0GQxLh6THepHa8Q4CKIrIHa/Ls\nRdv+nj0htxu4AywB9gBhIrLbcXLNgflYOeK+jaO/r4A9IjIrlut9sRLbrQbOOtjfidApIvuBbvF6\ndQaDIdFJzFUNInJCRPbak+6Bti2TiKywJ+1XiEhG2y4i8rk9eb9HRBI8M+uyUIOqnsCK10acvxTL\ntRYx3Pt2LM1Gfwh+rcNxGaxJtYNxaOoD9HEwlYx2/RfglxjuuwQkrzU1BsN/mEQONdSJNhnfF1hl\nhzz72ud9sLIR+9slACsNfED0xuJDUo94EwX7zZkL9EtqLQaDwfW4ONTQAvjePv4eaOlgn6EWW4EM\nEjUjcbz5VzheVR2lqvlUdWNSazEYDK5FAC9xXoAsIhLoULrG0JwCy0UkyOF6dlU9C2D/zWbbc2Et\ngY0g2LY9MmY/XoPB4FmI4BW/WMOleCwnq66qZ0QkG7BCRGINVRLzFhAJWo/7rxjxGgyG/xaJtUmO\nqp6x/14AfgMqY61u8gOw/0bsUhQMOG6ckZuo6/vjjXG8BoPBo7BCDeK0OG1HJLW9HQEikhpoCOzD\nWiEVscLqRWCefTwf6GyvbqgCXI8ISTwqJtRgMBg8jkRa1ZAd+M3eKsAH+FFVl4rIduBnEekC/I21\n7QHAYqyHuo4Ct4GXE9qxcbwGg8HjSIwHJOytCsrEYL/Mw0tXUWt/hTcfu2OM4zUYDB6Ipz8ybByv\nwWDwOIzjNRgMBjdirVrwbM9rHK/BYPAsPCC1jzOM4zUYDB6HcbwGg8HgVpL/to/OMI7XYDB4HGbE\na4gRVbgf/iCpZcRIKpJv6p+QuzElJUk+rB3SOKklxMr3gTHnd0tqLt++n6jtPcojwckV43gNBoPH\nEUdiWo/AOF6DweBxeLjfNY7XYDB4Hh7ud43jNRgMHoaYUIPBYDC4FcGEGgwGg8HteLjfNY7XYDB4\nHibUYDAYDG7Gw/2ucbwGg8Hz8HC/axyvwWDwLKzJNc92vSbZpcFg8CzsbSGdFafNiOQRkTUickBE\n/hSRd237EBE5LSK77NLU4Z5+InJURA6JSKOEvgQz4k0GvP9mV1YuW0yWrFlZs2UnAK+/3IljRw4D\ncOP6ddKlT8/Kjdu5cuUyXTt3YNfOQJ7r+AIjPvnMbTqDT53i1Vde5Py5c3h5efHKq6/x5tvvsnvX\nLt55qzt3797Fx8eHTydOplKlym7TFR4ezsst65A1hx/jps2JtI/9qDeL5v7Imj3BkbaVi37j689H\nIyL4Fy/BxxO+domms6eD6f3Oa1y6cB4vLy+ee/5lXnztTQ7+uYfBfd7l9q0QcuXJx9jJ00mTNl3k\nfWeCT/HM0xV4q2d/unR/zyXazp08xjeD3o48v3T6FM1ee59b16+yZ8MKxMuLtBky03ngWDJkzc7y\nWV+yfbmVaDc8LJxzJ4/yyeIgUqfL4BJ98SGRxrthQA9V3WFnGw4SkRX2tQmqOjZKnyJPAe2BEkBO\nYKWIFFHV8Eft2DjeZEC7ji/w8mvdebf7K5G2L7+dFXn80YDepE2XHoAnfJ+g14DBHDrwJwcP/OlW\nnd4+PowcM5Zy5cpz8+ZNqgdUpG69Bgzs34f+AwfRqHETli5ZzMB+fVi2co3bdM357gvyFy7CrZCb\nkbYDe3cScuN6lHp/nzjGjC8m8NXPS0mXPgNXLl90mSZvH2/6Dh5BidLlCAm5SetGNaheqy4DerxJ\nn0EjqFytJr/M/p6vp3zKe30GRd43cnAfatZt6DJdADnyFWLA94sBeBAeTr8WVShbqyGp0qXnf117\nALD6529Z/O3ndOw9nIadXqdhp9cB2LNxJat+mp6kThdIFM9rp2Y/ax/fFJEDQK44bmkB/KSq94Dj\nInIUqAxsedS+TaghGVClek0yZswY4zVVZf7vc2nZ5jkAUqVOTUDV6vj6PuFOiQD4+flRrlx5ANKm\nTUvRYsU5c+Y0IsLNGzcAa3Tu55fTbZounD3N5rXL+d9znSNt4eHhTBw1iLf6fBSl7rw539P6+VdJ\nl95yGpkyZ3WZrmzZ/ShRuhwAadKkpaB/Uc6fO8PxY0eoVLUGANVr1WP5onmR96xcsoDc+fLjX7S4\ny3RF52DgJrLkykdmv9w8mTptpP3+3Tsx/l7fvmIBlRo0d5u+mBG8xHkBsohIoEPpGmuLIvmBcsA2\n2/SWiOwRkekiEvE/Zy7glMNtwcTtqGPFON5kzrbNG8maNRsFC/kntZQonDxxgt27d1KpcgBjxk6g\nf7/e+BfMS7++vfh42Ai36ZgwrD9v9fkIkX/+Kf8ycxo16zUhS7YcUeqeOn6Mv08c5bXnGtGldQO2\nrFvpFo3Bp05yYO9uypSvRJFiT7Fq2SIAli74lbNnrDDI7du3mDZ5PG/16O8WTREErlwYxZHO++IT\n+resxh/L5tH81fej1L1/9w77t66jXJ0mbtUYHYlnAS6pakWH8lWM7YmkAeYC76nqDWAqUAgoizUi\nHufQdXQ0Ia/hP+V4RSSDiLyR1Doehd/nzqFl6+eSWkYUQkJC6NCuDWPGTiBdunRM+2oqYz4Zz5G/\n/mbMJ+Pp/vqrbtGxcfVSMmbOQrGSZSNtF8+fZdWS32nb+eHBTXh4GMEn/mLqrIUM/fRrRvR/l5vR\nwhGJza1bIbzTpSP9Px5DmrTpGD5+Kj9++yXPNqzOrVshpEyZEoCJnwzjxa5vkTp1GpfqcSQs9D57\nNq6kfN3IuSNadOvFiN83U7lRC9bOnRGl/p6NqyhUukLShxkg3p7XaTMiKbCc7ixV/RVAVc+rariq\nPgCmYYUTwBrh5nG4PTdwJiHy/2sx3gzAG8AUR6OIeCckQO5qwsLCWLxgHkvXPnIIyWWEhobSsV0b\n2nfoSMtWzwIwa+YMxo63JvmebdOWN7q95hYte4K2sWHVUjavW8H9e/e4FXKTjk2qkiKlL23qWSGR\nu3du06ZueX5ZvYNsOXJSomwlfFKkIGeefOQrWJhTJ47xVOnyLtEXGhrKO1060vzZdjR8pgUAhfyL\nMn3OAgCOHzvC2pVLAdi9I5BlC39n7NCB3LhxHS8vL3x9n+D5V7q5RBvAn1vWkrdICdJlejjkUqnB\n/5jcs0uUUW/gygVUbPA/l+l5FLwSYTmZWGvSvgEOqOp4B7ufHf8FaAXss4/nAz+KyHisyTV/4I+E\n9P1fc7yjgEIisgsIBUKwfkqUtZeMLFTVkgAi0hNIo6pDRKQQMBnICtwGXlPVg64Wu2HtKgr7FyVn\nrtyu7ipeqCrdu75K0WLFeOe9DyLtfn452bB+HbWers3aNaspVNg9YZE3eg3mjV6DAQjaupEfv5kY\nZVUDQJ3Sufll9Y7/b++8w62qjj78/kCqIqAJVhSU2IIBRRQrFooGRVFRSlSUWIixBNEYxR4sUVM0\nNj6/BHvswaBG0fioIBYf0MgAABY9SURBVLZgjSioiDEaRWyAKAKTP2Ydsr2h3HIqZ97nuc89d+99\n956z77m/PWvWrBkAduvVj4kT7mbfg4bw2SdzeHfmm2zQvkNBbDMzzhw5gk2+tzlHHnfi0u1zPv6I\ntb/TjiVLlnDNby9h0OHDAbh1/MSlx1x52Rharr56QUUXPF6bFdKP/jmTdu07Aj6Jtu7Gmyzdt2De\nF8x44RmOPOc3BbWptuQpq2Fn4DDglaQJAGcAgyV1xcMI7wDHApjZPyTdAbyGZ0QcX1+HrdqE93Sg\ns5l1lbQ7cH/6eWYKri+PscBxZjZD0g64x7xnzYNS8P4YgA3ab1Rro0YMP4wpk57gkzkf022rTTjl\n9LMYcviRjL/7zqWTalm233oz5s39goXfLOSh+//Cbffcz2ZbFH5CZspTk7n1lpvo3HlrdtjOJ47O\nu2AMV107llEjT2bxokU0a96c319zXcFtqQ89dtuLZyY9xqC+PWjcuBEnnH4+rduuVZBr/f3ZKYy/\n6zY22/L77N+rBwAjf3Eu77z9FreO81Bj7x/256BBh6/oNAVj4VcLeP25SQz9+Zil2+695ld8OOtt\nGjUSa627AUNO++++Fx9/mC2335VmLVqWwtxvk6f27mY2iWVr+AMr+J0xwJjl7a8tMqtXbLgiSeI6\nwcw6J+E9x8z2qLkv/TwKWAO4DJgNvJE5VTMzW6HSddmmm5VTiCBLm5ZNSm3Ccnn53cLGXBtK29Wb\nltqE5fK3mR+V2oRlctFR/Zk17eW8LTXrsk03e+Cxlf9vbdi22d/NbLt8XTefVJvHW5P5mdeL+PZk\nYy5fqxHwmZl1JQiCsqCyFwxXWVYDMBdotZx9HwLtJK0tqRmwL0BKL5kpaSB4QF5Sl6JYGwTBMsnH\nkuFSUlUer5nNkTRZ0qvAAlxsc/u+kXQ+nkA9E8hOng0FrpE0GmgC/Al4qXiWB0GQpdKL5FSV8AKY\n2ZAV7LsCuGIZ22cCexfSriAIak9ly24VCm8QBJVNJYQSVkYIbxAEFUeEGoIgCIpMZctuCG8QBBVI\nhTu8IbxBEFQaQhXu84bwBkFQUXjPtVJb0TBCeIMgqDhCeIMgCIpMhBqCIAiKiASNKlt3Q3iDIKhA\nQniDIAiKS4QagiAIikyEGoIgCIpNCG8QBEFxqfRQQ1W1/ikmkmYDs/J0uu8AH+fpXPmmnG2D8rav\nWmzb2Mz+t5VxPZH0V9y+lfGxmZVlOdcQ3gpA0vPl2juqnG2D8rYvbKteqq31TxAEQckJ4Q2CICgy\nIbyVwdhSG7ACytk2KG/7wrYqJWK8QRAERSY83iAIgiITwhsEQVBkQniDIAiKTAhvBaEarVVr/hwE\nQWUQwlshSJKlmVBJawNYBc+MLuuhIanon8dV5eG1qryPaiGyGioMSScAOwIfAI8DD5rZN6W1qm7k\nHiKS+gLfB1oAl5vZV6WwI73eD3dEPgCmmtmiYtpSFzL3b31giZn9O7s9z9caAMwHGpnZX/N57mom\nPN4KQtJAYCAwAugD7FJpogvuqUvaBxgDvAAcAlxeCjsAJI0CRgLdgEuAXsW2pS6k+9cP+AtwkaSJ\nkhoXQHR/CowC1gLulrRrPs9fzYTwljG54WNmCL4BcDEwAHgfODPtX7ckBtaDzJB4b2AQsCYwF7io\nxv5i2dMe6G5mewBf497dw5JaFNOOuiCpK/63PwD4G9ARWCOzv0H3UM7GQG9gT/xz9zjwlKQmDTl3\n4ITwlik1ho05YX0buBA43Mz6mtk3kk4BjitFfLSetErfhYvHCcCRZvaepAOBIUW252tgkaRxeAjn\nQDNbAvRLolyOfAlcA+wK/BToY2afS9oF8hL7FzAbeA84G+gJHGxmi4EjJG3WwPNXPZXyz1o1JG8j\nG3s8GbhV0urAW8A0YIKkbpIGA0OBO5NYlDWSOgGjJXUEbgMOBW4wsxmSdsIfKu8WyZZDJPU0s4+A\nN/FY82lmtlDSUbjglFWcV1JnSVsDC4EzgNOAnmb2dgoDnCNpowZeYxfgODP7EmgJnGJm/c3sS0lD\ngB8D8xr2ToIohF5+NDWzrwEkDceH4wPNbL6k6cD/4bHI84AFwBFm9o+SWVs32qbvx+Ie24HAdZJ2\nA7YDRpnZk0WyZWPgrHSPxwNNgOslPQ/sBRxiZh8UyZaVkh68+wNb417u6cD/A73TJNtPgTPMrF4P\nrjRiErAV0EXSIOAnwJqSHgVeBXYCjjKz9xv6fqqdyGooI5JHeDHuZcySNBL3xr4CugBHA78HbsSH\nyEqeSVkjqYuZvZRebwfsh2cyXIx7Vc3wB860QszM17BlEzN7O70+ATgMF5icsKwGTDezdwplQ22p\neS8kdQb6A53wycCeeLihBfBnM5tY3/snaSMze1dSS3wCtxvwtJndKqk/sBiYlrt3QcMI4S0j0jDx\neKA9cArQHRcFgD8AS/AY6MhyEIYVkUl5agFcDbQ2swPTvu2Bc4B/AVflRLmQdqTX3fERxGQzuydt\nGwmcCgwxs8cKZUd9kbQz8GMzOzL9vCVwEO6xn2dm7+XhGusDU/AQw4PJux6arnMLcGs5p9dVIhHj\nLSPSMPEqfBLtcnwm+WDgIDO7A/gMn2j7umRG1oKM6O4D3ITHI03SjQBm9izwd6A57s0X1I70eijQ\nD/gE2Dl5cZjZr/FRxWmSmhfKlvogaVs8r7iXpLEAZjYNeAbYBhgjqVVDJlbTg2dH/G90oaQ+Zjbf\nzMbi4ZcuZDImgjxhZvFVoi88ptZoGdvXwofhfwI2Stt+AkwFti613bV8b9sm+3ukn78L3AU8gOcg\nP4WncRXDlh2ACel1C3xS6nJgGO4BX4v3BSv5fcvY/ANcYNsBTYHXgOvTvq54fHfLBl6jN3BH5jM2\nBA+59AP2Bf4MrFfqe7EqfkWooYRIWsPM5qXXx+I5rY3M7BJJrYGfAx3weN56wOdWpjE2SU3NbGF6\nvQ6+KOJCYEczezVtb4yLXFPgHjMbXwRbugOj8TDNoeZZC9/BY6W7AZ3x9LzXCmFLXciMFDYCHgUu\nM7Pr0r4WwPPA63gI6hir40oySU2BTmb2mqRh+APoLTPbL3PMwXiYawFwspm9nIe3FtQghLdEpKHu\n/mY2PKWMDQDOwifPXjGzoZJa4au7WuL/aGWZMpYE9RDgU+BD3HMaC5wEbAicZGazMsc3N7OvCjGR\nlh5Y2/LfybLZeMbEXrjoP2GeIdLYzBZLamNmn+XThrqS/s7rmqfVdQfmAOfjnvoPzGxBOq4pHmKY\nn3uY1fE6nfB4+wfARvi8wcl4St8VmeNaA4vMbH7D3lmwPEJ4S4C8yM3tuDAtwgX3GOBE/J/N8DX4\nB6d/yhbm+aZli6RN8Zj0asBuZjY9rX4aBmyBpzrNLIId6+EPgf2ATfDh+NeSTgO+B9wNPFlOopLu\n0z3AI8Au+GfhdVwkO+KLOvKSOyvpsnT+n5vZNSkOfyzwqJldmY9rBCsnJtdKw0JccM9OX2cC2+Me\n8AA8vruHpJvNbG4FiK7wSavX8QnAbgDJy70en7z6dTEmr8xzbz/APcP7gHXS9l/hi08OxyeTyoZ0\nn27CQ0oPm+dlG+6NvoEvYc7XBNe1eObMMZIONbMHgQuAH6UFOUERCOEtAWY2F4/h9QNmZIbhU9L3\nTfFiLaNLYF6dMedTfEHEQOBUeYEV8DDJI8DPrEDVxyS1qbHpLnyxwRzgaEnd0vZb8AnKOg/Ti8CL\n+CKIkUkQl6QQw2jgCXxlXYMxszfN7CbSA19eIW5D3Bl4Oh/XCFZOhBpKRBpedsJjulcDD+Kr0t7B\n45F7mtmbJTNwJdRI1WoCLM7FoCX1AMbhgtsNON7MphbIjgOAo/CJswWSVku2WErHOhSfKGqLZ4uM\nyNewPZ9kJtb64pkGA/Hl0yOA03Nx3jxfc2/gUrww0HCrnBWQFU8Ib4lJ4nA7nkc5Ca8ENacY8dCG\nkuoDvJELhUjaAF9ddzk+eXMccJ+ZTSzQ9VvgXuxfcU92upl9kfbtiI/o5gE98Gpo51oBF2vUFUmr\nmdkiSc1SHLodnte8PR4S+Ay4oFDZH8mGdvigZXahrhH8LyG8ZYCkLnh5v1+YJ65XBJIuB9YysyMl\nrQU8C/zGzK5K+xuZ2ZICZS+0NC/cMgzPouiI5wV/JikX3z3WzB5IxzexMqldnGLi38eLv/dN2zrg\nWQa/M7PxkjYEGpsvHS/oMuqg+ESMtwxIXlhPPO5bSfwRmJsmzRbicdyc6CoXeiiA6G4OnCzpu8DH\neOGYiaSJNGBz4LCc6CYbSi66SqSY+Kv4vctNaP0CeCTn3ZrZe7nYf4juqkd4vMFKSQsiNjezJyT1\nBNYHxieP817gBTM7P3N8o0LmHEvaE4+BzgSeA/6NFwVvi9cVeLEYdtSWXN5yer00b1hefnJjMzsn\nl1ectoeHu4oTHm+wQtJk1f7AiBTT/RrPA71I3jLnHGALSW3SEJpCiV3m/H8D7sSX0/bAu3HcDHwD\nDJSXmSyYHXUhLUb4g6TekpoBz0n6maQf4jnFgyXtFaJbXYTwBivEvCrVRGAyvhhiAb7G/2p8sccY\nPI1su0IKRk1BSuJ7N76U+vhk19V4+truxcgZriWr4V758fiE4wC8KtvZ+APsFVJecfLQQ3SrgAg1\nBMulRsrYhvjwvjNwSxI+5G1gBuJivL+ZfV5gOw7HY7nT8WyGzsARuJj9ERe6b0o9S69v14vYFK8L\ncTBeyvHZ5AkPxVeq7Q5sY2YflsreoLiE8AbLJJNX2h0vDzgHL1d5PN6l4C4zezh3LF4t61Qzm1NA\nm07GY7m34pkMzwC/xBecnIgvjPhtqUMM8toVe+OV0N7HH0o3AvsAP8SzGR7LZH1cgqcQ/qpkRgdF\nJUINwTJJors3cAO+/HYaXpv1buAl4DBJfdLhO+GeW16H98rUmU2edRe89fqa+Ge3BV7nYgbwW9wT\nL3lcF1/uOx1vz/NnfCJyJr6ibgJwkqQ9MrZ+jq8eC6qE6LkWLBN5IZ9RePnEDrjw/tPMPpR0F17a\nMdeT7C18pd2/8mlDZiVcR7zj7Xn44oL+eNvxH+GlDRfjRXhKPnzLpdFJmoPX43gNt/lFM/tI0u1A\nY7zw+iv4ZGVzvJpbUCVEqCH4H5J3+RleUOZrfFh/hHnFsUPx2gEfmZdVzHvKlrzj8EZm9id5X7ST\ngMfw4unCa8qeIe8qsQ1waTnERzPhmX74fZuKe7JnA8+Z11neAF/s8balppHltLgjKA7h8QbAt0Rj\nBzxF7CQ8a6E3sH7K2d0WHz7PsNSBt0BD+7Z4utoWuHD1xT3cTnh4Ibd4oi/QuxxEF5aGZ/bFq8ud\nZmafSJqLt3M6QdLNeKnKEWb2fmYxRYhulREeb7AUeQfggfiw+DZ508MpeH+02bgIn1vI2gEZW3oD\nv8Y73R6dcmAH4i2EOuJe97NWz3bmhSDdrzvxMovP4rHvrfA6xYbXvX3IzB4qmZFBWRDCG2S93ZPw\nrIVxeM2A+fJCNIPwofM7ZvZUsZL8Je2PV2w7MYUdGuO5xO2BK8zsk0LbUBfSw2EsngGyKd6NYzvg\nXjO7IHNcLJKockJ4q5iM4H43l/cq6Qg8v/Q84HkzK2lH4xQvvQi4MIlvI2ANS1XISknm/m2LTzZ+\nik+o7YMvo54sqRde0HwIMK9Msi6CEhMx3iomE5M8Ic2wTzazG+S9vc4ELpX0RG45a4lsvF/SEmCs\npEVmdhdQctGFpfevL3AlniY2GK+d+3tYGi75DR7vLQubg/IghLeKkbQ7vuT3ILzjRQ9J7c3sijRs\nHo2vtvq0dFaCmT2YCsq8VUo7siTPuxVwKt6N94E0eXZnKilxC6kbh3l7nSBYSghvlZGtgoU3oRyE\nl1HcGF99dkAaQv9O0njzlj4lxwpUTL2uZOKzTYC5+Oq5Bem+Tk1x8qPSyGFUeLrBsgjhrRIktTJv\nnLk4VRlbD18C/AXe++0g86Lb/YFukjqY2TslNLksSeGFA/BOG28BO+MLIp7HhfhLwORV3eaWzNCg\nrAnhrQIktQTul/Q7vBrWVcALwBKgNbAtMFXSU/hn4rIQ3W+TmUhrg2dW3IKniO2CtzhqmbIudgHO\nMq/qFgTLJLIaqgRJA/DFD3OB0Wb2tKRNcG+3J57YvxC4xMzuLZ2l5Yuk7fH0sLVz6WFphHAOXjbz\nJqBpymaIlLFguYTHWyWY2b2S5uGFWnrhrbz/iXeyfQP34lqmegIhGomMp9sDuB6YBbSTNAmYZGb3\nSVoTzwI5w1IH47h/wYqI6mRVRJqgGgYMkzQ4LVX9FF9629xSt+AQjf+SWUZ9HjDIzPrhdYAPBHZK\ndRZuBnpZGbaND8qT8HirjOT5LgJukDQQL4Zzrpl9XGLTypnWwF5AH7zm7/l4qt0RuPPyWL4rswWr\nNuHxViFm9hfgx3jRmWvMbEIqZh4sA/OC7wcBwyUNSSOFC/Ammx+V1LigIonJtSpG0lrlVu+gnJE3\nqLwAuNLMxpXYnKCCCeENgjqQshguxicoPyzlcuqgcgnhDYI6ki0qFAT1IYQ3CIKgyMTkWhAEQZEJ\n4Q2CICgyIbxBEARFJoQ3CIKgyITwBnlD0mJJL0p6VdKdqSpafc+1u6QJ6XV/Saev4Ng2kn5Sj2uc\nK2lUbbfXOGacpIPrcK0Okl6tq43BqkkIb5BPFphZVzPrjFc6Oy67U06dP3Nmdp+ZXbyCQ9oAdRbe\nICgVIbxBoXgS6JQ8vWmSrgamAu0l9ZE0RdLU5BmvASBpb0mvp8pfB+ZOJGmYpFwfs3Uk3SvppfS1\nE76gYdPkbV+ajjtV0nOSXpZ0XuZcZ0p6Q9IjeOeNFSLp6HSelyTdXcOL7yXpSUnTU+86JDWWdGnm\n2sc29EYGqx4hvEHeSd0X9sGLroML3I1mtg0wHy8w08vMtsU7N4yU1Bxv5b4fsCuw7nJOfwXwuJl1\nwQu4/wOvM/xW8rZPldQH+B6wPdAV76ixm6RueKujbXBh716Lt3OPmXVP15sGDM/s64DXMu4HXJve\nw3DgczPrns5/tKSOtbhOUEVEdbIgn7SQ9GJ6/STew219YJaZPZ229wC2AianujxNgSl4/7eZZjYD\nIDWOPGYZ19gTOBwgLdf9XFLbGsf0SV8vpJ/XwIW4FXCvmX2ZrnFfLd5TZ0m/xMMZawAPZfbdkdq1\nz5D0dnoPfYAfZOK/rdO1p9fiWkGVEMIb5JMFZtY1uyGJ6/zsJmCimQ2ucVxXvJVOPhBwkZldV+Ma\nJ9fjGuOAA8zsJUnDgN0z+2qey9K1TzCzrEAjqUMdrxuswkSoISg2TwM7S+oE3g9O0mbA60BHSZum\n4wYv5/cfBUak322cuj/Mxb3ZHA8BR2VixxtIagc8AQyQ1EJSKzyssTJaAR9IagIMrbFvoKRGyeZN\n8E4eDwEj0vFI2kzS6rW4TlBFhMcbFBUzm508x9skNUubR5vZdEnH4E05PwYmAZ2XcYqTgLGShgOL\ngRFmNkXS5JSu9WCK824JTEke9zzgR6n9+u3Ai3gLnydrYfJZeAv3WXjMOivwbwCPA+sAx5nZV5Ku\nx2O/U1ON49nAAbW7O0G1EEVygiAIikyEGoIgCIpMCG8QBEGRCeENgiAoMiG8QRAERSaENwiCoMiE\n8AZBEBSZEN4gCIIi8x/FqW3KYadROgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11ac52160>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(mul_tfidf_train, mul_y_train)\n",
"pred = clf.predict(mul_tfidf_train)\n",
"\n",
"#print(pred[:20])\n",
"\n",
"score = metrics.accuracy_score(mul_y_train, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(mul_y_train, pred, labels=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])\n",
"plot_confusion_matrix(cm, classes=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.238\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEmCAYAAAAnRIjxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd8FFXXwPHfSUJvAULvTUCQIkhX\nioC9dxABUWyIXbG3R0WfVx57b4io2EVEihQFpBcpioAURXovCZBy3j/uBJeQZEPY3dmF8/WzH3Zn\nZmdONvHsnTN37hVVxRhjTOjF+R2AMcYcqyzBGmNMmFiCNcaYMLEEa4wxYWIJ1hhjwsQSrDHGhIkl\nWBNRIlJERL4TkZ0i8vlR7KeniIwLZWx+EZFTReQPv+MwoSfWD9ZkR0R6AHcCDYDdwALgKVWdepT7\n7QXcCrRT1bSjDjTKiYgC9VR1hd+xmMizFqw5jIjcCbwAPA1UAKoDrwEXhGD3NYBlx0NyzQsRSfA7\nBhNGqmoPexx8AKWAPcBluWxTCJeA13mPF4BC3rpOwFrgLmATsB7o6617HDgApHrH6Ac8BnwUsO+a\ngAIJ3us+wEpcK3oV0DNg+dSA97UDZgM7vX/bBaybDDwJTPP2Mw5IyuFny4z/3oD4LwTOBpYB24AH\nArZvBUwHdnjbvgIU9Nb97P0se72f94qA/d8HbACGZS7z3lPHO8bJ3uvKwBagk99/G/Y48oe1YE1W\nbYHCwNe5bPMg0AZoBjTFJZmHAtZXxCXqKrgk+qqIlFbVR3Gt4hGqWlxV380tEBEpBrwEnKWqJXBJ\ndEE225UBvve2LQsMAb4XkbIBm/UA+gLlgYLA3bkcuiLuM6gCPAK8DVwNtABOBR4RkdretunAHUAS\n7rM7HbgZQFVP87Zp6v28IwL2XwbXmu8feGBV/ROXfIeLSFHgfeADVZ2cS7wmSlmCNVmVBbZo7qfw\nPYEnVHWTqm7GtUx7BaxP9danqupoXOutfj7jyQAai0gRVV2vqkuy2eYcYLmqDlPVNFX9BFgKnBew\nzfuqukxVU4DPcF8OOUnF1ZtTgU9xyfNFVd3tHX8J0ARAVeeq6gzvuKuBN4GOefiZHlXV/V48h1DV\nt4HlwEygEu4LzcQgS7Amq61AUpDaYGVgTcDrNd6yg/vIkqCTgeJHGoiq7sWdVt8IrBeR70WkQR7i\nyYypSsDrDUcQz1ZVTfeeZybAjQHrUzLfLyIniMgoEdkgIrtwLfSkXPYNsFlV9wXZ5m2gMfCyqu4P\nsq2JUpZgTVbTgX24umNO1uFObzNV95blx16gaMDrioErVXWsqnbDteSW4hJPsHgyY/onnzEdiddx\ncdVT1ZLAA4AEeU+uXXdEpDiurv0u8JhXAjExyBKsOYSq7sTVHV8VkQtFpKiIFBCRs0TkOW+zT4CH\nRKSciCR523+Uz0MuAE4TkeoiUgq4P3OFiFQQkfO9Wux+XKkhPZt9jAZOEJEeIpIgIlcAJwKj8hnT\nkSgB7AL2eK3rm7Ks3wjUPuxduXsRmKuq1+Fqy28cdZTGF5ZgzWFUdQiuD+xDwGbgb2AA8I23yX+A\nOcBCYBEwz1uWn2ONB0Z4+5rLoUkxDtcbYR3uynpHvAtIWfaxFTjX23YrrgfAuaq6JT8xHaG7cRfQ\nduNa1yOyrH8MGCoiO0Tk8mA7E5ELgDNxZRFwv4eTRaRnyCI2EWM3GhhjTJhYC9YYY8LEEqwxxoSJ\nJVhjjAkTS7DGGBMmNtBEmBQsnqhFylbyO4xs1SpbNPhGPvlt7U6/Q8hVpSj+7BILF/A7hGyt/WsN\nW7duCdY3OM/iS9ZQTTvsBrjDaMrmsap6ZqiOmx+WYMOkSNlKtBv0gd9hZOujXi38DiFHTe79zu8Q\ncvXANc39DiFHF55YJfhGPujesU1I96dpKRSqH7THG/sWvBrsjrqwswRrjIktIhAX73cUeWIJ1hgT\neyQ2Lh9ZgjXGxB4JWUk3rCzBGmNijJUIjDEmPISYKRHERpTGGHOQuBJBsEewvYi8JyKbRGRxwLL/\nishSEVkoIl+LSGLAuvtFZIWI/CEiZ+QlUkuwxpjYI3HBH8F9gBu5LNB4oLGqNsHNwXY/gIicCFwJ\nNPLe85qIBK1TWII1xsQYrwYb7BGEqv6MGwYzcNm4gNk4ZgBVvecXAJ960/ysAlbg5qLLlSVYY0xs\nEfJaIkgSkTkBj/5B9pzVtcAP3vMquHGRM63l0CmJsmUXuYwxsSdvJYAtqtoyX7sXeRBIA4ZnLspm\ns6CDaVuCNcbEGIH48HXTEpHeuBkyTtd/ZyRYC1QL2KwqeZiHzhJslKmSWJj7u9U9+LpSycIMm72W\nBhWKUzWxMADFCyaw50AaAz5fnNNuwmLgTdcxbsxoksqVZ+qsBQBs37aN6/r04K+/1lC9eg3eHfoJ\niaVLRzSuTP0616FH+5oI8PG01bwz6U9OrFKSwVc1p2iheNZuS2bA+3PYsy+3GcnD44GLOlC4aHHi\n4uOIi0/ggfdHMnfC94x690U2rF7BoHe/oUbDJhGPC+D2W65n/JjRJJUrx08z3O/18YcGMX7MKAoU\nLEjNWrV54dV3KJWYGGRPERLGbloiciZwH9BRVZMDVo0EPhaRIbhZjOsBs4Ltz2qwUeafHfsY8Pli\nBny+mIFfLGZfWjq/rNzG4PErDi6funIbv6zcHvHYruzZmxFfHzqP4ItDnuO0jl2YveB3TuvYhReH\nPJfDu8OrfqUS9Ghfk3OenUy3pyfS9aSK1CpXjP9efTJPf7uYrk9N5IcF67mpaz1f4gO489WPeejD\n0Tzw/kgAKtepzw3PvE7dZkGvlYTVFT2u4ZMvD/29dux8OpNnLGDSL/OoXaceLw151qfochCablqf\n4GZRri8ia0WkH/AKbiLL8SKyQETeAFDVJcBnwG/AGOCWgKndc2QJNoo1q1KK9Tv3s2nPgUOWn1a3\nDJNXRGI+v0O163AqpUsfOoP0D99/xxU9ewFwRc9ejB41MuJxAdSrWIJ5q7axLzWd9AxlxvItnNms\nMnXKF2fG8q0ATFm6ibObV/YlvuxUqlmXijXq+B0GbdufethZR6fTu5GQ4E5wW5zSmvXrIjEDel6F\nrBfBVapaSVULqGpVVX1XVeuqajVVbeY9bgzY/ilVraOq9VX1h9z2nckSbBTrWLcMP63YesiyxpVK\nsD05lXU79/sU1aE2b95IxYpu3NuKFSuxZcsmX+JYun43beomUbpYQQoXiKdLo4pULl2EP9bvonsT\nF9+5zatQuXQRX+ITEV687Rqe7nMeU7752JcY8uuTjz6gS7c89auPnND0gw27Y74GKyIDcXPVz1PV\nw6Y+FpFOwN2qem6kY8tNQpzQumZp3p/59yHLO9Ure1jSNbBiw25eHb+MT25tz979afz2z07S05U7\nh83jycubcMfZ9Rm3cAOpaf7MonzPm1+QWK4Cu7Zt4cXbelGxRh3qNW/tSyxH4oX/PkNCQgKXXN7D\n71D+lccSQDQ45hMscDNwltc5OGa0rJ7In1uS2ZHy7wWZOIF2tcow8IvIXtzKTblyFdiwYT0VK1Zi\nw4b1JCWV9y2WT39Zw6e/rAFg0Pknsn5HCn9u3EOPl38BoHb54pzeuIIvsSWWc8ctWSaJZh3PYNVv\nv0Z9gh3x8YeMHzuaz0eORaItocXIYC/R0Y4OE69AXRsYKSL3icgvIjLf+7d+Ntt39ArbC7ztSnjL\n7xGR2d79yY9HIvZOdcsyefmhddbmVUuxdkcKW/YeyOFdkXfm2ecyYvgwAEYMH8ZZ55znWyxlixcE\noHLpIpzVrDLfzF57cJkI3HZWfYZNWR3xuPanJLNv756Dz3+fOYUqtQ/784sqE38cyysv/B9DP/2K\nokWjbZocsRJBNFDVG71uF52BA8DzqpomIl2Bp4FLsrzlbtzVwWkiUhzYJyLdcV0yWuE6iIwUkdO8\n2+wO4d0p0h+gcJmK+Y67UEIczauV5KWfD210d6xblsnL/SsPXN/3aqZN+YltW7dwUv2a3PfAI9x2\n5730630VHw17n6pVq/Heh5/6Ft/b/VtTulhB0tKVB0f8ys6UVPp1rkOf02oDMHrBOkZMXxPxuHZt\n28Ibg24AICM9nVO6n0+jth2ZP3ksI4Y8xp4d23jlrmupdsKJDHzhw4jHd+O1V/PL1J/ZtnULzRvW\n4p77H+GlIc9x4MB+rrjwLABatGzNcy+8GvHYchRtLeocyL/9aI9NIrIaaAkUAV7CJUsFCqhqg8Aa\nrIgMAi7C3b3xlaquFZH/Ay4Fdni7LA48o6rv5nbcUjUaqs3JdeSifU6uR2xOriPWvWMbfp0/N2QZ\nMS6xuhbqcG/Q7fZ9f+vc/N7JFSrHdAs2iyeBSap6kYjUBCZn3UBVB4vI98DZwAyvpSu4hPpmBGM1\nxuQodgbcjo5CRWSUAjI78/XJbgMRqaOqi1T1WWAO0AAYC1zrlQwQkSoi4t+VHGOM1WCj0HPAUBG5\nE5iYwza3i0hnIB13x8YPqrpfRBoC070rqXuAqwF/OnwaY2KmBnvMJ1hVrek93QKcELDqYW/9ZLxy\ngaremsM+XgReDFeMxpgjYNN2G2NM+ERdv9wcWII1xsQUN962JVhjjAk9ESTOEqwxxoSFtWCNMSZM\nLMEaY0w4CFYiMMaYcBDEWrDGGBMulmCNMSZMLMEaY0w4WA3WGGPCx1qwxhgTBnaRyxhjwshKBMe5\nEoUT6Fg/ye8wsrVrX1rwjXxSrLg/02rn1QmJJfwOIUeFEqJjDNSs4kLd2pTQlAhE5D3gXGCTqjb2\nlpUBRgA1gdXA5aq6XdwBX8QNxp8M9FHVecGOEZ2/EWOMyYWIBH3kwQfAmVmWDQImqGo9YIL3GuAs\n3HRT9XDz7r2elwNYgjXGxBRBiIuLC/oIxpu4dFuWxRcAQ73nQ4ELA5Z/qM4MIFFEKgU7hiVYY0zs\nkTw8IElE5gQ8+udhzxVUdT2A92/m9FBVgL8DtlvrLcuV1WCNMbEl7zXYLSGcVTa7AwadktsSrDEm\n5uSlBJBPG0Wkkqqu90oAmXPvrQWqBWxXFVgXbGdWIjDGxJ68lQjyYyTQ23veG/g2YPk14rQBdmaW\nEnJjLVhjTMwJUTetT4BOuFrtWuBRYDDwmYj0A/4CLvM2H43rorUC102rb16OYQnWGBNTjqAbVq5U\n9aocVp2ezbYK3HKkx7AEa4yJOWGswYaUJVhjTOyJjTtlLcEaY2KPDfZijDFhIAJxNtiLya9ne3Sk\nUNFixMXFExcfz4DXvwHgl68/ZPo3w4iLj6dB686cdcN9EY3r3oE3MGn8D5RNKseYKXMB+G3Rrzx0\nz63s37ef+IQEnnzuBZqefEpE48p0TfsaXN66KgJ8NmstQ6eu4bbudTm9UQVUla17DjDos0Vs2rU/\nonEd2L+P23qdR+qBA6SnpdHxjPPoc+sgbrv6XJL37gFgx9bNNGhyMk++MiyisWX12ssvMGzoe4Bw\nYqPGvPrmuxQuXNjXmA5nwxWao3T98x9RrFSZg6//nD+d3375kdveHkVCwULs2b414jFdemUvrul3\nI3cPuO7gssFPPMjAux+kU9czmDR+DIMff5BPvh0X8djqVSjO5a2rcunL00lNV97t14LJSzfzzk+r\neHHcCgB6ta/BLV3r8OhXv0U0tgIFCzHk/a8pUqw4aampDLz6HFqd2pUXPxp1cJtHB/ahfZezIhpX\nVuvW/cObr7/CjLmLKFKkCH17XclXn4+gR6/ewd8cYTGSX+1Gg1gx87uP6XTlDSQULARA8dJlIx5D\nq3YdSCxd5pBlgrBn9y4Adu/eSfmKQce/CIs65Yvx61872JeaQXqGMmvldro1qsDe/ekHtylaMB4N\nenNj6IkIRYoVByAtLZW01NRDWmDJe3czf+YU2nc9O/LBZZGWlsa+lBTS0tJITk6mYiV/fp+58koE\nwR7RwFqwUUhEeO/ePiBC63OvotW5V7Jl7WpWLZrN2PeGUKBgQc664X6qNWjid6g8/NR/6X35eTzz\n2P1kZGTwxehJvsSxfOMe7jjzBBKLFmBfajodG5Rj8dqdANxxRj0ubFGZ3fvS6PXmLF/iS09P58ZL\nT+efv1Zx4VXX0rBpi4Prpo4fzcltTqNYcX/Hmq1cuQq33nYnJzWoReEiRejcpRtdunb3NabsCLFT\ngw17C1ZEaorI4jDst4+IvHIE2yeKyM2hjiMcbnxxBLe+OZK+z7zH9G8/YtXCWWSkp5GyZxc3v/IF\nZ90wiE+eHIj60RzLYvj7b/HQk88x7dcVPPTkc9x3+02+xPHnpr28PXkl71/fknf7tWTp+l2kZbjP\n539jl9Px6Z/4bv56erWr4Ut88fHxvP31ZD6btJCli+axatnvB9dNHP0VXc652Je4Au3Yvp3Ro0ay\nYMkKfl/xN8nJexnxyXC/w8qWSPBHNIjqEoGIhLKFnQhkm2BFJD6ExzlqJZMqAK4M0KhDN/5eupCS\n5SrSuEN3RIRqDZoiIuzdmXUoy8j7csRwzjzXDZl59gWXsHDeHN9i+WL2P1z04nR6vjGLncmprNmy\n95D1381fR/eTKvgUnVO8ZCmatmrPrKkTANi5fRtLF86jTcduvsYFMHnSBGrUrEVSuXIUKFCA886/\niFkzp/sd1uFiqEQQqQSbICJDRWShiHwhIkVF5BERmS0ii0XkLW9KBkRksog8LSI/AbeJSDkR+dLb\ndraItA/csYiUEJFVIlLAe11SRFZnvg4wGKgjIgtE5L8i0klEJonIx8CirC1tEblbRB7zntcRkTEi\nMldEpohIg3B9UAdSktmfvOfg8+VzplKhZj0ate/Gn/NnALD571Wkp6UechHMLxUqVmLmL1MA+GXK\nZGrWrutbLGWKFQSgUmJhujeuwKgF66mRVPTg+tNPLM/KTXtzenvY7Ni2hT27XLli/74U5k3/meq1\n6gHw09hvadOpOwUL+X+lvmq1asyZPZPk5GRUlZ8mT6R+/bD9qeebELIZDcIuUjXY+kA/VZ3mzYNz\nM/CKqj4BICLDcHPjfOdtn6iqHb11HwP/U9WpIlIdGAs0zNyxqu4WkcnAOcA3wJXAl6qamiWGQUBj\nVW3m7bcT0MpbtkpEauYS/1vAjaq6XERaA68BXbJu5A3o2x8gsXzlvHwuh9mzfQvDHnUN7Yz0NJqd\nfj71W3UkLfUAX/53EC/0O4v4hAJcdt9/I/5HNLD/NcycNoXt27bQrkkdbrv3YZ4e8ipPPngPaelp\nFCpUiKeG5LlqE3KvXNOMxKIFSUvP4PFvfmNXShpPXdqYWuWKkaGwbnsKj361JOJxbd28kWfvH0BG\nejoZGRl0OvMC2nY+A4BJo7/mqutvi3hM2Wl5SmvOv/BiOrU/hfj4BJo0bUbva6/3O6xsRE8CDUbC\nXcfzEtfPqlrde90FGAgMA+4FigJlgJdVdbCXLB9V1Z+87Tdx6LiL5YAGwCVAS1Ud4LVq71XVC0Rk\nOnC9qh5S9/XiGBUwuVkn7zidc1h/N1Ac+D9gM/BHwO4KqWpDclG1/kma2X812lzRJOhA7L4547nJ\nfoeQqzev9aePb140r57odwjZ6tyhNfPnzQlZRixaub7WvyH4lFgLHjt9bggH3M6XSLVgs2ZxxbUC\nW6rq396peOA5UuB5XBzQVlVTAncQ+A3mtYxrikhHIF5VF4tINf5tEb8BjMkmrsDjpHFoySQznjhg\nR2bL1xjjsyi6iBVMpGqw1UWkrff8KmCq93yLiBQHLs3lveOAAZkvRCSnRPch8AnwPoCq/q2qzbzH\nG8BuILd+MBuB8iJSVkQK4UoWqOouYJWIXOYdX0SkaS77McaEUSzVYCOVYH8HeovIQlw54HXgbWAR\nrm46O5f3DgRaehfIfgNuzGG74UBpXJI9jKpuBaZ5F9X+m836VOAJYCYwClgasLon0E9EfgWW4GaY\nNMb4JFa6aYW9RKCqq4ETs1n1kPfIun2nLK+3AFdks90HuHnNM3UAvlDVHbnE0iPLoslZ1r8EvJTN\n+1Zx+PzpxhifREs3rGCOiTu5RORl4CzclA7GmGNZ3meV9d0xkWBV9Va/YzDGRIarwfodRd4cEwnW\nGHM8iZ47tYKxBGuMiTlWIjDGmHCIol4CwUT1YC/GGJOVG64wLugjT/sSuUNElnjdNz8RkcIiUktE\nZorIchEZISIF8xurJVhjTMwJRT9YEamC18/eu0U+HjeWybO48U/qAduBfvmN0xKsMSbmhPBOrgSg\niDc0alFgPW4gpy+89UOBC/MbpyVYY0xMEQk+Fmxeehmo6j+4wZz+wiXWncBc3Ngjad5ma4F8j45k\nCdYYE3PyWCJIEpE5AY/+h+5DSuNue68FVAaK4W5YyirfQw7m2ItARErm9kZvEBRjjIm4uLyVALYE\nGa6wK7BKVTcDiMhXQDsgUUQSvFZsVQ4dLvWI5NZNawkucwf+JJmvFaie34MaY8zRCFE3rb+ANiJS\nFEgBTgfmAJNwI/x9CvQGvs3vAXJMsKpaLb87NcaYcBGB+BDcyaWqM0XkC2Aebjzo+bjZS74HPhWR\n/3jL3s3vMfJ0o4GIXAnUVtWnRaQqUEFV5+b3oMeDgvFC1VKF/A4jW9F8l+Hf40f5HUKuhp0cvbNB\nnFwjOmc0CIdQ3cmlqo8Cj2ZZvBI3ndRRC3qRS9zU2J2BXt6iZNwMAcYY44tjaTzYdqp6sojMB1DV\nbUdzZ4MxxhwNAeKjJYMGkZcEmyoicXhdFUSkLJAR1qiMMSYnUTQlTDB56Qf7KvAlUE5EHsfNp/Vs\nWKMyxphcHDMlAlX9UETm4vqMAVyWdUpsY4yJFCE0vQgiIa/DFcYDqbgygd39ZYzx1TFTIhCRB3Ez\ntVbG3dXwsYjcH+7AjDEmO3kpD0RL/s1LC/ZqoIWqJgOIyFO4ARGeCWdgxhiTk2OpF8GaLNsl4Dri\nGmOML2KlRJDbYC//w9Vck4ElIjLWe90d15PAGGMiTojuuxED5daCzewpsAR3b26mGeELxxhjgoih\nfrC5DfaS7wEOjDEmnI6ZabtFpA7wFHAiUDhzuaqeEMa4jnsZ6ek80uscSpevyF0vfMCT113MvuS9\nAOzatoXajZpxx/OR/Q68Z+ANTBz3A2WTyjFuqhvr55Z+V7Pyz+Uurp07KFkqkR8mz4xIPG882pOz\nTmvM5m27aXnZ0wA8ffuFnH1aYw6kprNq7Rb6P/oRO/ekANC4XmVeeegqShQrTEaG0uHq59h/IC23\nQ4RMkQJx9G1VlSqlCqEK78/6hwPpGVzTsgoF4oUMVYbNWceqbSkRiSc7y5f9Qd9ePQ6+XrN6Jfc/\n/Bg3D7jNt5iyc6yUCDJ9APwHN7XCWUBf7FbZsBv7ybtUrlWXlL17AHj4na8Ornvxnv606Ng94jFd\nemUveve7kTtvue7gslff/ejg8/88fB8lSpaKWDzDvpvBGyN+4p0nrzm4bMKMpTz88kjS0zP4z8AL\nuOfa7jz00rfEx8fx3n960+/hD1m07B/KlCpGalp6xGLtcXJlFq3fzWvT/iI+TigYL9zUvjojl2xk\n0fo9nFSpBJc1q8hzE1dFLKas6p1Qn6kz3Rdneno6DetU59zz8z0dVVjFSokgLzcNFFXVsQCq+qeq\nPoQbXcuEybaN61kwbSIdL7zqsHUpe/fw25xfaNHpjIjH1bpdB0qVLpPtOlXl+2+/5PyLL49YPNPm\n/cm2ncmHLJswYynp6e77f9aiVVSp4Ibw69q2AYuX/8OiZf8AsG3nXjIy8j0TyBEpnBDHCeWKMWXl\ndgDSM5SU1AxQKJwQD0DRAnHsSIlMazovfpo0gVq1a1O9eg2/QzmMiOumFewRDfLSgt0v7uviTxG5\nEfgHKB/esI5vHz3/GFcOfIB9e/cetm7upDE0OqU9RYqX8CGynM2aPo2kchWoVaeu36EcdM0Fbfli\n3DwA6lUvjyqMfPUWkkoX54uxcxky9MeIxFGueEF270/j2tZVqZZYmDXbUvh43jo+mb+eOzvW5Irm\nFRGEp3/8MyLx5MWXn3/GJZdd6XcYOYqS/BlUXlqwdwDFcfOHtweuB64NZ1AiUlNE8jzegYg8JiJ3\ne88biMgCEZnv1Y8Dt+skIu1CHW8ozZ/yIyXLlKVWwybZrp8+7lvannFBhKMKbuRXn3H+xZf5HcZB\n9/Y7g/T0DD4dPRuAhPh42jWvTd8HP+D0a4dwfpemdGoVmcsI8SLUKF2Eycu38vjYFexPy+CcE8vT\nuW4ZPp2/nrtH/sGn89fTt1XViMQTzIEDB/hh9HdcePGlfoeSoxBO2x1WQROsqs5U1d2q+peq9lLV\n81V1WiSCy6cLgW9VtbmqZm0SdMJNanYYb1503y37dQ7zfh7PHee15dUHb+G32dN4/eGBAOzesZ2V\nSxbQtEMXn6M8VFpaGmO//5ZzL4qO/yF7nteas09rTJ8HPzi47J9NO5gydwVbd+wlZV8qY6YuoXmD\nyMyKtC0lle0pqaz0LmDNWbuT6qUL065maeaudXOHzv57J7XKFolIPMGMHzuGps2aU75CBb9DyZYg\nxMcFf0SD3G40+JpcpqtV1YvDEtG/4kXkbVxC/Ac3ve7VQH+gILAC6JV5Cy+AiJwN3A6ki8hpqto5\nYF1N4EZv3dXArUA/YBvQHJgnIruBPar6f957FgPnqupq7z0DvWPPBG5W1ZBfJbliwCCuGDAIgN/n\nTGf0R29y05MvATDrx1E069CVgoUK57aLiJv600Rq1z2BSpX9b4F1a9eQu/p0pft1L5KyL/Xg8vG/\n/MYdvbtSpHABDqSmc2qLurz80aSIxLRrXxrbklOpWKIgG3Yf4MQKxVm3cz/lihWkfvli/LFpLw0r\nFGPj7gMRiSeYLz//NKrLA0TRWAPB5NZqeyViUWSvHnCVql4vIp8BlwBfqerbAN6EZP2AlzPfoKqj\nReQNApJkwLrVWdeJSD/gBKCrqqaLyGPZBSIiDYErgPaqmioirwE9gQ+zbNcf9wVA2Yqhn7tpxriR\nnNfn5pDvN69uvf4aZkybwvZtW2hzUh3uuO9hrri6D999/XlEL25lGvpMH05tUY+kxOKsGPMkT74x\nmnv6dqdQwQRGvT4AgFmLVjPwqU/ZsTuFlz6ayNSP7kVVGTt1CWOmLolYrMPnrqN/22rExwmb9xzg\nvZlrWfDPLq46uTLxAqkZytCgFEF9AAAgAElEQVTZayMWT06Sk5OZNPFH/vfy636HkqtoKQEEk9uN\nBhMiGUg2VqnqAu/5XKAm0NhLrIm4uvDYEBzn8zy0RE8HWgCzvV9sEWBT1o1U9S3crJTUPrHJUV+i\nbtiyLQ1btj34+sG3Pj/aXR6Vl9/+MNvlz7/ydoQjcXrf/8Fhy4Z+Mz3H7T8dPftgTTbS/t6xjyfG\nHVqxWr4lmSfGrfAlnpwULVqUVWsP+9OOKsfalDF+2R/wPB2X1D4ALlTVX0WkD66mmiMRuQV3UQ7g\n7Bw2C7xUn8ahdenMc3EBhqqqDdNoTBSIkhJrULE2eHYJYL2IFMCdoudKVV9V1WbeYx2w29tHTlYD\nJwOIyMlALW/5BOBSESnvrSsjItHXQdCY40ScBH9EgzwnWBEpFM5A8uhh3AWm8cDSfLz/O+AirxvX\nqdms/xIoIyILgJuAZQCq+hvwEDBORBZ6x6+Uj+MbY46SG1A7NN20RCRRRL4QkaUi8ruItPUaUONF\nZLn3b+n8xpqXsQhaAe8CpYDqItIUuE5Vb83vQYNR1dVA44DXgResDqu+q+pj2T3PZrtlQGAH0ylZ\n1qfghmPM7r0jgBG5Bm6MiYj40J17vwiMUdVLRaQgUBR4AJigqoNFZBAwCLgvPzvPS5gvAecCWwFU\n9VfsVlljjE/cYC8S9BF0PyIlgdNwDUhU9YCq7sB1CR3qbTYU17c+X/KSYONUdU2WZZEbJcMYY7KI\ny8MDSBKROQGP/ll2UxvYDLzv3fn5jogUAyqo6noA7998Dw2Ql14Ef3tlAhWReFwH/WX5PaAxxhwN\nkTzfqbVFVVvmsj4Bd1H7VlWdKSIv4soBIZOXFuxNwJ1AdWAj0MZbZowxvgjRrLJrgbWqmjmA8Re4\nhLtRRCq540glsunznldBW7CqugmI4vvmjDHHm1B0w1LVDSLyt4jUV9U/cDcU/eY9egODvX+/ze8x\n8tKL4G2yGZNAVbPWM4wxJuwEQjmYy63AcK8HwUrchAJxwGferfR/AfkeJi4vNdjAQTMLAxcBf+f3\ngMYYc1RCeCOBdzt+dnXa00Ox/7yUCA7p+ykiw3Ad7Y0xxhdClNyqFUR+xiKoBdhtosYYXwiQECM3\n+eelBrudf2uwcbjxU0PalcEYY45EzA9XCODNxdUUN+A1QIaqRmamOGOMycYxM223qqqIfK2qLSIV\nkDHG5EpC2osgrPJSyZjlDd1njDG+y2zBxsJwhbnNyZWgqmlAB+B6EfkTNzi14Bq3lnSNMb6IkRJs\nriWCWbjbxvI9kszxLF7iKFOooN9hZCuxaAG/Q8hZ+VrBt/FR4QLxfoeQo737o3MMpoyQX7YR4o6B\nbloCkM3U18YY4xuRkI4HG1a5JdhyInJnTitVdUgY4jHGmKDyMt5rNMgtwcbjZm6NjZ/EGHNcEI6N\nGux6VX0iYpEYY0wexUo3raA1WGOMiSZC7EyHnVuCDcloMsYYE1JyDNwqq6rbIhmIMcbkhQDxsZ5g\njTEmWsVGerUEa4yJQTHSgLUEa4yJLYJYicAYY8Il5i9yGX+lp6dz+5XdKVu+Io+9OpwNa9fw7L03\nsGfnDuo0PIm7nnmVAgX8G+tg+bI/6Nurx8HXa1av5P6HH+PmAbdFLIY37jyDs1rXZvOOZFreMBSA\ni089gQd7taVBtbKcOnA485ZvBKB6hZIseLsPy9ZuB2DW0vUMfOnHHPcdakUKxNHz5MpULlUIFIbN\nXceJFYrTvlYiu70xBEYu2cSSDXsiFlOmOwf058exo0lKKsfE6fMPWffGy0N48pH7WbTiH8qUTYp4\nbDmJjfRqCTZqjfzobarVqkfy3t0AvP+//3BhrxvoeNZFvPLEPYz76mPOuaKPb/HVO6E+U2fOBdyX\nQcM61Tn3/MiOCzRs3GLeGDmfd+456+CyJau3cOUTI3llYLfDtl+5fidtbh4WyRAPuqxpRX7buId3\nZq4lXqBgQhwnVijOxOXb+HH5Vl9iynT5Vb3oe/1N3HbjtYcs/2ft3/w8eQJVqlb3KbIcxFA3rVjp\nr3tc2bJhHbOnjOeMS3oCoKosnDWVDt3OA+D08y9nxsQf/AzxED9NmkCt2rWpXj2yU7VNW/wP23bv\nO2TZH39vY7nXSo0WhRPiqJtUlF9W7wAgXSElNcPnqP7Vpv2pJJYufdjyxx68hwcfeybqkllmN61g\nj2hgLdgo9NZzD9P3jkdISXani7t2bKNYiZLEJ7hfV1LFymzdtN7PEA/x5eefccllV/odRlA1K5Zi\n+qu92J28n8eHTmPa4n+CvykEkooVYM/+dHq1qEzVxEL8tX0fn/+6AYCOdUrTukYp1mxP4cuFG6Mm\n8Y4b/R2VKlWm0UlN/A4lW9GRPoOLmRasiNQUkR4BrzuJyKgj3McDoY8stGb9NI5SZZKo16jpvwuz\nHU8zOv7EDhw4wA+jv+PCiy/1O5Rcbdi2lxOufou2twzjvjcn88GgcyhRNDI17DgRqiUWZsrK7Twz\nYRUH0jPoXj+Jn1du45ExK3j6x5Xs2pfGJU0qRCSeYFKSk3lpyLPcff+jfoeSI5Hgj7ztR+JFZH5m\nLhGRWiIyU0SWi8gIETmqP5KYSbBATaBHsI2CyDbBihMVn8Vv82cxc9JY+p7RkmfvuYGFs6bx1rMP\ns3f3LtLT0gBXQihbvqLPkTrjx46habPmlK8QHckhJwdS0w+WE+av2MTKdTuoV+Xw0+Jw2JGSyo6U\nVFZvTwFg3trdVE8szO796Shuyuapq3ZQs3SRiMQTzOpVK/lrzWq6nXoKrZucwPp1azmjYxs2bdzg\nd2hAyEsEtwG/B7x+FvifqtYDtgP9jibWsCUVr8W5VETeEZHFIjJcRLqKyDTv26GViJQRkW9EZKGI\nzBCRJt57O4rIAu8xX0RKAIOBU71ldwQcJ87bX7mA1ytEJClLPIOBIt77h3vx/S4irwHzgGoisidg\n+0tF5APveTkR+VJEZnuP9uH63Prc/hAfTljA+2PncN9/36RJq/bc8+zrnHRKe6aO/w6ACSM/o3Xn\nM8MVwhH58vNPY6I8kFSqCHHeCEw1K5aibpVEVm3YGZFj79qfzvaUNMoXd42hBuWLsX73fkoW/rdC\n16xyCdbt2h+ReIJp2KgxC5evZebCZcxcuIxKlasy9qcZlK8QHV/qrids8P+C7kWkKnAO8I73WoAu\nwBfeJkM5yhldwl2DrQtcBvQHZuNaoB2A83Gtyb+B+ap6oYh0AT4EmgF3A7eo6jQRKQ7sAwYBd6vq\nueBKBACqmiEiHwE9gReArsCvqrolMBBVHSQiA1S1mff+mkB9oK+q3uwty+nneBH3rTZVRKoDY4GG\nWTcSkf7ez0q5SlWP6IMKpu8dD/HcvTcw7OXB1G5wEmdcfLSN+aOXnJzMpIk/8r+XX/fl+EMHncOp\nTaqSVKoIKz7qz5PDfmH77n0MubkLSaWK8NWTF7Hwz82c/+CXdDipKg9f04609AzS05VbX/qR7Vku\nkIXTZwvW07dVFRLihC17D/DhnHVc3rQiVRMLA7B1byofz/enrn5zv15Mn/Yz27ZuoUWj2tw96GGu\n6tXXl1jyKo8N1CQRmRPw+i1VfSvg9QvAvUAJ73VZYIc3FyHAWqDK0cQZ7gS7SlUXAYjIEmCCNxX4\nItwpfw3gEgBVnSgiZUWkFDANGCIiw4GvVHVtkCuZ7wHf4j6wa4H38xjfGlWdkYftugInBsRQUkRK\nqOruwI28X95bAPUaNTvqiYianNKeJqe4xnKlajX53ydjj3aXIVW0aFFWrd3k2/F7D/4+2+Ujf1lx\n2LJvpi7nm6nLwx1Sjtbu3M+zE1cdsmzonHU+RXOo197NvevazIXLIhRJ3ojkebCXLaraMvt9yLnA\nJlWdm9lYI/sLG0f1/3G4E2zgOU9GwOsM79hph73DzVg7WES+B84GZohI19wOoqp/i8hGrxXcGugp\nIvHAXG+Tkar6SDZv3Zt1VwHPCwc8jwPaqmpKbnEYYyIjBL2w2gPni8jZuP/XS+IaaIkBM2pXBY7q\nW9DvCzs/407tM0/5t6jqLhGpo6qLVPVZYA7QANjNv0357LwDfAR8pqrp3qOZ98hMrqkiktuUqhtF\npKF3weuigOXjgAGZL0Sk2RH+nMaYEDraGqyq3q+qVVW1JnAlMFFVewKTgMwuMb1xZ8b55neCfQxo\nKSILcRexenvLb/cujP0KpAA/AAuBNBH5NfAiV4CRuDnEcisPvAUs9EoP2RkEjAImAoEFsYGZcYrI\nb8CNefrpjDEhF+YbDe4D7hSRFbia7LtHE2vYSgSquhpoHPC6Tw7rLsjmvbfmsNussyxMDnjeFHdx\na2kuMd2H+wAzNc6y/gv+vYIYuHwLcEVO+zXGRFYob9RS1cl4uURVVwKtQrXvY+JOLhEZBNyEV24w\nxhzb8tINKxocEwlWVQfjSgzGmGOcADEyqeyxkWCNMccREeKiZDCXYCzBGmNiTmykV0uwxpgY40oE\nsZFiLcEaY2JOjORXS7DGmNhjvQiMMSZMrAVrjDFhYgnWGGPCQLASgTHGhMcRTAnjN0uwxpiYYwnW\nGGPCIm9TwkQDS7DGmJhjLdjj3IH0DFbvSvY7jGx1KVDe7xBy1P68dn6HkKuk4tH7v4xmO737sUew\nW2WNMSZsgszRFzUswRpjYk6M5FdLsMaY2BMj+dUSrDEmxoiVCIwxJiwEKxEYY0zYxEh+tQRrjIk9\nViIwxpgwiZH8SpzfARhjzJGSPDyC7kOkmohMEpHfRWSJiNzmLS8jIuNFZLn3b+n8xmkJ1hgTU9xF\nLgn6yIM04C5VbQi0AW4RkROBQcAEVa0HTPBe54slWGNMbPGGKwz2CEZV16vqPO/5buB3oApwATDU\n22wocGF+Q7UabBR6+JIOFC5aHImLIz4+gfveG8lXrzzN4mkTiC9QgHJVanD1A/+laImSvsW4b98+\nunY+jQP795OWnsZFF1/Kw48+7ls8VRML89AZJxx8XalUIYbO/JsFa3dxe+faFCkQz4Zd+3hm3AqS\nU9MjHt9L13ShYNFixMXFERcfz3Uvf8WGlUsZ/dKjHNiXTGKFKlx07/9RqFjxiMd214D+/DjuB5KS\nyjHhl3kAPD/4ST4e9j5lyyYBcN/DT3B6tzMjHltO8liCTRKROQGv31LVt7Ldn0hNoDkwE6igquvB\nJWERyffgHZZgo9RtL39M8cQyB183PKUDF9x4L/EJCXzz2mDGDXuNC2/O95nLUStUqBBjxk+kePHi\npKam0qVjB7qfcRat27TxJZ61O/Zx44iFAMQJfNqnBVNXbuPRM+vz5rQ1LFy3izMbluPykyvzwcy/\nfYnxmmeHUrTUv7/TUf97kG7X30eNJq1YMPYLfvniHTr3vj3icV3Woxd9rr+J22/qd8jy62+8lRtv\nvSPi8eRJ3jLsFlVtGXRXIsWBL4HbVXVXKHsoWIkgRjRsfRrxCe77sGaj5mzftMHXeESE4sVdays1\nNZW01NSo6TrTvGop1u3ax6bdB6haujAL1+0CYO7fOzm1Tpkg746crf+sovpJpwBQ6+T2LJ02zpc4\n2rQ7lcTS+b6O4wMhToI/8rQnkQK45DpcVb/yFm8UkUre+krApvxGagk2CokIr9xxDYOvPY+p3358\n2Prp339Go7YdfYjsUOnp6bRu0YzqlcvTpWs3WrVu7XdIAHSul8SkZVsBWL01hXa1XPI4rW5ZyhUv\n5EtMIjD8gX68PeBi5o0eAUD5GiewbMYEAH7/eQy7Nq/3JbacfPDO63Tt0JK7BvRnx47tfodzUF56\nEOSxF4EA7wK/q+qQgFUjgd7e897At/mN9bhKsCKSKCI3+x1HMHe+/gWD3h/FLc+/z89fDWP5gpkH\n140Z+grx8Qmc0j3fdfeQiY+PZ+bcBaxYvZY5s2exZPFiv0MiIU5oW6s0P61wCfb/Jqzg/JMq8trl\nJ1G0QDxpGRm+xNVnyCdc/+rX9PjP28z+bjhrFs3mvDufYs53H/P2gIvZn7KX+ISCvsSWnWuu7c+0\neb8z7udZlK9YkScfus/vkA4VigwL7YFeQBcRWeA9zgYGA91EZDnQzXudL8dbDTYRuBl4LXChiMSr\nauSvfOQgsVwFAEqUTqLpaWew5rdfqdesNTNGf8niaRMZ+NLwqDkdB0hMTOS0jp0YN24MjRo39jWW\nVjUSWb55LztSUgH4e8c+Bo38HYAqiYVpXdOfU+ESZd3vtFhiWRq068a6PxbS9tJ+9Hz6PQC2rl3F\nilmTfYktO+XKVzj4vMc119Lnyot9jOZweS0B5EZVp5JzKj79qA/AcdaCxX0T1fG+qWZ7nYw/BhaJ\nSE0ROdgEE5G7ReQx73kdERkjInNFZIqINAhXgPtTktm3d8/B57/PmkKl2vVZMuMnxg9/gxuefZuC\nhYuE6/B5tnnzZnbs2AFASkoKEyf8SP36YftY8qxzvSQmLd9y8HViEdeGEODqllUZtTjytesD+5LZ\nn7zn4POV86ZRrmY99u5wrWzNyGDKJ6/T4pwrIx5bTjZu+LdcMWbUSOo3bORjNIcLTQM2/I63Fuwg\noLGqNhORTsD33utVXjeNnLwF3Kiqy0WkNa4F3CXrRiLSH+gPUKZC5XwFuHvbFt564AYA0tPSOaX7\n+TRq05FHL+9EWuoBXr69FwC1GjXnqnufytcxQmHD+vVcf21v0tPTydAMLrn0cs4+51zf4gEolBBH\ni+qleGHyyoPLOtdL4oImFQGY+uc2xvy+OeJx7d2+lc+euAWAjPR0Gnc+l7otT2PmN0OZ852rsTdo\n342m3S+JeGwAt1zXi+nTprBt6xZaNqrDXYMeYvq0n1myaCEiQrXqNRg85BVfYstWDE3bLcfLPD5w\nsK/bKFVt7CXYR1W1c9Z13uu7geLA/wGbgT8CdlXIu/sjRzUaNNH73hsZ4p8gNK5tVdPvEHJ07hvT\n/Q4hV+3qlfU7hBzd0KqG3yFk6+wu7fh1/tyQpcSmzVvo6EnB/06qli40Ny/dtMLpeGvBZrU34Hka\nh5ZMCnv/xgE7VLVZxKIyxuQqRhqwx10NdjdQIod1G4HyIlJWRAoB5wKo6i5glYhcBq5rh4g0jUi0\nxphsheJW2Ug4rlqwqrpVRKZ5F7NScEk1c12qiDyBu1VuFbA04K09gddF5CGgAPAp8GvkIjfGBIqm\nXjS5Oa4SLICq9shl3UvAS9ksXwVEz43YxhznYiO9HocJ1hgT26KpBBCMJVhjTMyxEoExxoRJbKRX\nS7DGmBgUIw1YS7DGmFgjSIy0YS3BGmNiipuTy+8o8sYSrDEm5liCNcaYMLESgTHGhIGIm3ctFliC\nNcbEHkuwxhgTHlYiMMaYMLESgTHGhIslWGOMCY9YKREcV1PGRJKIbAbWhGh3ScCWoFv5I5pjg+iO\n73iJrYaqlgvRvhCRMbj4gtmiqr4OM2oJNgaIyBy/5xbKSTTHBtEdn8V27DvepowxxpiIsQRrjDFh\nYgk2NrzldwC5iObYILrjs9iOcVaDNcaYMLEWrDHGhIklWGOMCRNLsMYYEyaWYGOIZJlKM+trY0x0\nsQQbI0RE1LsiKSJlATSGr1Bm9+UgIhH/ezxWvqSOlZ/jWGO9CGKMiNwKtAXWAz8BP6hqqr9RHZnM\nLwsROQNoBBQBnlfVfX7E4T0/D9fgWA/MU9W0SMZyJAI+v8pAhqpuCFwe4mNdBOwF4lR1TCj3fTyw\nFmwMEZHLgMuAm4DuQIdYS67gWt4ichbwFDAfuBx43o84AETkbuBOoAXwLNA10rEcCe/zOwf4DnhG\nRMaLSHwYkusA4G6gDPCliJwayv0fDyzBRrHM076AU+cqwGDgImAd8KC3vqIvAeZDwKnsmcCVQElg\nN/BMlvWRiqcacIqqdgb241pr40SkSCTjOBIi0gz3u78QmAjUAooHrD+qz1CcGkA3oAvu7+4n4BcR\nKXA0+z7eWIKNUllO9zIT6ErgaeAaVT1DVVNF5C7gRj/ql/lUwvtXcEniVqCvqq4VkYuBHhGOZz+Q\nJiIf4EovF6tqBnCOl3yjUTLwOnAqMADorqo7RaQDhKQ2L8BmYC3wCNARuFRV04HeInLCUe7/uBEr\n/1MeN7zWQ2Bt8HbgYxEpBvwJ/A6MEpEWInIV0BP43EsKUU1E6gIPiUgt4BPgCmCoqi4XkXa4L4+/\nIhTL5SLSUVU3AStwteB7VfWAiFyLSyxRVYcVkcYichJwAHgAuBfoqKorvdP3R0Wk+lEeowNwo6om\nA0WBu1T1fFVNFpEewHXAnqP7SY4fNuB29CmoqvsBRKQf7jT6MlXdKyLLgLdxtcLHgRSgt6ou8S3a\nI1Pa+/cGXAvsYuBNETkNaAncrapTIhRLDeBh7zP+FigAvCMic4DTgctVdX2EYgnK+4K9ADgJ12od\nBLwLdPMudg0AHlDVfH1BeWdAApwINBWRK4GbgZIiMgFYDLQDrlXVdUf78xwvrBdBFPFaeINxrYY1\nInInrnW1D2gKXA+8AnyIO7UVr6UR1USkqar+6j1vCZyH6zkwGNdKKoT7Yvk9HFfCs8RSW1VXes9v\nBXrhEklmAkkAlqnq6nDFkFdZPwsRaQycD9TFXZTriCsTFAG+UdXx+f38RKS6qv4lIkVxF1JbADNU\n9WMROR9IB37P/OxM3liCjSLe6d0tQDXgLuAU3P/8AO8BGbga5Z3RkAByE9CVqAjwGlBKVS/21rUC\nHgX+AV7NTL7hjMN7fgrujGCaqn7lLbsTuAfooaqTwhVHfolIe+A6Ve3rvW4IXIJrgT+uqmtDcIzK\nwHRcaeAHr7Xc0zvOcODjaO62Fs2sBhtFvNO7V3EXs57HXbm9FLhEVT8DduAueO33Lcg8CEiuZwHD\ncPVCFZEPAVR1FjAXKIxrnYc1Du95T+AcYBvQ3muVoapDcGcJ94pI4XDFkh8icjKuX25XEXkLQFV/\nB2YCzYGnRKTE0Vzg9L5g2uJ+R0+LSHdV3auqb+HKJk0J6KFgjpCq2sOnB67mFZfN8jK40+dPgere\nspuBecBJfsedx5/tZC/+Nt7rcsAXwGhcH95fcN2jIhFLa2CU97wI7uLQ80AfXIv2Ddy8Ub5/bgEx\nN8El0vJAQeA34B1vXTNc/bXhUR6jG/BZwN9YD1yp5BzgXOAboJLfn0UsP6xE4CMRKa6qe7znN+D6\nhMap6rMiUgq4D6iJq7dVAnZqlNbARKSgqh7wnlfA3TzwNNBWVRd7y+Nxyawg8JWqfhuBWE4BHsKV\nV65Q10sgCVfLPA1ojOv29ls4YjkSAS3/6sAE4P9U9U1vXRFgDrAUVzrqr0d4Z5WIFATqqupvItIH\n90Xzp6qeF7DNpbjyVApwu6ouDMGPdtyyBOsT7xT1AlXt53XFugh4GHcRa5Gq9hSREri7nYri/oeK\nyq5YXuK8HNgObMS1hN4CbgOqArep6pqA7Qur6r5wXNDyvphO5t+LVptxPRROxyX3n9X1yIhX1XQR\nSVTVHaGM4Uh5v+eK6rqrnQJsBZ7AtbybqGqKt11BXGlgb+aX1hEepy6uHr4eqI6r69+O6yr3UsB2\npYA0Vd17dD+ZsQTrA3GDtYzAJaA0XGLtDwzE/U+luHvML/X+5yuirr9m1BKROriacQJwmqou8+4G\n6gM0wHUhWhWBOCrhkv15QG3cafR+EbkXqAd8CUyJpuThfU5fAT8CHXB/C0txybAW7uaHkPQ9FZH/\n8/Z/n6q+7tXJbwAmqOrLoTiG+Zdd5PLHAVxifcR7PAi0wrVoL8LVXzuLyEequjsGkqvgLh4txV2I\nawHgtVrfwV1EGhKJi0jq+q6ux7X0RgIVvOXP4W7SuAZ3USdqeJ/TMFwpaJy6fs2Ka13+gbt1N1QX\nmt7A9VTpLyJXqOoPwJPA1d6NKyaELMH6QFV342ps5wDLA06fp3v/1sENOvKQD+EdMXW2424cuAy4\nR9xAIeDKGz8Cd2iYRssSkcQsi77AdcrfClwvIi285cNxFwqP+PQ6Ahbgbha400t8GV5p4CHgZ9yd\nZkdNVVeo6jC8L3ZxI5pVxX3pzwjFMcy/rETgE++0sC6u5voa8APuLq3VuHphF1Vd4VuAQWTpAlUA\nSM+sEYtIG+ADXGJtAdyiqvPCFMeFwLW4C1gpIpLgxaJeN6crcBdsSuN6Z9wUqtPtUAq4wHUG7sr+\nZbjbhm8CBmXWYUN8zDOB/+IGuOmnsXNHYMywBOszLwmMwPVDnIobuWhrJOqVR8u7//2PzBKGiFTB\n3W32PO4iyo3ASFUdH6bjF8G1SsfgWqbLVHWXt64t7gxtD9AGN3rXYxrGmxqOlIgkqGqaiBTy6sTl\ncf2CW+FO5XcAT4art4UXQ3ncScjmcB3jeGYJNgqISFPcsHP3q+vgHRNE5HmgjKr2FZEywCzgf6r6\nqrc+TlUzwtRboKi6AUj64Hot1ML1q90hIpn11xtUdbS3fQGNkrFzvZp1I9wg42d4y2riruq/qKrf\nikhVIF7dLdNhvX3YhI/VYKOA16rqiKvLxpL3gd3exasDuDprZnKVzJJBGJJrfeB2ESkHbMENgDIe\n74IWUB/olZlcvRh8T67i8WrWi3GfXeaFpfuBHzNbq6q6NrM2b8k1dlkL1gTl3ThQX1V/FpGOQGXg\nW68F+TUwX1WfCNg+Lpx9dkWkC65GuQqYDWzADT5dGnff/IJIxJFXmf1+vecH+92KGxaxhqo+mtkv\n11tuLdZjhLVgTa68i0YXADd5Ndf9uH6Uz4ibauVRoIGIJHqnvoQrqQXsfyLwOe420ja42R0+AlKB\ny8QNfxi2OI6E12n/PRHpJiKFgNkicoeInI3rk3uViJxuyfXYZAnW5ErdKErjgWm4mwZScPewv4a7\nKeIpXPesluFMDFkTj5dkv8TdQnyLF9druG5hnSLR5zaPEnCt7FtwF/4uwo0i9gjui2oRXr9cr8Vt\nyfUYYiUCk6MsXbGq4k7LGwPDvQSHuOlDLsMl3QtUdWeY47gGV2tdhus90BjojUta7+MSWqrfV8Xl\n0PEQ6uDGPbgUN8TgLL4B3WsAAAjZSURBVK9l2xN351YnoLmqbvQrXhMelmBNtgL6ZZ6CG7ZuK24Y\nxVtwo95/oarjMrfFje50j6puDWNMt+NqrR/jeg7MBP6DuzFjIO4Gghf8Lg3I/7d3xjFe12Ucf71l\nGhAnWEtLRx1CrhxLwDCGVlTHAVGEEgvMgkmYtNG1Bq0pFkRLjVaJlcb8A0xnhkYR5m7VmAIeU3dC\nWQgMlblsgkkESDXx6Y/nOfp6nXRHv+/9ftzveW233f1+3/t+Pr/vbs893+f7PO+3azNMxpW7nsf/\n+dwJTAE+incPbCx0WdyMt+Z9u2qbTkohSwRJl0RwnQyswcdOd+DaoPcD24HPSGqOw8fjmVhFb8tV\n0DmNTPki3FL7TPxvdwCu47Ab+D6eWVe97oqPue7CbV1+gT8QfAafMNsAtEj6UGGvB/FpqqSPkZ5c\nSZfIBWkW4bJ+jXiAfc7MXpB0Hy452OFZtQefPPtzJfdQmAwbhjucLsOb8KfhdtJX4ZJ7x3Axmarf\njnW0p0n6K6438Sd8z9vMbJ+ke4F+uMD3H/CHhv1x9bGkj5ElguS/iGzxb7gwyj/x2/E55gpZn8Jn\n4/eZy/1VvBVK7jD7djP7qdw3qwXYiIt0C9c0vU7uUjAaWFEL9ctCWWUqft3a8cz0a8Bj5jq/5+FD\nEU9bmAfW0hBEUlkyg02A1wSH9+GtVy14l8BE4NzoeR2D3/butnBcLemW/Cy8DexdeICahGesI/Cy\nQMeQwSRgYi0EVzheVvkYrob2FTN7SdIh3AZooaS7cAnFBWb2fGHoIINrHyUz2OQ4csfXmfjt7D1y\n87s23D9rPx5sl5Y5G1/Yy0Tgu7iz6fzoIZ2JW88Mw7PoR+0kbarLIK7XWlz+71G8Nn0hrpNruO5q\nq5m1Vm2TSa+SATYpZq8teJfAanwm/ohcUGUWfsv7rJk90lvN8JI+gSuMfTHKBf3wXtyhwEoze6ns\nPfSE+CewCu+4GI67O7wXWGdmywvH5TBBnZABto4pBNa3dPSNSpqD92cuAx43s6o62EY980bgWxFk\nTwMGWahmVZPC9RuDP/Q7gD/YmoKPD2+R1IQLZ18JHK6RLoekl8gabB1TqBkujCfaW8xsjdz76Xpg\nhaSHO8Y4q7THByS9CqyS9IqZ3QdUPbjC8es3CbgVb7+ajWu3/gCOlzm+h9dja2LPSe+SAbaOkTQB\nH3WdgTsojJM01MxWxu3uEnz66ED1dglm9mAIo+yp5j6KRCbdACzG3Vd/HQ+x1oZkwt2Eu4O5LUtS\nh2SArTOKqk24GeEsXN7vHfg01vS49b1F0i/NrWCqjpUk2t1TCvXT04FD+DTZ0biu7VHHvjruBBZl\n5lrfZICtEyQ1mBsoHgtVrLfho69/x73BZpiLO08DLpbUaGbPVnHLNUmUBabjzg17gEvxwYHH8YD7\nMmByFbJDVdtoUhNkgK0DJA0EHpB0C67e9EPgCeBVYDAwBmiX9Aj+N/GdDK6vpfBAawjeyXA33np1\nGW6NMzC6HC4DbjBXIUvqnOwiqBMkXY4PCRwClpjZVknn49nrB/EG+H8BN5vZuurttHaRdAnedvXm\njraryPi/jss5/gQ4I7oHshUryQy2XjCzdZIO44IjTbhF83O4c+lOPCsbGPPyGRyCQuY6DrgD2Auc\nLWkzsNnM1ks6E++6uM7CsTavXwKpplVXxIOiucBcSbNjRPMAPnLa38IdNoPDfyiMDy8DZpnZVFyH\n9gpgfOgI3AU0WQ3agSfVJTPYOiMy2VeANZJm4qIuS83sxSpvrZYZDHwEaMY1Z7+Bt7DNwZOUjZVW\nEkv6BpnB1iFm9ivgc7h4ym1mtiFEs5MuMBcWnwHMk3RlZP7LcbPFfVXdXFLT5EOuOkbSm2ptnr+W\nkRsVLgduNbPVVd5OcgqQATZJekB0DdyEPyh8oZpjxEntkwE2SXpIURwnSU5EBtgkSZKSyIdcSZIk\nJZEBNkmSpCQywCZJkpREBtgkSZKSyACbVAxJxyRtk/SkpLWh4nWy55ogaUN8P03SV09w7BBJXziJ\nNZZKWtTd1zsds1rSJ3uwVqOkJ3u6x+TUJgNsUkmOmtkoMxuJK3NdW3xTTo//5sxsvZnddIJDhgA9\nDrBJUjYZYJOy2ASMiMxth6QfAe3AUEnNktoktUemOwhA0mRJT4VS1RUdJ5I0V1KHz9U5ktZJ2h5f\n4/HG/+GRPa+I4xZLekzS7yUtK5zrekk7Jf0Wd3I4IZLmx3m2S7q/U1beJGmTpF3hbYakfpJWFNb+\n/P97IZNTlwywScUJNf8puLg3eCC708xGA0dwoZQmMxuDOwF8WVJ/3KL748D7gbe+zulXAg+Z2UW4\nUPgfcZ3bPZE9L5bUDLwTuAQYhTs0fEDSxbhFzmg8gI/txsf5uZmNjfV2APMK7zXiWrpTgdvjM8wD\nDprZ2Dj/fEnDurFO0gdJNa2kkgyQtC2+34R7fJ0L7DWzrfH6OOBCYEvoy5wBtOH+YM+Y2W6AMBC8\npos1Pgx8FiDGVA9KOqvTMc3x9UT8PAgPuA3AOjN7OdZY343PNFLSN/EyxCCgtfDez8KGe7ekp+Mz\nNAPvKdRnB8fau7qxVtLHyACbVJKjZjaq+EIE0SPFl4DfmNnsTseNwi1YKoGAG83sx53W+NJJrLEa\nmG5m2yXNBSYU3ut8Lou1F5pZMRAjqbGH6yZ9gCwRJL3NVuBSSSPA/cIkXQA8BQyTNDyOm/06v/87\nYEH8br9wEziEZ6cdtAJXF2q750k6G3gYuFzSAEkNeDnif9EA/EXS6cCnO703U9JpsefzcWeIVmBB\nHI+kCyS9sRvrJH2QzGCTXsXM9kcmeI+kN8TLS8xsl6RrcHPGF4HNwMguTtECrJI0DzgGLDCzNklb\nog3qwajDvhtoiwz6MHBV2GrfC2zDrV82dWPLN+DW3HvxmnIxkO8EHgLOAa41s39IugOvzbaHxu5+\nYHr3rk7S10ixlyRJkpLIEkGSJElJZIBNkiQpiQywSZIkJZEBNkmSpCQywCZJkpREBtgkSZKSyACb\nJElSEv8GmCutT8r42l8AAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11acb8f60>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(mul_tfidf_train, mul_y_train)\n",
"pred = clf.predict(mul_tfidf_test)\n",
"score = metrics.accuracy_score(mul_y_test, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(mul_y_test, pred, labels=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])\n",
"plot_confusion_matrix(cm, classes=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.231\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEmCAYAAAAnRIjxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd8FNUWwPHfSUIVhEAIHeldQUCU\nakdQivhAlCJiQRTbU1RUxIIFnr2giA0EVCyoiIgigggiHUR6FVB6kRBqyHl/3AkuIY2Q3dmF8/Wz\nn+zOzM6cbOTsnXPv3BFVxRhjTM6L8jsAY4w5VVmCNcaYILEEa4wxQWIJ1hhjgsQSrDHGBIklWGOM\nCRJLsCakRCSfiHwjIv+IyGcnsZ8uIvJDTsbmFxFpJiLL/Y7D5DyxcbAmLSLSGbgPqA4kAAuAZ1R1\n2knutxtwF9BYVZNOOtAwJyIKVFHVVX7HYkLPWrDmOCJyH/AK8CxQHCgHvAm0y4HdnwWsOB2Sa1aI\nSIzfMZggUlV72OPoAygE7AU6ZrBNHlwC/tt7vALk8dZdBGwE7ge2ApuAHt66J4FDwGHvGDcDTwAj\nA/ZdHlAgxnt9I7AG14peC3QJWD4t4H2NgdnAP97PxgHrpgADgOnefn4A4tL53VLifzAg/quBK4EV\nwE7gkYDtGwIzgN3etm8Aub11U73fJdH7fTsF7P8hYDMwImWZ955K3jHqea9LAduBi/z+f8MeJ/6w\nFqxJrRGQF/gyg20eBS4A6gJ1cEmmX8D6ErhEXRqXRAeLSKyqPo5rFY9W1QKq+l5GgYjIGcBrQCtV\nLYhLogvS2K4I8K23bVHgJeBbESkasFlnoAcQD+QG+mRw6BK4z6A00B94B+gK1AeaAf1FpKK37RHg\nv0Ac7rO7FLgDQFWbe9vU8X7f0QH7L4JrzfcMPLCqrsYl31Eikh/4ABimqlMyiNeEKUuwJrWiwHbN\n+BS+C/CUqm5V1W24lmm3gPWHvfWHVXU8rvVWLZvxJAO1RSSfqm5S1cVpbHMVsFJVR6hqkqp+DCwD\n2gRs84GqrlDV/cCnuC+H9BzG1ZsPA5/gkuerqprgHX8xcA6Aqs5V1d+8464D3gYuzMLv9LiqHvTi\nOYaqvgOsBGYCJXFfaCYCWYI1qe0A4jKpDZYC/gx4/ae37Og+UiXofUCBEw1EVRNxp9W9gE0i8q2I\nVM9CPCkxlQ54vfkE4tmhqke85ykJcEvA+v0p7xeRqiIyTkQ2i8geXAs9LoN9A2xT1QOZbPMOUBt4\nXVUPZrKtCVOWYE1qM4ADuLpjev7Gnd6mKOcty45EIH/A6xKBK1X1e1W9HNeSW4ZLPJnFkxLTX9mM\n6US8hYuriqqeCTwCSCbvyXDojogUwNW13wOe8EogJgJZgjXHUNV/cHXHwSJytYjkF5FcItJKRP7n\nbfYx0E9EiolInLf9yGwecgHQXETKiUgh4OGUFSJSXETaerXYg7hSw5E09jEeqCoinUUkRkQ6ATWB\ncdmM6UQUBPYAe73W9e2p1m8BKh73roy9CsxV1VtwteUhJx2l8YUlWHMcVX0JNwa2H7AN2ADcCXzl\nbfI0MAf4HVgEzPOWZedYE4HR3r7mcmxSjMKNRvgb17N+IV4HUqp97ABae9vuwI0AaK2q27MT0wnq\ng+tAS8C1rkenWv8EMFxEdovItZntTETaAS1xZRFwf4d6ItIlxyI2IWMXGhhjTJBYC9YYY4LEEqwx\nxgSJJVhjjAkSS7DGGBMkNtFEkMTkL6S5ChX3O4w0VYo/4TH/IbNk7Ta/Q8hQmVKF/Q4hXUXz5/E7\nhDStX7+OHdu3ZzY2OMuizzxLNem4C+COo/u3fa+qLXPquNlhCTZIchUqTqWbB/sdRpo+793Y7xDS\ndW73t/wOIUMPPv4fv0NIV9d65fwOIU0XNzk/R/enSfvJUy3TEW8cWDA4syvqgs4SrDEmsohAVLTf\nUWSJJVhjTOSRyOg+sgRrjIk8kmMl3aCyBGuMiTBWIjDGmOAQrERgjDHBIVYiMMaYoLEWrDHGBIPV\nYI0xJjgEKxEYY0zQWInAGGOCQSDaSgQmm7o1Lsd/6pdGgZVb9tJvzGL+U7803RqXo1zR/DR9dgq7\n9x0OeVyb/tpI33tuZfvWLUhUFNd27cENt/Tm+aceZfLE8eTKnZuyZ1Xg2ZeHcGah4E+KMuS+K2h1\nQSW27d5Hg57DAIgtmJcRj7bmrOKF+HPLP3R9+ht27z1I60aV6N+9KcmqJB1J5sG3JvPr4lDcE/Ff\nyUeO8Pwt7ShUrDi9/vcey+dM56s3B6LJyeTJl5+ujz5PsTLlQxpTakMGv8bwD94DVW7ocTO333mP\nr/GkKYKGaUVGlKeR+IJ56NKoHJ3emkn712cQJdDq7OLMX7+bWz6Yy1+7Mp9FKFiiY2J4sP9zfDt1\nHqPHTeajYe+wasVSGje/hLGTZ/P1pJmUr1iFoa+/GJJ4RkxcTLtHPj9mWZ9ODZkyfz1n93iPKfPX\n06eTm2hk8vz1NOw1nAtu/5BeL07gzftahCTGQFM++4DiZ1U6+nr0C4/Rvf/L9B32LfUvb8uE4f5O\nDrRk8R8M/+A9Jk2dwS8z5/H9d9+yetVKX2NKl0jmjzBgCTYMxUQJeXJFER0l5MsVzbaEgyzblMDf\nuw/4Gld88RLUOqcuAGcUKEilytXYsmkTTS66lJgYdzJUp/55bNkUmpbh9EUb2Zlw7GfSulFlRk5c\nDMDIiYtp07gyAIkH/m3xn5E3F6G+Fd2urZtYPGMyjdp0OrpMRDiQuBeAA4kJFIqLD21QqaxYvozz\nzjuf/PnzExMTQ5OmzRk39qvM3xhy3iiCzB5hwEoEYWZrwkGGTVvHj32acSApmV9X7eDXVTv9Dus4\nf234k6V/LKROvQbHLB/z8QhatfNvSr/42Pxs3pkIwOadiRQrnP/ourZNKvPUTc0oVig/1zw2JqRx\njXltAO1u78vBfYlHl13f9zneeuAmcufJS94zCnDf21+ENKbUatSsxdNPPMbOHTvImy8fE7//jrqp\n/r5hw0oE4UFE7haRpSIyKp31F4nIuLTW+eHMvDFcXCOeK16cxiWDppIvVzSt65TwO6xjJCbu5e5b\nutD3qUEUKHjm0eVDXv0f0THRtLmmUwbv9s/Y6auoe/MHXPvk1/Tv3jRkx/1j+iQKFC5KuepnH7N8\n8uj3uf359xnw5a+cf2UHvnz9mZDFlJZq1Wtwz30P0L51Szq0u5JaZ9chJiY8WoLHyEp5IExKBKdD\nC/YOoJWqrvU7kKy4oFIR/tq1n11eJ9akJVupW64w4xZu9jky5/Dhw9xzSxfaXNOJFle2O7r8q09H\nMeXHCXwwehzi4//cW3fto0SRM9i8M5ESRc5g2+59x20zfdFGKpYqTNEz87FjT/Br2msWzeWP6ZNY\n8tsUDh86yIHEvbz1wE1s/XMN5Wu5kku9S67irT49gh5LZrrdeBPdbrwJgKf6P0qp0mV8jigdYVIC\nyMwp3YIVkSFARWCsiDwkIr+KyHzvZ7U0tr9QRBZ4j/kiUtBb/oCIzBaR30XkyWDGvOmfA5xTphB5\nc7k/zfmVirBmW2Im7woNVaXf/XdQsUo1brztrqPLf5k8kXcHv8Sbw0aTL3/+DPYQfN/+tpqul9cC\noOvltRg3YxUAFQNu9VK3cjy5Y6JCklwB2vZ6kAFf/sqTn/9Cjydeo2r9RvR8bij7ExPYun4NAMvn\nTDumA8wv27ZuBWDDhvWMG/sVHa69zueI0iKuRJDZI7O9iLwvIltF5I+AZUVEZKKIrPR+xnrLRURe\nE5FVXh6ol5VIT+kWrKr2EpGWwMXAIeBFVU0SkcuAZ4HUxcI+QG9VnS4iBYADItICqAI0xA0QGSsi\nzVV1aurjiUhPoCdArjOz12GxaOMeJi7ewqd3XMCRZGXZpj18NnsjXS4oS49m5YkrkJsxdzbilxXb\nefyrJdk6RnbNmzWDsZ9/TNUatWh/WSMA7n34CZ597AEOHTzIzZ3aAq6j64lBrwU9nuEPX0Wzc8oS\nVygfq0bdxoAR03nhk5mM7NeG7i3PZsPWPXR5+hsA2jetSufLanL4SDIHDibR7Rl/q0LRMTFc/+Cz\nvNvvDkSiyF+wEF0eHuRrTAA3dO7Irp07icmVi+dffo3CsbF+h5S2nDlLGga8AXwYsKwvMElVB4pI\nX+/1Q0ArXB6oApwPvOX9zDhMDXV3aoiJyDqgAZAPeA33ASmQS1Wri8hFQB9Vbe19oO2BUcAYVd0o\nIi8AHYDd3i4LAM+p6nsZHTdfyapq9+Q6ceF+T67n7Z5cJ+ziJuczf96cHKsbRRUup3maPpjpdge+\nvWuuqmbYSyci5YFxqlrbe70cuEhVN4lISWCKqlYTkbe95x+n3i6j/Z/SLdhUBgCTVbW996FOSb2B\n9631LXAl8JvX0hVcQn07hLEaY9KV5cle4kRkTsDroao6NJP3FE9Jml6STTkVLQ1sCNhuo7fMEqyn\nEJAyQPPGtDYQkUqqughYJCKNgOrA98AAERmlqntFpDRwWFW3hiJoY0wasjZMa3tmLdgTOWIayzI9\n/T+dEuz/gOEich/wUzrb3CsiFwNHgCXAd6p6UERqADO83vG9QFfAEqwxfgneSJUtIlIyoESQ8u98\nI1A2YLsywN+Z7eyUT7CqWt57uh2oGrDqMW/9FLxygareRRpU9VXg1WDFaIw5AcG9bfdYoDsw0Pv5\ndcDyO0XkE1zn1j+Z1V/hNEiwxphTT06MtRaRj4GLcLXajcDjuMT6qYjcDKwHOnqbj8f1zawC9gFZ\nGrRsCdYYE1HcfNsnn2BV9fp0Vl2axrYK9D7RY1iCNcZEFhEkKjwuhc2MJVhjTMTx83LsE2EJ1hgT\ncSzBGmNMMAhWIjDGmGAQxFqwxhgTLJZgjTEmSCzBGmNMMFgN1hhjgsdasMYYEwTWyWWMMUFkJYLT\nXGzBPHRsXt7vMNJUMF8uv0NI37Z1fkeQob0Hk/0OIV25o8PzFns53tgUKxEYY0zQWII1xpggEISo\nqPBsradmCdYYE3kiowFrCdYYE2GsBmuMMcFjJQJjjAmWyGjAWoI1xkQeKxEYY0wQiNiVXMYYEzRW\ngzXGmGCJjAasJVhjTOSxEoExxgSBCETZZC8mu16+4WLy5DsDiYoiKjqG294Yw0/DX2HZjEmICGcU\nLsrVfQZyZtHiIY3r/jt78uP344mLK8akGfMBeHHgAD768H2KFo0D4KHHnuLSFq1CEs+Qx7vQqnlt\ntu1MoEHHZwG45rJzebTXlVSvUJxm3V5g3pL1AFxyfnUG3N2W3LliOHQ4iUde+YqfZ68ISZwAgzpf\nSJ78ZxAVFU1UdDR3vvUVi34ez4/DX2Pb+tXcMXgMZaqdHbJ40rN7925697qVJYv/QER4a+h7nH9B\nI7/DSsU6ucxJ6v6/DzmjUJGjrxt3uIVLut8LwG9ffcjPIwfT5p6nQhpTx+u7ceOtt3Nvr5uOWX7r\n7XfR6677QhoLwIhvfmPI6J95d8ANR5ctXv03193/Dm/0u/6YbXfs3kuHe99m07Z/qFmpJN+82ZtK\nV/QLaby3vjjymL9p8fJV6frkm3z5cmjjyMgD99/L5S2uYNQnn3Ho0CH27dvnd0hpipD8agk2UuQ9\no8DR54cP7PPlG/yCJs3YsH5dyI+bnunzVlOuZJFjli1fuyXNbRcu33j0+ZLVm8iTO9fR1qxf4s+q\n7Nux07Jnzx6m/zKVoe9+AEDu3LnJnTu3z1GlIYJKBJEx1uE0IwgjHrmJt3u3Z874T44un/TBS7zU\npTm///QNF99wj48RHmvYO0O4rEl97r+zJ7t37/I7nEy1v6wuC5dvCGlyFRHef/BGXu/VjlnjPsn8\nDT5Yu3YNccWKcdutN9GoYT3u6HULiYmJfod1HMEl2Mwe4SDoCVZEyovIH0HY740i8sYJbF9YRO7I\n6TiC4aaXP6bX4K/o8sy7zB47inWLZgNwaY/7uG/UVM65pA2zxo7wOUrnhpt6Mn3+Un74ZTbxxUsw\noN9DfoeUoRoVS/D03e248+nQJrler47mrrfH0uO595nx9UjW/j4rpMfPiiNJSSyYP49be/Zixqx5\n5M9/Bi8+P9DvsNIkkvkja/uR/4rIYhH5Q0Q+FpG8IlJBRGaKyEoRGS0i2W7Gh3ULVkRysoRRGEgz\nwYpIdA4e56SldF4VKFyU6k0u569lvx+z/uyL27Bk2g9+hHacYvHFiY6OJioqis7db2LB3Nl+h5Su\n0vGFGf1ST255bARrN24P6bHPjPP+prFFqdX0cjak+puGg1Kly1C6TBnOa3g+AO2v6cCC+fN9jioN\nkjMtWBEpDdwNNFDV2kA0cB0wCHhZVasAu4CbsxtqqBJsjIgMF5HfReRzEckvIv1FZLb3zTFUvKKi\niEwRkWdF5GfgHhEpJiJfeNvOFpEmgTsWkYIislZEcnmvzxSRdSmvAwwEKonIAhF5XkQuEpHJIvIR\nsCh1S1tE+ojIE97zSiIyQUTmisgvIlI9WB/UoQP7OLhv79Hnq+dOJ758FXb8te7oNst/m0Rc2YrB\nCuGEbNm86ejzCeO+plqNWj5Gk75CBfIx5vVe9H99LDMWrgnpsQ/tD/ib7t/HyjnTKF6+SkhjyIoS\nJUpQpkxZVixfDsCUyZOoXqOGz1EdT/j3ctmMHlkUA+TzGnP5gU3AJcDn3vrhwNXZjTVUnVzVgJtV\ndbqIvI9rSb6hqk8BiMgIoDXwjbd9YVW90Fv3Ee7bZJqIlAO+B47+1VU1QUSmAFcBX+G+gb5Q1cOp\nYugL1FbVut5+LwIaesvWikj5DOIfCvRS1ZUicj7wJu6PcAwR6Qn0BCgUXyorn8tx9u7azugnewOQ\nfOQIZ1/chirnNWf0U3eyfeNaJCqKwvGlaH33k9na/8nofXM3Zkyfys4d22lQqyL3932MGdOmsnjR\nQkSEsuXOYuDLg0MWz/DnbqRZ/SrEFS7AqgkDGDBkPLv+SeSlhzoSF1uAMa/14vflf9G292B6Xdec\nSmWL0ffWlvS9tSUAbW5/g2279gY9zr27tjPicXfylHwkibqXtqVawwtZPO0Hxr7+JIn/7GT4I7dQ\nsnINbho0LOjxZOSFl1/jphu7cujQISpUqMiQd973NZ60ZTmBxonInIDXQ1V1aMoLVf1LRF4A1gP7\ngR+AucBuVU0p0G8ESmc30lAl2A2qOt17PhLXLF8rIg/ivjWKAIv5N8GODnjvZUDNgA/0TBEpmGr/\n7wIP4hJsD+DWLMY1S1XXZrSBiBQAGgOfBcSQJ61tvT/eUIBSVc/WLMZwjCIly3H7kG+OW96pf5bL\nzUEz+L3j677Xd+vhQyRO94eHpbl87OTjT78Hvfs9g979PsgRpa1IqXLc886445bXatqCWk1b+BBR\n+urUqcu0GeFb5kmRxU6s7araIL2VIhILtAMqALuBz4C0BnFn698yhC7Bpg5Qca3ABqq6wTsVzxuw\nPrDrMgpopKr7A3cQ+A3mtYzLi8iFQLSq/iEiZfk3YQ8BJqQRV+Bxkji2ZJISTxTuG61uBr+fMSZU\nTqATKxOXAWtVdRuAiIzBNaYKi0iM14otA/yd3QOEqgZbTkRSLge5HpjmPd/utRA7ZPDeH4A7U16I\nSHqJ7kPgY+ADAFXdoKp1vccQIAFI3fINtAWIF5GiIpIHV7JAVffgWtsdveOLiNTJYD/GmCDKwRrs\neuACr09IgEuBJcBk/s1J3YGvsxtrqBLsUqC7iPyOKwe8BbwDLMKd1md0TnI30MDrIFsC9Epnu1FA\nLC7JHkdVdwDTvU6159NYfxh4CpgJjAOWBazuAtwsIgtxpYx2GcRrjAmynBimpaozcZ1Z83C5KApX\n4nsIuE9EVgFFgfeyG2fQSwSqug6omcaqft4j9fYXpXq9HeiUxnbDgGEBi5oCn6vq7gxi6Zxq0ZRU\n618DXkvjfWuBlunt1xgTWjl1IYGqPg48nmrxGlwH+Ek7JS6VFZHXccXpK/2OxRgTZHZX2dBS1bv8\njsEYExquBut3FFlzSiRYY8zpJHzmGsiMJVhjTMSxEoExxgRDzo2DDTpLsMaYiOKmKwzreaqOsgRr\njIk41oI1xpggsRqsMcYEgYiNIjDGmKCJkAZs+glWRM7M6I3eJCjGGBNyURGSYTNqwS7GTSsY+Juk\nvFagXBDjMsaYdEVIfk0/wapq2VAGYowxWSEC0adSDVZErgMqquqzIlIGKK6qc4MbWmRLOqJsSUh9\n15rwkDcmjMcQRoXV/SePs3xL+N3GOkW4dvwEI6pIGUWQ6b80cbfGvhjo5i3ah7tDgDHG+CKnbtsd\nbFlpwTZW1XoiMh9AVXeezH3CjTHmZAgQHS4ZNBNZSbCHRSQK775aIlIUSA5qVMYYk54Tuy23r7JS\njBsMfAEUE5EncffTGhTUqIwxJgOnTIlAVT8Ukbm4OzACdFTVP4IbljHGpE04xUYRANHAYVyZIIy7\noI0xp4NTpkQgIo/i7tRaCneP8I9E5OFgB2aMMWnJSnkgXPJvVlqwXYH6qroPQESeAeYCzwUzMGOM\nSc+pNIrgz1TbxeBua2uMMb6IlBJBRpO9vIyrue4DFovI997rFriRBMYYE3ICREgfV4Yt2JSRAouB\nbwOW/xa8cIwxJhMRNA42o8le3gtlIMYYk1XhOu9CapnWYEWkEvAMUBPIm7JcVasGMa7T2oCWlTmQ\nlEyyQrIqg35aS+lCebj+3JLkiYli577DfDDrLw4khfaCurtvv4UfJownrlg802YtAODrLz/nf88O\nYMXypfww5VfOrdcgZPEM6X89rZrVYtvOvTToNBCAay6ry6M9W1K9QnGa3fAS85ZuAKBIofx89L+b\nqF+zHCO/mcl///dFyOIEyJcrih4Ny1CmcF5U4f2ZG6lfthB1SxckKVnZmnCI92ZuYP9hfy+SrFa5\nPAULFCQ6OpqYmBimz5zjazxpOVVKBCmGAU8DLwCtgB7YpbJB98rUP0k8dOTo6671SjFm0RZWbt9H\no7MKc1nVooxbsi2kMV3XpTs333YHvXvedHRZjRq1GDbqU+6/546QxgIw4ptZDPn0F959suvRZYtX\nbeK6B97njUeuPWbbAweTeOqt8dSsVJJalUqEOlS61C/FH5v28ub09URHCbmjhbybo/h84SaSFTrW\nKUHrmvF8tnBzyGNLbcKPk4mLi/M7jAzlVIlARAoD7wK1cX1MNwHLgdFAeWAdcK2q7srO/rNy0UB+\nVf0eQFVXq2o/3OxaJoTiC+Zm5fZ9ACzbupdzS2d4w4mgaNy0GbGxRY5ZVrV6DapUrRbyWACmz1/N\nzn/2HbNs+botrPxz63Hb7jtwiF8XrOHAodBPIZk3JoqqxQowdc1OAI4kK/sPJ7N4816S1W2zesc+\nYvPnCnlskUjEDdPK7JFFrwITVLU6UAdYCvQFJqlqFWCS9zpbstKCPSju62K1iPQC/gLis3tAkzkF\n7mrqbhjxy9pdTF+7m017DnJOyQL8vmkv55Y5k9h8dju1SFGsQG4SDiZx8/llKBubjz937mfU3L84\ndESPbtOsYhFmrd/tY5SOiNCmVQtEhJtvvY2bb+3pd0hpyokGrHdbrObAjQCqegg4JCLtgIu8zYYD\nU4CHsnOMrPwr/S9QALgbV4sthGtGB42IlAfGqWrtLG7/BLBXVV8QkerAJ7g81UFVVwdsdxFwSFV/\nzemYc9KLU9bxz4EkCuSJ5u6mZ7El4RAj5v7NtXVKcGWNYvy+KYGkZM18RyYsREcJZ8XmY9Tcv1iz\nYz+d65XiqprxfLloCwCta8ZzJFmZsc7/BPvTz9MpVaoUW7dupXXLy6lWvTpNmzX3O6zjZLFEECci\ngUXkoao6NOB1RWAb8IGI1MFdQHUP7oYCmwBUdZOIZLtBmZXJXmZ6TxP4d9LtcHY18LWqPp7GuouA\nvcBxCVZEYlQ1KcixZck/B1wYew8eYeHfCZSPzcePK3fw+rT1AMQXyE3tEgX9DNGcgJ37DrNr32HW\n7NgPwOwNu7mqhvs326RCLHVKF+T5n8Lj2p1SpUoBEB8fT9ur2zN79qywS7CCZHWyl+2qmlGvawxQ\nD7hLVWeKyKucRDkgvQOkSUS+xJsDNi2qek1OBpKGaBF5B2iMK0u0w1222xPIDawCuqVcwgsgIlcC\n9wJHRKS5ql4csK480Mtb1xW4C7gZ2AmcC8wTkQS8lrD3nj+A1qq6znvP3d6xZwJ3qOq/vVA5JHe0\nG+N3MCmZ3NFCjeJnMH7pNgrkiWbvwSMI0Kp6HL+syVbN3fhgz4Ekdu47TImCediccJCaxQvy956D\n1C5ZgFY1ijFo0upjygV+SUxMJDk5mYIFC5KYmMiPE3/gkX79/Q7reDk318BGYGNAI/JzXILdIiIl\nvdZrSeD4on4WZdSCfSO7O80hVYDrVfVWEfkU+A8wRlXfARCRp3EJ8vWUN6jqeBEZQkCSDFi3LvU6\nEbkZqApcpqpHvFLDcUSkBtAJaKKqh0XkTaAL8GGq7XrivgAoEFcyW790wbwx3HaBu99kVBTMWb+H\nJVsSubhyEZpXjAVgwd8JzPgz9KeTt/boyvRffmbnju2cXa08Dz3Sn9jYIvR94F52bN9G5w7tqH1O\nHT77anxI4hn+zA00a1CZuMIFWDX+SQa8/R279uzjpQf+Q1xsAca8ehu/r9hI2zvdHY6WfdOfgmfk\nJXeuGNpcdA6te7/JsrVbQhLryLl/0bNRWWKihW17D/Hebxvpf0VlckUJfS6uCMDq7fv4cM5fIYkn\nLVu3bKFTh/YAJB1JotN1nWlxRUvf4slITowiUNXNIrJBRKqp6nLgUmCJ9+gODPR+fp3dY2R0ocGk\n7O40h6xV1QXe87m4IRO1vcRaGFcX/j4HjvNZFlqilwL1gdneHzYfaXyrefWdoQDxlWpnq0myI/Ew\nz046/nRx8qqdTF61Mzu7zDHvfDAyzeVXtb06xJE43R/9MM3lYyf/nuby6m2eCmY4Gdqw+wBP/bDq\nmGV9xy33KZq0VahYkVnzFvodRqZy+JYxdwGjvNtgrcENQ40CPvUaYOuBjtndeTh3RR8MeH4El9SG\nAVer6kIRuZF/e/rSJCK9gVu9l1ems1ngbUKTOHboWsqFFQIMV1WbptGYMJBTFxp4jbi06rSX5sT+\nI23y7ILAJhHJhTtFz5CqDlZf5Z+4AAAgAElEQVTVut7jb1xHXUa9Q+twRW9EpB5QwVs+CeiQ0pso\nIkVE5Kzs/xrGmJMRJZk/wkGWE6yI5AlmIFn0GK6DaSKwLBvv/wZoLyILRKRZGuu/AIqIyALgdmAF\ngKouAfoBP4jI797xs1dkNcacFDehtmT6CAdZmYugIfAebvxrOW+82C2qelewglLVdbhL11JeB3ZY\nvZXG9k+k9TyN7VYA5wQs+iXV+v246RjTeu9o3OVzxhifRUfIuXdWwnwNaA3sAFDVhdilssYYn7jJ\nXiTTRzjISidXlKr+marJnePjP40xJqsipAGbpQS7wSsTqIhE44Y1rAhuWMYYkzaRLF/J5busJNjb\ncWWCcsAW4EdvmTHG+CJMKgCZyspcBFuB60IQizHGZEmENGCzNIrgHdKYk0BVw3MeM2PMKU3glCoR\n/BjwPC/QHtgQnHCMMSYTYXQhQWayUiI4ZuyniIzADbQ3xhhfCJGRYbMzF0EFwC4TNcb4QoCYCBmn\nlZUa7C7+rcFG4eZPzdFJaY0x5kSEy6WwmckwwXr34qqDm/AaIFlV/Z8Z2Bhz2jplbtutqioiX6pq\n/VAFZIwxGZLIGUWQlUrGLG/qPmOM8V1KCzYSpivM6J5cKTcBbArcKiKrcZNTC65xa0nXGOOLCCnB\nZlgimIWbfNqf+4FEuDy5hKrF8ma+oQ/OyBvGN7LIF953y122PnxvNrkr8ZDfIaQp528xL0SdAsO0\nBEBVV4coFmOMyZRI5MwHm1GCLSYi96W3UlVfCkI8xhiTqXCZ7zUzGSXYaNydWyPjNzHGnBaEU6MG\nu0lV/bvPsTHGpCNShmllWoM1xphwIpwadzTIkfuCG2NMjpJT4FJZVd0ZykCMMSYrBIiO9ARrjDHh\nKjLSqyVYY0wEipAGrCVYY0xkESRiSgSR0hlnjDFHiUimjyzuJ1pE5ovIOO91BRGZKSIrRWS0iOQ+\nmTitBRumko8c4aWeV1OoWHFuHfguK+f9ytdvDuRI0iHKVK3NdQ8OJDrG3z9ftcrlKVigINHR0cTE\nxDB95pyQHn/Iw+1p1bga23Yl0uCG1wGILZiPEU914qwShflz82669v+E3QkH+O/1TenUog4AMdFR\nVD+rGGVbP8euhP1Bj7NckXwMaFvz6OvShfPyzrR1jJ7zFx3qlaJDvdIcUeXX1TsZPGVN0ONJ7b+9\ne/Lj9+OJK1aMyTPmH7PurddfYsBjD7No9V8ULRoX8tjSk4Pt13uApcCZ3utBwMuq+omIDAFuBt7K\n7s6tBRumpn4+jOJnVQIgOTmZj559gBsef5WHhk2gSPHSzP5+jM8ROhN+nMzMuQtCnlwBRoyfT7v7\nhx+zrE/X5kyZu4azr3+FKXPX0KdrcwBe/ngaF/QYzAU9BtP/7R/4ZcG6kCRXgPU799N92Fy6D5tL\nj+FzOXA4mZ9XbKdeucI0rxJHtw/m0OW9OXw0y597iXbq3I1Rn39z3PK/Nm5g6uRJlC5TzoeoMiA5\n04IVkTLAVcC73msBLgE+9zYZzklOdmUJNgzt3rqJJb9N5oLW1wKwb88uYnLnJr5sBQCqNmjK7z9P\n8DPEsDB94Tp27jk2SbZuVp2R380DYOR382jTrMZx77v2snP49MffQxJjag3OiuWv3fvZvOcg15xb\nkhG/refwETfb1K59h32J6YImzYiNjT1u+ROPPEC/J58LuzGnKcO0MnsAcSIyJ+DRM9WuXgEeBJK9\n10WB3d40rQAbgdInE6sl2DD05RtP06bXQ4i4P88ZhYpwJCmJ9ctcUlj483fs3rrJzxAB14po06oF\njRvW5713hvodDgDxsQXYvGMvAJt37KVYbIFj1ufLk4vLz6/CV1MW+xEel9coxsSlWwEoG5ufOmUL\n8W63c3nz+jrUKBE+UzV+P/4bSpQsRa2zz/E7lDRJFh7AdlVtEPA4+j+piLQGtqrq3FS7Te2k5lqM\nmAQrIuVFpHPA64tSCtMnsI9Hcj6ynLX4158oWLgoZaudfXSZiHBD/1f56o1nePm29uTNfwZR0f6X\nz3/6eTozZs/jq3Hf8fZbg5n2y1S/Q8rUVU2qMWPR+pCVBwLFRAlNK8cxadk2wF1PXzBPDLeMmM8b\nU9bwdLvjW9t+2LdvH6+9OIgHHnnc71DSJZL5IxNNgLYisg74BFcaeAUoLCIp/7jKAH+fTJwRk2CB\n8kDnzDbKRJoJVpyw+CzW/jGXP36dxFOdmvPhU/ewct4MRj59H+Vr1+PuN0bz37e/pGKdhsSVKe93\nqJQqVQqA+Ph42l7dntmzZ/kcEWzdtZcSRV2rtUTRAmzbtfeY9R0vO4fPfCoPNKpYhOVbEo6WArYl\nHGTKiu0ALNmUQLJC4Xy5fIkt0J9r17D+z3Vc1vQ8Gp5dlU1/b+SKCy9g65bNfocGnFCJIF2q+rCq\nllHV8sB1wE+q2gWYDHTwNusOfH0ysQYtqXgtzmUi8q6I/CEio0TkMhGZ7g2BaCgiRUTkKxH5XUR+\nE5FzvPdeKCILvMd8ESkIDASaecv+G3CcKG9/xQJerxKRuFTxDATyee8f5cW3VETeBOYBZUVkb8D2\nHURkmPe8mIh8ISKzvUeTYH1urXs+wBOfT6f/6Knc0P9VqtRrRNd+L5Gwy/1DTDp0kJ8+epsm7a4P\nVghZkpiYSEJCwtHnP078gVq1avsaE8C305bRtZW7m1HXVvUY98uyo+vOPCMPTeuW55tflvoS2+U1\n44+WBwCmrtxOg7Nc7bNsbD5yRQu79/tThw1Uo1ZtFq3ayKxFK5i1aAUlS5Xh+59/I754Cb9D80iW\n/sumh4D7RGQVrib73slEGuzzzMpAR6AnMBvXAm0KtMW1JjcA81X1ahG5BPgQqAv0AXqr6nQRKQAc\nAPoCfVS1NbgSAYCqJovISKALrol/GbBQVbcHBqKqfUXkTlWt672/PFAN6KGqd3jL0vs9XsUN3Zgm\nIuWA74Hjzue8InpPgNjipU7og8rM5E/eYfGvk1FNpkm7LlSp1zhH93+itm7ZQqcO7QFIOpJEp+s6\n0+KKliGNYfgT19KsbgXiCudn1ZgHGPDeT7wwciojn7qO7lfVY8OWf+jy2CdHt2/bvCaTZq1i34HQ\nJ7E8MVE0LB/LoAkrji775vfNPHplNUbe1ICkI8kM+HZ5yOMCuP3mbsyYNpWdO7ZTv2ZF7u/7GJ1v\n6OFLLFmVk/1uqjoFmOI9XwM0zKl9i2pO3y/H27FLYBNVtYr3+kPge1UdJSIVgTG4AvJ/vF8KEdkA\n1AZuB9oDo4AxqrrRS6ipE2wfVW0tImWBr1W1noh8AoxU1ePqsyKyV1ULBMQ3WVUrpLO+A9BaVW8U\nka0cW4spBlRX1YT0fv+y1c/W+4ee1NlF0PRqXNHvENIVe1E/v0PIUN02l/sdQrrG9GrkdwhpanlR\nIxbOn5tjKbFq7br6+qcTMz9urfi5qtogp46bHcFuwR4MeJ4c8DrZO3bSce9wd6wdKCLfAlcCv4nI\nZRkdRFU3iMgWrxV8PtBFRKKBlB7CsaraP423JqbeVcDzwDsWRgGNVDX0PSPGmOOE2cixdPndsTMV\nd2qf0iLdrqp7RKSSqi5S1UHAHKA6kABkNI7lXWAk8KmqHvEedb1HSnI9LCIZ9SJsEZEaXodX+4Dl\nPwB3prwQkbon+HsaY3JQEGuwOcrvBPsE0EBEfsd1YnX3lt/rdYwtBPYD3wG/A0kisjCwkyvAWNw9\nxD7I4HhDgd9FZFQ66/sC44CfgMCBpnenxCkiS4BeWfrtjDE5LidGEYRK0EoEqroOV09NeX1jOuva\npfHeu9LZbeq7LEwJeF4H17m1jHSo6kO4XsIUtVOt/5x/L5MLXL4d6JTefo0xoRUm+TNT/o9WzwEi\n0hfXMdbF71iMMcEXLiWAzJwSCVZVB+JKDMaYU5wAEXJT2VMjwRpjTiMiREVIjcASrDEm4kRGerUE\na4yJMK5EEBkp1hKsMSbiREh+tQRrjIk8NorAGGOCxFqwxhgTJJZgjTEmCNwtYSIjw1qCNcZElqzd\nEiYsWII1xkQcS7DGGBMU4TMdYWYswRpjIo61YE9zBXPF0LRsUb/DiDiNrr3S7xAydGWdcLnx3/Ei\n5eqmkyXYpbLGGBM0GdygNKxYgjXGRJwIya+WYI0xkSdC8qslWGNMhBErERhjTFAIViIwxpigiZD8\nagnWGBN5IqVEEOV3AMYYc6JEMn9kvg8pKyKTRWSpiCwWkXu85UVEZKKIrPR+xmY3TkuwxpiII1l4\nZEEScL+q1gAuAHqLSE2gLzBJVasAk7zX2WIJ1hgTUVwnl2T6yIyqblLVed7zBGApUBpoBwz3NhsO\nXJ3dWK0Ga4yJLEGYrlBEygPnAjOB4qq6CVwSFpH47O7XEmyY2fz3Rp7o04sd27YiUVG0v6471/e4\nnRVLFzGw333sS0ykZJmyDHj5HQoUPNPXWKtVLk/BAgWJjo4mJiaG6TPn+BZL2cJ5eaxVtaOvSxbK\nw7DfNhB3Rm4aVYjlcLKy6Z8DDJq4isRDR0Ie37OdmpMn/xlIVDTR0dHcM/Rr9u3Zzcgn72bX5o3E\nlihD1ydeJ3/BQiGP7d7etzJxwnjiihXj598WADDo6ceZMP4boqKiiIuL59W33qVEyVIhjy09Wcyv\ncSIS+D/lUFUdety+RAoAXwD3quqenOxAsxJBmImJieHeR57ms4mz+OCLiXw+4l3WrFzG033vpveD\nj/PJhF+5uEVrRrzzmt+hAjDhx8nMnLvA1+QKsGH3AXp+vJCeHy+k1ycLOXg4mWmrdzJ3w25uGrWA\nWz9ayIZdB+jcoIxvMfZ6eRT3vTeOe4Z+DcBPHw2hcr3GPDTqJyrXa8zkj4b4Elenzjfw8Rfjjll2\nx933M/nXeUyaNofLW17JS4Oe8SW2dGWtCLtdVRsEPNJKrrlwyXWUqo7xFm8RkZLe+pLA1uyGaQk2\nzMTFl6B67boAnFGgIOUrV2Xb5k2sX7uKeg2bANCw6cVMnvCNn2GGtXplC/H3PwfYknCQOev/IVnd\n8qWbEyhWILe/wQVYMv1HGrS8BoAGLa9h8bSJvsTRqEkzCsce21Fe8Mx/z472JSaG2ch+IUoyf2S6\nF9dUfQ9YqqovBawaC3T3nncHvs5upJZgw9jfG/9k+eJF1Kpbn4pVazD1x/EATBr/FVs2/eVzdK6j\noU2rFjRuWJ/33jmuceCbi6vE8dOK7cctb1Urnll/7vIhIkCEdx64kVd6tuW3bz4GIGHnds4s6sp7\nZxaNZ++uHf7Elo7nnnqMejUr8sVnH/Pgo4/7Hc5RWWm8ZvHroAnQDbhERBZ4jyuBgcDlIrISuNx7\nnS2nVYIVkcIicoffcWTFvsS9PHTHDdz32LMUKHgm/Qe9wWcj3qVb2wvZl7iXXLly+R0iP/08nRmz\n5/HVuO94+63BTPtlqt8hERMlNK5YhJ9XHpusujQozZFk5cflxyfeUOj9xqfc+85Ybhn0Pr9+NZI1\nC2f5EseJeLj/AOYtWcN/Ol7P+0Pf9DucY+VAhlXVaaoqqnqOqtb1HuNVdYeqXqqqVbyfO7Mb5mmV\nYIHCwHEJVkSifYglXUmHD/PQHTfQsm1HLmnZFoDylaryxodfMmLsz7Ro04HS5Sr4HCWUKuU6PeLj\n42l7dXtmz/Y/aTQsX5iV2xLZtf/w0WUtqhfjggpFeOb7lb7FVSiuOAAFYuOo3bQF65cupGCROPbs\ncOW9PTu2UiA2PCdob9/xOr4d+6XfYRwjJ0oEoXC6JdiBQCXvVGC2dxXHR8AiESkvIn+kbCgifUTk\nCe95JRGZICJzReQXEakerABVlQF976R8pap0ueXOo8t3bt8GQHJyMu8Pfp7/dO4RrBCyJDExkYSE\nhKPPf5z4A7Vq1fY1JoBLqhbjp4BW6nlnFea6BqXpN24pB5OSfYnp0P59HNi39+jzFXN+oUSFqtRs\nfClzJrh+lTkTxlCzyWW+xJeWNav//TL6/rtxVK5SLYOtQy+HSgRBd7oN0+oL1FbVuiJyEfCt93qt\nNw4uPUOBXqq6UkTOB94ELkm9kYj0BHoClChVNlsBLpzzG+O/HE3lajXpfFVTAHr36c/6dav5fMS7\nAFx0RRvadOyarf3nlK1bttCpQ3sAko4k0em6zrS4oqWvMeWJiaJ+2UK8/NPqo8vuvrACuaKjeP7q\nWgAs2ZzAK5PXhDSuhF3bGf7Y7QAkHznCuZe2ofr5F1K2+jmMfPIuZo//lMLFS9HtiTdCGleKXjd1\n5ddpU9m5Yzvn1qjAAw/3Z9IP37Fq1QqioqIoU7Yc/3t5sC+xpSmCbtstqup3DCHjJdFxqlrbS7CP\nq+rFqdd5r/sABYAXgG3A8oBd5fEur0tXzbPP1Q/HTsnZXyCH1C4b+rGWWXXlm7/6HUKGwvmeXN3O\nzd6XerC1uPACFs6fm2Mpsc659XX85BmZblcmNs9cVW2QU8fNjtOtBZtaYsDzJI4tmeT1fkYBu1W1\nbsiiMsZkKEIasKddDTYBKJjOui1AvIgUFZE8QGsAVd0DrBWRjuDGzolInZBEa4xJU07MphUKp1UL\nVlV3iMh0rzNrPy6ppqw7LCJP4a5FXgssC3hrF+AtEekH5AI+ARaGLnJjTKBImQ/2tEqwAKraOYN1\nrwHHXYOqqmsBf3twjDFHRUZ6PQ0TrDEmsoVTCSAzlmCNMRHHSgTGGBMkkZFeLcEaYyJQhDRgLcEa\nYyKNIBHShrUEa4yJKO6eXH5HkTWWYI0xEccSrDHGBImVCIwxJghEICoy8qslWGNMBLIEa4wxwWEl\nAmOMCRIrERhjTLBYgjXGmOCIlBLBaXXLmFASkW3Anzm0uzjAn/tNZy6cY4Pwju90ie0sVS2WQ/tC\nRCbg4svMdlX1dZpRS7ARQETm+H1vofSEc2wQ3vFZbKe+0+2WMcYYEzKWYI0xJkgswUaGoX4HkIFw\njg3COz6L7RRnNVhjjAkSa8EaY0yQWII1xpggsQRrjDFBYgk2gkiqW2mmfm2MCS+WYCOEiIh6PZIi\nUhRAI7iHMq0vBxEJ+f+Pp8qX1Knye5xqbBRBhBGRu4BGwCbgZ+A7VT3sb1QnJuXLQkSuAGoB+YAX\nVfWAH3F4z9vgGhybgHmqmhTKWE5EwOdXCkhW1c2By3P4WO2BRCBKVSfk5L5PB9aCjSAi0hHoCNwO\ntACaRlpyBdfyFpFWwDPAfOBa4EU/4gAQkT7AfUB9YBBwWahjORHe53cV8A3wnIhMFJHoICTXO4E+\nQBHgCxFplpP7Px1Ygg1jKad9AafOpYGBQHvgb+BRb30JXwLMhoBT2ZbAdcCZQALwXKr1oYqnLHCe\nql4MHMS11n4QkXyhjONEiEhd3N/+auAnoAJQIGD9SX2G4pwFXA5cgvv/7mfgVxHJdTL7Pt1Ygg1T\nqU73UhLoGuBZ4AZVvUJVD4vI/UAvP+qX2VTQ+ym4JHEX0ENVN4rINUDnEMdzEEgSkWG40ss1qpoM\nXOUl33C0D3gLaAbcCbRQ1X9EpCnkSG1egG3ARqA/cCHQQVWPAN1FpOpJ7v+0ESn/KE8bXushsDZ4\nL/CRiJwBrAaWAuNEpL6IXA90AT7zkkJYE5HKQD8RqQB8DHQChqvqShFpjPvyWB+iWK4VkQtVdSuw\nClcLflBVD4nITbjEElZ1WBGpLSJnA4eAR4AHgQtVdY13+v64iJQ7yWM0BXqp6j4gP3C/qrZV1X0i\n0hm4Bdh7cr/J6cMm3A4/uVX1IICI3Iw7je6oqokisgJ4B1crfBLYD3RX1cW+RXtiYr2ft+FaYNcA\nb4tIc6AB0EdVfwlRLGcBj3mf8ddALuBdEZkDXApcq6qbQhRLprwv2HbA2bhWa1/gPeByr7PrTuAR\nVc3WF5R3BiRATaCOiFwH3AGcKSKTgD+AxsBNqvr3yf4+pwsbRRBGvBbeQFyr4U8RuQ/XujoA1AFu\nBd4APsSd2orX0ghrIlJHVRd6zxsAbXAjBwbiWkl5cF8sS4PRE54qloqqusZ7fhfQDZdIUhJIDLBC\nVdcFK4asSv1ZiEhtoC1QGdcpdyGuTJAP+EpVJ2b38xORcqq6XkTy4zpS6wO/qepHItIWOAIsTfns\nTNZYgg0j3uldb6AscD9wHu4fP8D7QDKuRnlfOCSAjAQMJcoHvAkUUtVrvHUNgceBv4DBKck3mHF4\nz8/DnRFMV9Ux3rL7gAeAzqo6OVhxZJeINAFuUdUe3usawH9wLfAnVXVjDhyjFDADVxr4zmstd/GO\nMwr4KJyHrYUzq8GGEe/0bjCuM+tFXM9tB+A/qvopsBvX4XXQtyCzICC5tgJG4OqFKiIfAqjqLGAu\nkBfXOg9qHN7zLsBVwE6gidcqQ1Vfwp0lPCgieYMVS3aISD3cuNzLRGQogKouBWYC5wLPiEjBk+ng\n9L5gGuH+Rs+KSAtVTVTVobiySR0CRiiYE6Sq9vDpgat5RaWxvAju9PkToJy37A5gHnC233Fn8Xer\n58V/gfe6GPA5MB43hvdX3PCoUMRyPjDOe54P1zn0InAjrkU7BHffKN8/t4CYz8El0nggN7AEeNdb\nVxdXf61xkse4HPg04P+xzrhSyVVAa+AroKTfn0UkP6xE4CMRKaCqe73nt+HGhEap6iARKQQ8BJTH\n1dtKAv9omNbARCS3qh7ynhfHXTzwLNBIVf/wlkfjklluYIyqfh2CWM4D+uHKK53UjRKIw9UymwO1\nccPelgQjlhMR0PIvB0wCXlDVt711+YA5wDJc6ainnuCVVSKSG6isqktE5EbcF81qVW0TsE0HXHlq\nP3Cvqv6eA7/aacsSrE+8U9R2qnqzNxSrPfAYrhNrkap2EZGCuKud8uP+QYXlUCwvcV4L7AK24FpC\nQ4F7gDLAPar6Z8D2eVX1QDA6tLwvpnr822m1DTdC4VJccp+qbkRGtKoeEZHCqro7J2M4Ud7fuYS6\n4WrnATuAp3At73NUdb+3XW5caSAx5UvrBI9TGVcP3wSUw9X178UNlXstYLtCQJKqJp7cb2YswfpA\n3GQto3EJKAmXWHsCd+P+USnuGvMO3j++fOrGa4YtEamEqxnHAM1VdYV3NdCNQHXcEKK1IYijJC7Z\ntwEq4k6jD4rIg0AV4Avgl3BKHt7nNAb4EWiK+39hGS4ZVsBd/JAjY09F5AVv/w+p6ltenfw2YJKq\nvp4TxzD/sk4ufxzCJdb+3uNRoCGuRdseV3+9WERGqmpCBCRXwXUeLcN1xNUH8Fqt7+I6kV4KRSeS\nurGrm3AtvbFAcW/5/3AXadyA69QJG97nNAJXCvpB3bhmxbUul+Mu3c2pjqYhuJEqPUWkk6p+BwwA\nunoXrpgcZAnWB6qagKuxXQWsDDh9nuH9rISbdKSfD+GdMHV24S4c6Ag8IG6iEHDljR+B/2qQZssS\nkcKpFn2OG5S/A7hVROp7y0fhOgpP+PQ6BBbgLha4z0t8yV5poB8wFXel2UlT1VWqOgLvi13cjGZl\ncF/6v+XEMcy/rETgE++0sDKu5vom8B3uKq11uHrhJaq6yrcAM5FqCFQu4EhKjVhELgCG4RJrfaC3\nqs4LUhxXAzfhOrD2i0iMF4t6w5w64TpsYnGjM27PqdPtnBTQwXUFrme/I+6y4duBvil12Bw+Zkvg\nedwENzdr5FwRGDEswfrMSwKjceMQp+FmLtoRinrlyfKuf1+eUsIQkdK4q81exHWi9ALGqurEIB0/\nH65VOgHXMl2hqnu8dY1wZ2h7gQtws3c9oUG8qOFEiUiMqiaJSB6vThyPGxfcEHcqvxsYEKzRFl4M\n8biTkG3BOsbpzBJsGBCROrhp5x5WN8A7IojIi0ARVe0hIkWAWcDLqjrYWx+lqslBGi2QX90EJDfi\nRi1UwI2r3S0iKfXX21R1vLd9Lg2TuXO9mnUt3CTjV3jLyuN69V9V1a9FpAwQre6S6aBePmyCx2qw\nYcBrVV2Iq8tGkg+ABK/z6hCuzpqSXCWlZBCE5FoNuFdEigHbcROgTMTr0AKqAd1SkqsXg+/JVTxe\nzfoP3GeX0rH0MPBjSmtVVTem1OYtuUYua8GaTHkXDlRT1akiciFQCvjaa0F+CcxX1acCto8K5phd\nEbkEV6NcC8wGNuMmn47FXTe/IBRxZFXKuF/v+dFxt+KmRTxLVR9PGZfrLbcW6ynCWrAmQ16nUTvg\ndq/mehA3jvI5cbdaeRyoLiKFvVNfgpXUAvb/E/AZ7jLSC3B3dxgJHAY6ipv+MGhxnAhv0P77InK5\niOQBZovIf0XkStyY3OtF5FJLrqcmS7AmQ+pmUZoITMddNLAfdw37m7iLIp7BDc9qEMzEkDrxeEn2\nC9wlxL29uN7EDQu7KBRjbrMoBtfK7o3r+GuPm0WsP+6LahHeuFyvxW3J9RRiJQKTrlRDscrgTstr\nA6O8BIe424d0xCXddqr6T5DjuAFXa12BGz1QG+iOS1of4BLaYb97xeXY+RAq4eY96ICbYnCW17Lt\ngrty6yLgXFXd4le8JjgswZo0BYzLPA83bd0O3DSKvXGz3n+uqj+kbIub3ekBVd0RxJjuxdVaP8KN\nHJgJPI27MONu3AUEr/hdGhA3N0NL3Mxdf+O+fD4EWgFX4kYPTA4YZTEINzTvf74FbYLCSgQmTV5y\nbQkMx112uhQ3N+gXwEKgm4i08DZvjGuJ5ehpuQTMc+q1lOvgbql9Ju7/3Xy4eRxWAq/gWta+111x\nl7muwN3W5Stch+Ba3BVm44B7ROTigFj/wV1NZU4xdk8ukyZxE9L0wU3rVx6XYDeo6hYR+Rw35WDK\nPatW4648+ysnYwi4MqwC7g6nT+IG4bfF3U66K27KvSO4yWR8Px1LGZ4mIjtw800swcW8QFW3isho\nIBo3wfciXKdhXtzsY+YUYyUCcxyvtbgbNzHKQdzpeHd1M2R1wl0bv1XddH85PhRK3B1my6nqJ+Lu\nm3UPMBk3Sbfg5jR9RNxdCs4Fng+H+mVAWeUq3Oc2D9cy7Q/MVjfPb2ncRRFr1Lt5YDhdBGFylrVg\nDXBMcjgfN/TqHtwogR5YjQAAAAdMSURBVMuBUt6Y13q4096V6t1xNUin5LG4YWDVcQnqClyLtTKu\nLJBykcEVwOXhkFzhaFmlNW42tAdVdafI/9s7/1gtyzKOf74yTYkTVEv7MRaIueVYIYZjasXqdMq5\nDCU2sJZMwqSNbE22lrggWtpoNbGfzD+wdK7QWKS5s2pNAWHmEJpNwVGytlrhYnZAWhO//XFdh57O\nkM6h85735bzXZzvbOc/znOe+32dn17me+76u71cDhA3QCkn3EhKKy23/udF0UMF1nFIZbHEchePr\nQuJ19n6F+d0Owj/rIBFsV7eyN74xlw8B3yScTZdlDelCwnpmOpFFP+FTtKluBfm8NhHyf08Qa9MX\nETq5JnRX+233t22SxZhSAbZoZq83E1UCG4me+CMKQZVFxCvv87YfH6tieEkfIxTGPpfLBROIWtyp\nwHrbf2/1HEZC/hPYQFRczCDcHd4DbLa9tnFdNRN0CRVgu5hGYH3TYN2opOuJ+sw1wJO22+pgm+uZ\ntwNfyyB7BjDJqZrVThrPbzax6XeI2Ni6kmgf3i6plxDOvg443CFVDsUYUWuwXUxjzXBF7mhvt32P\nwvvpVmCdpMcG2zjbNMeHJb0CbJD0su0HgLYHVzj+/D4M3EWUXy0mtFu/DceXOb5FrMd2xJyLsaUC\nbBcjaR7R6rqAcFCYK2mq7fX5uruK6D461L5Zgu1HUhhlfzvn0SQz6R5gJeG++ovcxNqUkgn3ke4O\nDluWogupANtlNFWbCDPCRYS839uJbqz5+ep7p6SfOaxg2o5bJNo9Uhrrp2cCA0Q32dF8rrtyHfuG\nfBO4pTLX7qYCbJcgqcdhoHgsVbHeQrS+/oPwBlvgEHe+GrhE0jTbz7dxyh1JLgvMJ5wb9gOXE40D\nTxIB9yXAChWygbZNtOgIKsB2AZImAg9LupNQb/oO8BTwCjAZmA3skvQ48TfxjQqu/01jQ2sKUclw\nH1F6dQVhjTMxqxyuAG5zqJAVXU5VEXQJkq4hmgQGgFW2d0o6n8he308UwP8L+Lrtze2baeci6VKi\n7OqNg2VXmfF/mZBz/BFwVlYPVClWURlst2B7s6TDhOBIL2HR/CfCuXQvkZVNzH75Cg5JI3OdC9wN\nHADOlbQN2GZ7i6TXEVUXX3I61tbzK6DUtLqK3ChaAiyRtDhbNA8RLadnO91hKzj8h0b78Bpgke2r\nCB3aa4HLUkfgXqDXHWgHXrSXymC7jMxkXwbukbSQEHVZbfuFNk+tk5kMfBDoIzRnv0KUsF1PJCm/\nGW0lsWJ8UBlsF2L758CnCfGU79l+KEWzixPgEBZfACyVdF1m/msJs8W/tXVyRUdTm1xdjKQ3dFo/\nfyejMCpcC9xle2Obp1OcBlSALYoRkFUDdxAbhX9tZxtx0flUgC2KEdIUxymKk1EBtiiKokXUJldR\nFEWLqABbFEXRIirAFkVRtIgKsEVRFC2iAmwxakg6Jmm3pKclbUoVr1O91zxJD+X3V0v64kmunSLp\ns6cwxmpJtwz3+JBrNkr6+AjGmibp6ZHOsTi9qQBbjCZHbc+yPZNQ5rqpeVLBiP/mbG+xfcdJLpkC\njDjAFkWrqQBbtIqtwAWZuT0j6bvALmCqpD5JOyTtykx3EoCkj0h6NpWqrh28kaQlkgZ9rs6TtFnS\nnvy6jCj8n5HZ87q8bqWk30r6naQ1jXvdKmmvpF8RTg4nRdKyvM8eSQ8Oycp7JW2VtC+9zZA0QdK6\nxtif+X8fZHH6UgG2GHVSzf9KQtwbIpD90PbFwBFCKKXX9mzCCeALks4mLLo/CrwXePOr3H498Kjt\ndxNC4b8ndG73Z/a8UlIf8A7gUmAW4dDwPkmXEBY5FxMBfM4wPs5Pbc/J8Z4BljbOTSO0dK8Cvp+f\nYSnwou05ef9lkqYPY5xiHFJqWsVoco6k3fn9VsLj663AAds78/hc4CJge+rLnAXsIPzB/mj7OYA0\nELzxBGN8APgUQLapvijp9UOu6cuvp/LnSUTA7QE2234px9gyjM80U9JXiWWISUB/49xP0ob7OUl/\nyM/QB7yrsT47OcfeN4yxinFGBdhiNDlqe1bzQAbRI81DwC9tLx5y3SzCgmU0EHC77R8MGePzpzDG\nRmC+7T2SlgDzGueG3ss59grbzUCMpGkjHLcYB9QSQTHW7AQul3QBhF+YpAuBZ4HpkmbkdYtf5fd/\nDSzP352QbgIDRHY6SD9wQ2Nt922SzgUeA66RdI6kHmI54n/RA/xF0pnAJ4acWyjpjJzz+YQzRD+w\nPK9H0oWSXjuMcYpxSGWwxZhi+2BmgvdLek0eXmV7n6QbCXPGF4BtwMwT3OJmYIOkpcAxYLntHZK2\nZxnUI7kO+05gR2bQh4FPpq32j4HdhPXL1mFM+TbCmvsAsabcDOR7gUeB84CbbP9T0t3E2uyu1Ng9\nCMwf3tMpxhsl9lIURdEiaomgKIqiRVSALYqiaBEVYIuiKFpEBdiiKIoWUQG2KIqiRVSALYqiaBEV\nYIuiKFrEvwEZTQgEsUeetgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11ad15ac8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(mul_tfidf_train, mul_y_train)\n",
"pred = clf.predict(mul_tfidf_valid)\n",
"score = metrics.accuracy_score(mul_y_valid, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(mul_y_valid, pred, labels=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])\n",
"plot_confusion_matrix(cm, classes=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuration 3 (!no train)\n",
"* model a - test - dataset2 - [performance measures]\n",
"* model b - test - dataset1 - [performance measures]"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 true\n",
"1 false\n",
"2 false\n",
"3 half-true\n",
"4 pants-fire\n",
"Name: y, dtype: object\n"
]
}
],
"source": [
"#print(type(bin_tfidf_train),type(mul_tfidf_test))\n",
"\n",
"tmp_mul_tfidf_test = bin_tfidf_vectorizer.transform(mul_X_test)\n",
"\n",
"print(mul_y_test[:5])"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf.fit(bin_tfidf_train, bin_y_train)\n",
"pred = clf.predict(tmp_mul_tfidf_test)"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1267\n",
"['true' 'true' 'true' 'true' 'true' 'true' 'true' 'true' 'true' 'true'\n",
" 'true' 'true' 'true' 'true' 'true' 'true' 'false' 'true' 'true' 'true'\n",
" 'true' 'true' 'true' 'false' 'false' 'true' 'false' 'true' 'true' 'false']\n"
]
}
],
"source": [
"print(len(pred))\n",
"pred = np.array(pred, dtype=object)\n",
"\n",
"pred[pred == \"FAKE\"] = \"false\"\n",
"pred[pred == \"REAL\"] = \"true\"\n",
"\n",
"print(pred[:30])\n",
"#pred[pred['FAKE']==false]"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.169\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEmCAYAAAAnRIjxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXuc1eP2x9+fmZRSJNeaKd0ole6F\nJCEpJdeIUDiuuZ+Qy49w0HE5jlwP5zg4OpXrQShx3EkXIvdKOd2EkG66TOv3x/Od2k0zs/dMe8/e\nu1nvXt/X7P18n/18P3s3s/bzXc961pKZ4TiO4ySfnHQLcBzH2VpxA+s4jpMi3MA6juOkCDewjuM4\nKcINrOM4TopwA+s4jpMi3MA6FYqk6pJelLRU0lNbMM5ASa8mU1u6kHSgpK/TrcNJPvI4WKc4JJ0M\nXAY0B5YB04GbzezdLRz3VOBCoIuZrdtioRmOJAP2NLNZ6dbiVDw+g3U2Q9JlwF+BW4DdgAbA/cBR\nSRh+D+CbymBcE0FSlXRrcFKImfnhx4YD2AFYDvQvpU81ggFeGB1/BapF57oD84E/Aj8Ai4DTo3M3\nAGuAtdE1zgSGA0/EjN0QMKBK9Hww8C1hFj0HGBjT/m7M67oAU4Cl0c8uMefeBG4C3ovGeRXYuYT3\nVqj/ihj9RwNHAN8APwNXx/TvDHwA/Br1vReoGp17O3ovK6L3e2LM+FcC3wP/KmyLXtMkukb76Hk9\n4Cege7p/N/wo++EzWKco+wPbAs+V0ucaYD+gLdCGYGSujTm/O8FQ5xGM6H2SdjSz6wmz4rFmVtPM\n/lGaEEnbASOB3mZWi2BEpxfTrw7wUtR3J+AvwEuSdorpdjJwOrArUBUYWsqldyd8BnnAdcDDwClA\nB+BA4DpJjaO+BcClwM6Ez+5Q4HwAM+sW9WkTvd+xMePXIczmz469sJnNJhjfUZJqAP8EHjWzN0vR\n62QobmCdouwE/GSl38IPBG40sx/M7EfCzPTUmPNro/NrzexlwuytWTn1rAdaSapuZovM7PNi+vQB\nZprZv8xsnZmNBr4Cjozp808z+8bMVgFPEr4cSmItwd+8FhhDMJ53m9my6PqfA60BzGyamU2KrjsX\n+BtwUALv6XozWx3p2QQzexiYCXwI1CV8oTlZiBtYpyhLgJ3j+AbrAd/FPP8uatswRhEDvRKoWVYh\nZraCcFt9LrBI0kuSmiegp1BTXszz78ugZ4mZFUSPCw3g4pjzqwpfL2kvSeMkfS/pN8IMfedSxgb4\n0cx+j9PnYaAVcI+ZrY7T18lQ3MA6RfkA+J3gdyyJhYTb20IaRG3lYQVQI+b57rEnzWyCmR1GmMl9\nRTA88fQUalpQTk1l4QGCrj3NbHvgakBxXlNq6I6kmgS/9j+A4ZELxMlC3MA6m2BmSwl+x/skHS2p\nhqRtJPWWdFvUbTRwraRdJO0c9X+inJecDnST1EDSDsBVhSck7SapX+SLXU1wNRQUM8bLwF6STpZU\nRdKJQAtgXDk1lYVawG/A8mh2fV6R84uBxpu9qnTuBqaZ2R8IvuUHt1ilkxbcwDqbYWZ/IcTAXgv8\nCMwDLgD+E3X5EzAV+BSYAXwUtZXnWhOBsdFY09jUKOYQohEWElbWDyJaQCoyxhKgb9R3CSECoK+Z\n/VQeTWVkKGEBbRlhdj22yPnhwGOSfpV0QrzBJB0F9CK4RSD8P7SXNDBpip0KwzcaOI7jpAifwTqO\n46QIN7CO4zgpwg2s4zhOinAD6ziOkyI80USKqF1nJ6uX3yDdMoql+ja56ZbgpICPZy2O3ykN2Kpf\nsDXL48UGJ0zu9nuYrdtsA1wx1/1xgpn1StZ1y4Mb2BRRL78Bo158K90yimXvvO3TLcFJATse+dd0\nSyiW1e/fmdTxbN0qqjWLG/HG79Pvi7ejLuW4gXUcJ7uQICc77sLcwDqOk30oO5aP3MA6jpN9KGku\n3ZTiBtZxnCzDXQSO4zipQbiLwHEcJzXIXQSO4zgpw2ewjuM4qcB9sI7jOKlBuIvAcRwnZbiLwHEc\nJxUIcrPDRZAdXwOVkIKCAk46oisXnRH2XJ/RvxcDendlQO+u9OzcjMvOOjnNCuHVCeNp3bIZLZs3\n5fbbRqRbzia4tpLJ37km40ccx8d/O41pD57KkKNCBfMda1Zj3M3HMOPvgxh38zHUrlkNgAEHN2Py\n/QOZfP9A3rjzBPZplOYt/oVhWvGODMBnsBnK6H8+QKOmzVi+fBkAjzw1fsO5oeeeQvfD+qRLGhC+\nAC65aAgvvTKRvPx8uu7Xib59+7F3ixZp1eXa4rOuYD3DHn6b6bN/pGb1bXh/5Mm8/vH/OLVHC96c\nPo87nprK0P4dGXpCJ6595F3mfv8bPa94ml+Xr6Znx4bcd1EPul06psL0FkuW+GAzw8w7m7B40QLe\n+e8Ejh5w2mbnVixfxpT336Z7z/Qa2CmTJ9OkSVMaNW5M1apV6X/iAMa9+HxaNRXi2krn+19WMn32\njwAsX7WWr+b9TL2datJ3/8Y88doXADzx2hccuX8ohjvpy0X8unw1AJO/WkTezjUrVO/mRFEE8Y4M\nwA1sBnLHjcO4+KobySnmNueNCePofMBB1KyV3pSDCxcuID+//obneXn5LFiwII2KNuLaEqfBrtvT\ntskuTPn6e3atvR3f/7ISCEZ4lx1qbNZ/8OEtmTB1bgWrLIYscRFkhooUIukiSV9KGlXC+e6SxhV3\nLh28/fp46uy0Cy32aVfs+fEvPE2vfsdXsKrNKa4asTLkts21JcZ2227D6Gv7cPnf3mLZyjVx+3dr\nnc+gnq249pF3K0BdKUiJHXGHUX1Jb0T24XNJF0ftdSRNlDQz+rlj1C5JIyXNkvSppPbxrrHVG1jg\nfOAIM8uKuvKfTJ3EW6+9Qp8D9uGqC89g6vtvc80lZwHw6y8/8/kn0+h68OFpVhlmXvPnz9vwfMGC\n+dSrVy+Nijbi2uJTJTeH0df2ZewbX/H8+7MB+OHXFey+Y5i17r5jDX5cunJD/1YNd+aBS3rQ/8YX\n+HnZ7xWudzOS4yJYB/zRzPYG9gOGSGoBDANeN7M9gdej5wC9gT2j42zggbgyy/7OsgdJDwKNgRck\nXSnpfUkfRz+bFdP/IEnTo+NjSbWi9sslTYm+tW5IpeYLrxzO+Elf8tJ7M7j1nkfo2KUbN//1YQBe\ne+k5DjykF9W23TaVEhKiY6dOzJo1k7lz5rBmzRqeGjuGPn37pVsW4NoS4cFLevD1vJ8Z+dzHG9pe\nmvQtp/QIi22n9GjBuA++BaD+LrUY8399OfP2Ccxa8GuFa90cJcVFYGaLzOyj6PEy4EsgDzgKeCzq\n9hhwdPT4KOBxC0wCakuqW9o1tuooAjM7V1Iv4GBgDXCnma2T1AO4BTiuyEuGAkPM7D1JNYHfJfUk\nfGN1JgSIvCCpm5m9XfR6ks4mfLOxe179oqe3mAkvPsvg8y5N+rjloUqVKtx1970c2edwCgoKGDT4\nDFq0bJluWYBri0eXlvUY2KMFM+b8yKR7w43d9Y+9xx1PTuWJq49g0OEtmffjMgbe/BIAV528L3Vq\nbctfhxwChCiErhePrlDNm5Fkt4qkhkA74ENgNzNbBMEIS9o16pYHzIt52fyobVGJ4xbnE9qakDQX\n6AhUB0YSjKUB25hZc0ndgaFm1lfSMOAYYBTwrJnNl3QHcDxQ+NVdE7jVzP5R2nVbtG5nXpPLqUgy\nuSbX+qXzkmYRc2o3sGpdr4jb7/eXLvwO+Cmm6SEze6hov2gy9RZws5k9K+lXM6sdc/4XM9tR0kuE\nv/13o/bXgSvMbFpJGrbqGWwRbgLeMLNjom+rN4t2MLMR0Yd4BDApmumK8KH+rQK1Oo5TIgkne/nJ\nzDqWOpK0DfAMMMrMno2aF0uqG81e6wI/RO3zgdhb03xgYWnjb9U+2CLsABTGwwwuroOkJmY2w8z+\nDEwFmgMTgDOibzkk5cXcMjiOkw6S4INVCN/4B/Clmf0l5tQLwKDo8SDg+Zj206Jogv2ApYWuhJKo\nTDPY24DHJF0G/LeEPpdIOhgoAL4AXjGz1ZL2Bj6IwmmWA6ew8VvNcZyKJjk+2AOAU4EZkqZHbVcD\nI4AnJZ0J/A/oH517mXB3OwtYCZwe7wJbvYE1s4bRw5+AvWJO/V90/k0id4GZXVjCGHcDd6dKo+M4\nZSBJZbsjX2pJlvrQYvobMKQs19jqDazjOFsfmbJxJB5uYB3HySpCvm03sI7jOMlHQjluYB3HcVKC\nz2Adx3FShBtYx3GcVCDcReA4jpMKhHwG6ziOkyrcwDqO46QIN7CO4zipwH2wjuM4qcNnsI7jOCnA\nF7kcx3FSiLsIKjk5EjWq+cfrVBw75OenW0KxLKlaNbkDKjkuAkmPAH2BH8ysVdQ2Fiis11cb+NXM\n2kZJ+r8Evo7OTTKzc+Ndwy2A4zhZR5JcBI8C9wKPFzaY2Ykx17gTWBrTf7aZtS3LBdzAOo6TVQiR\nk7PlxVjM7O1oZrr5NYIFPwE4ZEuuUZlKxjiOs7WgBA7YWdLUmOPsMlzhQGCxmc2MaWsk6WNJb0k6\nMJFBfAbrOE52kbgPNm7Rw1I4CYitTb4IaGBmSyR1AP4jqaWZ/VbaIG5gHcfJOpLhIigJSVWAY4EO\nhW1mthpYHT2eJmk2oQTV1FJ1pkyl4zhOqkjMRVBeegBfmdn8DZeTdpGUGz1uDOwJfBtvIDewjuNk\nHZLiHgmMMRr4AGgmaX5URRZgAJu6BwC6AZ9K+gR4GjjXzH6Odw13ETiOk1UkakDjYWYnldA+uJi2\nZ4BnynoNN7CO42QdqfTBJhM3sI7jZB/ZsVPWDazjONmHJ3txHMdJARLkZEmyl+xwZFQiVv/+O/17\nd+OoQ/el70EdGXn7nzY5f9M1f6R9k13TpG5TXp0wntYtm9GyeVNuv21EuuVsgmsrmbsGdeCzO/vy\n5vDDNrT97ex9ee26Hrx2XQ+m3Nqb167rAUCVXDHy9I68cf1hvH1jTy7s3aykYSuQ+BEEmTLD9Rls\nhlG1WjUeffplttuuJmvXrmXgUT3odkhP2nbozIzpH7Fs6a/plghAQUEBl1w0hJdemUhefj5d9+tE\n37792LtFi3RLc21xGPv+dzzyxmzuOaPThrZzHvpww+Ph/Vvz26q1ABzZIZ+qVXI5+IaJVK+ay9s3\n9OQ/k+cxb8nKCtNbHBliP+PiM9gMQxLbbVcTgHVr17Ju7VokUVBQwO03XcPQ//tTnBEqhimTJ9Ok\nSVMaNW5M1apV6X/iAMa9+Hy6ZQGuLR6TZv7EryvWlHj+yI75PDd5HgAG1KiWS26O2HabXNYUrGdZ\nZHzTRuQiiHdkAm5gM5CCggKO7rEfB+zTkC4HHUKb9p0Y9ciDHNLzCHbdrW665QGwcOEC8vPrb3ie\nl5fPggUL0qhoI66t/Oy358789NvvzPlhOQDjps1n5eoCPr2jL9P+fAQPTPiGX1em18AKN7AbkNRQ\n0mcpGHewpHvL0L+2pPOTrSMV5Obm8p/XJvHmR9/w6cfTmPLBu4x/8TlOOfO8dEvbgJlt1pYpfi/X\nVn6O6Vx/w+wVoF3DOhSY0ebycXS+6hXO7bkXDXbeLo0KA1L8IxPI6BlslHQhWdQGijWwhXuMM43t\nd6hN5y4H8uH7b/O/ubPpuf8+HNJpb1atWknP/fdJq7a8vHzmz9/4h7hgwXzq1auXRkUbcW3lIzdH\nHNE+j+enbtiCz7H71ueNz75nXYHx07LVTJn1E20b7phGlbiLoBiqSHpM0qeSnpZUQ9J1kqZI+kzS\nQ1GCWyS9KekWSW8BF0dJFp6J+k6RdEDswJJqSZojaZvo+faS5hY+j2EE0ETSdEm3S+ou6Q1J/wZm\nFJ1pSxoqaXj0uImk8ZKmSXpHUvNUfVA///Qjv0ULWb+vWsUHb79By9btePfTOfx3ypf8d8qXVK9e\ng1c/mJEqCQnRsVMnZs2aydw5c1izZg1PjR1Dn7790qqpENdWPrrtvSuzFi1j0S+rNrQt+HkVXZuH\nqJUaVXPp0HgnZi5ali6JQJTLxaMINqEZcKaZvRfVwTkfuNfMbgSQ9C9CbZwXo/61zeyg6Ny/gbvM\n7F1JDYAJwN6FA5vZMklvAn2A/xASNTxjZkUdRcOAVoUlHyR1BzpHbXNKymwe8RAhucNMSfsC91NM\npvMooe/ZAPXy6hc9nRA//vA9wy4+m4KCAmz9enr1O46DD+tdrrFSSZUqVbjr7ns5ss/hFBQUMGjw\nGbRo2TLdsgDXFo8HzupMl712oU7Nanx02xHc/sIXjH53Lkd3rs9zU+Zt0veRN2Zx9+BOvHXDYQgx\n5r25fLlgaQkjVxSZY0DjoeJ8Qkm9QDBcb5tZg+j5IcBFwL+AK4AaQB3gHjMbERnL683sraj/D8DC\nmCF3AZoDxwEdzeyCaFZ7hZkdJekD4Cwz28TvG+kYF1PcrHt0nYNLOD8UqAncAfzIxmJnANXMbG9K\noVWb9vbMhHcT+owqmj12rpFuCU4KaHje0+mWUCxL/nMla3+cnTSLWKNeM2t2zgNx+00ffui0LUi4\nnRQqykVQ1IobYRZ4vJntAzwMbBtzfkXM4xxgfzNrGx15ZrbJPYqZvQc0lHQQkGtmn0mqH7kDpksq\nqfpj7HXWsennUagnh6iyZMxRqnF1HCeFJLDAlcgEV9Ijkn4o4hocLmlBjO04IubcVZJmSfpa0uGJ\nSK0oA9tA0v7R45OAwqndT5JqAseX8tpXgQsKn0gqqarj44Qcjv8EMLN5MQbxQWAZUKuU6ywGdpW0\nk6RqBJcFUUmIOZL6R9eXpDaljOM4TgpJog/2UaBXMe13xdiOlwnXa0FwP7aMXnN/IovjFWVgvwQG\nSfqU4A54gDBrnUHwm04p5bUXAR2jBbIvgJJmo6OAHdk8US4AZrYEeC9aVLu9mPNrgRuBD4FxwFcx\npwcCZyok2/0cOKoUvY7jpJhkzGDN7G0gbtLsiKOAMWa22szmALMIazilkvJFLjObCxS3D/Da6Cja\nv3uR5z8BJxbT71HCN1AhXYGnzazEvaRmdnKRpjeLnB8JjCzmdXMo/pvOcZw0kGAY1s6SYmtmPWRm\nDyXwugsknUaot/VHM/sFyAMmxfSZH7WVylaRi0DSPUBv4Ih4fR3HyXJSW1X2AeAmwjrRTcCdwBkU\nn4E2boTAVmFgzezCdGtwHKdiCD7Y1IxtZos3XEd6mOAuhDBjjY29zGfT6KZiyeidXI7jOJsTfxdX\neXdySYpN9nEMUBhh8AIwQFI1SY0IVWUnxxtvq5jBOo5TuUjGRgOFqrLdCb7a+cD1QPcoUsmAucA5\nAGb2uaQngS8IIZ1DzKwg3jXcwDqOk10kKZlLCVVl/1FK/5uBm8tyDTewjuNkFSFdYXZ4N93AOo6T\ndWRJKgI3sI7jZB/ZkuzFDazjOFmFlDn5XuPhBtZxnKwjSyawJRtYSduX9sIoCYrjOE6Fk5MlFra0\nGeznhFiw2HdS+NyABinU5TiOUyJZYl9LNrBmVr6U/I7jOClECvXDsoGEfLCSBgCNzewWSfnAbmY2\nLbXSshszWL027kYPx0kaS6e+mW4JxVKwIvk1vLIliiButK5CaeyDgVOjppXAg6kU5TiOUxrZUrY7\nkRlsFzNrL+ljADP7WVLVFOtyHMcpFgG5mWJB45CIgV0rKYco96GknYD1KVXlOI5TEhlUljseiWzo\nvQ94BthF0g2Eelp/Tqkqx3GcUthqXARm9rikaUCPqKl/0ZLYjuM4FYVIThSBpEcIxU1/MLNWUdvt\nwJHAGmA2cLqZ/SqpIaG24NfRyyeZWUn1ATeQaEqaXGBtdNHsSGPjOM5WSwqryk4EWplZa+Ab4KqY\nc7Njqs3GNa6QWBTBNYRKrfUIZRL+Lemq0l/lOI6TGhJxD5S3qqyZvWpm66Knkwg2r9wkssh1CtDB\nzFYCSLoZmAbcuiUXdhzHKS8JRhGUt6psIWcAY2OeN4qiqX4DrjWzd+INkIiB/a5IvyrAt2UQ6TiO\nk1RSWFW2cPxrCKVhRkVNi4AGZrZEUgfgP5JaxsvJUlqyl7sIoVkrgc8lTYie9yREEjiO41Q4AlK5\nU1bSIMLi16FmZgBmthpYHT2eJmk2sBcwtcSBKH0GWxgp8DnwUkz7pHLqdhzH2XJSGAcrqRdwJXBQ\noVs0at8F+NnMCiQ1JlSVjXsnX1qylxKLfzmO46STZCTcLqGq7FVANWBiZMQLw7G6ATdKWgcUAOea\n2c/FDhyrMwERTSSNkfSppG8Kj3K/KychCgoKOKF3Vy4Y3B+Aqy46k37d23Nsj325buj5rF27Ns0K\n4dUJ42ndshktmzfl9ttGpFvOJri2ksnfrTbjH7qIj5+5lmlPX8OQk7oDcMslRzP92WuZPPYqxt55\nFjvUrA5Ax5Z7MGnMMCaNGcaHY4fR7+DWFa45lkIXQbwjHmZ2kpnVNbNtzCzfzP5hZk3NrH7RcCwz\ne8bMWppZGzNrb2YvJqI1kZjWR4F/Ru+rN/AkMCaRwZ3yM+qRB2jcdK8Nz484+gSef2Maz0ycxOrf\nV/HcmMfSqC58AVxy0RCef/EVPv70C54aM5ovv/girZoKcW2ls65gPcP+8iztjvsTB512B+ec2I3m\njXfn9Ulf0aH/LXQ+8VZmfvcDl5/RE4DPZy/kgIG3sd+AERw15H7uufYkcnPTGw6fpDjYlJPIp1TD\nzCYAmNlsM7uWkF3LSRGLFy3gndcncMyAQRvaDjzk8A2/OK3admDxooVpVAhTJk+mSZOmNGrcmKpV\nq9L/xAGMe/H5tGoqxLWVzvc//cb0r+YDsHzlar6a8z31dqnN65O+oqAgpBmZPGMOebvVBmDV72s3\ntFerug3Ruk/akEKYVrwjE0jEwK5W+DqYLelcSUcCu6ZYV6XmtuHDuPTqG4ut/b527VrGPTuWAw7q\nUcwrK46FCxeQn78xJ3teXj4LFixIo6KNuLbEaVC3Dm2b5TPls7mbtJ921P5MeG/jzLpTqz2Y9vQ1\nTH3qai66ecwGg5susiUXQSIG9lKgJnARcABwFiEAN2VIaigp4XwHkoZLGho9bi5puqSPJTUp0q+7\npC7J1ptM3nrtFersvDMtWrcr9vwt11xGh85daL9vet9GcbOYTLktc22JsV31qoy+4w9cfsczLFvx\n+4b2K848nIKC9Yx5ecqGtimffUeH42+m6ym3cfkZPalWNb31UrPFRZBIspcPo4fL2Jh0O5M5Gnje\nzK4v5lx3YDnwftETkqrEbJFLG9OnfsibE1/h3Tcmsnr176xYtoyrLv4Dt979dx6861Z++fkn/m/E\nqPgDpZi8vHzmz5+34fmCBfOpV69eGhVtxLXFp0qVHEbfcRZjX5nK8//9ZEP7wCP35Yhureh9zshi\nX/f1nMWsWLWGlk3r8dEX/6souZsglP0lYyQ9R5QDtjjM7NiUKNpIrqSHgS7AAuAowrbds4GqwCzg\n1CKxakcAlwAFkrqZ2cEx5xoC50bnTgEuBM4k7EVuB3wkaRmw3MzuiF7zGdDXzOZGr7kouvaHwPlm\nlvSaMBcPG87Fw4YDMOWDd3jsbyO59e6/8+zox3j/7dd5aPSLxboOKpqOnToxa9ZM5s6ZQ728PJ4a\nO4ZH//XvdMsCXFsiPHj9QL6e8z0jn/jvhrbDuuzNHwf3oOcf7mbV7xujVPaotxPzF/9CQcF6GtTd\nkb0a7sZ3C5dUuOYNZJALIB6lzWDvrTAVxbMncJKZnSXpSeA44FkzexhA0p8IBvKewheY2cuSHiTG\nSMacm1v0nKQzCbsxekQBxMOLEyJpb+BE4AAzWyvpfmAg8HiRfmcTvgCom5fcmpF/uvoS6ubV57Sj\ng+/1kF5Hcu4lw5J6jbJQpUoV7rr7Xo7sczgFBQUMGnwGLVq2TJueWFxb6XRp25iBffdlxjcLmDQm\n/A5df+8L3Hl5f6pVrcK4By4AYPKMuVx08xi6tGvM0NN7snZdAevXGxffMpYlv66oUM1FyRQXQDxK\n22jwekUKKYY5ZjY9ejwNaAi0igxrbYJfeEISrvNUAjPRQ4EOwJToP7Y68EPRTlEiiYcAWrZuv8VL\nrZ32P5BO+x8IwEdzftnS4ZJOr95H0Kv3EemWUSyurWTen/4t1dtdsFn7hHdvKLb/6JemMPqlKcWe\nSwdbW8mYdLE65nEBwag9ChxtZp9IGkzwqZaIpCGERTmAkn6jY7+K17Hpwt+2hUMBj5mZp2l0nAwg\nS1ywWZc8uxawSNI2hFv0UjGz+2J2ZCwkLNTVKuUlc4H2AJLaA42i9teB4yXtGp2rI2mP8r8Nx3G2\nhGTs5KoIEjawkqqlUkiC/B9hgWki8FU5Xv8icEwUxnVgMeefAepImg6cR8hojpl9AVwLvCrp0+j6\ndctxfcdxtpAQ57qVhGlJ6gz8A9gBaCCpDfAHM7swVaLMbC7QKuZ57ILVA8X0H17c42L6fQPEbqR+\np8j5VYR0jMW9diybJt91HCdNpHmnbsIkInMkITfiEgAz+wTfKus4TpoIyV4U98gEEjGwOWb2XZG2\npMd/Oo7jJEpOAkc8JD0i6YfYXaPR+spESTOjnztG7ZI0UtKsKLNg+0R1xmNe5CYwSbmSLiHyTTqO\n41Q0UtjJFe9IgEfZvKrsMOB1M9uTsLhdGGzemxCbvych1n0zV2VxJGJgzwMuAxoAi4H9ojbHcZy0\nkKqqsoQdo4W5QB8jbL0vbH/cApOA2pLiLnQnkovgB2BAfLmO4zgVQ4JhWOWpKrubmS0CMLNFhaGZ\nQB4wL6bf/KhtUWmDJRJF8DDF5CQws7PjvdZxHCfZCBJ1AZS7qmwJly1K3N2aiezkei3m8bbAMWxq\nyR3HcSqO1G4kWCypbjR7rcvGLfHzgdgEI/lA3Kz3ibgINon9lPQvQqC94zhOWlCxE8qk8AIwCBgR\n/Xw+pv0CSWOAfYGlha6E0ihPLoJGgG8TdRwnLQiokoSNBiVUlR0BPBll2vsf0D/q/jIhn8ksYCVw\neiLXSMQH+wsbfQ05hFW39OXJcxyn0pOMrbBmdlIJpw4tpq8BQ8p6jVINbFSLqw0h4TXAekt3xTPH\ncSo1hWW7s4FSDayZmaTnzKxDRQlyHMcpFSUcRZB2EvFkTE50W5jjOE6qKZzBZkO6wtJqchUWAewK\nnCVpNiE5tQiTWze6juOkhQyn2fdTAAAgAElEQVTJ5RKX0lwEkwnJp48upY9TArk5onaNbdItw6lM\nbFsz3QqKJyc3yQOKnNSFaSWV0gysAMxsdgVpcRzHiYuUPflgSzOwu0i6rKSTZvaXFOhxHMeJS6bk\ne41HaQY2l1C5NTveieM4lQKxdfhgF5nZjRWmxHEcJ0GyJUwrrg/WcRwnkxDZUw67NAO72XYxx3Gc\ntKPkbJWtCEo0sGZWNNO34zhO2hGQm+0G1nEcJ1PJDvPqBtZxnCwkGRNYSc2A2HzXjYHrgNrAWcCP\nUfvVZvZyea7hBtZxnKxCKCkuAjP7GmgLICmXkDXwOUKu17vM7I4tvYYbWMdxso4ULHIdCsw2s++S\nOXa2RDtUGhYumMeJRx3OIfu3pccB7Xnkb/cCcNef/0TnVo3p3X1fenffl/9OHJ9mpfDqhPG0btmM\nls2bcvttI9ItZxNcW8nk77oD4+89i49HX8a0UZcy5IQDADj2kH2YNupSVrx3C+2b523ymlZNdufN\nh85j2qhLmfLEJVSrmt65mRI4iKrKxhylFWodAIyOeX6BpE8lPSJpx/Lq9BlshpGbW4VrbxzBPm3a\nsXzZMvoe2oWu3UPE3JnnXsg5F1yaZoWBgoICLrloCC+9MpG8/Hy67teJvn37sXeLFumW5trisK5g\nPcNGvsT0bxZSs0ZV3v/nhbw+eSafz/6eAVf9i3uvPHaT/rm5OTwy/ETOvOFJZsxaRJ3ta7B2XUGF\n6d2MxMO0EqoqK6kq0A+4Kmp6ALiJUMnlJuBO4IzySPUZbIax2+512adNOwBq1qpF072as3hR3OKV\nFc6UyZNp0qQpjRo3pmrVqvQ/cQDjXnw+/gsrANdWOt8vWcb0b8Lv1PKVa/hq7o/U22V7vv7uR2b+\n76fN+vfovCefzfqeGbNCjb+ff1vJ+vXpK2xSGKYV7ygDvYGPzGwxgJktNrMCM1sPPAx0Lq9WN7AZ\nzLz/fcfnM6bTtkMnAB7/x4Mc3q0TQy86h6W//pJWbQsXLiA/f2MV47y8fBYsWFDKKyoO15Y4DXbf\nkbZ71WPK5/NK7LNng50xM1646wzef/RCLhvYrQIVFk+CLoJEOYkY90BUrruQY4DPyqszawyspIaS\nTo553l3SuDKOcXXylaWGFcuXc+7gk7ju5tupVWt7Tjn9LN6e+gWvvPkhu+62Ozddl966k8WVZsuU\n3TWuLTG2q16V0bcO5PK/vsiylatL7FclN4cubRpy+vAxHHrOg/Q7qCXdOzapQKWbI8U/EhtHNYDD\ngGdjmm+TNEPSp8DBQLn9clljYIGGwMnxOsWhWAOrQMZ8FmvXruXc00/i6ONPpHffkO98l113Izc3\nl5ycHE469Qw++WhqWjXm5eUzf/7GWc+CBfOpV69eGhVtxLXFp0puDqNvOYWxE6bz/Fufl9p3wQ9L\neefjOSxZupJVq9cy/oOvadcsr9TXpJJkugjMbKWZ7WRmS2PaTjWzfcystZn1M7NF5dWaMqMSzTi/\nkvR3SZ9JGiWph6T3JM2U1FlSHUn/iVbrJklqHb32IEnTo+NjSbUI9coPjNoujblOTjTeLjHPZ0na\nuYieEUD16PWjIn1fSrof+AioL2l5TP/jJT0aPd5F0jOSpkTHAan63MyMKy4+l6Z7NeOs8y/e0L74\n+43/xxNeep5mzdO7YNOxUydmzZrJ3DlzWLNmDU+NHUOfvv3SqqkQ1xafB685nq+/+4GRY96N23fi\nhzNp1XR3qlfbhtzcHA5s14gv5yyuAJUloYT+ZQKpjiJoCvQHzgamEGagXQkrdlcD84CPzexoSYcA\njxMCf4cCQ8zsPUk1gd+BYcBQM+sLwUUAYGbrJT0BDAT+CvQAPjGzTbz1ZjZM0gVmVhhY3BBoBpxu\nZudHbSW9j7sJgcfvSmoATAD2LtopCgM5GyAvxs9WFqZ++D7PPvlvmrdoRe/u+wJw+TU38MKzT/LF\nZ58iifz6e3DLnfeUa/xkUaVKFe66+16O7HM4BQUFDBp8Bi1atkyrpkJcW+l0ab0HA3u3Z8asRUx6\n7CIArn9wAtWqVuEvl/Vj59rb8eydg/n0m0X0u/QRfl22ipGj3+HdRy7AzJjwwdeMf//rCtVclAzx\n+MRFxfmEkjJwMGATzWzP6PnjwAQzGyWpMcHnYcBxZvZt1Gce0Ao4j+BcHgU8a2bzI4Na1MAONbO+\nkuoDz5tZe0ljgCfMbDP/rKTlZlYzRt8bZtaohPPHA33NbLCkH4DYpfxdgOZmtqyk99+6bQcb9/p7\nZfrMKopdd9g23RKcFLDjgen1y5fE6hmPsX75oqSZxL1atbV7npwYt1+vlrtOSyRMK5WkegYb6zlf\nH/N8fXTtdcW8xsxshKSXgCOASZJ6lHYRM5snaXE0C94XGBhtfZsWdXnBzK4r5qUrig4V8zjWCuUA\n+5vZqtJ0OI5TMWTLDDbdCztvE27tC2ekP5nZb5KamNkMM/szMBVoDiwDapUy1t+BJ4Anoxi2AjNr\nGx2FxnWtpNJKvS6WtHe04HVMTPurwAWFTyS1LeP7dBwniWSLDzbdBnY40DEKhxgBDIraL4kWxj4B\nVgGvAJ8C6yR9ErvIFcMLhBpi/yzleg8Bn0oaVcL5YcA44L9A7MrhRYU6JX0BnJvQu3McJ+mkYKNB\nykiZi8DM5hL8qYXPB5dw7qhiXnthCcMWrbLwZszjNoTFra9K0XQlcGVMU6si558Gni7mdT8BJ5Y0\nruM4FUuG2M+4bBW5CCQNIyyMDUy3FsdxUk+muADisVUYWDMbQXAxOI6zlSMgS4rKbh0G1nGcSoRE\nTpb4CNzAOo6TdWSHeXUD6zhOlhFcBNlhYt3AOo6TdWSJfXUD6zhO9pGsKAJJcwmbmAqAdWbWUVId\nQrXZhsBc4AQzK1cC5nRvNHAcxykzycoHG3FwtOOzMG/BMOD1KI/K69HzcuEG1nGcrCPJBrYoRwGP\nRY8fA44u70BuYB3HySpCSZiEchEkUlXWgFclTYs5v1thku3o567l1eo+WMdxsovEZ6iJVJU9wMwW\nStoVmCipxK325cFnsI7jZB3JchGY2cLo5w/Ac4QKsosLCx9GP38or043sI7jZBnJKRkjabuoHBWS\ntgN6EirIvsDGzH6DgHLXVXcXgeM4WUeS4mB3A56LSkVVAf5tZuMlTQGelHQm8D9C2aty4QY2RRSs\nN5auKq5gQ/rZdYd0K3BSQd39u6VbQrEs+PaZpI4nkrNVNipV1aaY9iVsnhq1XLiBdRwn6yilQGlG\n4QbWcZysI0vsqxtYx3Gyjyyxr25gHcfJMuQuAsdxnJQg3EXgOI6TMrLEvrqBdRwn+3AXgeM4TorI\nEvvqBtZxnOwjS+yrG1jHcbKLsMiVHSbWDazjONnFlifUrjA8m1aGUlBQwAm9DuCCwccDMPrRv9Gn\naxta16/FLz//lGZ1gVcnjKd1y2a0bN6U228bkW45m+DaSubPJ+7D5BsO5ZXLD9yk/bSue/DasG6M\nv+JAruzbDICj2tdj3B+7bjhm3dGbvevVqnDNRVECRybgM9gMZdQ/7qdR02asWP4bAG077ke3Q3tx\n5glHpFlZoKCggEsuGsJLr0wkLz+frvt1om/ffuzdokW6pbm2ODw9ZT6Pv/sdd5y8Mc/Jfk3rcFir\n3Tji9ndZU7CenWpWBeD5jxby/EcLAWhWtxZ/O6MDXy5cVmFaSyRTLGgcfAabgXy/aAFv/3cCx540\naEPb3q3akFd/jzSq2pQpkyfTpElTGjVuTNWqVel/4gDGvVjutJlJxbXF0fDtL/y6cu0mbQO77MGD\nr89mTcF6AJYsX7PZ645sV5cXI2ObXkSO4h9xR5HqS3pD0peSPpd0cdQ+XNICSdOjo9yzGjewGcht\nw6/ksqtvIicnc/97Fi5cQH5+/Q3P8/LyWbBgQRoVbcS1lZ1Gu2xHp8Z1ePbiLowesi+t62+e07JP\n27q8+HH6DWwi7oEEJ7jrgD+a2d7AfsAQSYW3EndFlWbbmtnL5dWauX/BKUBSbUnnp1tHabz12ivU\n2WkXWrRul24ppWJmm7Vlysquays7uTli+xrbcOzd73Pri19xz2mb/v61abADv69dzzffL0+TwiIk\nwcKa2SIz+yh6vAz4EshLpsxKZWCB2sBmBlZSbhq0FMv0qZN4c+LL9Nq/JVcMGczk997mqov+kG5Z\nm5GXl8/8+fM2PF+wYD716tVLo6KNuLay8/3S35nw6fcAfPq/paw3o852VTecP7JdvQxxDwQSdBEk\nUlUWAEkNgXbAh1HTBZI+lfSIpB3LrbO8L8xSRgBNIr/KlMj/8m9ghqSGkj4r7ChpqKTh0eMmksZH\npX3fkdQ8VQIvHnYDr035mvEffM5t9z1K5wO6cevIv6fqcuWmY6dOzJo1k7lz5rBmzRqeGjuGPn37\npVsW4NrKw8QZi9l/z52A4C7YJjeHn1cEP6wEvdvsnhHugUISnMD+ZGYdY46Hih1Lqgk8A1xiZr8B\nDwBNgLbAIuDO8uqsbFEEw4BWZtZWUnfgpej5nOgbrCQeAs41s5mS9gXuBw4p2in6hjwboG5e/aKn\nt4hRjzzAPx/4K0t+XMzxh+1P10N6csPt9yX1GmWhSpUq3HX3vRzZ53AKCgoYNPgMWrRsmTY9sbi2\n0rn7lLbs27QOO25XlfeuO5i7J8zkqcnz+POA1rxy+YGsLVjP5aM/3dC/c+M6fL/0d+b9vKpCdZZI\nEuNgJW1DMK6jzOxZADNbHHP+YWBcuccvzie0tRIZ0XFm1ioysNeb2cFFz0XPhwI1gTuAH4GvY4aq\nFjnGS6Rl6/Y25uW3k/wOksOeu9dMtwQnBbS4otxrMSllwaiLWb14ZtIczW3adbCX3/ggbr/8HatN\nM7OOJZ1XcH4/BvxsZpfEtNc1s0XR40uBfc1sQHm0VrYZbFFWxDxex6Yuk22jnznAr2bWtsJUOY5T\nKkmy1gcApxJchNOjtquBkyS1BQyYC5xT3gtUNgO7DChpG8piYFdJOwHLgb7AeDP7TdIcSf3N7Kno\nW6+1mX1SQZodxylCMlwEZvYuxdvqpN0KVCoDa2ZLJL0XLWatIhjVwnNrJd1IWEWcA3wV89KBwAOS\nrgW2AcYAbmAdJ01kQmhbIlQqAwtgZieXcm4kMLKY9jlAr1TqchwncbLDvFZCA+s4TnajLMqm5QbW\ncZysw10EjuM4KSI7zKsbWMdxspAsmcC6gXUcJ9sQypI5rBtYx3GyilCTK90qEsMNrOM4WYcbWMdx\nnBThLgLHcZwUIEFOdthXN7CO42QhbmAdx3FSQ7a4CCpbRQPHcbYCchT/SARJvSR9LWmWpGFJ15ns\nAR3HcVJOEooeRrX47gN6Ay0IeWBblP6qsuEG1nGcrEMJ/EuAzsAsM/vWzNYQ0pAelVSdlalkTEUi\n6UfguyQNtzPwU5LGSjaZrA0yW19l0baHme2SpLGQNJ6gLx7bAr/HPH8otvChpOOBXmb2h+j5qYTy\nMBckS6svcqWIJP9CTS2ttlA6yWRtkNn6XFv5MLNk5WYubpqb1Bmnuwgcx6mszAdiyz/nA0mtTe4G\n1nGcysoUYE9JjSRVBQYALyTzAu4iyA4eit8lbWSyNshsfa4tjZjZOkkXABOAXOARM/s8mdfwRS7H\ncZwU4S4Cx3GcFOEG1nEcJ0W4gXUcx0kRbmCzCBUppVn0ueM4mYUb2CxBkixakZS0E4Bl8QplcV8O\nkir893Fr+ZLaWt7H1oZHEWQZki4E9gcWAW8Br5jZ2vSqKhuFXxaSDgdaAtWBO83s9zgvTYmO6PGR\nhAnHIuAjM1tXkVrKQsznVw9Yb2bfx7Yn+VrHACuAHDMbn8yxKwM+g80iJPUH+gPnAT2BrtlmXCHM\nvCX1Bm4GPgZOAO5Mhw4ASUOBy4AOwJ+BHhWtpSxEn18f4EXgVkkTJeWmwLheAAwF6gDPSDowmeNX\nBtzAZjCFt30xt855wAjgGMKWvmui87unRWA5iLmV7UXYObM9sAy4tcj5itJTH+hkZgcDqwmztVcl\nVa9IHWVBUlvC//3RwH+BRkDNmPNb9BkqsAdwGHAI4ffuLeB9SdtsydiVDTewGUqR271CA/otcAtw\nmpkdbmZrJf0RODcd/styUiv6KYKRuBA43czmSzoWOLmC9awG1kl6lOB6OdbM1gN9IuObiawEHgAO\nBC4AeprZUkldISm+eQE/EvbqXwccBBxvZgXAIEl7beH4lYZs+aOsNESzh1jf4CXAvyVtB8wGvgTG\nSeog6SRgIPBUZBQyGklNgWslNQJGAycCj5nZTEldCF8e/6sgLSdIOsjMfgBmEXzBV5jZGklnEAxL\nRvlhJbWStA+wBrgauAI4yMy+jW7fr5fUYAuv0RU418xWAjWAP5pZPzNbKelk4A/A8i17J5UHz0WQ\neVQ1s9UAks4k3Eb3N7MVkr4BHib4Cm8AVgGDkr1/OoXsGP08hzADOxb4m6RuQEdgqJm9U0Fa9gD+\nL/qMnwe2Af4uaSpwKHCCmS2qIC1xib5gjwL2IcxahwH/AA6LFrsuAK42s3J9QUV3QCJk9m8jaQBw\nPrC9pNeBz4AuwBlmltSMU1szHkWQQUQzvBGEWcN3ki4jzK5+B9oAZwH3Ao8Tbm0VzTQyGkltzOyT\n6HFH4EhC5MAIwiypGuGL5ctUrIQX0dLYzL6NHl8InEowJIUGpArwjZnNTZWGRCn6WUhqBfQDmhIW\n5Q4iuAmqA/8xs4nl/fwkNTCz/0mqQVhI7QBMMrN/S+oHFABfFn52TmK4gc0gotu7IYQclX8EOhH+\n+AEeAdYTfJSXZYIBKI2YUKLqwP3ADmZ2bHSuM3A9sAC4r9D4plJH9LgT4Y7gPTN7Nmq7DLgcONnM\n3kiVjvIi6QDgD2Z2evR8b+A4wgz8BjObn4Rr1AM+ILgGXolmywOj64wC/p3JYWuZjPtgM4jo9u4+\nwmLWnYSV2+OB48zsSeBXwoLX6rSJTIAY49ob+BfBX2iSHgcws8nANDYv6ZESHdHjgUAf4GfggGhW\nhpn9hXCXcIWkbVOlpTxIak+Iy+0h6SEAM/sS+BBoB9wsqdaWLHBGXzD7E/6PbpHU08xWRKVVtiHc\nOdUsbQynFMzMjzQdBJ9XTjHtdQi3z2OABlHb+cBHwD7p1p3ge2sf6d8ver4L8DTwMiGG931CeFRF\naNkXGBc9rk5YHLoTGEyY0T5IqBuV9s8tRnNrgiHdFagKfAH8PTrXluB/3XsLr3EY8GTM79jJBFdJ\nH6Av8B+gbro/i2w+3EWQRiTVNLPl0eNzCDGhOWb2Z0k7AFcCDQn+trrAUstQH5ikqhYqcyJpN8Lm\ngVuA/c3ss6g9l2DMqgLPmtnzFaClE3Atwb1yooUogZ0JvsxuQCtC2NsXqdBSFmJm/g2A14E7zOxv\n0bnqwFTgK4Lr6Gwr484qhaz9Tc3sC0mDCV80s83syJg+xxPcU6uAS8zs0yS8tUqLG9g0Ed2iHmVm\nZ0ahWMcA/0dYxJphZgMl1SLsdqpB+IPKyFCsyHCeAPwCLCbMhB4CLibUObrYzL6L6b+tmf2eigWt\n6IupPRsXrX4kRCgcSjDub1uIyMg1swJJtc3s12RqKCvR//PuFsLVOgFLgBsJM+/WZrYq6leV4BpY\nUfilVcbrNCX4wxcBDQh+/UsIoXIjY/rtAKwzsxVb9s4cN7BpQCFZy1iCAVpHMKxnAxcR/qiMsMf8\n+OiPr7qFeM2MRVITgs+4CtDNzL6JdgMNBpoTQojmVICOugRjfyTQmHAbvVrSFcCewDPAO5lkPKLP\n6VngNaAr4XfhK4IxbETY/JCU2FNJd0TjX2lmD0R+8nOA183snmRcw9mIL3KlhzUEw3pddFwDdCbM\naI8h+F8PlvSEmS3LAuMqwuLRV4SFuA4A0az174RFpL9UxCKShdjVRYSZ3gvAblH7bYRNGqcRFnUy\nhuhz+hfBFfSqhbhmI8wuvyZs3U3WQtODhEiVsyWdaGavADcBp0QbV5wk4gY2DZjZMoKPrQ8wM+b2\n+YPoZxNC0pFr0yCvzFjgF8LGgf7A5QqJQiC4N14DLrUUZcuSVLtI09OEoPwlwFmSOkTtowgLhWW+\nva4AphM2C1wWGb71kWvgWuBtwk6zLcbMZpnZv4i+2BUymuUTvvQnJeMazkbcRZAmotvCpgSf6/3A\nK4RdWnMJ/sJDzGxW2gTGoUgI1DZAQaGPWNJ+wKMEw9oBGGJmH6VIx9HAGYQFrFWSqkRaLApzOpGw\nYLMjITrjvGTdbieTmAWuwwkr+/0J24bPA4YV+mGTfM1ewO2EBDdnWvbsCMwa3MCmmcgIjCXEIb5L\nyFy0pCL8lVtKtP/960IXhqQ8wm6zOwmLKOcCL5jZxBRdvzphVjqeMDP9xsx+i87tT7hDWw7sR8je\nNdxSuKmhrEiqYqF0dLXIT7wrIS64M+FW/lfgplRFW0QadiXchPyYqmtUZtzAZgCS2hDSzl1lIcA7\nK5B0J1DHzE6XVAeYDNxlZvdF53PMbH2KogVqWEhAMpgQtdCIEFf7q6RC/+s5ZvZy1H8by5DcuZHP\nuiUhyfjhUVtDwqr+3Wb2vKR8INfClumUbh92Uof7YDOAaFZ1EMEvm038E1gWLV6tIfhZC42rCl0G\nKTCuzYBLJO0C/ERIgDKRaEELaAacWmhcIw1pN66KiHzWnxE+u8KFpauA1wpnq2Y2v9A378Y1e/EZ\nrBOXaONAMzN7W9JBQD3g+WgG+RzwsZndGNM/J5Uxu5IOIfgo5wBTgO8Jyad3JOybn14ROhKlMO43\nerwh7lYhLeIeZnZ9YVxu1O4z1q0En8E6pRItGh0FnBf5XFcT4ihvVSi1cj3QXFLt6NaXVBm1mPH/\nCzxF2Ea6H6G6wxPAWqC/QvrDlOkoC1HQ/iOSDpNUDZgi6VJJRxBick+SdKgb160TN7BOqVjIojQR\neI+waWAVYQ/7/YRNETcTwrM6ptIwFDU8kZF9hrCFeEik635CWFj3ioi5TZAqhFn2EMLC3zGELGLX\nEb6oZhDF5UYzbjeuWxHuInBKpEgoVj7htrwVMCoycCiUD+lPMLpHmdnSFOs4jeBr/YYQPdAKGEQw\nWv8kGLS16V4V16b5EJoQ8h4cT0gxODma2Q4k7NzqDrQzs8Xp0uukBjewTrHExGV2IqStW0JIoziE\nkPX+aTN7tbAvIbvT5Wa2JIWaLiH4Wv9NiBz4EPgTYWPGRYQNBH9Nt2tAITdDL0LmroWEL5/Hgd7A\nEYTogTdioiz+TAjNuy1top2U4C4Cp1gi49oLeIyw7fRLQm7QZ4BPgFMl9Yy6dyHMxJJ6W66YPKfR\nTLkNoaT29oTf3eqEPA4zgb8SZtZp97sStrl+Qyjr8h/CguAcwg6zccDFkg6O0bqUsJvK2crwmlxO\nsSgkpBlKSOvXkGBg55nZYklPE1IOFtasmk3YebYgmRpidoY1IlQ4vYEQhN+PUE76FELKvQJCMpm0\n344VhqdJWkLIN/EFQfN0M/tB0lggl5DgewZh0XBbQvYxZyvDXQTOZkSzxV8JiVFWE27HB1nIkHUi\nYW/8DxbS/SU9FEqhwmwDMxujUDfrYuANQpJuEXKaXq1QpaAdcHsm+C9j3Cp9CJ/bR4SZ6XXAFAt5\nfvMImyK+tah4YCZtgnCSi89gHWAT47AvIfTqYkKUwGFAvSjmtT3htnemRRVXU3RLviMhDKw5wUAd\nTpixNiW4BQo3GRwOHJYJxhU2uFX6ErKhXWFmP0taRigDdKGkJwgpFM8zs4Uxmw7cuG6l+AzW2YBC\nxdf+hNvZ0QrF7z4g1M/6kWBsh6dyb3yMlsOAvxAqm54VxZD2J5SeaUSYRU+2cpapTgXR5/UUIf3f\nZIJvugUhT64R8q5OMLMJaRPpVChuYJ3Y2evFhCiBRwl74lcoJFQZQLjlnWtm71dUMLykowgZxi6K\n3AW5hFjc+sBIM/s51RrKQvQl8BAh4qIJobpDR+A5M7sppp9vJqgkuIGtxMQY1l0K40YlDSLEZ94A\nTDWztFawjfyZtwK3REY2B6hpUdasdBLz+bUnLPr9QljY6k3YPvyepB6ExNknA8szJMrBqSDcB1uJ\nifEZXhitaL9nZo8p1H66Brhd0tuF2zjTpPElSeuBhyStM7OngbQbV9jw+R0O3EMIvzqJkLv1Xtjg\n5riL4I/NCM1OxeIGthIjqTthq+txhAoK+0mqb2Yjo9vdawm7j35Jn0ows1eixCiz06kjlmgmXQu4\nnFB99eVoEeupKGXCKKLqDhbKsjiVEDewlYzYrE2EYoQDCOn99iDsxjo6uvW9W9LzFkrBpB1LUdLu\nshLjP90GWEbYTbYq+lw/ivzYZ0R3AkN95lq5cQNbSZBUy0IBxYIoK1ZdwtbX3wi1wY6zkNy5H9BB\nUkMzm5tGyRlJ5BY4mlC5YTZwAGHjwFSCwV0JmEIWsmVpE+pkBG5gKwGSagAvSbqbkL3pPuBjYD2w\nA9Ae+EjS+4TfiTvcuG5KzIJWbUIkwyhC6FVXQmmcGlGUQ1fg/yxkIXMqOR5FUEmQdAxhk8Ay4Foz\nmySpMWH2ehAhAH4N8Gczey59SjMXSZ0JYVc7FYZdRTP+6wnpHP8FVI2iBzwUy/EZbGXBzJ6TtJyQ\ncKQHoUTzPELl0q8Js7Ia0X55Nw4RMTPX/YC/A98Bu0p6F3jXzF6QtD0h6uJqiyrW+ufngGfTqlRE\nC0WDgcGSToq2aP5C2HK6rUXVYd04bCRm+/ANwAAz60PIQ3ss0CXKI/AE0MMysBy4k158BlvJiGay\n64DHJPUnJHUZbmY/pVlaJrMDcCjQk5Bz9kZCCNsgwiTljWRnEnO2DnwGWwkxsxeBPxCSpzxgZuOi\npNlOMVhILH4ccKakk6OZ/02EYos/pFWck9H4IlclRlKdTNvPn8koFCq8CbjHzB5NsxwnC3AD6zhl\nIIoaGEFYKFyczm3ETubjBtZxykhschzHKQ03sI7jOCnCF7kcx3FShBtYx3GcFOEG1nEcJ0W4gXUc\nx0kRbmCdpCGpQNJ0SdqFbzUAAANRSURBVJ9JeirK4lXesbpLGhc97idpWCl9a0s6vxzXGC5paKLt\nRfo8Kun4MlyroaTPyqrRyW7cwDrJZJWZtTWzVoTMXOfGnlSgzL9zZvaCmY0opUttoMwG1nFSjRtY\nJ1W8AzSNZm5fSrof+AioL6mnpA8kfRTNdGsCSOol6asoU9WxhQNJGiypsM7VbpKek/RJdHQhBP43\niWbPt0f9Lpc0RdKnkm6IGesaSV9Leo1QyaFUJJ0VjfOJpGeKzMp7SHpH0jdRbTMk5Uq6Peba52zp\nB+lkL25gnaQTZfPvTUjuDcGQPW5m7YAVhEQpPcysPaESwGWStiWU6D4SOBDYvYThRwJvmVkbQqLw\nzwl5bmdHs+fLJfUE9gQ6A20JFRq6SepAKJHTjmDAOyXwdp41s07R9b4Ezow515CQS7cP8GD0Hs4E\nlppZp2j8syQ1SuA6zlaIZ9Nykkl1SdOjx+8QanzVA74zs0lR+35AC+C9KL9MVeADQn2wOWY2EyAq\nIHj2/7d396xRRFEYx/+PIiJmERtT2GgMASFIehuxSB0Li2AIIYJkixA/gHaCn0GxsBNtBBtZgoUx\nIVZGu7xAwMoiNuJL0sixuEcYhoijZFIszw+22HvvzJnZ4nA5zM7ZJ8ZVYBog/6b6RdLp2prx/Kzl\n9wFKwu0AzyPiR8Z40eCeRiXdo5QhBoBeZe5ZtuHekrSd9zAOXKrUZ09l7M0GsazPOMHaQdqNiLHq\nQCbR79UhYDEiJmvrxigtWA6CgPsR8aAW4/Z/xHgMTETEB0kzwJXKXP1ckbHnI6KaiJF07h/jWh9w\nicAO21vgsqRhKP3CJI0A68B5SRdy3eQfjn8FdPPYo9lN4Ctld/pbD5it1HbPSjoDLAHXJJ2Q1KGU\nI/6mA3ySdAy4UZu7LulIXvMQpTNED+jmeiSNSDrZII71Ie9g7VBFxE7uBJ9IOp7DdyJiU9ItSnPG\nz8AyMLrPKRaAh5JuAj+BbkSsSlrJx6BeZh32IrCaO+hvwFS21X4KvKe0fnnT4JLvUlpzf6TUlKuJ\nfAN4DQwCcxGxJ+kRpTb7Lt+xuwNMNPt1rN/4ZS9mZi1xicDMrCVOsGZmLXGCNTNriROsmVlLnGDN\nzFriBGtm1hInWDOzlvwCAdo7IVKwydUAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x119875c50>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"score = metrics.accuracy_score(mul_y_test, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(mul_y_test, pred, labels=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true' ])\n",
"plot_confusion_matrix(cm, classes=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [],
"source": [
"clf = MultinomialNB()\n",
"#from sklearn.neural_network import MLPClassifier\n",
"\n",
"#clf = MLPClassifier(hidden_layer_sizes=(40,20))"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 true\n",
"1 false\n",
"2 false\n",
"3 half-true\n",
"4 pants-fire\n",
"5 true\n",
"6 true\n",
"7 barely-true\n",
"8 true\n",
"9 barely-true\n",
"Name: y, dtype: object\n"
]
}
],
"source": [
"tmp_bin_tfidf_test = mul_tfidf_vectorizer.transform(bin_X_test)\n",
"\n",
"print(mul_y_test[:10])"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [],
"source": [
"clf.fit(mul_tfidf_train, mul_y_train)\n",
"pred = clf.predict(tmp_bin_tfidf_test)"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['half-true' 'false' 'false' 'half-true' 'half-true' 'barely-true' 'false'\n",
" 'mostly-true' 'half-true' 'false' 'half-true' 'half-true' 'false' 'false'\n",
" 'false' 'half-true' 'half-true' 'half-true' 'false' 'true']\n"
]
}
],
"source": [
"print(pred[:20])\n",
"\n",
"bin_y_test = np.array(bin_y_test, dtype=object)\n",
"\n",
"bin_y_test[bin_y_test == \"FAKE\"] = \"false\"\n",
"bin_y_test[bin_y_test == \"REAL\"] = \"true\""
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.220\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVgAAAEmCAYAAAAnRIjxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXeYVFXSh9/fzACCIEFEJShJRWUF\nBXNCREVFMWFCBXNEXcMa1uy64rrqZ17jmhNGVDDrohgAERExgIIKIgoKAhKH+v44Z6AZZ7p7hu7p\nbqjX5z7ce8+551S3UF23Tp0qmRmO4zhO5inKtQCO4zirKq5gHcdxsoQrWMdxnCzhCtZxHCdLuIJ1\nHMfJEq5gHcdxsoQrWKdGkVRX0ouSZksatBLj9JX0WiZlyxWSdpb0Va7lcDKPPA7WqQhJRwLnAB2A\nOcAY4Boze28lxz0aGADsYGZLVlrQPEeSARuZ2cRcy+LUPG7BOn9C0jnA/wH/BNYFNgDuAHpnYPgN\nga9XB+WaDpJKci2Dk0XMzA8/lh1AQ2Au0CdJnzoEBfxjPP4PqBPbugFTgHOBn4FpwLGx7UpgEbA4\nznE8cAXwSMLYrQEDSuJ1f+BbghU9CeibcP+9hOd2AEYCs+OfOyS0vQNcDQyP47wGNK3ks5XJ/7cE\n+Q8A9gG+Bn4FLk7ovw3wATAr9r0NqB3bhsXPMi9+3sMSxr8A+Al4uOxefKZdnGOreN0cmAF0y/Xf\nDT+qfrgF65Rne2AN4Lkkff4ObAd0BjoRlMwlCe3rERR1C4ISvV1SYzO7nGAVP2lm9c3svmSCSFoT\nuAXY28waEJTomAr6NQFejn3XBm4EXpa0dkK3I4FjgWZAbeC8JFOvR/gOWgCXAfcARwFdgJ2ByyS1\njX1Lgb8CTQnf3e7AaQBmtkvs0yl+3icTxm9CsOZPSpzYzL4hKN9HJdUD/gs8YGbvJJHXyVNcwTrl\nWRuYYclf4fsCV5nZz2b2C8EyPTqhfXFsX2xmQwjW2ybVlGcp0FFSXTObZmafV9BnX2CCmT1sZkvM\n7HHgS2C/hD7/NbOvzWw+8BThx6EyFhP8zYuBJwjK82YzmxPn/xzYAsDMPjazD+O8k4G7gF3T+EyX\nm9nCKM8KmNk9wATgI2B9wg+aU4C4gnXKMxNomsI32Bz4LuH6u3hv2RjlFPQfQP2qCmJm8wiv1acA\n0yS9LKlDGvKUydQi4fqnKsgz08xK43mZApye0D6/7HlJG0t6SdJPkn4nWOhNk4wN8IuZLUjR5x6g\nI3CrmS1M0dfJU1zBOuX5AFhA8DtWxo+E19syNoj3qsM8oF7C9XqJjWb2qpntQbDkviQonlTylMk0\ntZoyVYU7CXJtZGZrARcDSvFM0tAdSfUJfu37gCuiC8QpQFzBOitgZrMJfsfbJR0gqZ6kWpL2lvSv\n2O1x4BJJ60hqGvs/Us0pxwC7SNpAUkPgorIGSetK2j/6YhcSXA2lFYwxBNhY0pGSSiQdBmwGvFRN\nmapCA+B3YG60rk8t1z4daPunp5JzM/CxmZ1A8C3/Z6WldHKCK1jnT5jZjYQY2EuAX4AfgDOA52OX\nfwCjgLHAZ8DoeK86c70OPBnH+pgVlWIRIRrhR8LK+q7EBaRyY8wEesW+MwkRAL3MbEZ1ZKoi5xEW\n0OYQrOsny7VfATwoaZakQ1MNJqk30JPgFoHw/2ErSX0zJrFTY/hGA8dxnCzhFqzjOE6WcAXrOI6T\nJVzBOo7jZAlXsI7jrJZIWkPSCEmfSvpc0pXx/gOSJkkaE4/O8b4k3SJpoqSxkrZKNYcnmsgSqlXP\nVKdhrsWokI7tm6fulCPGTfol1yIkZaNW+RuSWrd2ca5FqJAfvv+OmTNmpIoNTpvitTY0W/KnDXB/\nwub/8qqZ9UzSZSHQ3czmSqoFvCdpaGw738yeLtd/b2CjeGxLiIHeNpkMrmCzhOo0pM5f+uVajAp5\n6aUrci1CpWxy9N25FiEpd9x4ZK5FqJROLRvlWoQK6bFLUh1UZWzJfOpskjLijQVjbk+6o85CCNXc\neFkrHsnCqnoDD8XnPpTUSNL6ZjatsgfcReA4TmEhQVFx6iNs+R6VcJz056FULGkMIXPa62b2UWy6\nJroBbpJUJ95rQYgJL2MKK27H/hNuwTqOU3goLdtwhpl1TdYh5pzoLKkR8JykjoTdhD8Rsq7dTchu\ndhUVb4FOupHALVjHcQoPKfVRBcxsFiFvcM+Ytc1ikp3/EtJxQrBYWyU81pIUOThcwTqOU2Ck7SJI\nPkrIpdEontcFegBfSlo/3hMh6dG4+Mhg4JgYTbAdMDuZ/xXcReA4TqEh0nURpGJ9Qp6IYoKx+ZSZ\nvSTpLUnrxJnGsDwvxBBCZYuJhJSXx6aawBWs4zgFRtVdABVhZmOBLSu4372S/gacXpU5XME6jlN4\nZMaCzTquYB3HKTCUlo81H3AF6zhOYSEy4iKoCVzBOo5TeLiLwHEcJxsIigvDRVAYPwOrOHVql/Du\nfafz0UNn8fGjf+WSE3qs0H7jOfvzy5tXLrvesXMb3n9gAHPevYYDd+tYY3IuWLCA/ffYiZ67bkOP\nHbfixoFXAzB82Dvss9v27LFTF845/QSWLElW8Ttz1KlVzLs3HcpHtx3Bx3f25ZK+Yc/7Kb22YNy9\nxzB/yJmsvdYay/qvVa82T1++37L+R++xaY3ICfDsQ3dxwn47c3yvnXjmwVBi65svxzHg8L05Yf9d\nuOTUvsybO6fG5Clj6pQfOGCfHuzQ5S/stHUn7rrjFgBeeO5pdtq6E83Wqs2Y0aNqXK6klIVppTry\nALdg84CFi5bQ84x7mDd/ESXFRbx11ym89sFXjPj8B7bq0IKG9ddYof8PP83ipKsHcXbfXWpUzjp1\n6vD4c6+wZv36LF68mEP27c4u3Xtw7hkn8NizQ2nbfiNuuPYqnn7iEQ4/qn/W5Vm4uJSeFz3HvAWL\nw/f270N4bdR3fDB+GkNGTOK16w5eof/Jvbbgy+9ncsiVL9J0rbp8es/RPPH2VyxesjSrck76+guG\nDHqE2556lVq1anPhiYex7a57cMOlf+Xk86+g0zY7MvSZR3nqvts49qyLUg+YQYpLSrjyn/+iU+et\nmDtnDrvvvC3duvdg000354FHn+Lcs/5UAi0/KBAfbH6oeYd58xcBUKukmJKSYsygqEj884x9+Pvt\nQ1fo+/1PvzHum59YurRm66lJYs369QFYsngxixcvobi4mNp16tC2/UYA7NytO0Nfej7ZMBll3oLF\nANQqKaKkuAjD+PTbX/j+5z9bgwbUr1sbgDXr1uK3OQtYUppd5Qrw/bdfs2mnLqxRtx7FJSV02noH\nhr8xhCmTJrLF1jsA0GWHbrz7ek0UwV2R9dZbn06dQ1rT+g0asPEmHZj2449s3GFT2m+8SY3Lkx6Z\n2clVE7iCzROKisSHD57J90Mu4a0RExg5/gdOPWQHXn7vC36aWfOvjpVRWlrK3t22ZatNN2Dnbt3p\nvNXWLFm8mLGffAzAkBefY9rUKTUmT1GR+PDWI/j+sRN465PvGfnV9Er7/ufFT+nQqgnfPnI8o+44\nkvPuGkZN1PxsvdGmjB31AbN/+5UF8//go2Fv8PNPU2m90aa8/9YrAAx7dTC/TJuafWGS8P13k/ls\n7Bi6dN0mdedcUyAugvyQIotIOlPSF5IeraS9m6SaNx3KsXSpsV2/W2jf+1q6btaKHTu34aDuf+GO\nQe/nWrQVKC4uZug7H/Hh2ImMGT2Kr78cz633PMRVl/6N/ffYiTXrN6C4pOY8T0uXGtsNeJz2x9xP\n143XY7MNK0+IvcdWGzL2219oe9R9bHvG49x06q40iBZtNtmw3cYcfsIALjj+EC468TDaddic4uIS\nzrvmZgY/dj+nHrw7f8ybS0mt7MtSGXPnzuXYow7lHwNvoMFaa+VMjrRIJ9FLnrgQVgcf7GnA3mY2\nKdeCpMPsuQsYNvpbdt2qLW1brs3ng84HoN4atRg36Dw69vl3jiUMNGzYiO133IV33nyNk8/4K0+/\n9CYAw95+g0nfTKhxeWbPW8Swz6awZ5cNGf/drxX2OXqPTblhULC0v502m8nTf2eTVo0Z9XXlVm+m\n2PuQo9j7kKMAuO+mf9B03eZs0HYjrrtvEABTJn3DR/97PetyVMTixYs59qhDOeTQI+jV+8CcyFBl\n8sQFkIpV2oKV9B+gLTBY0gWS3pf0SfzzTw4mSbsm1OH5RFKDeP98SSNjAt4ryz+3sjRttOayhaw1\n6pTQfev2fPLVVNr0uoYOB11Hh4Ou448Fi3OuXGfO+IXZs2cBsGD+fN4b9hbtN9qEGb/8DMDChQu5\n85Yb6NvvxBqRp+ladWm4ZrD61qhdTPfOrfhqym+V9v/hlzl06xyyzTVrVJeNWzRm0k+za0TW32aG\nUjjTf5zCe6+/TPd9D1p2b+nSpTzynxvpdVjNV8AwM84+/UQ23qQDpw74a43PXz1UMC6CVdqCNbNT\nJPUEdgMWATeY2RJJPYB/AgeXe+Q84HQzGy6pPrBA0p6EGjzbEAJEBkvaxcyGlZ8vZkwPWdNrp/+a\ntd7aDbjnskMpLhJFEs+89RlDh39Zaf8um7bkyYFH06hBXfbZqQOXnLAHXfrelPZ81eXn6T9xzhkn\nsrS0lKVLl9Kr98Hsvtc+XHP5Rbz52lBs6VKOOvZEdtylW9ZlAVivST3uOXfP5d/buxMYOmIyp+3f\niXMO6cK6jesx8vYjeWXUd5x285sMfHwkd5+zByPvOBIh/v7f4cz8fUGNyHrlWcfy+6zfKCmpxYBL\nr6NBw0Y8+9BdvPDY/QDstMe+9Dyo5svRfPTBcJ56/FE227wj3XboAsDfL/8HixYu5KLzz2bmjF84\n8pDebL5FJwY9P6TG5auUPHEBpEJWE17+HCJpMtAVqAvcQlCWBtQysw6SugHnmVkvSRcCBwKPAs+a\n2RRJ/wYOAWbFIesD15rZfcnmLaq/vuVrTa6vvCZXtXnJa3JVmR67bMuY0R9nTCMWNdrA6uz0t5T9\nFrw84ONUFQ2yzSptwZbjauBtMztQUmtC9vIVMLOBkl4m5Hz8MFq6IijUu2pQVsdxKqVwkr3kh6Oi\nZmgIlMXB9K+og6R2ZvaZmV0HjAI6AK8Cx0WXAZJaSGpWA/I6jlMZ7oPNO/5FyF5+DvBWJX3OlrQb\nUAqMB4aa2UJJmwIfhAoSzAWOIlShdBwnFxSID3aVV7Bm1jqezgA2Tmi6NLa/Q3QXmNmASsa4Gbg5\nWzI6jlMFVDguglVewTqOs+qhArFg88NR4TiOkyYh37ZSHinHkdaQNELSp5I+L4txl9RG0keSJkh6\nUlLteL9OvJ4Y21unmsMVrOM4hYWEilIfabAQ6G5mnYDOQM9Yjvs64CYz2wj4DTg+9j8e+M3M2gM3\nxX5JcQXrOE7BkQkL1gJz42WteBjQHXg63n8QOCCe947XxPbdlWIiV7CO4xQcaSrYppJGJRwnVTBO\nsaQxhKig14FvgFlmVpY1fgrQIp63AH4AiO2zgbWTyemLXI7jFBYiXRfAjFQ7ucysFOgsqRHwHFBR\nmYuy7a4VTZp0K6xbsI7jFBQitfVa1SgDM5tFCNfcDmgkqcz4bAn8GM+nAK0AYntDoOLUbRFXsI7j\nFBwZiiJYJ1quSKoL9AC+AN4m5B8B6Ae8EM8Hx2ti+1uWIpmLuwgcxyk4MhQHuz5hd2cxwdh8ysxe\nkjQeeELSP4BPgLLETvcBD0uaSLBcD081gStYx3EKi/R9sEkxs7HAlhXc/5aQnrT8/QVAn6rM4QrW\ncZyCo1B2crmCdRynoChb5CoEXME6jlNwZMJFUBO4gs0SW27SguHvDsy1GAXHb4PPzLUIToYpzrS1\nKXcROI7jZA1XsI7jOFlAiKKiwgjhdwXrOE7hURgGrCtYx3EKDPfBOo7jZA93ETiO42SLwjBgXcE6\njlN4uIvAcRwnC1QnHWGucAXrOE7B4T5Yx3GcbFEYBqwrWMdxCg93ETiO42QBCYoKJNlLYTgyVmNe\ne/UVtth8Ezbv0J7r/5VfyWNcturhsq0sma/JlS1cweYxpaWlnH3m6bzw4lA+GTueQU88zhfjx+da\nLMBlqy4uW2aQUh/5gCvYPGbkiBG0a9eeNm3bUrt2bfocdjgvvfhC6gdrAJeterhsGSC6CFIdKYeR\nWkl6W9IXkj6XdFa8f4WkqZLGxGOfhGcukjRR0leS9ko1hyvYPObHH6fSsmWrZdctWrRk6tSpOZRo\nOS5b9XDZVh6RGQULLAHONbNNCeW6T5e0WWy7ycw6x2MIQGw7HNgc6AncEQsmVkrWFayk1pLGZWHc\n/pJuq0L/RpJOy7Qc2aSiisD54lty2aqHy5YZMuEiMLNpZjY6ns8hlOxukeSR3sATZrbQzCYBE6mg\nOGIieW3BSspklEMjoEIFm+pXKFe0aNGSKVN+WHY9deoUmjdvnkOJluOyVQ+XLQOk7yJoKmlUwnFS\npUNKrQkVZj+Kt86QNFbS/ZIax3stgB8SHptCcoVcYwq2RNKDUeCnJdWTdJmkkZLGSbpb8adS0juS\n/inpf8BZktaR9EzsO1LSjokDS2ogaZKkWvF6LUmTy64TGAi0iz6V6yV1i/6Xx4DPylvaks6TdEU8\nbyfpFUkfS3pXUocsflfL6Lr11kycOIHJkyaxaNEiBj35BPv22r8mpk6Jy1Y9XLaVRyzfLpsiimCG\nmXVNOO6ucDypPvAMcLaZ/Q7cCbQDOgPTgBsSpi7Pn83+BGoqDnYT4HgzGy7pfoIleZuZXQUg6WGg\nF/Bi7N/IzHaNbY8R/CHvSdoAeBXYtGxgM5sj6R1gX+B5go/kGTNbXE6GC4GOZtY5jtuNYN53NLNJ\n8ResMu4GTjGzCZK2Be4AupfvFH8hTwJotcEG6XwvSSkpKeGmm29jv333orS0lH79j2OzzTdf6XEz\ngctWPVy2TJC5MKxoiD0DPGpmzwKY2fSE9nuAl+LlFKBVwuMtgR+TjV9TCvYHMxsezx8BzgQmSfob\nUA9oAnzOcgX7ZMKzPYDNEr7QtSQ1KDf+vcDfCAr2WODENOUaEX0plRJ/3XYABiXIUKeivvEX8m6A\nLl26Jv1lS5eee+9Dz733Sd0xB7hs1cNlW3kysdEgvjXfB3xhZjcm3F/fzKbFywOBsjfbwcBjkm4E\nmgMbASOSzVFTCra8sjGCFdjVzH6Ir+JrJLTPSzgvArY3s/mJAyT+gkXLuLWkXYFiMxsnqRXLFfZ/\ngFcqkCtxniWs6DIpk6cImFVm+TqOk2MyF+e6I3A0wUU4Jt67GDhCUmeCnpoMnAxgZp9LegoYT9AX\np5tZabIJakrBbiBpezP7ADgCeI9gFc6IFuIhwNOVPPsacAZwPYCkzmY2poJ+DwGPA1cDmNkPBB8K\n8bm1gfKWbyLTgWax31yCy+IVM/s9+nj7mNmg+Ku3hZl9mu6Hdxwnc5T5YFcWM3uPiv2qQ5I8cw1w\nTbpz1NQi1xdAP0ljCe6AO4F7gM8Ir/Ujkzx7JtA1LpCNB06ppN+jQGOCkv0TZjYTGB4X1a6voH0x\ncBVhFfEl4MuE5r7A8ZI+JbgyeieR13GcLFMoO7mybsGa2WRgswqaLolH+f7dyl3PAA6roN8DwAMJ\nt3YCnjazWUlkObLcrXfKtd8C3FLBc5MIgcWO4+QBhZLsZZXIpiXpVmBvIP+9847jrBxeVbZmMbMB\nuZbBcZyaIfhgcy1FeqwSCtZxnNWJtHMN5BxXsI7jFBzuInAcx8kGeRQlkApXsI7jFBQhXWFe56la\nhitYx3EKDrdgHcdxsoT7YB3HcbKA5FEEjuM4WaNADNjKFayktZI9GBPTOo7j1DhFBaJhk1mwnxPS\ndSV+krJrA1Y+o7TjOE41KBD9WrmCNbNWlbU5juPkCgmKC8QHm1YwmaTDJV0cz1tK6pJdsRzHcSon\nzZpcOSelglUojb0bIfM3wB+ECgGO4zg5YVXKB7uDmW0l6RMAM/tVUu0sy+U4jlMhAorzRYOmIB0F\nu1hSEbGuViypsjSrUjmO41RGHrkAUpGOD/Z2QlnbdSRdSaindV1WpXIcx0lCJlwEklpJelvSF5I+\nl3RWvN9E0uuSJsQ/G8f7knSLpImxhNVWqeZIacGa2UOSPiaUzwboY2bjkj3jOI6TLUTGogiWAOea\n2WhJDYCPJb0O9AfeNLOBki4ELgQuIFRN2Sge2xJqC26bbIJ0U9IUA4uBRVV4xnEcJytkIorAzKaZ\n2eh4PodQnLUFoajpg7Hbg8AB8bw38JAFPgQaSVo/2RzpRBH8nVCptTnQEnhM0kUppXccx8kC6bgH\non5tKmlUwnFS5WOqNbAloar0umY2DYISBprFbi2AHxIemxLvVUo6i1xHAV3M7I8oyDXAx8C1aTzr\nOI6TcdKMIphhZl1TdZJUn7DOdLaZ/Z7E+q2owZKNnc7r/nesqIhLgG/TeM5xHCcrZGqjgaRaBOX6\nqJk9G29PL3v1j3/+HO9PARJ3uLYEfkw2fqUKVtJNkm4kbCz4XNK9ku4BPgNmpSW94zhOhhFQpNRH\nynGCFr4P+MLMbkxoGgz0i+f9gBcS7h8Towm2A2aXuRIqI5mLoCxS4HPg5YT7H6YW3XEcJ0tkLg52\nR8IO1c8kjYn3LgYGAk9JOh74HugT24YA+wATCYbnsakmSJbs5b7qy+04jpM9MpFw28zeo2K/KsDu\nFfQ34PSqzJFOFEE7SU/EwNqvy46qTOJUn9defYUtNt+EzTu05/p/Dcy1OCvgslUPl23lyJSLoCZI\nZ5HrAeC/hM+1N/AU8EQWZXIipaWlnH3m6bzw4lA+GTueQU88zhfjx+daLMBlqy4uW2ZYZbJpAfXM\n7FUAM/vGzC4hZNdysszIESNo1649bdq2pXbt2vQ57HBeevGF1A/WAC5b9XDZVh4phGmlOvKBdBTs\nwrja9o2kUyTtx/LAWyeL/PjjVFq2XB4V0qJFS6ZOnZpDiZbjslUPly0zFEq6wnQU7F+B+sCZhFW3\nE4HjsimUpNaS0s53IOkKSefF8w6Sxkj6RFK7cv26Sdoh0/Jmi+BTX5F8efVx2aqHy5YZCsVFkE6y\nl4/i6RyWJ93OZw4AXjCzyyto6wbMBd4v3yCpxMyWZFm2KtGiRUumTFm+M2/q1Ck0b948hxItx2Wr\nHi7byiNUMCVjklWVfY4k28DM7KCsSLSc4rixYQdgKiHRwlHASUBtQiza0WVbeAEk7QOcDZRK2sXM\ndktoaw2cEtuOAgYAxwO/EvYgj5Y0B5hrZv+Oz4wDepnZ5PjMmXHuj4DTzKw0i5+frltvzcSJE5g8\naRLNW7Rg0JNP8MDDj2VzyrRx2aqHy5YB8sgFkIpkFuxtNSZFxWwEHGFmJ0p6CjgYeNbM7gGQ9A+C\ngry17AEzGyLpPyQoyYS2yeXbYiDxxkAPMyuVdEVFgkjaFDgM2NHMFku6A+gLPFSu30mEHwBabbDy\nRXdLSkq46ebb2G/fvSgtLaVf/+PYbPPNV3rcTOCyVQ+XLTPkiwsgFck2GrxZk4JUwCQzK9td8THQ\nGugYFWsjgl/41QzMMygNS3R3oAswMv6Prcvy/cnLMLO7gbsBunTpmjQJRLr03Hsfeu69TyaGyjgu\nW/Vw2VaOVa1kTK5YmHBeSlBqDwAHmNmnkvoTfKqVIul0wqIchC1uFTEv4XwJKy78rVE2FPCgmXma\nRsfJAwrEBVtwybMbANNiBpy+qTqb2e1m1jkePxIW6hokeWQysBVALAfRJt5/EzhEUrPY1kTShtX/\nGI7jrAyr0k4uACTVyaYgaXIpYYHpdeDLajz/InBgDOPauYL2Z4AmMfHDqcDXAGY2HrgEeE3S2Dh/\n0kzmjuNkhxDnuoqEaUnahpDSqyGwgaROwAlmNiBbQpnZZKBjwnXigtWdFfS/oqLzCvp9DWyRcOvd\ncu3zgT0refZJ4MmkgjuOUyMUF8i7dzpi3gL0AmYCmNmn+FZZx3FyREj2opRHPpDOIleRmX1XzuTO\navyn4zhOMgrEgE1Lwf4Q3QQmqZgQoO/pCh3HyQnSKrCTK4FTCW6CDYDpwBvxnuM4Tk7IEw9AStLJ\nRfAzcHgNyOI4jpMWBWLAphVFcA8V5CQws0prjDuO42QLQcG4CNLxFb9BCLR/ExhOyAW7MOkTjuM4\n2SKNTQZpVpW9X9LPialRY+rTqTFWfkxMIFXWdpGkiZK+krRXOqKm4yJYIfZT0sOEQHvHcZycoEpr\nFVaJBwhJrR4qd/+m8smiJG1GcJVuDjQH3pC0cao8JtWJdmgD+DZRx3FygoCSotRHKsxsGCFdaTr0\nBp4ws4VmNomQLnWbVA+l44P9jeU+2KIo0IVpCuU4jpNx0twK21TSqITru2PGu1ScIekYYBRwrpn9\nBrQAPkzoMyXeS0pSBRtrcXUiJLwGWGoV1ZVwHMepIcrKdqfBDDPrWsXh7wSuJhiVVwM3EEpkVTRj\nSl2Y1JCOyvQ5MyuNhytXx3Fyi0IUQaqjOpjZ9KjrlgL3sNwNMAVoldC1JfBjqvHS8cGOiKn7HMdx\nck6ZBZuNdIWSErPkHQiURRgMBg6XVEdSG0LFlRGpxktWk6usCOBOwImSviEkpxbBuHWl6zhOTsjE\nTi5JjxOS9jeVNAW4HOgmqTPh9X8ycDKAmX0eS1eNJyTmPz2dmnzJfLAjCMmnD1iJz+A4jpNhRFEG\nwrTM7IgKbt+XpP81wDVVmSOZglUc9JuqDOg4jpNNpMLJB5tMwa4j6ZzKGs3sxizI4ziOk5J8yfea\nimQKtphQubUwPonjOKsFYtXIpjXNzK6qMUkcx3HSpFCSvaT0wTqO4+QTYtWoaLB7jUnhOI6TLkp7\nq2zOqVTBmlm6SRAcx3FqDAHFha5gHcdx8pXCUK+uYB3HKUAKxIB1Bes4TmEh5C4Cx3GcbFEoi1yF\nEu2w2vLaq6+wxeabsHmH9lz/r4G5FmcFXLbq4bKtPErjyAdcweYxpaWlnH3m6bzw4lA+GTueQU88\nzhfjx+daLMBlqy4uWwaIYVqpjnzAFWweM3LECNq1a0+btm2pXbs2fQ47nJdefCHXYgEuW3Vx2Vae\nsjCtVEc+4Ao2j/nxx6m0bLk8iXqLFi2ZOnVqkidqDpeterhsmcFdBBlGUmtJRyZcd5P0UhXHuDjz\nkmWPiir05Murj8tWPVy2zCAPXyMoAAAgAElEQVSlPvKBglGwQGvgyFSdUlChglUg776LFi1aMmXK\nD8uup06dQvPmzXMo0XJcturhsq087iJgmcX5paR7JY2T9KikHpKGS5ogaRtJTSQ9L2mspA8lbRGf\n3VXSmHh8IqkBMBDYOd77a8I8RXG8dRKuJ0pqWk6egUDd+PyjUb4vJN0BjAZaSZqb0P8QSQ/E83Uk\nPSNpZDx2zNb3lkjXrbdm4sQJTJ40iUWLFjHoySfYt9f+NTF1Sly26uGyZQKl9V8+kO042PZAH+Ak\nYCTBAt0J2J9gTf4AfGJmB0jqDjwEdAbOI9S8GS6pPrAAuBA4z8x6QXARAJjZUkmPAH2B/wN6AJ+a\n2YxEQczsQklnmFnn+HxrYBPgWDM7Ld6r7HPcDNxkZu9J2gB4Fdi0fCdJJ8XPSqsNNqjSF1URJSUl\n3HTzbey3716UlpbSr/9xbLb55is9biZw2aqHy5YZMlST636gF/CzmXWM95oATxLemCcDh5rZbwrK\n4WZgH+APoL+ZjU45R7YqcUcF9rqZbRSvHwJeNbNHJbUFniUUFjvYzL6NfX4AOgKnEio6Pgo8a2ZT\nokItr2DPM7NekloBL5jZVpKeAB4xsz/5ZyXNNbP6CfK9bWZtKmk/BOhlZv0l/cyKJXrXATqY2ZzK\nPn+XLl1t+EejqvSdOc6qyI7bduXjj0dlzKTcuGNnu/Wp11P267l5s4/NrGtl7ZJ2AeYCDyUo2H8B\nv5rZQEkXAo3N7AJJ+wADCAp2W+BmM9s2lQzZ9jsuTDhfmnC9lGA9V/Slm5kNBE4A6gIfSuqQbBIz\n+wGYHq3gbYGhkooT3AyVJQ6fV36ohPM1Es6LgO3NrHM8WiRTro7jZJdMLHKZ2TCgfNbA3sCD8fxB\nlhd97U1QxGZmHwKNypX4rpBcL+wMI7zal1mkM8zsd0ntzOwzM7sOGAV0AOYADZKMdS/wCPCUmZXG\no0whXhb7LJZUK8kY0yVtGhe8Dky4/xpwRtlFLOvrOE6OyKIPdl0zmwYQ/2wW77cguDTLmBLvJSXX\nCvYKoKuksYRFrH7x/tlxYexTYD4wFBgLLJH0aeIiVwKDCTXE/ptkvruBsZIeraT9QuAl4C1gWsL9\nM8vklDQeOCWtT+c4TsapQhRBU0mjEo6TVnLa8qT0r2ZtkcvMJhP8qWXX/Stp613BswMqGbZ8lYV3\nEs47ERa3vkwi0wXABQm3OpZrfxp4uoLnZgCHVTau4zg1S5qLXDOS+WArYbqk9c1sWnQB/BzvTwFa\nJfRryYrrMhWSaws2I0Rn9DPARbmWxXGc7JNFF8Fglr9J9wNeSLh/TIyZ3w6YXeZKSMYqka4wLorl\nb+ofx3EyhoBMFJWV9DjQjeBKmAJcTtAjT0k6HvieEGYKMIQQQTCREKZ1bDpzrBIK1nGc1QiJogwE\nwprZEZU0/angq4V41tOrOocrWMdxCo782KeVGlewjuMUFMFFUBgq1hWs4zgFR4HoV1ewjuMUHvmS\nzCUVrmAdxyk43IJ1HMfJEq5gHcdxskAoCVMYGtYVrOM4hUUelYRJhStYx3EKDlewjuM4WSF/SsKk\nwhWs4zgFh1uwqzmffPE9jbc+I3XHHDD1vf/LtQiVstnZz+dahKT89+xdci1CpezcvmnqTjkg00Wp\nhG+VdRzHyRpJCpTmFa5gHccpOApEv7qCdRyn8CgQ/eoK1nGcAkPuInAcx8kKwl0EjuM4WaNA9Ksr\nWMdxCg93ETiO42SJTOlXSZOBOUApsMTMukpqAjwJtAYmA4ea2W/VGX+VKNvtOM7qhdI4qsBuZtbZ\nzLrG6wuBN81sI+DNeF0tXME6jlNQhEUupTxWgt7Ag/H8QeCA6g7kCtZxnMIipitMdQBNJY1KOE6q\nYDQDXpP0cUL7umY2DSD+2ay6oroPNg+oU7uEN+47m9q1SygpLua5Nz7hH/8Zwt1XHsXOXdoze+4C\nAE667GHGfj2VXt3+wmWn9mKpGUtKl/K365/m/THfZl3OAaeewGtDh9B0nWYMHzkGgOOPOZKJE74C\nYPbs2TRs2JD/ffBx1mUBaN64LrcetzXrNFwDM+PhYZO4982JbN6qIf86aivq1CqmtHQpFz76CZ9M\n/o2Dtm3FGT03AWDeglIueHQ046fMzopsN11yFiOGvU6jJk258/lhAMyZ/RvXnnsSP//4A82at+Ki\nG+6hQcNGAIwdMZy7r7uUJUuWsFbjJvzrgZrPyfD111/R/6gjll1PnvQtf7/sSk4fcFaNy5KKNO3T\nGQmv/ZWxo5n9KKkZ8LqkL1dWtkRcweYBCxctoedJtzBv/iJKSop46/5zeG34eAAu/r/nee6NMSv0\nf/ujr3jpnc8A6LhRcx657jg6H/SPrMt5RN9+nHDyaZx24nHL7t330GPLzi+96HzWWqth1uUoY8lS\n44pBY/ns+1msWaeE1y7dnWHjp3PpwVtww4tf8Na4n9i943pcesgWHPTv//H9jD848Pr/MfuPxXTv\nuB7/ProL+1z7VlZk63HA4ex35PHccPHyhD9P3XsrnbfbmUNPOJOn7r2FQffdynHnXMrc32dz+z8u\n5Oq7HqfZ+i2ZNfOXrMiUio033oT3R4wGoLS0lI3btmK//av9dpxdMrTIZWY/xj9/lvQcsA0wXdL6\nZjZN0vrAz9Ud310EecK8+YsAqFVSTElJMWaV5yAq6wuwZt06JOmaUXbYaWcaN25SYZuZ8fyzT3NQ\nn8NqRhjg59kL+Oz7WQDMW7iECdPmsF6juhhGgzWC7dCgXi1+mjUfgFHfzGT2H4sB+PjbmazfuG7W\nZPtL1+2XWadlfPj2K/ToHb6fHr0P44O3hgLwzpBn2aHHPjRbvyUAjdZeJ2typcs7b71Jmzbt2GDD\nDXMtSgWIIqU+Uo4irSmpQdk5sCcwDhgM9Ivd+gEvVFdSt2DzhKIi8f5jF9Cu1Trc9eQwRo77jhP7\n7MwVp+/HRSfuzTsjvuKSWwazaPESAPbfbQuuGrA/6zRpwEFn/ifH0sMHw99jnWbNaNd+o5zM32rt\nenRs1YjRk37lsic+5fGzd+ayPltQJLHfwLf/1P/Indrw1rifalTGWTN/ock66wLQZJ11mf3rDACm\nTv6GJUuWcEH/A5n/x1x69z2R3XsfWqOylefpQU/S57DDcypDZWQwXeG6wHNxQawEeMzMXpE0EnhK\n0vHA90Cf6k6wWilYSY2AI83sjlzLUp6lS43tDh9Iw/p1efLGE9ms3fpcdutgfprxO7VrlXD7pUdw\n7rE9uPbuVwAY/PZYBr89lh23asdlp+3LvqfcllP5nxn0BAf3yc0/yHp1irn31O257MkxzF2whH7d\n2nL5U5/y8uip7N+1JTf268KhN727rP+Om6zDETu1pvd17+RE3vKUlpYycfynXHvv0yxcuIBz++7L\nJp260LJ1u5zIs2jRIoa8/CJXXv3PnMyfFhnQsGb2LdCpgvszgd1XfobVz0XQCDit/E1JxTmQpUJm\nz53PsFET2HOHzfhpxu8ALFq8hIde+JCum7f+U//ho7+hbcumrN1ozRqWdDlLlizh5cHPc8DB1f6h\nrzYlxeK+U7fn2Y++Z8gnPwJw6PateXn0VAAGj5rClm2WuzU2bdGQG47pQv/b3+e3eYsqHDNbNFp7\nHX79ZToAv/4ynYZNQoLspuuuT5cdu7NGvTVp2HhtOnbZjklffV6jsiXy2qtD6dx5S5qtu27OZEhF\nJlwENcHqpmAHAu0kjZE0UtLbkh4DPpPUWtK4so6SzpN0RTxvJ+mVGMrxrqQOmRSqaeP6NKwf/IFr\n1KlF92034avJ01mv6VrL+uy/2xaM/yYokLatlmeu79yhJbVrlTBz1rxMilQl/vf2m2y08Sa0aNGy\nxue+qV9XJkybw12vT1h276fZ89lh4+DH3KlDM779eS4ALZrU5f7TtueM+0fy7fS5NS7rdt324o0X\nngTgjReeZLvdeob7u/Xk89EfUrpkCQvm/8FXn42mVdvcuFoAnn7qCQ45ND/dA2VkeKNB1litXASE\nHRkdzayzpG7Ay/F6kqTWSZ67GzjFzCZI2ha4A+hevlOMowuxdLXqpy3Uek3X4p6rjqa4qIiiIvHM\n66MZ+u44ht41gKaNGyDB2K+mMOCaJwA4cPfOHNlrWxYvKWXBwsUcfcH9ac+1MpzY/yiGv/s/Zs6c\nQceNW3Ph3y/jqH7H8ezTT9bo4lYZ27Rfmz7bb8j4KbN447IeAFz77DjOe+hjrj68MyVFYuHipZz/\nUAgbO6fXZjReszYD+24JQGnpUva6JjtRBNedfzJjR77P77N+5ejdO3PUaefT54QBXHvuibz27GOs\ns34LLr7xXgA2aLcxXXbszmkH7UZRkdjr4L603mjTrMiVij/++IO33nyDm2/LvV+/UgqobLeSrVav\nakQl+pKZdYwK9nIz2618W7w+D6gP/Bv4BfgqYag6Zpb0X0BRvWZWZ5PcLlRUhtfkqj5ek6vq7LLD\nNoz+eFTGVGKnLbvYkLc/SNmvZeM6H6cRB5tVVjcLtjyJ79VLWNFlskb8swiYZWada0wqx3GSUiAG\n7Grng50DNKikbTrQTNLakuoAvQDM7HdgkqQ+AAr8aeXRcZyaI82tsjlntbJgzWympOFxMWs+QamW\ntS2WdBXwETAJSNwy1xe4U9IlQC3gCeDTmpPccZxEPB9snmJmRyZpuwW4pYL7k4Ce2ZTLcZz0KQz1\nuhoqWMdxCpt8cgGkwhWs4zgFh7sIHMdxskRhqFdXsI7jFCAFYsC6gnUcp9AQKhAb1hWs4zgFRajJ\nlWsp0sMVrOM4BYcrWMdxnCzhLgLHcZwsIEFRYehXV7CO4xQgrmAdx3GyQ6G4CFa3bFqO46wCFCn1\nkQ6Sekr6StJESRdmXM5MD+g4jpN1MlAzJtbiux3YG9gMOELSZpkU0xWs4zgFh9L4Lw22ASaa2bdm\ntoiQhrR3RuVcnUrG1CSSfgG+y9BwTYEZGRor0+SzbJDf8q0usm1oZutkaCwkvUKQLxVrAAsSru82\ns7sTxjkE6GlmJ8Tro4FtzeyMTMnqi1xZIsN/oUblurZQZeSzbJDf8rls1cPMMpWbuSIzN6MWp7sI\nHMdZXZkCtEq4bgn8mMkJXME6jrO6MhLYSFIbSbWBw4HBmZzAXQSFwd2pu+SMfJYN8ls+ly2HmNkS\nSWcArwLFwP1m9nkm5/BFLsdxnCzhLgLHcZws4QrWcRwnS7iCdRzHyRKuYAsIlSulWf7acZz8whVs\ngSBJFlckJa0NYAW8QlnRj4OkGv/7uKr8SK0qn2NVw6MICgxJA4DtgWnA/4ChZrY4t1JVjbIfC0l7\nAZsDdYEbzGxBikezIkc8349gcEwDRpvZkpqUpSokfH/NgaVm9lPi/QzPdSAwDygys1cyOfbqgFuw\nBYSkPkAf4FRgT2CnQlOuECxvSXsD1wCfAIcCN+RCDgBJ5wHnAF2A64AeNS1LVYjf377Ai8C1kl6X\nVJwF5XoGcB7QBHhG0s6ZHH91wBVsHlP22pfw6twCGAgcSNjS9/fYvl5OBKwGCa+yPQk7Z9YC5gDX\nlmuvKXlaAVub2W7AQoK19pqkujUpR1WQ1Jnw//4A4C2gDVA/oX2lvkMFNgT2ALoT/t79D3hfUq2V\nGXt1wxVsnlLuda9MgX4L/BM4xsz2MrPFks4FTsmF/7KaNIh/iqAkBgDHmtkUSQcBR9awPAuBJZIe\nILheDjKzpcC+UfnmI38AdwI7A2cAe5rZbEk7QUZ88wJ+IezVvwzYFTjEzEqBfpI2XsnxVxsK5R/l\nakO0HhJ9g2cDj0laE/gG+AJ4SVIXSUcAfYFBUSnkNZLaA5dIagM8DhwGPGhmEyTtQPjx+L6GZDlU\n0q5m9jMwkeAL/puZLZJ0HEGx5JUfVlJHSX8BFgEXA38DdjWzb+Pr++WSNljJOXYCTjGzP4B6wLlm\ntr+Z/SHpSOAEYO7KfZLVB89FkH/UNrOFAJKOJ7xG9zGzeZK+Bu4h+AqvBOYD/TK9fzqLNI5/nkyw\nwA4C7pK0C9AVOM/M3q0hWTYELo3f8QtALeBeSaOA3YFDzWxaDcmSkvgD2xv4C8FqvRC4D9gjLnad\nAVxsZtX6gYpvQCJk9u8k6XDgNGAtSW8C44AdgOPMLKMZp1ZlPIogj4gW3kCC1fCdpHMI1tUCoBNw\nInAb8BDh1VbR0shrJHUys0/jeVdgP0LkwECClVSH8MPyRTZWwsvJ0tbMvo3nA4CjCYqkTIGUAF+b\n2eRsyZAu5b8LSR2B/YH2hEW5XQlugrrA82b2enW/P0kbmNn3kuoRFlK7AB+a2WOS9gdKgS/Kvjsn\nPVzB5hHx9e50Qo7Kc4GtCf/4Ae4HlhJ8lOfkgwJIRkIoUV3gDqChmR0U27YBLgemAreXKd9syhHP\ntya8EQw3s2fjvXOA84EjzeztbMlRXSTtCJxgZsfG602BgwkW+JVmNiUDczQHPiC4BoZGa7lvnOdR\n4LF8DlvLZ9wHm0fE17vbCYtZNxBWbg8BDjazp4BZhAWvhTkTMg0SlOvewMMEf6FJegjAzEYAH/Pn\nkh5ZkSOe9wX2BX4FdoxWGWZ2I+Et4W+S1siWLNVB0laEuNweku4GMLMvgI+ALYFrJDVYmQXO+AOz\nPeH/0T8l7Wlm82JplVqEN6f6ycZwkmBmfuToIPi8iiq434Tw+vwEsEG8dxowGvhLruVO87NtFeXf\nLl6vAzwNDCHE8L5PCI+qCVm2BV6K53UJi0M3AP0JFu1/CHWjcv69Jci8BUGRNgNqA+OBe2NbZ4L/\nddOVnGMP4KmEv2NHElwl+wK9gOeB9XP9XRTy4S6CHCKpvpnNjecnE2JCi8zsOkkNgQuA1gR/2/rA\nbMtTH5ik2hYqcyJpXcLmgX8C25vZuHi/mKDMagPPmtkLNSDL1sAlBPfKYRaiBJoSfJm7AB0JYW/j\nsyFLVUiw/DcA3gT+bWZ3xba6wCjgS4Lr6CSr4s4qhaz97c1svKT+hB+ab8xsv4Q+hxDcU/OBs81s\nbAY+2mqLK9gcEV9Re5vZ8TEU60DgUsIi1mdm1ldSA8Jup3qEf1B5GYoVFeehwG/AdIIldDdwFqHO\n0Vlm9l1C/zXMbEE2FrTiD9NWLF+0+oUQobA7QbkPsxCRUWxmpZIamdmsTMpQVeL/5/UshKttDcwE\nriJY3luY2fzYrzbBNTCv7EerivO0J/jDpwEbEPz6ZxNC5W5J6NcQWGJm81bukzmuYHOAQrKWJwkK\naAlBsZ4EnEn4R2WEPeaHxH98dS3Ea+YtktoRfMYlwC5m9nXcDdQf6EAIIZpUA3KsT1D2+wFtCa/R\nCyX9DdgIeAZ4N5+UR/yengXeAHYi/F34kqAM2xA2P2Qk9lTSv+P4F5jZndFPfjLwppndmok5nOX4\nIlduWERQrJfF4+/ANgSL9kCC/3U3SY+Y2ZwCUK4iLB59SViI6wIQrdZ7CYtIN9bEIpKF2NVpBEtv\nMLBuvP8vwiaNYwiLOnlD/J4eJriCXrMQ12wE6/IrwtbdTC00/YcQqXKSpMPMbChwNXBU3LjiZBBX\nsDnAzOYQfGz7AhMSXp8/iH+2IyQduSQH4lUZC/xG2DjQBzhfIVEIBPfGG8BfLUvZsiQ1KnfraUJQ\n/kzgREld4v1HCQuFVX69rgHGEDYLnBMV39LoGrgEGEbYabbSmNlEM3uY+MOukNGsJeFH/8NMzOEs\nx10EOSK+FrYn+FzvAIYSdmlNJvgLu5vZxJwJmIJyIVC1gNIyH7Gk7YAHCIq1C3C6mY3OkhwHAMcR\nFrDmSyqJslgMczqMsGDTmBCdcWqmXrczScIC116Elf0+hG3DpwIXlvlhMzxnT+B6QoKb461wdgQW\nDK5gc0xUAk8S4hDfI2QumlkT/sqVJe5//6rMhSGpBWG32Q2ERZRTgMFm9nqW5q9LsEpfIVimX5vZ\n77Fte8Ib2lxgO0L2rissi5saqoqkEgulo+tEP3EzQlzwNoRX+VnA1dmKtogyNCO8hPySrTlWZ1zB\n5gGSOhHSzl1kIcC7IJB0A9DEzI6V1AQYAdxkZrfH9iIzW5qlaIF6FhKQ9CdELbQhxNXOklTmfz3Z\nzIbE/rUsT3LnRp/15oQk43vFe60Jq/o3m9kLkloCxRa2TGd1+7CTPdwHmwdEq2pXgl+2kPgvMCcu\nXi0i+FnLlKvKXAZZUK6bAGdLWgeYQUiA8jpxQQvYBDi6TLlGGXKuXBWJPutxhO+ubGHpIuCNMmvV\nzKaU+eZduRYubsE6KYkbBzYxs2GSdgWaAy9EC/I54BMzuyqhf1E2Y3YldSf4KCcBI4GfCMmnGxP2\nzY+pCTnSpSzuN54vi7tVSIu4oZldXhaXG++7xbqK4Bask5S4aNQbODX6XBcS4iivVSi1cjnQQVKj\n+OpLtpRawvhvAYMI20i3I1R3eARYDPRRSH+YNTmqQgzav1/SHpLqACMl/VXSPoSY3CMk7e7KddXE\nFayTFAtZlF4HhhM2Dcwn7GG/g7Ap4hpCeFbXbCqG8oonKtlnCFuIT49y3UEIC+tWEzG3aVJCsLJP\nJyz8HUjIInYZ4YfqM2JcbrS4XbmuQriLwKmUcqFYLQmv5R2BR6OCQ6F8SB+C0u1tZrOzLMcxBF/r\n14TogY5AP4LS+i9BoS3O9aq4VsyH0I6Q9+AQQorBEdGy7UvYudUN2NLMpudKXic7uIJ1KiQhLnNr\nQtq6mYQ0iqcTst4/bWavlfUlZHc638xmZlGmswm+1scIkQMfAf8gbMw4k7CB4P9y7RpQyM3Qk5C5\n60fCj89DwN7APoTogbcToiyuI4Tm/StnQjtZwV0EToVE5doTeJCw7fQLQm7QZ4BPgaMl7Rm770Cw\nxDL6Wq6EPKfRUu5EKKm9FuHvbl1CHocJwP8RLOuc+10J21y/JpR1eZ6wIDiJsMPsJeAsSbslyDqb\nsJvKWcXwmlxOhSgkpDmPkNavNUHB/mBm0yU9TUg5WFaz6hvCzrOpmZQhYWdYG0KF0ysJQfj7E8pJ\nH0VIuVdKSCaT89exsvA0STMJ+SbGE2QeY2Y/S3oSKCYk+P6MsGi4BiH7mLOK4S4C509Ea3EWITHK\nQsLreD8LGbIOI+yN/9lCur+Mh0IpVJjdwMyeUKibdRbwNiFJtwg5TS9WqFKwJXB9PvgvE9wq+xK+\nt9EEy/QyYKSFPL8tCJsivrVYPDCfNkE4mcUtWAdYQTlsSwi9OosQJbAH0DzGvG5FeO2dYLHiapZe\nyRsTwsA6EBTUXgSLtT3BLVC2yWAvYI98UK6wzK3Si5AN7W9m9qukOYQyQAMkPUJIoXiqmf2YsOnA\nlesqiluwzjIUKr72IbzOPq5Q/O4DQv2sXwjK9ops7o1PkGUP4EZCZdMTYwxpH0LpmTYEK3qEVbNM\ndTaI39cgQvq/EQTf9GaEPLlGyLv6qpm9mjMhnRrFFayTaL2eRYgSeICwJ36eQkKVwwmvvJPN7P2a\nCoaX1JuQYezM6C4oJsTitgJuMbNfsy1DVYg/AncTIi7aEao7dAWeM7OrE/r5ZoLVBFewqzEJinWd\nsrhRSf0I8ZlXAqPMLKcVbKM/81rgn1HJFgH1LWbNyiUJ399WhEW/3wgLW3sTtg8Pl9SDkDj7SGBu\nnkQ5ODWE+2BXYxJ8hgPiivZwM3tQofbT34HrJQ0r28aZIxlflrQUuFvSEjN7Gsi5coVl399ewK2E\n8KsjCLlbb4Nlbo6bCP7YvJDZqVlcwa7GSOpG2Op6MKGCwnaSWpnZLfF19xLC7qPfciclmNnQmBjl\nm1zKkUi0pBsA5xOqrw6Ji1iDYsqER4nVHSyUZXFWQ1zBrmYkZm0iFCM8nJDeb0PCbqwD4qvvzZJe\nsFAKJudYlpJ2V5UE/2ktYA5hN9n8+L2Ojn7s4+KbwHluua7euIJdTZDUwEIBxdKYFWt9wtbX3wm1\nwQ62kNx5f6CLpNZmNjmHIucl0S1wAKFywzfAjoSNA6MICvcPwBSykM3JmaBOXuAKdjVAUj3gZUk3\nE7I33Q58AiwFGgJbAaMlvU/4O/FvV64rkrCg1YgQyfAoIfRqJ0JpnHoxymEn4FILWcic1RyPIlhN\nkHQgYZPAHOASM/tQUluC9borIQB+EXCdmT2XO0nzF0nbEMKu1i4Lu4oW/+WEdI4PA7Vj9ICHYjlu\nwa4umNlzkuYSEo70IJRo/oFQufQrglVWL+6Xd+UQSbBctwPuBb4Dmkl6D3jPzAZLWosQdXGxxYq1\n/v054Nm0ViviQlF/oL+kI+IWzd8IW07XsFgd1pXDchK2D18JHG5m+xLy0B4E7BDzCDwC9LA8LAfu\n5Ba3YFczoiW7BHhQUh9CUpcrzGxGjkXLZxoCuwN7EnLOXkUIYetHMFLeznQmMWfVwC3Y1RAzexE4\ngZA85U4zeykmzXYqwEJi8YOB4yUdGS3/qwnFFn/OqXBOXuOLXKsxkprk237+fEahUOHVwK1m9kCO\nxXEKAFewjlMFYtTAQMJC4fRcbiN28h9XsI5TRRKT4zhOMlzBOo7jZAlf5HIcx8kSrmAdx3GyhCtY\nx3GcLOEK1nEcJ0u4gnUyhqRSSWMkjZM0KGbxqu5Y3SS9FM/3l3Rhkr6NJJ1WjTmukHReuvfL9XlA\n0iFVmKu1pHFVldEpbFzBOplkvpl1NrOOhMxcpyQ2KlDlv3NmNtjMBibp0giosoJ1nGzjCtbJFu8C\n7aPl9oWkO4DRQCtJe0r6QNLoaOnWB5DUU9KXMVPVQWUDSeovqazO1bqSnpP0aTx2IAT+t4vW8/Wx\n3/mSRkoaK+nKhLH+LukrSW8QKjkkRdKJcZxPJT1TzirvIeldSV/H2mZIKpZ0fcLcJ6/sF+kULq5g\nnYwTs/nvTUjuDUGRPWRmWwLzCIlSepjZVoRKAOdIWoNQons/YGdgvUqGvwX4n5l1IiQK/5yQ5/ab\naD2fL2lPYCNgG6AzoULDLpK6EErkbElQ4Fun8XGeNbOt43xfAMcntLUm5NLdF/hP/AzHA7PNbOs4\n/omS2qQxj7MK4tm0nMdtmCYAAAHqSURBVExSV9KYeP4uocZXc+A7M/sw3t8O2AwYHvPL1AY+INQH\nm2RmEwBiAcGTKpijO3AMQNymOltS43J99ozHJ/G6PkHhNgCeM7M/4hyD0/hMHSX9g+CGqA+8mtD2\nVCzDPUHSt/Ez7AlskeCfbRjn/jqNuZxVDFewTiaZb2adE29EJTov8RbwupkdUa5fZ0IJlkwg4Foz\nu6vcHGdXY44HgAPM7FNJ/YFuCW3lx7I49wAzS1TESGpdxXmdVQB3ETg1zYfAjpLaQ6gXJmlj4Eug\njaR2sd8RlTz/JnBqfLY4VhOYQ7BOy3gVOC7Bt9tCUjNgGHCgpLqSGhDcEaloAEyTVAvoW66tj6Si\nKHNbQmWIV4FT/7+9O8RpKIiiMPwfVcEC2AFJExbDAnAkhCo2ABsheMIK0ASauqa2uOoiERhyETMC\nUfHMIJr/k5OXeZMnTm5uXub250lyluRkwnt0hKxg9a+qat8rwacks758V1UfSa5pwxk/gXfg/MAW\nt8BDkivgB1hU1SrJsv8G9dL7sHNg1SvoL+Cyj9V+Bja00S9vE458TxvNvaP1lP8G+RZ4BU6Bm6r6\nTvJI682u+x27e+Bi2tfRsfGyF0kaxBaBJA1iwErSIAasJA1iwErSIAasJA1iwErSIAasJA3yCyFq\nxMpH1DMCAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1a1c1935c0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"score = metrics.accuracy_score(bin_y_test, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(bin_y_test, pred, labels=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])\n",
"plot_confusion_matrix(cm, classes=['false', 'barely-true' , 'half-true' , 'mostly-true' , 'true'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuration 4\n",
"* model c - train - [performance measures][0:4]\n",
"* model c - test - [performance measures][0:4]"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.860\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xm8FXX9x/HX+wICiooIKosIIqJG\ngkq4lqaluKWW+nPLJfelNJcktdwyzcyKXEpzwSWVyoXcyX1DWUQRCQERQVBABFERAT+/P2YuHvAu\n58Lce+ae+372mMc95zvfmfkOxNvv/c7MdxQRmJlZdipK3QAzs3LjYDUzy5iD1cwsYw5WM7OMOVjN\nzDLmYDUzy5iD1aokqbWk/0iaL+mfq7CfwyU9nmXbSkXStyVNKHU7LP/k+1gbN0mHAWcCmwELgDHA\nZRHx/Cru98fAT4EdImLJKjc05yQF0DMiJpW6Ldb4ucfaiEk6E/gT8FtgfaArcB2wXwa73wh4qymE\najEkNS91G6wRiQgvjXAB1gY+AQ6qoU5LkuCdkS5/Alqm63YBpgNnAbOAmcAx6bqLgS+AxekxjgUu\nAu4o2Hc3IIDm6fejgbdJes1TgMMLyp8v2G4HYAQwP/25Q8G6p4FLgRfS/TwOtK/m3Crb/4uC9u8P\n7AW8BcwFziuo3x94CZiX1r0GWC1d92x6Lp+m5/t/Bfs/F3gfuL2yLN2mR3qMrdPvnYA5wC6l/v+G\nl9Iv7rE2XtsDrYD7aqhzPrAd0BfoQxIuFxSs34AkoDuThOe1ktaJiAtJesH3RESbiLippoZIWgMY\nBOwZEWuShOeYKuq1Ax5K664LXA08JGndgmqHAccA6wGrAWfXcOgNSP4MOgO/Bm4EjgC2Ab4N/FrS\nxmndpcDPgfYkf3a7AacARMR30jp90vO9p2D/7Uh67ycUHjgiJpOE7p2SVgduAW6NiKdraK81EQ7W\nxmtdYE7U/Kv64cAlETErImaT9ER/XLB+cbp+cUQ8TNJb67WS7fkS6C2pdUTMjIhxVdTZG5gYEbdH\nxJKIuAv4H7BvQZ1bIuKtiFgIDCH5j0J1FpOMJy8G7iYJzT9HxIL0+OOALQEiYlREDE+P+w7wN2Dn\nIs7pwohYlLZnORFxIzAReBnoSPIfMjMHayP2IdC+lrG/TsDUgu9T07Jl+1ghmD8D2tS1IRHxKcmv\nzycBMyU9JGmzItpT2abOBd/fr0N7PoyIpennyuD7oGD9wsrtJW0q6UFJ70v6mKRH3r6GfQPMjojP\na6lzI9Ab+EtELKqlrjURDtbG6yXgc5JxxerMIPk1tlLXtGxlfAqsXvB9g8KVEfFYRHyfpOf2P5LA\nqa09lW16byXbVBfXk7SrZ0SsBZwHqJZtarxlRlIbknHrm4CL0qEOMwdrYxUR80nGFa+VtL+k1SW1\nkLSnpCvTancBF0jqIKl9Wv+OlTzkGOA7krpKWhv4ZeUKSetL+kE61rqIZEhhaRX7eBjYVNJhkppL\n+j9gC+DBlWxTXawJfAx8kvamT15h/QfAxl/bqmZ/BkZFxHEkY8d/XeVWWllwsDZiEXE1yT2sFwCz\ngWnAacD9aZXfACOB14GxwOi0bGWONQy4J93XKJYPwwqSuwtmkFwp35n0wtAK+/gQ2Cet+yHJFf19\nImLOyrSpjs4muTC2gKQ3fc8K6y8CBkuaJ+ng2nYmaT9gAMnwByR/D1tLOjyzFluj5QcEzMwy5h6r\nmVnGHKxmZhlzsJqZZczBamaWsbKbWELNW4dWW7PUzbA62mrzrqVugq2E0aNHzYmIDlntr9laG0Us\n+dpDblWKhbMfi4gBWR07S+UXrKutSctetd4tYznzwsvXlLoJthJat9CKT9KtkliysOh/v5+Puba2\nJ+dKpuyC1cwaM4Ea/wilg9XM8kNARbNSt2KVOVjNLF9U2xQO+edgNbMc8VCAmVn23GM1M8uQcI/V\nzCxbco/VzCxzvivAzCxLvnhlZpYt4aEAM7PMucdqZpYlDwWYmWWvwkMBZmbZ8VwBZmZZ81CAmVn2\nfFeAmVnG3GM1M8uQ/EirmVn23GM1M8uSfFeAmVnmPBRgZpYhz8dqZpY138dqZpY9DwWYmWXMPVYz\nswypPO4KaPz/aTCz8lL5kEBtS627UStJr0h6TdI4SRen5d0lvSxpoqR7JK2WlrdMv09K13cr2Ncv\n0/IJkvao7dgOVjPLFUlFLUVYBOwaEX2AvsAASdsBvwP+GBE9gY+AY9P6xwIfRcQmwB/TekjaAjgE\n+AYwALhOUo3dagermeVG8maWbII1Ep+kX1ukSwC7Av9KywcD+6ef90u/k67fTcmB9gPujohFETEF\nmAT0r+nYDlYzyw/VYSlmd1IzSWOAWcAwYDIwLyKWpFWmA53Tz52BaQDp+vnAuoXlVWxTJV+8MrMc\nKfrXfID2kkYWfL8hIm4orBARS4G+ktoC9wGbV7GfWHbwqtdVV14tB6uZ5UpFRdG/SM+JiH7FVIyI\neZKeBrYD2kpqnvZKuwAz0mrTgQ2B6ZKaA2sDcwvKKxVuU/U5FHsGZmYNIasxVkkd0p4qkloD3wPG\nA08BB6bVjgIeSD8PTb+Trn8yIiItPyS9a6A70BN4paZju8dqZvlRh/HTInQEBqdX8CuAIRHxoKQ3\ngbsl/QZ4FbgprX8TcLukSSQ91UMAImKcpCHAm8AS4NR0iKFaDlYzyw3VbYy1RhHxOrBVFeVvU8VV\n/Yj4HDiomn1dBlxW7LEdrGaWK1kFayk5WM0sVxysZmZZEqjCwWpmlin3WM3MMpTlxatScrCaWa44\nWM3Mstb4c9XBamY5IvdYzcwyV4e5AnLLwWpmueGLV2Zm9aHx56pntyqllqs157nbz+blewYy6l/n\nc8FJey23/upzD2L2C39Y9v3Ks37I8LsHMvzugbx+/6+Z+eyVAHTtuA4v3PkLht+d7Oe4A3dq0PNo\nyk487id07bQe2/Tt/bV1f7z6Klq3EHPmzAEgIjjzjJ/xjc024Vtbbcmro0c3dHPzT5m+mqVk3GMt\noUVfLGHACYP4dOEXNG9ewZM3n8njL7zJK2PfYesturJ2m9bL1f/FH+5d9vnkQ3amT68uAMyc/THf\nPfpqvli8hDVar8aof53PQ8+MZebs+Q16Pk3Rj486mpNOOY3jfnLkcuXTpk3jyf8OY8OuXZeVPfbo\nI0yeNJE3xk/klZdf5menncxzL77c0E3OvbyHZjHcYy2xTxd+AUCL5s1o3rwZEUFFhfjtGftz/p/v\nr3a7gwdsw5BHRwGweMlSvlicvGmi5WotqCiD/2M2Fjt9+zu0a9fua+W/OPvnXHb5lcuFxINDH+Cw\nI45EEttutx3z589j5syZDdncRsE9VltlFRXixX+cS48NO/C3e55lxBtTOfXQXXjombG8P+fjKrfp\n2nEdNuq0Lk+PmLCsrMv6bbl30Mn02LAD5/3pfvdWS+jB/wylU6fObNmnz3LlM2a8R5cuX01E37lz\nF2a89x4dO3Zs6CbmmucKKFI6i/dhEXFdQxyvMfnyy2C7Q65g7Tatuefq49lx6x788Ptbsfvxf652\nm4P22Ib7nxjDl19+9dqd6R/Mo///XU7HDmsz5Orjue+/rzJr7oKGOAUr8Nlnn/G7yy/jwUce/9q6\nZDL65eW959XQGkNvtBgNNRTQFjhlxcLa3s3dlMz/ZCHPjpzIzv02ZeMNOzBu6IX876GLWb1VC954\n4MLl6h64xzYMeXRklfuZOXs+b05+nx237tEQzbYVvD15MlPfmUL/bfrQa5NuvDd9Otv335r333+f\nzp27MH36Vy/7fO+96XTs1KmErc2nchgKaKhgvQLoIWmMpBGSnpL0D2CspG6S3qisKOlsSReln3tI\nelTSKEnPSdqsgdrbINqv02bZBapWLVuw67a9eHX8NLp//zw22/tCNtv7Qj77fDG997t42TY9N1qP\nddZaneGvTVlW1nm9trRq2QKAtmu2Zvu+G/PWO7Ma9mQMgN7f/CbvzpjFhEnvMGHSO3Tu0oWXXhnN\nBhtswN77/oB/3HEbEcHLw4ez1lprexigCuUQrA01xjoQ6B0RfSXtAjyUfp8iqVsN290AnBQREyVt\nC1wH7LpiJUknACcA0KJNti2vRxu0X4sbL/kxzSoqqKgQ/x42mkeee6PGbQ4e0I9/PjZqubJe3Tfg\nijMPIAiE+NNtTzBuUo0vkbSMHHnEoTz3zNPMmTOHHt268KtfX8zRPzm2yroD9tyLxx55mG9stgmr\nt16dv/39lgZubSOR78wsiqoa98n8IEl4PhgRvdNgvTAivrviuvT72UAb4CpgNjChYFctI6Kq94Iv\nU7H6etGy18EZn4HVt49GXFPqJthKaN1Co4p9BXUxWq7fMzofXv31hUJT/rh3psfOUqnuCvi04PMS\nlh+SaJX+rADmRUTfBmuVmZWUlNwp09g11BjrAmDNatZ9AKwnaV1JLYF9ACLiY2CKpIMAlOhTzT7M\nrCwUN77qMVYgIj6U9EJ6kWohSZhWrlss6RLgZWAK8L+CTQ8Hrpd0AdACuBt4rSHabGalkfPMLEqD\nDQVExGE1rBsEDKqifAowoD7bZWb5kvfeaDH8SKuZ5YeSHmsxS627kjZMb+0cL2mcpNPT8oskvZfe\n/jlG0l4F2/xS0iRJEyTtUVA+IC2bJGlgbcf2I61mlhsi04tXS4CzImK0pDWBUZKGpev+GBFXLXds\naQvgEOAbQCfgv5I2TVdfC3wfmA6MkDQ0It6s7sAOVjPLlayCNSJmAjPTzwskjQc617DJfsDdEbGI\n5ML5JKB/um5SRLwNIOnutG61weqhADPLj7oNBbSXNLJgOaHa3Sb3y29FcpEc4DRJr0u6WdI6aVln\nYFrBZtPTsurKq+VgNbPcEHV6pHVORPQrWG6ocp9SG+DfwBnpbZzXAz2AviQ92srZ5KvqKkcN5dXy\nUICZ5Ui296hKakESqndGxL0AEfFBwfobgQfTr9OBDQs27wJUPhteXXmV3GM1s1zJ8K4AATcB4yPi\n6oLywplvDgAqJ+gYChwiqaWk7kBP4BVgBNBTUndJq5Fc4Bpa07HdYzWzXMmwx7oj8GOSWfTGpGXn\nAYdK6kvy6/w7wIkAETFO0hCSi1JLgFMjYmnaptOAx4BmwM0RMa6mAztYzSw3spwrICKep+rx0Ydr\n2OYy4LIqyh+uabsVOVjNLFfK4MErB6uZ5Us5PNLqYDWzXCmDXHWwmlmOyD1WM7NMJQ8IlLoVq87B\namY5orJ4g4CD1cxyxUMBZmZZKvKpqrxzsJpZblROwtLYOVjNLFccrGZmGSuDXHWwmlmOZDhXQCk5\nWM0sN5TxfKyl4mA1s1wpg1x1sJpZvlSUQbI6WM0sV8ogVx2sZpYfKvdJWCStVdOG6dsOzcwy1azM\n7woYx9df/Vr5PYCu9dguM2uiyqDDWn2wRsSG1a0zM6sPIrnlqrEr6vXXkg6RdF76uYukbeq3WWbW\nVFWouCXPag1WSdcA3yV5jSzAZ8Bf67NRZtZEKXlAoJglz4q5K2CHiNha0qsAETFX0mr13C4za6Jy\nnplFKSZYF0uqILlghaR1gS/rtVVm1iSJ8rgroJgx1muBfwMdJF0MPA/8rl5bZWZNVlZDAZI2lPSU\npPGSxkk6PS1vJ2mYpInpz3XSckkaJGmSpNclbV2wr6PS+hMlHVXbsWvtsUbEbZJGAd9Liw6KiDdq\nPSszszpStm8QWAKcFRGjJa0JjJI0DDgaeCIirpA0EBgInAvsCfRMl22B64FtJbUDLgT6kfzmPkrS\n0Ij4qLoDF3VXANAMWAx8UYdtzMzqrEIqaqlNRMyMiNHp5wXAeKAzsB8wOK02GNg//bwfcFskhgNt\nJXUE9gCGRcTcNEyHAQNqPIfaGifpfOAuoBPQBfiHpF/WelZmZitBRS5Ae0kjC5YTqt2n1A3YCngZ\nWD8iZkISvsB6abXOwLSCzaanZdWVV6uYi1dHANtExGdpAy8DRgGXF7GtmVmd1OFWqjkR0a+I/bUh\nuU50RkR8XMP+q1qx4tOnheXVKubX+qksH8DNgbeL2M7MrE4k0ayiuKXI/bUgCdU7I+LetPiD9Fd8\n0p+z0vLpQOETp12AGTWUV6vaYJX0R0lXkzwQME7S3yXdCIwF5hV1VmZmdVR5Aau2pfb9SMBNwPiI\nuLpg1VCg8sr+UcADBeVHpncHbAfMT4cKHgN2l7ROegfB7mlZtWoaCqi88j8OeKigfHjtp2RmtnIy\nfKpqR5InRsdKGpOWnQdcAQyRdCzwLnBQuu5hYC9gEkmH8hhY9lDUpcCItN4lETG3pgPXNAnLTSt3\nLmZmK0dkNw9ARDxP1eOjALtVUT+AU6vZ183AzcUeu9aLV5J6AJcBWwCtCg60abEHMTMrVt7nAShG\nMRevbgVuIUn+PYEhwN312CYza8LqcLtVbhUTrKtHxGMAETE5Ii4gme3KzCxTEpneFVAqxdzHuii9\nujZZ0knAe3x1Q62ZWabKYSigmGD9OdAG+BnJWOvawE/qs1Fm1nSVQa4WNQnLy+nHBXw12bWZWeZE\ncfMA5F1Nb2m9jxoe24qIH9ZLi8ys6cp2dquSqanHek2DtSJDvXp0ZvC/Lyt1M6yOOhw+uPZK1iSU\n9RhrRDzRkA0xMxPQrJyD1cysFHJ+J1VRHKxmlitNKlgltYyIRfXZGDNr2pKZqxp/shbzBoH+ksYC\nE9PvfST9pd5bZmZNUoWKW/KsmEdaBwH7AB8CRMRr+JFWM6snWc3HWkrFDAVURMTUFbrnS+upPWbW\nhAlonvfULEIxwTpNUn8gJDUDfgq8Vb/NMrOmqgxytahgPZlkOKAr8AHw37TMzCxTKvLV1nlXzFwB\ns4BDGqAtZmZNo8eavkDwa3MGRES17/A2M1tZeb/iX4xihgL+W/C5FXAAMK1+mmNmTVnyzqvGn6zF\nDAXcU/hd0u3AsHprkZk1XYJmxdwEmnMr80hrd2CjrBtiZgbJnKyNXTFjrB/x1RhrBTAXGFifjTKz\npinL11+XUo3Bmr7rqg/Je64AvkzfvW1mVi/KIVhrHM1IQ/S+iFiaLg5VM6tXkopaitjPzZJmSXqj\noOwiSe9JGpMuexWs+6WkSZImSNqjoHxAWjZJUlG/rRczTPyKpK2L2ZmZ2aqoHArIaBKWW4EBVZT/\nMSL6psvDAJK2ILlf/xvpNtdJapY+bXotsCewBXBoWrdGNb3zqnlELAF2Ao6XNBn4ND33iAiHrZll\nS9Aso7GAiHhWUrciq+8H3J1OjTpF0iSgf7puUkS8DSDp7rTumzXtrKYx1leArYH9i2yYmdkqqePF\nq/aSRhZ8vyEibihiu9MkHQmMBM6KiI+AzsDwgjrT0zJY/r796cC2tR2gpmAVQERMLqKhZmaZqMPz\nAXMiol8dd389cCnJnU6XAn8AfgJV3uMVVD1cWuu1ppqCtYOkM6tbGRFX17ZzM7O6ERX1eB9rRHyw\n7EjJ4/oPpl+nAxsWVO0CzEg/V1derZouXjUD2gBrVrOYmWVK1O9E15I6Fnw9AKi8Y2AocIiklpK6\nAz1JhkNHAD0ldZe0GskFrqG1HaemHuvMiLhkpVpvZrYyMnztiqS7gF1IxmKnAxcCu0jqS/Lr/DvA\niQARMU7SEJKLUkuAUyNiabqf04DHSDqbN0fEuNqOXesYq5lZQxGZ3hVwaBXFN9VQ/zLgsirKHwYe\nrsuxawrW3eqyIzOzLJT17FYRMbchG2JmBk1komszs4YiinscNO8crGaWH6KoeQDyzsFqZrnS+GPV\nwWpmOSKgmXusZmbZKoNcdbCaWZ4UN9dq3jlYzSw3fFeAmVk9cI/VzCxjjT9WHaxmliOS7wowM8uc\nhwLMzDLW+GPVwWpmOVMGHVYHq5nlR3K7VeNPVgermeWKe6xmZplSeU90bWbW0DwUYGaWtVV4A2ue\nOFjNLFccrGZmGZOHAixLd99yPQ/ccxtBsN//Hcmhx5zCEw/fz42DruCdSRO45d4n2XzLrQB4+fmn\nuPbKi1iyeDHNW7TgZwMvod8OO5f4DJqGzuuuzg2n7sT6bVvz5ZdwyxNvcf0j4+m90Tr8+bjtWKNV\nC96d/QnH/uU5FixczME7def0fXsv275313XYaeB/GDv1I364fTfOOeCbNKuo4LFXp/OrO0eV8MxK\nT0BGb78uKQdrTkye8CYP3HMbt9z3BM1brMYZx/yIHXfZg4033ZzfXXc7V1xwxnL1267Tjj/ceDcd\n1u/I5AlvcvoxP+LBF8eXqPVNy5KlwXm3j+S1KXNp06o5z12+D0++PoNrTtyB828fyQvjP+DHu2zC\n6ft+g98MGcOQ56cw5PkpAGyxYVvuPmdXxk79iHZtWvKbI7bhOwMfZM6CRfztlB3ZufcGPPPG+yU+\nw9Iqh7sCymHqw7LwzuS36L1VP1q1Xp3mzZuzVf8deebxB+m+SS822rjn1+r3+kYfOqzfEYCNN92c\nRYs+54tFixq62U3SB/MW8tqU5O3wn3y+hAnvzadTu9Xp2XEtXhj/AQBPjp3Bfttu9LVtD9qxO/96\nIQnZbuu3YdLMj5mzIPl7e2rszCq3aWpU5P9q3Y90s6RZkt4oKGsnaZikienPddJySRokaZKk1yVt\nXbDNUWn9iZKOKuYcHKw5sfGmm/PqKy8y/6O5fL7wM158ZhgfzJxe1LZPPjqUXltsyWotW9ZzK21F\nXTuswZbd2zFy0hzGT5vH3v02BOCA7brRed01vlb/h9t3558vJsH69vsL2LTT2nTtsAbNKsQ+3+pK\nlyq2aUoqhwKKWYpwKzBghbKBwBMR0RN4Iv0OsCfQM11OAK6HJIiBC4Ftgf7AhZVhXJN6C1ZJP5M0\nXtKd1azfRdKD9XX8xqb7Jr048sTT+elR+3P6MT+i52a9ada89pGat98az7VXXsjA3/ypAVpphdZo\n2Zw7zvwuAwePYMHCxZzy1xc4fvfNePbyfWjTugWLlyxdrn6/Tdqz8IsljJ82D4B5n37Bz/8+nFtP\n35nHLx7Au7M/YcnSKMWp5Eix/dXakzUingXmrlC8HzA4/TwY2L+g/LZIDAfaSuoI7AEMi4i5EfER\nMIyvh/XX1OcY6ynAnhExpR6PUVZ+cPCR/ODgIwG47qpLWG+DTjXW/2Dme/zi5CO48Pd/pctG3Rui\niZZq3kzccdYuDHn+bYa+8i4Ab834mP1/OwyATTquxR5bdVlumx/t8NUwQKVHRk/nkdHJbybH7NaT\npV828WCt232s7SWNLPh+Q0TcUMs260fETICImClpvbS8MzCtoN70tKy68hrVS49V0l+BjYGhks6V\n9KKkV9Ofvaqov7OkMenyqqQ10/JzJI1Ixzwuro+25sncObMBeH/GNJ5+7D/svu+B1dZd8PE8zjzu\nYE4559f06bddQzXRUteetCMT3pvPNQ+9uays/VqtgCQYzvnhltw8bMKydRIcsN1G/OvF5YO1cpu2\na6zGcbtvxuAnJzZA6/NNRS7AnIjoV7DUFqq1HXZFUUN5jeqlxxoRJ0kaAHwX+AL4Q0QskfQ94LfA\nj1bY5Gzg1Ih4QVIb4HNJu5OMd/QnObmhkr6Tdu+XI+kEknERNui0YX2cUoMYeOqRzJ83l+bNm3PO\nRVex1tptefqx/3DVJecyb+4cfn7cwWy6xTcZdOu9/PO2G5k+dQo3X/N7br7m9wAMuvU+2rXvUOKz\nKH/b91qPw77TgzemzuWF3+0LwMV3jaZHx7U4Yfek3zD0lXe5/elJy7bZcfP1mTH3M96Z9cly+7ry\n6P58c6NkyO6Kf7/GpJkfN9BZ5JOo9zcIfCCpY9pb7QjMSsunA4Xh0QWYkZbvskL507UdRBH186uH\npHeAfkBrYBBJSAbQIiI2k7QLcHZE7CNpIHAAcCdwb0RMl3QVcCAwL91lG+DyiLippuNu/s2tYvAD\nT9fDGVl92u2X95e6CbYSPhly9KiI6JfV/jb/5lZxy/1PFVV3+03WqfXYkroBD0ZE7/T774EPI+KK\nNHfaRcQvJO0NnAbsRXKhalBE9E8vXo0CKu8SGA1sExErjt0upyHuY70UeCoiDkhP8ukVK6Qn+RDJ\nSQ1Pe7YiCdK/NUAbzSwnsnryStJdJL3N9pKmk1zdvwIYIulY4F3goLT6wyT5Mwn4DDgGICLmSroU\nGJHWu6S2UIWGCda1gffSz0dXVUFSj4gYC4yVtD2wGfAYcKmkOyPiE0mdgcURMauqfZhZechqJCAi\nDq1m1W5V1A3g1Gr2czNwc12O3RDBeiUwWNKZwJPV1DlD0neBpcCbwCMRsUjS5sBL6cvFPgGO4Ksx\nETMrQ43/uat6DNaI6JZ+nANsWrDqV+n6p0mHBSLip9Xs48/An+urjWaWQ2WQrJ4rwMxyQyqPuQIc\nrGaWK40/Vh2sZpY3ZZCsDlYzy5Hi5gHIOwermeVKGQyxOljNLD8K5gFo1BysZpYrKoMuq4PVzHKl\nDHLVwWpm+VIGuepgNbMcKZNBVgermeWKb7cyM8uQ8BirmVnmHKxmZhnzUICZWcbcYzUzy1gZ5KqD\n1cxypgyS1cFqZrmR3Mba+JPVwWpm+SGoaPy56mA1s5xxsJqZZckTXZuZZa4cbreqKHUDzMwqqQ5L\nUfuT3pE0VtIYSSPTsnaShkmamP5cJy2XpEGSJkl6XdLWK3seDlYzy5cskzXx3YjoGxH90u8DgSci\noifwRPodYE+gZ7qcAFy/sqfgYDWzXKmQilpWwX7A4PTzYGD/gvLbIjEcaCup40qdw6q0zswsa3Xo\nsLaXNLJgOaGK3QXwuKRRBevXj4iZAOnP9dLyzsC0gm2np2V15otXZpYfqtPFqzkFv95XZ8eImCFp\nPWCYpP/VfPSviaJbU8A9VjPLmewGWSNiRvpzFnAf0B/4oPJX/PTnrLT6dGDDgs27ADNW5gwcrGaW\nG5UTXRez1LovaQ1Ja1Z+BnYH3gCGAkel1Y4CHkg/DwWOTO8O2A6YXzlkUFceCjCzXMnwNtb1gfvS\n12k3B/4REY9KGgEMkXQs8C5wUFr/YWAvYBLwGXDMyh7YwWpmubKKV/yXiYi3gT5VlH8I7FZFeQCn\nZnFsB6uZ5UsZPHnlYDWzXCmDXHWwmll+FHthKu8crGaWK57dyswsa40/Vx2sZpYvfoOAmVmmPNG1\nmVmmKp+8auz8SKuZWcbcYzVhrreRAAAHxUlEQVSzXCmHHquD1cxyxWOsZmYZknxXgJlZ9hysZmbZ\n8lCAmVnGfPHKzCxjZZCrDlYzy5kySFYHq5nlhsjuDQKlpORtBOVD0mxgaqnbUU/aA3NK3Qirs3L+\ne9soIjpktTNJj5L8eRVjTkQMyOrYWSq7YC1nkkYW8R51yxn/vTU9nivAzCxjDlYzs4w5WBuXG0rd\nAFsp/ntrYjzGamaWMfdYzcwy5mA1M8uYg9XMLGMOVrMGIi3/SNGK3618OFjNGoAkRXqlWNK6AOEr\nx2XLdwU0QpIOAD4FKiLi0VK3x4on6afA9sBM4BngkYhYXNpWWdbcY21kJJ0GnA20A/4t6dslbpIV\nSdJBwEHAycDuwE4O1fLkYG0klNgI+D6wK9CZpMfzoqQWJW2cValyDFVS5b+zzsAVwAHADOD8dP0G\nJWmg1RsHa+MhYDYwHfg1sDNwYEQsBY6StGkpG2fLKxxTBSqD823gt8CREbFHRCyWdBZwUkH4Whnw\nfKyNgKSdgC0j4jpJqwPHRkSrdN1hwHHAw6VsoyUqe6kFF6rOAPaXtDcwGRgPjJC0DbApcDjw44j4\nskRNtnrgi1c5lvZiBBwLbAM8BTwA3AG0Bd4AdgB+EhFjS9VO+4qklhGxKP18LHA8cFBETEuHbL5N\n8ne5M7AQuMR/d+XHwZpjkrpGxLtpL/Ugkn+QwyPiH5J+ACwFxkfE2yVtqAEgaROSMdSzImKqpDOB\nScDnQB+SkL0GuA1YRPLv77NStdfqj8d1ckpSJ+A5SXum//j+RdJDPUrSkcDDEfGQQzVXviD5df9y\nSR1JQvUU4EySt1qcB+wCtI2IhQ7V8uUeaw6lPZ2pQCuSW6vOjYjH03VPAq8Cl0bEvNK10qoiqStw\nArAxyW1VS0mGXD+V9D3gEuBHETGzhM20euYea85I+j6wHTAiIu4Efg9cLWlvSfsAHwNXOVRLL70F\nbrl/QxHxLnA18C7wN6BdGqqnAFcCJzpUy597rCUmaTVgk4h4U9LRwC+AyRGxb0GdA4GzSC52nBER\nr5eksbYcSW0i4pP084nAWiRPw/1O0trAuUA3kqGAjsB8D900DQ7WEksveFxH8ohjV+Bm4AxgcEQM\nKqi3NrAkIj4tSUNtOenFw/0i4tj0lqoDgF+RXJwaGxGHS1oTuAxYHTjBt1Q1Hb6PtcQiYpKk10nG\n5c6NiNslzQFOTO8x/0tab35JG2rLpJOo/Aw4XVIvoB+wZ1o2GWgt6V8RcaCk84HWDtWmxcGaD38F\nXgPOlDQ3Iu6RNAu4TtKciLirxO2z5X0BLCF5Am4JydX+/iQ92O0l9QcekXRHRBwBLChdU60UHKw5\nEBGTgEmS5gGXpT9bkfwDHl7SxtnXRMQCSU8AF5JcSJwqqTvwUlqlB/A7YEip2mil5WDNkYj4j6TF\nwFUk0wIeGxFTStwsq9oQYDRwjaQPgUeArSTdAuwG7BoR75SwfVZCvniVQ5LWI7n3cXap22I1k7Q1\ncA/JcMDzJDNYfej/IDZtDlazVSSpD/Ak8MuIuKHU7bHSc7CaZUBSb2BhREwudVus9BysZmYZ8yOt\nZmYZc7CamWXMwWpmljEHq5lZxhysZmYZc7A2UZKWShoj6Q1J/0xf/7Ky+9pF0oPp5x9IGlhD3bbp\n3KR1PcZFks4utnyFOremUy8We6xukt6oaxvNKjlYm66FEdE3InqTzElwUuHKqiZxLkZEDI2IK2qo\n0pbkdSVmZcvBagDPAZukPbXxkq4jeQ5+Q0m7S3pJ0ui0Z9sGQNIASf+T9Dzww8odSTpa0jXp5/Ul\n3SfptXTZgeRlez3S3vLv03rnSBoh6XVJFxfs63xJEyT9F+hV20lIOj7dz2uS/r1CL/x7kp6T9Fb6\nJgYkNZP0+4Jjn7iqf5Bm4GBt8iQ1J5lLtPIVzL2A2yJiK5KJYC4AvhcRWwMjSaY2bAXcCOxL8jrn\nDarZ/SDgmYjoA2wNjAMGkrwhoW9EnCNpd6AnybR7fYFtJH1H0jbAIcBWJMH9rSJO596I+FZ6vPEk\nrw2v1I3kldN7A39Nz+FYkln9v5Xu//h0liqzVeLZrZqu1pLGpJ+fA24COgFTI6JyqsLtgC2AFyQB\nrEYyNd5mwJSImAgg6Q6SibpXtCtwJEBELAXmS1pnhTq7p8ur6fc2JEG7JnBf5ZtMJQ0t4px6S/oN\nyXBDG+CxgnVD0smmJ0p6Oz2H3YEtC8Zf106P/VYRxzKrloO16VoYEX0LC9LwLHz1i4BhEXHoCvX6\nAlk9Cy3g8oj42wrHOGMljnErsH9EvJa+P2yXgnUr7ivSY/80IgoDGEnd6nhcs+V4KMBqMhzYMX0v\nF5JWl7Qp8D+gu6Qeab1Dq9n+CZJXQFeOZ65FMpv+mgV1HgN+UjB22zmdNvFZ4ABJrdN3R+1L7dYE\nZkpqARy+wrqDJFWkbd4YmJAe++S0PpI2lbRGEccxq5F7rFatiJid9vzuktQyLb4gIt6SdALwkJL3\ncz0P9K5iF6cDN0g6FlgKnBwRL0l6Ib2d6ZF0nHVz4KW0x/wJcEREjJZ0DzAGmEoyXFGbXwEvp/XH\nsnyATwCeAdYHToqIzyX9nWTsdbSSg88G9i/uT8esep7dyswsYx4KMDPLmIPVzCxjDlYzs4w5WM3M\nMuZgNTPLmIPVzCxjDlYzs4z9P3Wjv5Jr3ijvAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1a1c871f60>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(concat_tfidf_train, concat_y_train)\n",
"pred = clf.predict(concat_tfidf_train)\n",
"score = metrics.accuracy_score(concat_y_train, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(concat_y_train, pred, labels=['true', 'false'])\n",
"plot_confusion_matrix(cm, classes=['true', 'false'])"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"clf = MultinomialNB()"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"accuracy: 0.749\n",
"Confusion matrix, without normalization\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVYAAAEmCAYAAAA5jbhCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XecFdXdx/HPdwEpAiIgoKBiQSw8\nit1YMfYKJtbYRbHFEks0amKPGn1ssUVDFMtjiSkSe9doFClib6CiKEovAirg7/ljZsllXXYvy+ze\n2bvft6957b0z586cYd3vnj1z5owiAjMzy05FqStgZlZuHKxmZhlzsJqZZczBamaWMQermVnGHKxm\nZhlzsFq1JLWW9C9JMyT9dSn2c7CkJ7OsW6lI2kbSB6Wuh+WfPI61cZP0C+A0YG1gFjAauDQiXlrK\n/R4KnARsGRHzl7qiOScpgF4RMabUdbHGzy3WRkzSacC1wO+BrsAqwE1A/wx2vyrwYVMI1WJIal7q\nOlgjEhFeGuECLAd8A+xXQ5mWJMH7ZbpcC7RMt/UDxgOnAxOBCcCR6bYLge+BeekxBgIXAHcX7Lsn\nEEDz9P0RwMckreZPgIML1r9U8LktgeHAjPTrlgXbngcuBl5O9/Mk0Hkx51ZZ/18X1H8AsDvwITAV\nOKeg/GbAK8D0tOwNwDLpthfTc5mdnu8BBfs/C/gKuKtyXfqZNdJjbJS+XwmYDPQr9f8bXkq/uMXa\neP0EaAX8o4Yy5wJbAH2BDUjC5byC7d1IAro7SXjeKGn5iDifpBV8f0S0jYjBNVVE0rLA9cBuEdGO\nJDxHV1OuI/BIWrYTcDXwiKROBcV+ARwJdAGWAc6o4dDdSP4NugO/A24DDgE2BrYBfidp9bTsAuBX\nQGeSf7sdgBMAImLbtMwG6fneX7D/jiSt90GFB46IsSShe4+kNsDtwB0R8XwN9bUmwsHaeHUCJkfN\nf6ofDFwUERMjYhJJS/TQgu3z0u3zIuJRktZa7zrW5wegj6TWETEhIt6ppswewEcRcVdEzI+Ie4H3\ngb0KytweER9GxFzgAZJfCoszj6Q/eR5wH0loXhcRs9LjvwOsDxARIyPi1fS4nwJ/ArYr4pzOj4jv\n0vosIiJuAz4ChgErkvwiM3OwNmJTgM619P2tBIwreD8uXbdwH1WCeQ7QdkkrEhGzSf58Pg6YIOkR\nSWsXUZ/KOnUveP/VEtRnSkQsSF9XBt/XBdvnVn5e0lqSHpb0laSZJC3yzjXsG2BSRHxbS5nbgD7A\nHyPiu1rKWhPhYG28XgG+JelXXJwvSf6MrbRKuq4uZgNtCt53K9wYEU9ExE4kLbf3SQKntvpU1umL\nOtZpSdxMUq9eEdEeOAdQLZ+pcciMpLYk/daDgQvSrg4zB2tjFREzSPoVb5Q0QFIbSS0k7SbpD2mx\ne4HzJK0gqXNa/u46HnI0sK2kVSQtB/ymcoOkrpL2TvtavyPpUlhQzT4eBdaS9AtJzSUdAKwLPFzH\nOi2JdsBM4Ju0NX18le1fA6v/6FM1uw4YGRFHk/Qd37LUtbSy4GBtxCLiapIxrOcBk4DPgV8C/0yL\nXAKMAN4E3gJGpevqcqyngPvTfY1k0TCsIBld8CXJlfLtSC8MVdnHFGDPtOwUkiv6e0bE5LrUaQmd\nQXJhbBZJa/r+KtsvAIZImi5p/9p2Jqk/sCtJ9wck34eNJB2cWY2t0fINAmZmGXOL1cwsYw5WM7OM\nOVjNzDLmYDUzy1jZTSyh5q1Dy7QrdTVsCW24ziqlroLVwahRIydHxApZ7a9Z+1Uj5v/oJrdqxdxJ\nT0TErlkdO0vlF6zLtKNl71pHy1jOvDzshlJXweqgdQtVvZNuqcT8uUX//H47+sba7pwrmbILVjNr\nzARq/D2UDlYzyw8BFc1KXYul5mA1s3xRbVM45J+D1cxyxF0BZmbZc4vVzCxDwi1WM7NsyS1WM7PM\neVSAmVmWfPHKzCxbwl0BZmaZc4vVzCxL7gowM8tehbsCzMyy47kCzMyyVh5dAY3/DMysvEjFLbXu\nRn+RNFHS2wXrOkp6StJH6dfl0/WSdL2kMZLelLRRwWcOT8t/JOnwYk7BwWpm+aKK4pba3QFUfcLA\n2cAzEdELeCZ9D7Ab0CtdBgE3QxLEwPnA5sBmwPmVYVwTB6uZ5UexrdUiWqwR8SIwtcrq/sCQ9PUQ\nYEDB+jsj8SrQQdKKwC7AUxExNSKmAU/x47D+Efexmlm+FN/H2lnSiIL3t0bErbV8pmtETACIiAmS\nuqTruwOfF5Qbn65b3PoaOVjNLEe0JKMCJkfEJtkd+EeihvU1cleAmeVLRl0Bi/F1+ic+6deJ6frx\nwMoF5XoAX9awvkYOVjPLj8r5WLO5eFWdoUDllf3DgYcK1h+Wjg7YApiRdhk8Aewsafn0otXO6boa\nuSvAzHIku3Gsku4F+pH0xY4nubp/OfCApIHAZ8B+afFHgd2BMcAc4EiAiJgq6WJgeFruooioekHs\nRxysZpYvGc1uFREHLWbTDtWUDeDExeznL8BfluTYDlYzy5cyuPPKwWpm+aElGhWQWw5WM8sXT3Rt\nZpYtOVjNzLKTPJnFwWpmlh1R/b1OjYyD1cxyRG6xmpllraLCw63MzDLlFquZWZbcx2pmli25j9XM\nLHsOVjOzjDlYzcyyJFCFg9XMLFNusZqZZcgXr8zM6oGD1cwsa40/Vx2sZpYjcovVzCxznivAzCxD\nvnhlZlYfGn+u0vjb3I3cLecfzLhnLmPEX89ZuG759m14+OZf8tZDv+Phm39Jh3atAWjfthUPXnss\nw+4/m5EPnsuhe28BwCorLs/L9/yaV+9L1h+979YlOZem6Nijj2KVlbqwcd8+C9ddctEFrL5qdzbf\nuC+bb9yXxx97FIApU6awy47b07lDW049+ZelqnK+pX2sxSx55mAtsbv+9Sr9T7xxkXVnHLkTz7/2\nAf/T/yKef+0DzjhyZwCO3X9b3v/4KzY/4HJ2OeY6Lj9tH1o0b8aESTPZ/oir2eLAy9n20Cs548id\nWHGF5UpxOk3OoYcfwUMPP/6j9Sed8iuGjRzNsJGj2XW33QFo1aoVv7vgYi674qqGrmaj4mC1pfby\nqLFMnTFnkXV79lufu/81DIC7/zWMvbZfH4AA2i7bEoBlW7dk2ow5zF/wA/PmL+D7efMBaLlMCypy\n/j9dOdl6m23p2LFjUWWXXXZZttp6a1q1alXPtWrcHKxWL7p0asdXk2cC8NXkmazQsR0At9z3Amuv\n1o2Pn7yUEX89hzOufJCIAKBH1w68dv9v+Oixi/nfO55mwqQZJau/wS033cCmG67PsUcfxbRp00pd\nnUZFFSpqybMGCVZJHSSd0BDHKmc7bbkOb34wntV3PpfND7yMa87ej3bLJq2f8V9PZ7MDLqNP/ws5\nZK/N6JKGsTW8Y449nnc/GMuwkaPptuKKnH3m6aWuUqNRbGvVLdZEB+BHwSqpWQMdv1GZOGUW3Tq3\nB6Bb5/ZMmjoLgEP33oKHnn0DgI8/n8ynX0yhd8+ui3x2wqQZvDv2K7baaI2GrbQt1LVrV5o1a0ZF\nRQVHDTyGESNeK3WVGhUHa/EuB9aQNFrScEnPSfo/4C1JPSW9XVlQ0hmSLkhfryHpcUkjJf1b0toN\nVN+SeuSFtzhkr80BOGSvzXn4+TcB+PyrafTbrDcAXTq2Y62eXfnki8l079KBVi1bANChXWt+0nd1\nPvx0Ymkqb0yYMGHh64f++Q/WXa9PDaWtqnII1oYax3o20Cci+krqBzySvv9EUs8aPncrcFxEfCRp\nc+Am4KdVC0kaBAwCoEXbbGtez4ZcdgTbbNyLzh3aMubxi7n4lke56vanuPuKozh8wE/4fMI0Dv71\nYAAuv+1xbr3wEIY/cA4SnHvdQ0yZPpufbr42l5+2D0EgxLV3PsM7Y74s8Zk1DYcdchD/fuF5Jk+e\nzBo9e/Db313Iiy88z5tvjEYSq/bsyR9v+tPC8r3X7MmsmTP5/vvv+dfQf/Lwo0+yzrrrlvAMcijf\nmVkUVV78qNeDJOH5cET0SYP1/IjYvuq29P0ZQFvgKmAS8EHBrlpGxDo1HauiTZdo2Xv/jM/A6tu0\n4TeUugpWB61baGREbJLV/lp27RXdD76uqLKfXLNHpsfOUqnuvJpd8Ho+i3ZJVI5FqQCmR0TfBquV\nmZWUBBU5v+JfjIbqY50FLO4y9ddAF0mdJLUE9gSIiJnAJ5L2A1BigwaprZmVSHmMCmiQFmtETJH0\ncnqRai5JmFZumyfpImAY8AnwfsFHDwZulnQe0AK4D3ijIepsZqWR88wsSoN1BUTEL2rYdj1wfTXr\nPwF2rc96mVm+5L01WgzPbmVm+SG3WM3MMiXK4+KVg9XMcqUcgtWTsJhZfqRdAcUsRe1O+pWkdyS9\nLeleSa0krSZpmKSPJN0vaZm0bMv0/Zh0e8+6noaD1cxyQ2R3S6uk7sDJwCbpDUjNgAOBK4BrIqIX\nMA0YmH5kIDAtItYErknL1YmD1cxyJPNxrM2B1pKaA22ACSS3xT+Ybh8CDEhf90/fk27fQXUcouBg\nNbNcWYKugM6SRhQsgwr3ExFfkNwa/xlJoM4ARpLc0Tk/LTYe6J6+7g58nn52flq+U13OwRevzCxX\nlqCROLmmuQIkLU/SCl0NmA78FditmqKVE6ZUd+A6TabiYDWz3Mh4roAdgU8iYlKyb/0d2BLoIKl5\n2irtAVROBTceWBkYn3YdLAdMrcuB3RVgZrmS4aiAz4AtJLVJ+0p3AN4FngP2TcscDjyUvh6avifd\n/mzUcfo/t1jNLFeyuqU1IoZJehAYRTKL3uskczw/Atwn6ZJ03eD0I4OBuySNIWmpHljXYztYzSxX\nsrylNSLOB86vsvpjYLNqyn4L7JfFcR2sZpYf8iQsZmaZSm4QKHUtlp6D1cxyRGUxV4CD1cxyxV0B\nZmZZ8nysZmbZqpyEpbFzsJpZrjhYzcwyVga56mA1sxzJdq6AknGwmlluiCWaazW3HKxmlitlkKsO\nVjPLl4oySFYHq5nlShnkqoPVzPJD5T4Ji6T2NX0wImZmXx0za+qalfmogHdInvdSeJaV7wNYpR7r\nZWZNVBk0WBcfrBGxckNWxMxMJEOuGruinnkl6UBJ56Sve0jauH6rZWZNVYWKW/Ks1mCVdAOwPXBo\numoOcEt9VsrMmiglNwgUs+RZMaMCtoyIjSS9DhARUyUtU8/1MrMmKueZWZRignWepAqSC1ZI6gT8\nUK+1MrMmSZTHqIBi+lhvBP4GrCDpQuAl4Ip6rZWZNVlNoisgIu6UNBLYMV21X0S8Xb/VMrOmSE3s\nCQLNgHkk3QFFjSQwM6uLcpgroJhRAecC9wIrAT2A/5P0m/qumJk1TSpyybNiWqyHABtHxBwASZcC\nI4HL6rNiZtY05b3/tBjFBOu4KuWaAx/XT3XMrCmTVBajAmqahOUakj7VOcA7kp5I3+9MMjLAzCxz\nZdBgrbHFWnnl/x3gkYL1r9ZfdcysqSvrroCIGNyQFTEzE/mfB6AYtfaxSloDuBRYF2hVuT4i1qrH\neplZE1UOLdZixqTeAdxO8stkN+AB4L56rJOZNWHlMNyqmGBtExFPAETE2Ig4j2S2KzOzTEnJXAHF\nLHlWzHCr75S0zcdKOg74AuhSv9Uys6aqHLoCignWXwFtgZNJ+lqXA46qz0qZWdNVBrla1CQsw9KX\ns/jvZNdmZpkTKou5Amq6QeAfpHOwViciflYvNTKzpqsJzG51Q4PVIkM9e3bj0sFnl7oatoS2/P2z\npa6C5URZ97FGxDMNWREzMwHNMgxWSR2APwN9SP4CPwr4ALgf6Al8CuwfEdPSi/TXAbuT3Mp/RESM\nqstxPbeqmeVKxk9pvQ54PCLWBjYA3gPOBp6JiF7AM+l7SMbp90qXQcDNdT6Hun7QzKw+ZBWsktoD\n2wKDASLi+4iYDvQHhqTFhgAD0tf9gTsj8SrQQdKKdTqHYgtKalmXA5iZFSt5NEvRz7zqLGlEwTKo\nyu5WByYBt0t6XdKfJS0LdI2ICQDp18px+d2Bzws+Pz5dt8SKeYLAZpLeAj5K328g6Y91OZiZWW2W\noMU6OSI2KVhurbKr5sBGwM0RsSEwm//+2V+d6trBix0ZVeM5FFHmemBPYApARLyBb2k1s3pS+UDB\n2pYijAfGF4zFf5AkaL+u/BM//TqxoPzKBZ/vAXxZl3MoJlgrImJclXUL6nIwM7OaCGguFbXUJiK+\nAj6X1DtdtQPwLjAUODxddzjwUPp6KHCYElsAMyq7DJZUMbe0fi5pMyAkNQNOAj6sy8HMzGqT8TDW\nk4B7JC1D8kipI0kalA9IGgh8BuyXln2UZKjVGJLhVkfW9aDFBOvxJN0BqwBfA0+n68zMMiVle0tr\nRIwGNqlm0w7VlA3gxCyOW8xcAROBA7M4mJlZbcrgxquiniBwG9VcGYuIqkMbzMyWWs6nWi1KMV0B\nTxe8bgXsw6JjvczMMpE886rxJ2sxXQH3F76XdBfwVL3VyMyaLkGzMrgftJgWa1WrAatmXREzM0jm\nZG3siuljncZ/+1grgKnUfPeCmVmdNInHX6fTaG1A8pwrgB/SIQlmZvWiHIK1xt6MNET/EREL0sWh\namb1agkmYcmtYrqJX5O0Ub3XxMyavMqugAznYy2Jmp551Twi5gNbA8dIGksyO4xIGrMOWzPLlqBZ\n3lOzCDX1sb5GMhPMgBrKmJllpilcvBJARIxtoLqYmZX9La0rSDptcRsj4up6qI+ZNWmioszHsTYD\n2lL9rNpmZpkT5d9inRARFzVYTczMGsEV/2LU2sdqZtZQRPmPCvjRRLBmZvWtrGe3ioipDVkRMzMo\n/z5WM7MGJYq7HTTvHKxmlh8i9/MAFMPBama50vhj1cFqZjkioJlbrGZm2SqDXHWwmlme5H+u1WI4\nWM0sNzwqwMysHrjFamaWscYfqw5WM8sRyaMCzMwy564AM7OMNf5YdbCaWc6UQYPVwWpm+ZEMt2r8\nyepgNbNccYvVzCxTKu+Jrs3MGpq7AszMsiZ3BZiZZc7BamaWMbkrwLL2w4IFnHvoHnRcoRtnXncH\nFw78Gd/OmQ3AjKmTWWO9vpx+9WDmzJrJjb89hSlffcGCBQvY49BB9Nv7gBLXvulZtVMbLv/5egvf\nd1++Nbc8/zHDP53OuXv0pnWLZkyY8S3n/v0dZn+/AIAjt1qVARuuyIIfgiuf+IhXxvq5nZUEZPn0\na0nNgBHAFxGxp6TVgPuAjsAo4NCI+F5SS+BOYGNgCnBARHxa1+M6WHPmsXsH073nmsyd/Q0A5w/+\n+8Jt15w5iI232xmAJ/86hB6r9+LMa29n5rQpnP6z7dh6t31o3mKZktS7qRo3ZQ4H3TocSALh8V9t\nxXPvT+YP+/bhmqfHMGrcdPr3XZHDtlyFm5//hNU6t2GX9bqw783DWKFdS24+ZEP2ufEVfogSn0iO\nZDwq4BTgPaB9+v4K4JqIuE/SLcBA4Ob067SIWFPSgWm5OrdUymHqw7Ix5esJjH7pWbYfcNCPts2d\n/Q3vDP8Pm/TbBUj+XJo7+xsigm/nzKZt+w5UNPPvyVLabLWOjJ82lwkzvmXVzm0YNW46AK9+PJUd\n1ukCQL/eK/DEOxOZtyD4cvq3jJ82hz7d29e02yZHRf5X636kHsAewJ/T9wJ+CjyYFhkCDEhf90/f\nk27fQUsxaYGDNUfu+t8LOOiUc1DFj78tw597nD6bbUWbtu0A2PmAI/jykzGcuMsmnHXAThx2xoVU\nVPM5azi7rNeFJ97+GoCxE2ez3VqdAdhx3S50bd8SgC7tWvL1zG8Xfubrmd+xQruWDV/ZnKrsCihm\nATpLGlGwDKqyu2uBXwM/pO87AdMjYn76fjzQPX3dHfgcIN0+Iy1fJ/X2kyjpZEnvSbpnMdv7SXq4\nvo7f2Ix68WnaL9+J1ddZv9rtrzzxEFvu0n/h+zdfeYFVe6/LjU+M4LJ7H+eOP/yWOd/MaqjqWhXN\nK8S2vTvz1LsTAbhw6Hvsv2kP7jl6E5ZdphnzFiR/61fXBgp3AxQotr0qgMkRsUnBcuvCvUh7AhMj\nYuQiO/+xKGLbEqvPvx1PAHaLiE/q8Rhl48M3RjDqxacY/fJzzPv+O+Z+M4sbzzuZEy+5nlnTpzH2\nndH86qrbFpZ/YegD7H3kCUii28qrscJKK/Plp2NYs8+GJTyLpmurNTvx/oRvmDp7HgCfTpnDifeM\nBmCVjq3ZulfSev165nd0bd9q4ee6tm/J5G++a/gK51V241i3AvaWtDvQiqSP9Vqgg6Tmaau0B/Bl\nWn48sDIwXlJzYDmgzlcV66XFmnYKrw4MlXSWpP9Iej392rua8ttJGp0ur0tql64/U9JwSW9KurA+\n6poXB550Njc8NpzrH36Fk35/I+ttuhUnXnI9AMOefpgNt96RZVr+9weyU7eVePu1lwGYMWUSE8aN\npUv3VUtSd4Nd+3Rd2A0AsHybFkDSDDp6m578beQXALzw4WR2Wa8LLZqJlTq0YuWObXj7i5mlqHJu\nqcilJhHxm4joERE9gQOBZyPiYOA5YN+02OHAQ+nroel70u3PRtT9b4l6abFGxHGSdgW2B74H/jci\n5kvaEfg98PMqHzkDODEiXpbUFvhW0s5AL2Azkn/HoZK2jYgXqx4v7VsZBNC5W/eqmxu9V54cyt5H\nnLDIup8dcwq3nH8aZ+2/I0Fw0Mnn0H75jiWqYdPWqnkFm6/ekUsfeX/hul37dGX/TXsA8Oz7k3ho\n9AQAPp40m6fenciDx2/Bgh9+4PLHPvCIgAKi3p8gcBZwn6RLgNeBwen6wcBdksaQtFQPXJqDaClC\nueYdS58CmwCtgetJQjKAFhGxtqR+wBnp2LKzgX2Ae4C/R8R4SVeR/OaYnu6yLXBZRAymBquvu35c\nevej9XFKVo+ufPjDUlfB6uD183cYGRGbZLW/df5nw7j9n88VVfYnay6f6bGz1BDjcy4GnouIfST1\nBJ6vWiAiLpf0CLA78GrashVJkP6pAepoZjlRDndeNcT4nOWAL9LXR1RXQNIaEfFWRFxBcpfE2sAT\nwFFp1wCSukvq0gD1NbMSkopb8qwhWqx/AIZIOg14djFlTpW0PbAAeBd4LCK+k7QO8Eo6Tvcb4BBg\nYgPU2cxKJOeZWZR6C9b0ahzAZGCtgk2/Tbc/T9otEBEnLWYf1wHX1VcdzSyHyiBZfQ+kmeWGlPlc\nASXhYDWzXGn8sepgNbO8KYNkdbCaWY4UN3NV3jlYzSxXyqCL1cFqZvlRzDwAjYGD1cxyZSnml84N\nB6uZ5UoZ5KqD1czypQxy1cFqZjlSJp2sDlYzyxUPtzIzy5BwH6uZWeYcrGZmGXNXgJlZxtxiNTPL\nWBnkqoPVzHKmDJLVwWpmuZEMY238yepgNbP8EFQ0/lx1sJpZzjhYzcyy5Imuzcwy5+FWZmYZKpM5\nWBysZpYzZZCsDlYzy5WKMugLcLCaWa40/lh1sJpZnsgXr8zM6kHjT1YHq5nlhie6NjOrB2WQqw5W\nM8sXjwowM8ta489VB6uZ5UsZ5KqD1czyQ2Uy3Kqi1BUwMyukIv+rdT/SypKek/SepHcknZKu7yjp\nKUkfpV+XT9dL0vWSxkh6U9JGdT0HB6uZ5YuKXGo3Hzg9ItYBtgBOlLQucDbwTET0Ap5J3wPsBvRK\nl0HAzXU9BQermeVKhYpbahMREyJiVPp6FvAe0B3oDwxJiw0BBqSv+wN3RuJVoIOkFet0DnX5kJlZ\n/Si2I0AAnSWNKFgGLXavUk9gQ2AY0DUiJkASvkCXtFh34POCj41P1y0xX7wys9xYwjuvJkfEJrXu\nU2oL/A04NSJmavEHqG5DFF2bAm6xmlnZktSCJFTviYi/p6u/rvwTP/06MV0/Hli54OM9gC/rclwH\nq5nlSuWQq9qW2vcjAYOB9yLi6oJNQ4HD09eHAw8VrD8sHR2wBTCjsstgSbkrwMxyJcOHCW4FHAq8\nJWl0uu4c4HLgAUkDgc+A/dJtjwK7A2OAOcCRdT2wg9XMckNFXvEvRkS8xOIHZu1QTfkATszi2A5W\nM8uXMrjzysFqZrmSYVdAyThYzSxXymGuAAermeVKGeSqg9XMcqYMktXBama5IcrjCQJKRhiUD0mT\ngHGlrkc96QxMLnUlbImV8/dt1YhYIaudSXqc5N+rGJMjYtesjp2lsgvWciZpRDH3Rlu++PvW9PiW\nVjOzjDlYzcwy5mBtXG4tdQWsTvx9a2Lcx2pmljG3WM3MMuZgNTPLmIPVzCxjDlazBqIqD1uq+t7K\nh4PVrAFIUjqRMpI6wcKJla0MeVRAIyRpH2A2UBERj5e6PlY8SScBPwEmAC8Aj0XEvNLWyrLmFmsj\nI+mXwBlAR+BvkrYpcZWsSJL2I3m+0vHAzsDWDtXy5GBtJNInR64K7AT8FOhO0uL5T/qIX8uZyj5U\nSZU/Z91JHmS3D8ljlc9Nt3crSQWt3jhYGw8Bk0ieff47YDtg34hYABwuaa1SVs4WVdinClQG58fA\n74HDImKXiJgn6XTguILwtTLg+VgbAUlbA+tHxE2S2gADI6JVuu0XwNEkj+61EqtspRZcqDoVGCBp\nD2As8B4wXNLGwFrAwcChEfFDiaps9cAXr3IsbcUIGAhsDDwHPATcDXQA3ga2BI6KiLdKVU/7L0kt\nI+K79PVA4Bhgv4j4PO2y2Ybke7kdMBe4yN+78uNgzTFJq0TEZ2krdT+SH8hXI+L/JO0NLADei4iP\nS1pRA0DSmiR9qKdHxDhJpwFjgG+BDUhC9gbgTuA7kp+/OaWqr9Uf9+vklKSVgH9L2i394XuQpIV6\nuKTDgEcj4hGHaq58T/Ln/mWSViQJ1ROA00ieanEO0A/oEBFzHarlyy3WHEpbOuOAViRDq86KiCfT\nbc8CrwMXR8T00tXSqiNpFWAQsDrJsKoFJF2usyXtCFwE/DwiJpSwmlbP3GLNGUk7AVsAwyPiHuBK\n4GpJe0jaE5gJXOVQLb10CNwiP0MR8RlwNfAZ8CegYxqqJwB/AI51qJY/t1hLTNIywJoR8a6kI4Bf\nA2MjYq+CMvsCp5Nc7Dg1It4sSWVtEZLaRsQ36etjgfYkd8NdIWk54CygJ0lXwIrADHfdNA0O1hJL\nL3jcRHKL4yrAX4BTgSERcX33ZvQqAAAEiUlEQVRBueWA+RExuyQVtUWkFw/7R8TAdEjVPsBvSS5O\nvRURB0tqB1wKtAEGeUhV0+FxrCUWEWMkvUnSL3dWRNwlaTJwbDrG/I9puRklragtlE6icjJwiqTe\nwCbAbum6sUBrSQ9GxL6SzgVaO1SbFgdrPtwCvAGcJmlqRNwvaSJwk6TJEXFvietni/oemE9yB9x8\nkqv9m5G0YH8iaTPgMUl3R8QhwKzSVdVKwcGaAxExBhgjaTpwafq1FckP8KslrZz9SETMkvQMcD7J\nhcRxklYDXkmLrAFcATxQqjpaaTlYcyQi/iVpHnAVybSAAyPikxJXy6r3ADAKuEHSFOAxYENJtwM7\nAD+NiE9LWD8rIV+8yiFJXUjGPk4qdV2sZpI2Au4n6Q54iWQGqyn+hdi0OVjNlpKkDYBngd9ExK2l\nro+VnoPVLAOS+gBzI2JsqetipedgNTPLmG9pNTPLmIPVzCxjDlYzs4w5WM3MMuZgNTPLmIO1iZK0\nQNJoSW9L+mv6+Je67qufpIfT13tLOruGsh3SuUmX9BgXSDqj2PVVytyRTr1Y7LF6Snp7SetoVsnB\n2nTNjYi+EdGHZE6C4wo3VjeJczEiYmhEXF5DkQ4kjysxK1sOVgP4N7Bm2lJ7T9JNJPfBryxpZ0mv\nSBqVtmzbAkjaVdL7kl4Cfla5I0lHSLohfd1V0j8kvZEuW5I8bG+NtLV8ZVruTEnDJb0p6cKCfZ0r\n6QNJTwO9azsJScek+3lD0t+qtMJ3lPRvSR+mT2JAUjNJVxYc+9il/Yc0AwdrkyepOclcopWPYO4N\n3BkRG5JMBHMesGNEbASMIJnasBVwG7AXyeOcuy1m99cDL0TEBsBGwDvA2SRPSOgbEWdK2hnoRTLt\nXl9gY0nbStoYOBDYkCS4Ny3idP4eEZumx3uP5LHhlXqSPHJ6D+CW9BwGkszqv2m6/2PSWarMlopn\nt2q6Wksanb7+NzAYWAkYFxGVUxVuAawLvCwJYBmSqfHWBj6JiI8AJN1NMlF3VT8FDgOIiAXADEnL\nVymzc7q8nr5vSxK07YB/VD7JVNLQIs6pj6RLSLob2gJPFGx7IJ1s+iNJH6fnsDOwfkH/63LpsT8s\n4lhmi+VgbbrmRkTfwhVpeBY++kXAUxFxUJVyfYGs7oUWcFlE/KnKMU6twzHuAAZExBvp88P6FWyr\nuq9Ij31SRBQGMJJ6LuFxzRbhrgCryavAVulzuZDURtJawPvAapLWSMsdtJjPP0PyCOjK/sz2JLPp\ntyso8wRwVEHfbfd02sQXgX0ktU6fHbUXtWsHTJDUAji4yrb9JFWkdV4d+CA99vFpeSStJWnZIo5j\nViO3WG2xImJS2vK7V1LLdPV5EfGhpEHAI0qez/US0KeaXZwC3CppILAAOD4iXpH0cjqc6bG0n3Ud\n4JW0xfwNcEhEjJJ0PzAaGEfSXVGb3wLD0vJvsWiAfwC8AHQFjouIbyX9maTvdZSSg08CBhT3r2O2\neJ7dyswsY+4KMDPLmIPVzCxjDlYzs4w5WM3MMuZgNTPLmIPVzCxjDlYzs4z9PwDNbmOVIKyxAAAA\nAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11ac93d30>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"clf.fit(concat_tfidf_train, concat_y_train)\n",
"pred = clf.predict(concat_tfidf_test)\n",
"score = metrics.accuracy_score(concat_y_test, pred)\n",
"print(\"accuracy: %0.3f\" % score)\n",
"cm = metrics.confusion_matrix(concat_y_test, pred, labels=['true', 'false'])\n",
"plot_confusion_matrix(cm, classes=['true', 'false'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}