Coursera_ML-with-Python_Capstone.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [],
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### About dataset"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets download the dataset"
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-07-12 11:54:54--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\n100%[======================================>] 23,101      --.-K/s   in 0.001s  \n\n2020-07-12 11:54:54 (17.3 MB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ],
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Data From CSV File  "
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    },
                    "execution_count": 3,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 10)"
                    },
                    "execution_count": 4,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Convert to date time object "
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Data visualization and pre-processing\n\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Let\u2019s see how many of each class is in our data set "
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['loan_status'].value_counts()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Lets plot some columns to underestand data better:"
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Solving environment: done\n\n## Package Plan ##\n\n  environment location: /opt/conda/envs/Python36\n\n  added / updated specs: \n    - seaborn\n\n\nThe following packages will be downloaded:\n\n    package                    |            build\n    ---------------------------|-----------------\n    seaborn-0.10.1             |             py_0         160 KB  anaconda\n    openssl-1.1.1g             |       h7b6447c_0         3.8 MB  anaconda\n    ca-certificates-2020.6.24  |                0         133 KB  anaconda\n    certifi-2020.6.20          |           py36_0         160 KB  anaconda\n    ------------------------------------------------------------\n                                           Total:         4.2 MB\n\nThe following packages will be UPDATED:\n\n    ca-certificates: 2020.6.24-0        --> 2020.6.24-0       anaconda\n    certifi:         2020.6.20-py36_0   --> 2020.6.20-py36_0  anaconda\n    openssl:         1.1.1g-h7b6447c_0  --> 1.1.1g-h7b6447c_0 anaconda\n    seaborn:         0.9.0-pyh91ea838_1 --> 0.10.1-py_0       anaconda\n\n\nDownloading and Extracting Packages\nseaborn-0.10.1       | 160 KB    | ##################################### | 100% \nopenssl-1.1.1g       | 3.8 MB    | ##################################### | 100% \nca-certificates-2020 | 133 KB    | ##################################### | 100% \ncertifi-2020.6.20    | 160 KB    | ##################################### | 100% \nPreparing transaction: done\nVerifying transaction: done\nExecuting transaction: done\n"
                }
            ],
            "source": "# notice: installing seaborn might takes a few minutes\n!conda install -c anaconda seaborn -y"
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Pre-processing:  Feature selection/extraction"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Lets look at the day of the week people get the loan "
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    },
                    "execution_count": 11,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Convert Categorical features to numerical values"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets look at gender:"
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets convert male to 0 and female to 1:\n"
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## One Hot Encoding  \n#### How about education?"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Feature befor One Hot Encoding"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Feature selection"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets defind feature sets, X:"
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X = Feature\nX[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "What are our lables?"
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 18,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "y = df['loan_status'].values\ny[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Normalize Data "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                },
                {
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Classification "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Downloading the Test set (we need to use it to evalute the best KNN among the  K)"
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-07-12 11:59:34--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\n100%[======================================>] 3,642       --.-K/s   in 0s      \n\n2020-07-12 11:59:34 (250 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ],
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Test set and clean it "
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [],
            "source": "test_df = pd.read_csv('loan_test.csv')\n\n#cleaning the Data\ntest_df['due_date'] = pd.to_datetime(test_df['due_date'])\ntest_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n\n#Adding the 'dayofweek' column\ntest_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\nbins = np.linspace(test_df.dayofweek.min(), test_df.dayofweek.max(), 10)\n\n#Replace the 'Gender' with Bool var\ntest_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n\n#Add the 'Weekend' column\ntest_df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n\n#Coding in One Hot the 'education' column\nFeature_test = test_df[['Principal','terms','age','Gender','weekend']]\nFeature_test = pd.concat([Feature_test,pd.get_dummies(test_df['education'])], axis=1)\nFeature_test.drop(['Master or Above'], axis = 1, inplace=True)\nFeature_test.head()\n\nX_test = Feature_test\n\ny_test = test_df['loan_status'].values"
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        0         0                     0   \n2       1000     30   43       1        0         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college  \n0        0  \n1        0  \n2        0  \n3        1  \n4        0  "
                    },
                    "execution_count": 22,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X_test.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                },
                {
                    "data": {
                        "text/plain": "array([[ 0.49362588,  0.92844966,  3.05981865,  1.97714211, -4.12310563,\n         2.39791576, -0.79772404, -0.86135677],\n       [-3.56269116, -1.70427745,  0.53336288, -0.50578054, -4.12310563,\n        -0.41702883, -0.79772404, -0.86135677],\n       [ 0.49362588,  0.92844966,  1.88080596,  1.97714211, -4.12310563,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.98251057, -0.50578054,  0.24253563,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.24253563,\n         2.39791576, -0.79772404, -0.86135677]])"
                    },
                    "execution_count": 23,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X_test = preprocessing.StandardScaler().fit(X_test).transform(X_test)\nX_test[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 51,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION'\n 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION'\n 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION'\n 'COLLECTION' 'COLLECTION' 'COLLECTION']\n[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0]\n"
                }
            ],
            "source": "le_status = preprocessing.LabelEncoder()\nle_status.fit(['PAIDOFF','COLLECTION'])\ny_test_binary = le_status.transform(y_test)\nprint(y_test)\nprint(y_test_binary)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Test set Accuracy for k= 1 is: 0.6296296296296297\nTest set Accuracy for k= 2 is: 0.48148148148148145\nTest set Accuracy for k= 3 is: 0.5555555555555556\nTest set Accuracy for k= 4 is: 0.48148148148148145\nTest set Accuracy for k= 5 is: 0.6666666666666666\nTest set Accuracy for k= 6 is: 0.6481481481481481\nTest set Accuracy for k= 7 is: 0.6666666666666666\nTest set Accuracy for k= 8 is: 0.6296296296296297\nTest set Accuracy for k= 9 is: 0.6666666666666666\nTest set Accuracy for k= 10 is: 0.6481481481481481\nTest set Accuracy for k= 11 is: 0.6851851851851852\nTest set Accuracy for k= 12 is: 0.6666666666666666\nTest set Accuracy for k= 13 is: 0.7592592592592593\nTest set Accuracy for k= 14 is: 0.7592592592592593\nTest set Accuracy for k= 15 is: 0.7592592592592593\nTest set Accuracy for k= 16 is: 0.7407407407407407\nTest set Accuracy for k= 17 is: 0.7407407407407407\nTest set Accuracy for k= 18 is: 0.7407407407407407\nTest set Accuracy for k= 19 is: 0.7407407407407407\n"
                }
            ],
            "source": "from sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics\nK = list(range(1,20))\nk_scores = []\nfor k in K:\n    #train model and predict\n    neigh = KNeighborsClassifier(n_neighbors = k).fit(X,y)\n    #predict the test set\n    yhat = neigh.predict(X_test)\n    #calculate the accuracy of this KNN\n    k_scores.append(metrics.accuracy_score(y_test, yhat))\n    print(\"Test set Accuracy for k=\", k, \"is:\", k_scores[k-1])\n    \n"
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8FPX5wPHPk/viSEhCuK+AnIqKyOVVFfCoeLRWbRXtYa1VW2sPba1WW1vbX2ut1lZRqVataJV6IqhVqwjIoQgEBJKAJEAgJ0cOcj2/P2aCS8hmj+xmczzv12tf7M58Z+bZyTLPzPf7ne+IqmKMMca0JirSARhjjOn4LFkYY4zxyZKFMcYYnyxZGGOM8cmShTHGGJ8sWRhjjPHJkoXpEESkr4i8LyIHRORP7bTN00WksD221RGJyFARURGJ8aPs1SKytJX5F4lIgYgcFJHjQxDbeyLybT/LqohUisg9bd1uMETkHRGpaW3/dAWWLDoI9z9HuYjERzqWCLkWKAF6quotkQ6mJc0PYG6yKReRy9zPKiLrRSTKo8xvROQJ933Twfn1Zut9WkR+1T7fImz+CNygqimq+kkEtn+cqv6i6YOIzBORzSLSKCJX+1q4tfIiEi8ifxaRXe7f+28iEts0X1W/BFwXwu/SIVmy6ABEZChwCqDABe28bZ9nle1kCLBRO8ldoiIyE3gJ+KaqLvCY1R+4zMfiU0RketiCi4whQE4wC4pIdIhjAfgUuB74OATlbwUmAeOBUcAJwO0hiLFTsWTRMVwFrACeAOZ6zhCRRBH5k4h8LiL7RGSpiCS682aIyDIRqXCrAK52pzc/Az6iCsE9u/2+iGwFtrrT/uKuY7+IrBGRUzzKR4vIz0Ukz60mWiMig0TkoeZVRiLyqoj8sKUvKSLTRGSV+z1Wicg0d3rT9/6pW41xVgvLxovIH0Vkh4jsEZGHPfZDqoi8JiLF7pnfayIy0GPZNBH5h8eZ4UvN1n2LiOwVkd0ico3Xv9IX5c8HngeuUNX/NJv9B+AuH0n4D8BvfG3H3dbVIvKhe2ZbISL57n682v177RWRuR7le4nIP9198bmI3N50peP+Hf8oIiUikg+c12xbvUTkcXc/7HSvilo9kLt/l4NANPCpiOS508e4v8MKEckRkQs8lnlCRP4uIotEpBI4w8c2+onIOhH5sT/7DEBVH1LV/wI1ISj/ZeABVS1T1WLgAeCb/sbSVViy6BiuAp5xX7NEpK/HvD8CJwLTgDTgp0CjiAwG3gAeBDKAicDaALZ5IXAyMNb9vMpdRxrwL+DfIpLgzvsRcDlwLtAT5z9KFfAkcLnHwSgdOBN4tvnGRCQNeB3nP1of4D7gdRHpo6pXu9/9D241xtstxPt7nLO6iUA2MAC4w50XBfwD5+x2MFAN/NVj2aeAJGAckAn82WNeFtDLXd+3gIdEJNXLPgPnwPE08BVVXdTC/IXAfuDqVtbxEDCqpaToxcnAOpz99i9gAXASzn74BvBXEUlxyz7ofp/hwGk4v62mBPgd4HzgeJwz5a80286TQL273uOBmUCr7QaqekhVm7Z9nKqOcKtoXgXexNnfNwLPiMgxHoteAdwD9ABaawsZCvwP+Kuq/rG1WMJI3Jfn54Ei0itC8USGqtorgi9gBlAHpLufPwNudt9H4Rz4jmthuduA/3hZ53vAtz0+Xw0s9fiswJd8xFXetF1gMzDHS7lNwNnu+xuARV7KXQmsbDZtOXC1+/4J4DdelhWgEhjhMW0qsM1L+YlAufu+H9AIpLZQ7nR3/8Z4TNsLTGllv+4HVgKJLcxXnAPtucAOIB7nCuIJd/5Qt0wMTpXHCnf608CvvGzzamCrx+cJ7jr6ekwrdb9zNHAIGOsx77vAe+77d4DrPObN9Iinr7tsosf8y4F3W/oNefvu7vtTgCIgymP+s03f0f1b/9PH7+89nBOK7cDlPsoe3nYL85Y2/cb8/P94VHn3b/ghzklZFvCRu81+3v6PdcWXXVlE3lzgTVUtcT//iy+qotKBBCCvheUGeZnurwLPD25VzCa3iqgC5+w03Y9tPYlzdov771NeyvUHPm827XOcM3pfMnCuDNa41RoVwGJ3OiKSJCKPuNUu+4H3gd5uFcogoExVy72su1RV6z0+VwEpXsoC/BLnoPqSeOmMoM4Vxw6cRntvHgX6isiXWynTZI/H+2p3G82npeD8veI4cj977uP+HPl39yw3BIgFdnvs40dwrgwC1R8oUNVGL3HQLA5vvg7sBF4IIgav3KrOptdgPxa5B/gE58p9GU5bVR3OiUW3Yckigtw690uB00SkSESKgJuB40TkOJzeQTXAiBYWL/AyHZyz8CSPz1ktlDnckOy2T/zMjSVVVXsD+/ji0ru1bT0NzHHjHYPzH6klu3AOSJ4G4xwMfCnBOSCOU9Xe7quXflH9cQtwDHCyqvYETm36am7saSLS24/t+KMS58qhF/CCePSKaeZ24Bcc+Xc4TFXrgLuAX3NkFUdblOAcxDz3s+c+3o2TPD3nNSnASYLpHvu4p6qOCyKOXcAg8egVxtF/a386MvwK5zv9y1fbSSDUqepseu3wo3y1qt6gqgNUdTjOldwaVW0IVUydgSWLyLoQaMBpN5jovsYAHwBXuWdm84H7RKS/20A51T2jfQY4S0QuFZEYEekjIhPd9a4FLnbPuLNx6uJb0wOnrroYiBGRO3DaJpo8BvxaREaK41gR6QOgqoU47R1PAS+qarWXbSzCqae/wo33a+73fs3XTnL3w6PAn0UkE0BEBojILI/4q4EKt23kTo9ld+O07fxNnIbwWBE5lTZQ1QPAbJwz6BYPZKr6HrCeZh0WmnkKp6pqdlvi8dhmA07D+z0i0kNEhuC0Nz3tFnkeuElEBrrtMrd6LLsbp43hTyLSU0SiRGSEiJwWRCgf4STVn7r7+3Sctp4FrS51tDrgq0Ay8FSz5NMqEYlz29wEiBWRhNaWb628+1vr7/72p+BcXd7pbV1dlSWLyJoL/ENVd6hqUdMLp3H26+L0qPkxzkFnFVCG09Ab5Z4RnYtzVl2GkyCOc9f7Z6AWp/riSZzE0polOAfULTjVBTUcWU1wH86B5k2cOvvHgUSP+U/i1KV7q4JCVUtxGldvwTkz+ylwvkf1my8/A3KBFW5V09s4VxMA97vxlOD0KlvcbNkrcQ48n+FUHbTYWysQqloBnI3T6P5PLwei23E6DHhbRwPOQcdrmSDciHOgzsepf/8XzgkHOAl3CU430Y9xGuM9XYVTjbURp83qBZw2n4Coai1OF/BzcP4mf8M5+fksyHVdjFMdNj+AhPEmzgnENGCe+761k4TWyo/AqX6qxPmt36qqbwb2TTo/cRtnjAmae6b+NDC0WT21MWEnIjU4VWgPqOovI7D9t4ApOB04zmzv7bcXSxamTdw6+wXAp6p6d6TjMcaEh1VDmaCJyBigAqeq4v4Ih2OMCSO7sjDGGOOTXVkYY4zxqaMMItdm6enpOnTo0EiHYYwxncqaNWtKVDXDV7kukyyGDh3K6tWrIx2GMcZ0KiLSfGSFFlk1lDHGGJ8sWRhjjPHJkoUxxhifLFkYY4zxyZKFMcYYnyxZGGOM8cmShTHGGJ+6zH0WxpjwamhU/vHhNvZX10U6FHomxvL1k4eQGBeyZyIZHyxZGGP8siK/lN+8vgkACdWz/YKkCi9+vJOHv3ECQ/okRzaYbsKShTHGL8vySoiJEtbeOZOU+MgeOt7dvJcfLljLlx9cyp+/NpEzx/SNaDzdgbVZGGP8siyvlGMH9op4ogA445hMXrtxBoPSkvjWk6u5783NNDTaCNrhZMnCGOPTwUP1rCvcx7QR6ZEO5bBBaUm8+L1pXDppIA+8k8vV/1hJeWVtpMPqssKaLERktohsFpFcEbm1hfl/FpG17muLiFR4zGvwmPdKOOM0xrRu1bYyGhqVaSP6RDqUIyTERvOHrxzHvRdP4KP8Ms5/cCnrCit8L2gCFrZkISLRwEM4D20fC1wuImM9y6jqzao6UVUnAg9y5APkq5vmqeoF4YrTGOPbsrwS4mKiOGFIaqRDadFlkwfzwvemAvCVvy9nwcodEY6o6wnnlcVkIFdV81W1Fuc5zXNaKX858GwY4zHGBGlZXiknDO5NQmzH7ap67MDevHrjDE4ensatC9fz0xc+paauIdJhdRnhTBYDgAKPz4XutKOIyBBgGPCOx+QEEVktIitE5EIvy13rllldXFwcqriNMR4qqmrZuHt/h2qv8CYtOY4nrpnMjV/K5vnVhXzl4WUUlFVFOqwuIZzJoqWe2N66K1wGvKCqnqcBg1V1EnAFcL+IjDhqZarzVHWSqk7KyPD5oCdjTBBW5JehSodrr/AmOkq4ZeYxPD53Ep+XVnH+g0t5d/PeSIfV6YUzWRQCgzw+DwR2eSl7Gc2qoFR1l/tvPvAecHzoQzTG+LI8r4TE2GiOHdg70qEE5MwxfXntxhn0753IN59Yxf1vb6HRutcGLZzJYhUwUkSGiUgcTkI4qleTiBwDpALLPaaliki8+z4dmA5sDGOsxhgvluWVctKwNOJiOl9P+yF9kln4vWlcdPwA7n97K996chUVVda9Nhhh++uraj1wA7AE2AQ8r6o5InK3iHj2brocWKCqnil/DLBaRD4F3gXuVVVLFsa0s+IDh9i692CnqYJqSWJcNH/66nH85sLxLM0t4fwHl7Jh575Ih9XphPVWTFVdBCxqNu2OZp9/1cJyy4AJ4YzNGOPb8vxSAKYO77zJAkBE+MaUIYzr35Prn/mYi/++jB/PHMXA1KRIhxYSvRNjmZYd3g4Ikb9v3xjTYS3PK6FHQgzj+veMdCghcfzgVF67cQY3LfiE3y76LNLhhMzEQb15yZKFMSZSlueVcvKwPsREd772Cm/6pMTz1DdPJr+kssuMJ5UQG/6/jyULY0yLdlZUs720iiunDo10KCEXFSVkZ6ZEOoxOpeucLhhjQmp5ntNe0Zkbt03oWLIwxrRoWV4JaclxHNO3R6RDMR2AJQtjzFFUlRV5pUwd3oeoqAg/Fs90CJYsjDFH+by0il37aphiVVDGZcnCGHOUZdZeYZqxZGGMOcry/FL69oxneHpypEMxHYQlC2PMEVSV5XklTB3eBxFrrzAOSxbGmCNs3XuQkoO1neL5Fab9WLIwxhyh6f6KqdZeYTxYsjDGHGFZXgkDUxMZlNY1BtkzoWHJwhhzWGOjsiK/zHpBmaNYsjDGHLZx9372VddZe4U5iiULY8xh1l5hvLFkYYw5bFleCcMzkunbMyHSoZgOxpKFMQaAuoZGVm6z9grTMksWxhgA1u/cR2Vtg7VXmBZZsjDGAF+0V0zp5M/bNuFhycIYAzjtFaOzepCWHBfpUEwHZMnCGMOh+gZWby+3KijjlSULYwyf7KjgUH2jdZk1XlmyMMawLK+UKIHJw9IiHYrpoCxZGGNYkVfKhAG96JUYG+lQTAdlycKYbq6qtp5PCsqZau0VphVhTRYiMltENotIrojc2sL8P4vIWve1RUQqPObNFZGt7mtuOOM0pjtbvb2cuga19grTqphwrVhEooGHgLOBQmCViLyiqhubyqjqzR7lbwSOd9+nAXcCkwAF1rjLlocrXmO6q2V5pcRECScNTY10KKYDC+eVxWQgV1XzVbUWWADMaaX85cCz7vtZwFuqWuYmiLeA2WGM1Zhua3l+KccP7k1SXNjOHU0XEM5kMQAo8Phc6E47iogMAYYB7wSyrIhcKyKrRWR1cXFxSII2pjvZX1PH+sIKptpd28aHcCaLlp70rl7KXga8oKoNgSyrqvNUdZKqTsrIyAgyTGO6r5X5ZTQq1rhtfApnsigEBnl8Hgjs8lL2Mr6oggp0WWNMkJbnlxIfE8Xxg3tHOhTTwYUzWawCRorIMBGJw0kIrzQvJCLHAKnAco/JS4CZIpIqIqnATHeaMSaEluWVcuKQVBJioyMdiungwpYsVLUeuAHnIL8JeF5Vc0TkbhG5wKPo5cACVVWPZcuAX+MknFXA3e40Y0yIlFXWsmn3fnt+hfFLWLs/qOoiYFGzaXc0+/wrL8vOB+aHLThjurmP8pseoWrtFcY3u4PbmG5qWV4pyXHRHDuwV6RDMZ2AJQtjuqlleSWcNCyN2Gg7DBjf7FdiTDe0Z38NecWV1l5h/GbJwphuaIXbXmEPOzL+smRhTAdXW9/Ij55fy9MrPsej02CbLMstpWdCDGP69QzJ+kzXZ8nCmA7ukf/lsfDjndz+0gZufm4tVbX1bV7nsvwSpgzvQ3RUS4MlGHM0SxbGdGC5ew/y4Du5nDehH7ecPYqXP93FxX9bxraSyqDXWVBWRUFZtbVXmIBYsjCmg2psVG5buI7EuGh+dcE4bjxzJE9cM5mi/TVc8OBS3swpCmq9y+3+ChMESxbGdFD/WrmDVdvLuf28MWT0iAfgtFEZvHrDDIamJ3PtU2v4vyWf0dAYWDvG8rxS+iTHMapvSjjCNl2UJQtjOqCifTXc+8ZnTM/uw1dOHHjEvEFpSfz7uqlcdtIgHno3j7nzV1J68JBf61VVlueVMnVEH0SsvcL4z5KFMR2MqnL7Sxuob2zkdxcd2+JBPSE2mnsvOZbfXzKBldvL+PKDS1lbUNHC2o60raSSov011mXWBMyShTEdzKL1Rby9aQ+3nH0Mg/sktVr2aycN5sXrphEVJVz68HKf3WuX5TW1V1jjtgmMJQtjOpCKqlrufGUDEwb04prpQ/1aZsLAXrx6wwymjujD7S9t4Mf/XkdNXUOLZZfnldKvVwJDfSQhY5qzZGFMB/LbRZsor6rj3ksmEBPAmE2pyXHMv/okfnDmSBZ+UsjFf1vGjtKqI8o0Nior8q29wgTHkoUxHcSHuSU8v7qQa08dzrj+gY8EGx0l3Hz2KObPPYnC8irOf/AD3vlsz+H5W/YeoLSy1p63bYJiycKYDqC6toHbFq5nWHoyPzhzZJvWdcboTF678RQGpibxzSdWc9+bm2loVJblWnuFCV5YH35kjPHP/W9vYUdZFQuunRKSR5wO7pPEwuuncftLG3jgnVw+KaigvkEZ0ieJganWXmECZ1cW5gi7Kqp56N1cDh5q+/hDbfFmThELVu6IaAwAL32yk1c/3RXWbWzYuY9HP8jn8smDmBLCKqKE2Gj+7yvH8tuLJvBRfhnL80utCsoEza4szBGeW1XAX/67lYUfF/LIlSeSndmjXbdf19DI7xZ9xvwPtwHOGXKk7glYV1jBj55fS6M6Dwq688vjQnLW76muoZGfvrCO9JR4bj1nTEjXDSAiXHHyYMb178k9r2/iq5MG+l7ImBbYlYU5QkF5FT0SYqioqmPOXz9k0frd7bbtvftruOLRFcz/cBtzpw5hSJ8kfr5wvdduoOFU19DIz15cT3pKPNeeOpxnVxbw1YeXU1he5XvhADz2wTY27t7P3XPG0ysxNqTr9nTcoN48f91UThySFrZtmK7NkoU5QmFZNWOyevLaTTMYldWD65/5mHte30h9Q2NYt7tyWxnnPbiUDTv385fLJnLXnPH87qIJbC+t4i//3RrWbbfk0Q/y2bR7P7++cDw/P3cM8648ke0llZz/4FL+t6U4JNvYVlLJ/W9vYfa4LGaPzwrJOo0JF0sW5ggF5VUMTEukX69Enrt2KnOnDuHRD7bx9cc+Yu+BmpBvT1V5fOk2Ln90BSnxMbz0/enMmTgAgGnZ6Vw6aSDz3s9nw859Id+2N/nFB7n/7a2cMz6LWeOcg/jMcVm8cuMMsnomcPU/VvLAf7fSGOAAfp5UnRFl42KiuGvOuFCFbkzY+EwWIvJVEenhvr9dRBaKyAnhD820t9r6Ror21xzuLeMcyMZz/9cm8mlhBec/sJTV28tCtr3KQ/Xc+Own/Pq1jZw1JpOXb5jOMVlHtpH84tyxpCbFcevCdWG/uoGmYcHXkxATxV0XHHkQH5aezMLrp3HhxAHc99YWvv3P1eyrqgtqO8+tKmBFfhk/P3cMfXsmhCJ0Y8LKnyuLX6rqARGZAcwCngT+Ht6wTCTsqqhGFQalJh4x/cLjB/Cf66eTFBfNZfNW8I8Pt7X58Z65ew8y5yGnTeTWc0bz8DdOpGfC0XX2vZJiuXvOODbs3H+40TucnltdwEfbyvjFeWPIbOEgnhQXw32XHsfdc8bxwdZivvzXpeTsCuyqZ+/+Gu5ZtIkpw9O47KRBoQrdmLDyJ1k0tS6eB/xdVV8G4sIXkomUArfxdlDa0f3wx/Trycs3zOCM0Znc9epGfrBgLZVBdq99Y/1u5vx1KeWVtTz9rZO57rQRrQ4/cc74LM4e25f73trC56XBPyHOlz37a/jtok1MHd6HSyd5P4iLCFdNHcqCa6dSW9/IxX9bxotrCv3ezp2v5HCovpHfXdzyiLLGdET+JIudIvIIcCmwSETi/VzOdDIFZdVAy8kCoFdiLI9840R+MusYXlu3i4v+9iH5xQf9Xn99QyO/XbSJ7z3zMSP79uC1m2YwLdt3t1gR4ddzxhMbFcVtC9e3+arGmzte3kBtfSO/vXiCXwfxE4ek8tpNMzhhcCq3/PtTfvGf9Ryqb73n1uINRbyxoYgfnjWSYenJoQrdmLDz56B/KbAEmK2qFUAa8JOwRmUioqC8ithoIauVOvSoKOH7Z2Tzz2+eTMnBWi7464cs3uD78Z7FBw7xjcc/Yt77+Vw5ZQjPfXcK/Xol+lyuSVavBG49dzTL8kr5dwBn8f5avGE3S3L28MOzRgV0EE9Pieepb03mu6cN55mPdnDpIyvYVVHdYtl91XXc8fIGxvbryXdOGR6q0I1pF16ThYj0dN8mAO8BpSKSBhwCVvuzchGZLSKbRSRXRG71UuZSEdkoIjki8i+P6Q0istZ9veLvFzLBKyiron/vRKKjfJ9VzxiZzms3zmBEZgrXPb2Ge9/4zGsD9JrPyzn/wQ9YW1DBfZcex68vHE98TOA3t11+0mAmD03jntc3hbRn1r7qOn75cg7j+vfkO6cMC3j5mOgobjtnDA9/4wTy9h7k/AeXsnRryVHl7n3jM0oOHuL3lxxLbAAjyhrTEbT2i206cK/BSQ5rPF4+k4WIRAMPAecAY4HLRWRsszIjgduA6ao6Dvihx+xqVZ3ovi7w8/uYNigor2ZQAOMG9e+dyPPfncLXTx7Mw//L48rHV1Li8XhPVeXJZdu5bN5yEmKj+c/107n4hODvII6KEn53yQSqaxu469WNQa+nuXvf2ERZZS2/v+TYgIYFb272+H68fMN00lPiuGr+Rzz0bu7h7rUr8kt5duUOvn3KcCYMDHxEWWMizev/DFU93/13mKoOd/9tevlzDT0ZyFXVfFWtBRYAc5qV+Q7wkKqWu9vaG9zXMKGws7yKgan+Vw0BxMdEc89FE/jTV4/j4x3lnP/AUj7eUU5VbT03P7eWO1/J4bRRGbxywwzG9Ovpe4U+jMhI4aYzs3l93W7e2rjH9wI+LM8r5dmVBXx7xjDGD2j7QXxERgr/uX465x/bn/9bsplrn1rD3gM13LZwPYPTkrj5rFFt3oYxkeDPfRbfavY5WkTu9GPdA4ACj8+F7jRPo4BRIvKhiKwQkdke8xJEZLU7/UIvsV3rllldXByau2q7q6raekoO1npt3PblkhMHsvD6acTFRPG1R5Zzzl8+4OVPd/GTWccw78pJIR3K4runjWB0Vg9uf2k9+2uCu88BoKaugdsWrmNwWhI/DOFBPDk+hr9cNpE7vzyW9zbv5bQ/vMe2kkp+e9EEEuNCO7aUMe3Fn2vuM0VkkYj0E5EJwArAn9HlWqr4bt6NJQYYCZwOXA48JiK93XmDVXUScAVwv4iMOGplqvNUdZKqTsrIyPAjJONNYbnTKBvolYWncf2dx3ueNiqTgzX1PHnNZL5/RjZRfrSBBCI2Oop7LzmW4gOH+MPiz4Jez1/+u5XtpVX87uLQH8RFhGumD2PBtVPonRTLN6YMZsbIyAyIaEwo+Bx1VlWvEJGvAeuBKuByVf3Qj3UXAp6d1QcCzcd6LgRWqGodsE1ENuMkj1Wqusvdfr6IvAccD+T5sV0ThIIy7/dYBKJXUiyPzZ1EQ6P61VAerImDenPN9GE8vnQbcyYO4KShgQ2Ql7NrH/Pez+fSSQOZ7kf33WBNGprGhz/7EnY7hens/KmGGgn8AHgR2A5cKSL+HFFWASNFZJiIxAGXAc17Nb0EnOFuJx2nWipfRFLd+zmapk8HQteiaY5yOFmE6ME44UwUTW6ZOYqBqYn87MV1AY1MW9/QyM9eXEdqUhw/Pzf0w4I3FxUldvOd6fT8qYZ6FWfIj+8CpwFbcRJBq1S1HrgB5x6NTcDzqpojIneLSFPvpiU4XXI3Au8CP1HVUmAMsFpEPnWn36uqlizCqKC8msTYaNJTOs/N+UlxMfz2ognkF1fy0Lu5fi83/8NtbNi5n7suGEfvpM7zfY2JJH8efjRZVfcDqHPr7J/8ve9BVRcBi5pNu8PjvQI/cl+eZZYBE/zZhgmNQrcnVGc7Az51VAYXnzCAv7+Xx3nH9mN0Vus9rj4vreS+t7Zw1pi+nDvBhgU3xl8+ryxUdb+IjHdvnrtKRK4CprZDbKYdFZRVt6lxO5J+ed5YeiXG8rMX19PQyrDhqsrP/7Oe2KgofnPh+E6XGI2JJH/aLO4EHnRfZwB/AOwmuS6moLyqzY3bkZKaHMedF4zj04IKnli23Wu5f68p5MPcUn52zmiyetmw4MYEwp82i68AZwJFqnoNcBwQH9aoTLvaV1XHgZr6kDVuR8KXj+3Hl0Zn8sclmw831nvae6CGe17fxElDU7li8uAIRGhM5+ZPsqhW1Uag3h0vai9go6B1IV8MTd45q6HAHZn2wvFECfzipQ1HjUx716sbqa5t4HcXHxvy+z6M6Q78SRar3RvlHsUZF+pjYGVYozLtqulMfGAnvrIAGNA7kZ/OHs37W4p5ae3Ow9Pf2riH19ft5qYzs8nOTIlghMZ0Xv7clHe9+/ZhEVkM9FTVdeENy7Sn1h561NlcOWUIL6/dyd2vbuTUkRnExkTxy5c2MDqrB9eeetQgAMYYP/nTdfYwVd0epjhMBBWWV9MjISak4zdFSlSU8Pu0RfhXAAAbbUlEQVRLjuXcBz7g7tc20iMhhj0Hanj4yhOJi7FhwY0JVkDJois6VN/A2h0VDEpLon/vzltn3xYFZVWdunG7uZF9e/D9M7K5/+2tAHxz+jAmDurtYyljTGu6/anWvuo6vjZvBUtyfD/trasqKK/u1I3bLbn+9GyO6duDQWmJ3DLThgU3pq0CurIQkWtVdV64gomEjJR4eibEkLvX/2dJdyWqSmF5FaeP6lqj9sbFRLHw+mnUNyrJ8d3+AtqYNgv0yuK6sEQRQSJCdmZKt00WxQcPUVPX2CUat5tLju8a7TDGdASBJosu2UE9OzOFvOLumSwKypznWHS1aihjTGgFmiy+HJYoIiw7M4WSg7VUVNVGOpR2V1ge2qHJjTFdU0DJQlULwxVIJDXdqNUdq6KanpA3oJMOImiMaR/dvjcUQHaG85TY7pgsCsqqSE+JIynOGoGNMd5ZssA5q46PieqeyaK8qtMP82GMCT9/hij/qoj0cN/fLiILReSE8IfWfqKjhOEZKeR2w0bugrLqLtkTyhgTWv5cWfxSVQ+IyAxgFvAk8PfwhtX+umP32YZGZVdFNYOsvcIY44M/yaLB/fc84O+q+jLQ5R5cnJ2Rws6KaqprG3wX7iJ276umvlHtysIY45M/yWKniDwCXAosEpF4P5frVEb2TUGVbnW/RVNPqM76OFVjTPvx56B/KbAEmK2qFUAa8JOwRhUBTd1nu1OyaHqOhd1jYYzxxWeyUNUqnKfjzXAn1QNbwxlUJAztk0x0lHSrdouC8mpE6Laj7Rpj/OdPb6g7gZ8Bt7mTYoGnwxlUJMTFRDEkLalbJYvCsir69Uyw5zwYY3zy5yhxEXABUAmgqruAHuEMKlJGdLMeUQXlVQy0xm1jjB/8SRa1qqqAAohIcnhDipzszBS2lVRS19AY6VDaRUFZtbVXGGP84k+yeN7tDdVbRL4DvA08Gt6wIiM7I4X6RuXz0qpIhxJ2h+ob2HOgxkabNcb4xZ8G7j8CLwAvAscAd6jqg/6sXERmi8hmEckVkVu9lLlURDaKSI6I/Mtj+lwR2eq+5vr3ddqmOw0ouKuiBlVsqA9jjF9aHT1ORKKBJap6FvBWICt2l30IOBsoBFaJyCuqutGjzEichvPpqlouIpnu9DTgTmASTvXXGnfZ8kBiCNSIbtR99otus3ZlYYzxrdUrC1VtAKpEpFcQ654M5KpqvqrWAguAOc3KfAd4qCkJqOped/os4C1VLXPnvQXMDiKGgKTEx9CvV0K3uLIoaHqOhTVwG2P84M+41DXAehF5C7dHFICq3uRjuQFAgcfnQuDkZmVGAYjIh0A08CtVXexl2QF+xNpm3WWMqIKyamKjhb49EyIdijGmE/AnWbzuvgLV0iNYtYXtjwROBwYCH4jIeD+XRUSuBa4FGDx4cBAhHm1ERgrPry6gsVGJiuqST5EFnCuLAb0Tie7C39EYEzo+k4WqPikicbhXAcBmVa3zY92FwCCPzwOBXS2UWeGub5uIbMZJHoU4CcRz2fdaiG0eMA9g0qRJRyWTYGRnplBV28Du/TUM6MJ3NheWVVkVlDHGb/7cwX06zvAeDwF/A7aIyKl+rHsVMFJEhrnJ5jLglWZlXgLOcLeTjpOQ8nHGopopIqkikgrMdKeFXXfpEVVYXm09oYwxfvOnGupPwExV3QwgIqOAZ4ETW1tIVetF5Aacg3w0MF9Vc0TkbmC1qr7CF0lhI85Q6D9R1VJ3O7/GSTgAd6tqWeBfL3CeyeK0URntscl2V3montLKWhtt1hjjN3+SRWxTogBQ1S0iEuvPylV1EbCo2bQ7PN4r8CP31XzZ+cB8f7YTSn2S4+idFNulryyahia3aihjjL/8SRarReRx4Cn389eBNeELKbJEhOyMFPK6cLKweyyMMYHyZ7iP7wE5wE3AD4CNwHXhDCrSRvbt2s/jtnssjDGB8ufKIgb4i6reB4fvzI4Pa1QRNiIjhbLKAsoqa0lL7nJPkKWgrJrE2Gj6dMHvZowJD3+uLP4LeNZXJOIMJthldfUeUQXlVQxKS0TE7rEwxvjHn2SRoKqHj5ru+y5df9HVk4V1mzXGBMqfZFEpIic0fRCRE4Hq8IUUef17JZIYG83WvQciHUrIqapzQ541bhtjAuBPm8UPgX+LSNPd1/2Ar4UvpMiLihJGZCZ3ySuLfdV1HDhUb43bxpiA+DPcxyoRGY3zLAsBPvNzuI9OLTsjhZXb2uU+wHZVUOZcFFo1lDEmEF6roUTkJBHJAnCTwwnAb4A/uc+b6NKyM1PYta+GykP1kQ4lpL7oNmvVUMYY/7XWZvEIUAvgjgV1L/BPYB/u4H1dWXYXfRDS4RvyrBrKGBOA1pJFtMd4TF8D5qnqi6r6SyA7/KFFVlftEVVYXk2vxFh6Jvg1YosxxgA+koWINLVpnAm84zHPn4bxTm1In2RioqTLJYuC8iobQNAYE7DWDvrPAv8TkRKcrrIfAIhINk5VVJcWGx3FkD5JXS9ZlFUxMrNHpMMwxnQyXpOFqt4jIv/F6Sr7pjtCLDhXIze2R3CRlp2ZwtYulCxUlcLyar40OjPSoRhjOplWq5NUdUUL07aEL5yOJTszhbc37aW2vpG4GH/uX+zYig8c4lB9ozVuG2MC1vmPgGGUnZlCQ6PyeWllpEMJicPdZu0eC2NMgCxZtCI7w6nb7yrtFk035Nk9FsaYQFmyaMWIzGSg6ySLQvfKYkBvu7IwxgTGkkUrkuJiGNA7scs8CKmgrJr0lHgS46IjHYoxppOxZOFDdmZKl7myaHqOhTHGBMqShQ/ZmSnkFR+ksVF9F+7gCsqrrHHbGBMUSxY+ZGemUFPXyM6Kzv0Ij/qGRnZV1NiVhTEmKJYsfOgqY0Tt3ldDQ6PalYUxJiiWLHzIzugayaKwvKnbrCULY0zgLFn4kJocR5/kuE6fLJpuyLNBBI0xwbBk4YcRmSlh7z5bU9fAYx/kU1PXEJb1F5ZVESXQv7clC2NM4CxZ+KGp++wXYymG3sKPd/Kb1zfx+rrdYVl/QXk1/XolEhttf3JjTODCeuQQkdkisllEckXk1hbmXy0ixSKy1n1922Neg8f0V8IZpy/ZGSnsq66j5GBt2LaxOKfoiH9DraDMnmNhjAle2B5iJCLRwEPA2UAhsEpEXlHVjc2KPqeqN7SwimpVnRiu+ALh2SMqo0d8yNe/r7qOZbklxMdE8f6WYioP1ZMcH9o/TUF5FaeMzAjpOo0x3Uc4rywmA7mqmq+qtcACYE4Ytxc2h5NFmNot3vlsD/WNys1nj+JQfSP/21Ic0vXX1DWwZ/8h6zZrjAlaOJPFAKDA43OhO625S0RknYi8ICKDPKYniMhqEVkhIhe2tAERudYts7q4OLQHWE/9eiWQHBdNXph6RC3eUERWzwS+PWMYfZLjWLwhtFVRuypstFljTNuEM1lIC9OatxC/CgxV1WOBt4EnPeYNVtVJwBXA/SIy4qiVqc5T1UmqOikjI3xVLCLi9IgKQ7Koqq3nf1uKmTWuLzHRUcwc15d3PtvLofrQ9YoqcO+xGGhXFsaYIIUzWRQCnlcKA4FdngVUtVRVD7kfHwVO9Ji3y/03H3gPOD6MsfqUnRGeZPG/zcXU1DUya3wWALPGZXHwUD0f5paEbBsFZe5Dj+zKwhgTpHAmi1XASBEZJiJxwGXAEb2aRKSfx8cLgE3u9FQRiXffpwPTgeYN4+1qRGYKRftrOFBTF9L1Ls4pIjUplslD0wCYNiKdHvExIa2KKiivIi46ir49EkK2TmNM9xK23lCqWi8iNwBLgGhgvqrmiMjdwGpVfQW4SUQuAOqBMuBqd/ExwCMi0oiT0O5toRdVu2pq5M4rrmTioN4hWeeh+gbe2bSXcyZkEePe/xAXE8WZYzJ5a+Me6hsaD09vi8KyagakJhIV1VLNoDHG+Ba2ZAGgqouARc2m3eHx/jbgthaWWwZMCGdsgRrp0X02VMliWV4pBw7VM9utgmoye3wWL63dxcrtZUwbkd7m7RSU2z0Wxpi2sdt5/TQ4LYm46KiQtlss2VBESnzMUQnh1FEZJMRGsSREVVEFZVU2gKAxpk0sWfgpJjqKoelJIUsWDY3Kmxv3cMboTBJij3zMaVJcDKeNymBJzp42P3Tp4KF6yqvq7MrCGNMmliwC4IwRdSAk61q1vYyyylrOaVYF1WT2+CyK9tfwaWFFm7ZT6I42azfkGWPawpJFALIzUthRVhWSkWEXbygiPiaK00a1fH/Il0b3JTZa2twrqqDMnmNhjGk7SxYBGJGZQqPC9tLKNq2nsVFZvKGIU0dleB0DqldiLNNGpLM4p6hNo90evsfCqqGMMW1gySIAoXrE6rqd+yjaX8PscS1XQTWZPT6Lz0ur+Kwo+KqvgvIqkuKiSUuOC3odxhhjySIAIzJSEGl7sli8oYiYKOHMMZmtljt7bF9EaFNVVEFZNYNSkxCxeyyMMcGzZBGAhNhoBqYmtilZqCqLN+xm6og+9E5q/Ww/PSWek4amsaQNz7goLK+yYT6MMW1mySJAbR0jasueg2wvrWKWjyqoJrPHZfFZ0QG2lQTeTqKqFJZX2wCCxpg2s2QRoOzMFPJLKmkI8v6HxRuKEIGZ4/r6Vb5pgMFgri4qquo4eKje7rEwxrSZJYsAZWemUFvfePj+hUAtzili0pBUMv0c1G9A70SOHdgrqHaLgqZ7LKzbrDGmjSxZBKgtPaI+L61k0+79fldBNZk1Lou1BRXs3lcd0HKH77GwaihjTBtZsghQdkYPILhk0XR1EGiyaBpoMNCxor64srBqKGNM21iyCFCvpFjSU+KDSxY5RYwf0DPgaqERGSmMzExhcYDtFgVlVfROiqVHQmxAyxljTHOWLIKQnZlMbnFgyaJoXw2f7KjweSOeN+eMz2LltjJKDx7yXdjl9ISyqwpjTNtZsghCtvs87kCG4Xhzo3NV0PzZFf6aNT6LRoW3N+3xe5mC8iprrzDGhIQliyCMzOzBgZp6ig/4f5a/eEMRIzKSyc7sEdQ2x/bryaC0RL97RTU2OvdYWE8oY0woWLIIQqA9osoqa/loW1nQVxUAIsLscVl8mFvKfj+eA1588BC19Y02gKAxJiQsWQShKVls9TNZvL1pDw2Nyjnj+7Vpu7PHZ1Hb0Mi7n+31WbZptNmBdmVhjAkBSxZByOwRT4/4GL+vLBZvKGJA70TG9e/Zpu0ePyiVjB7xflVFFdhDj4wxIWTJIggiwohM/8aIOlBTx9KtJcwen9XmkV+jooRZ4/ry3uZiqmtbfwBToXtDnvWGMsaEgiWLIGVnpvjVffbdzcXUNjS2qb3C0+xx/aiua+D9rcWtlisoryKjR/xRz/c2xphgWLIIUnZmCsUHDrGvuvXG5iUbikhPieeEwakh2e7Jw9PolRjr825u5zkWdlVhjAkNSxZBys7w3SOqpq6BdzfvZea4vkRHhebhQ7HRUZw9ti9vb9pDbX2j13IF5VXWbdYYEzKWLILU1CMqr5Vk8cHWEqpqG4K+a9ub2eOy2F9Tz4r80hbn1zc0sntfjTVuG2NCxpJFkAalJREXE9Vqu8XiDUX0TIhhyvA+Id32jJHpJMVFex0rave+Ghoa1QYQNMaEjCWLIEVHCcPTk71WQ9U1NPL2pj2cNbYvcTGh3c0JsdGcMTqTN3P2tPgQpqZ7LOzKwhgTKmFNFiIyW0Q2i0iuiNzawvyrRaRYRNa6r297zJsrIlvd19xwxhms1rrPrsgvZV91XciroJrMHpdFycFDrPm8/Kh5heXucyyszcIYEyJhSxYiEg08BJwDjAUuF5GxLRR9TlUnuq/H3GXTgDuBk4HJwJ0iEpruRCGUnZFCQXkVNXVH3/OweEMRibHRnDoqIyzbPmN0JnHRUS3eoFdQXkWUQFYv/57GZ4wxvoTzymIykKuq+apaCywA5vi57CzgLVUtU9Vy4C1gdpjiDFp2ZgqqkF9cecT0hkZlSc4ezhidEbb7HFLiYzhlZDpLcoqOGv22oKyKfr0SiY22WkZjTGiE82gyACjw+FzoTmvuEhFZJyIviMigQJYVkWtFZLWIrC4ubv0mtXA4PKBgs0buT3aUU3LwUMBPxAvUrPFZ7KyoZsPO/UdMLyivtsZtY0xIhTNZtHRjQfPW2FeBoap6LPA28GQAy6Kq81R1kqpOysgIT3VPa4alJxMlR99rsXhDEXHRUXxpdGZYt3/WGOf+jcU5u4+YXlBmz7EwxoRWOJNFITDI4/NAYJdnAVUtVdWmh0I8Cpzo77IdQUJsNIPSko6410JVWZxTxPTsPmF/nGlachwnD0s7ot2ipq6BvQcOWeO2MSakwpksVgEjRWSYiMQBlwGveBYQEc8xuy8ANrnvlwAzRSTVbdie6U7rcEY26xGVs2s/heXVIRsLypdzxmeRV1xJ7t4DAOysaOoJZdVQxpjQCVuyUNV64Aacg/wm4HlVzRGRu0XkArfYTSKSIyKfAjcBV7vLlgG/xkk4q4C73WkdzojMFLaVVFLf4Ay9sXhDEVECZ49tn2Qx020XeWO9c3Vx+DkWVg1ljAmhmHCuXFUXAYuaTbvD4/1twG1elp0PzA9nfKGQnZFCbUMjO8qqGJ6RwuKcIk4e1oe05Lh22X7fngmcMLg3i3OKuPHMkRQ03WNhycIYE0LWt7KNPB+xmrv3ALl7D7ZbFVST2eOzyNm1n4KyKgrLqoiLiSKzR3y7xmCM6dosWbTRCI/us0ty9gAwc1zfdo2hqYvukpwiCsqrGNg7kagQjXJrjDEQ5mqo7qBnQix9e8aTu/cgW/ccZOKg3vTr1b6Ny0P6JDOmX08WbyjiUH2jPXfbGBNydmURAtmZKSzPK2X9zn3tXgXVZPa4LNbsKCd370F76JExJuQsWYRAdkYKu/fVAIT9rm1vZo/PQhWq6xrsHgtjTMhZsgiBpkbu0Vk9GJaeHJEYRvVNObztgXZlYYwJMUsWIdDUyB2pKigAETm8fes2a4wJNWvgDoETh6Ty7RnDuOLkwRGNY+7UoTQ0KmP794xoHMaYrkeaD2/dWU2aNElXr14d6TCMMaZTEZE1qjrJVzmrhjLGGOOTJQtjjDE+WbIwxhjjkyULY4wxPlmyMMYY45MlC2OMMT5ZsjDGGOOTJQtjjDE+dZmb8kSkGPg80nH4kA6URDoIP3SWOKHzxGpxhlZniRM6fqxDVDXDV6Eukyw6AxFZ7c+dkpHWWeKEzhOrxRlanSVO6FyxtsaqoYwxxvhkycIYY4xPliza17xIB+CnzhIndJ5YLc7Q6ixxQueK1StrszDGGOOTXVkYY4zxyZKFMcYYnyxZhJiIDBKRd0Vkk4jkiMgPWihzuojsE5G17uuOCMW6XUTWuzEc9eQocTwgIrkisk5ETohAjMd47Ke1IrJfRH7YrEzE9qeIzBeRvSKywWNamoi8JSJb3X9TvSw71y2zVUTmRiDO/xORz9y/7X9EpLeXZVv9nbRDnL8SkZ0ef99zvSw7W0Q2u7/XW8MZZyuxPucR53YRWetl2XbbpyGjqvYK4QvoB5zgvu8BbAHGNitzOvBaB4h1O5DeyvxzgTcAAaYAH0U43migCOcmog6xP4FTgROADR7T/gDc6r6/Ffh9C8ulAfnuv6nu+9R2jnMmEOO+/31LcfrzO2mHOH8F/NiP30YeMByIAz5t/v+uPWJtNv9PwB2R3qehetmVRYip6m5V/dh9fwDYBAyIbFRBmwP8Ux0rgN4i0i+C8ZwJ5Klqh7lTX1XfB8qaTZ4DPOm+fxK4sIVFZwFvqWqZqpYDbwGz2zNOVX1TVevdjyuAgeHavr+87E9/TAZyVTVfVWuBBTh/h7BpLVYREeBS4NlwxtCeLFmEkYgMBY4HPmph9lQR+VRE3hCRce0a2BcUeFNE1ojItS3MHwAUeHwuJLKJ7zK8/+frCPuzSV9V3Q3OyQOQ2UKZjrZvv4lzFdkSX7+T9nCDW10230u1Xkfbn6cAe1R1q5f5HWGfBsSSRZiISArwIvBDVd3fbPbHOFUpxwEPAi+1d3yu6ap6AnAO8H0RObXZfGlhmYj0tRaROOAC4N8tzO4o+zMQHWnf/gKoB57xUsTX7yTc/g6MACYCu3Gqd5rrMPvTdTmtX1VEep8GzJJFGIhILE6ieEZVFzafr6r7VfWg+34RECsi6e0cJqq6y/13L/AfnEt5T4XAII/PA4Fd7RPdUc4BPlbVPc1ndJT96WFPU3Wd++/eFsp0iH3rNqyfD3xd3cr05vz4nYSVqu5R1QZVbQQe9bL9DrE/AUQkBrgYeM5bmUjv02BYsggxt67ycWCTqt7npUyWWw4RmYzzdyhtvyhBRJJFpEfTe5zGzg3Nir0CXOX2ipoC7GuqXokAr2dqHWF/NvMK0NS7aS7wcgtllgAzRSTVrVaZ6U5rNyIyG/gZcIGqVnkp48/vJKyatZNd5GX7q4CRIjLMvQq9DOfvEAlnAZ+pamFLMzvCPg1KpFvYu9oLmIFz+bsOWOu+zgWuA65zy9wA5OD02FgBTItAnMPd7X/qxvILd7pnnAI8hNPLZD0wKUL7NAnn4N/LY1qH2J84CWw3UIdzdvstoA/wX2Cr+2+aW3YS8JjHst8Ect3XNRGIMxennr/pd/qwW7Y/sKi130k7x/mU+/tbh5MA+jWP0/18Lk7vw7xwx+ktVnf6E02/TY+yEdunoXrZcB/GGGN8smooY4wxPlmyMMYY45MlC2OMMT5ZsjDGGOOTJQtjjDE+WbIwxgcROejx/lx3lNjBYdze3SJyVrjWb0wwrOusMT6IyEFVTRGRM3EekTlTVfMiHZcx7cmuLIzxg4icgjPUxHmBJgoR+Y6IrHIHOnxRRJLc6S+LyFXu+++KyDPu+ydE5Cvu+3tFZKM7iN4fQ/utjPGfXVkY44OI1AEHgNNVdV0Qy/dR1VL3/W9wRiN9UET6Ah8C1+AMETNFVctE5AngNeAdYDkwWlVVRHqrakVovpUxgbErC2N8qwOW4Qw9EYzxIvKBiKwHvg6MA2eAPOAO4F3gFlVt/myE/UAN8JiIXAy0OH6TMe3BkoUxvjXiPMjmJBH5eUsFRGSJ+4jMx1qY/QRwg6pOAO4CEjzmTcAZ96p/84XUeTDRZJwRjC8EFrflSxjTFjGRDsCYzkBVq0TkfOADEdmjqo83mz+rlcV7ALvdoeu/DuyEwyPknoPzgKz/icibqrqtaSH3mShJqrpIRFbgDPxnTERYsjDGT257wmzgfREpUdWWhh5vyS9xnpb4Oc7oqT1EJB6nwfwaVd0lIrcA80XkSx7L9QBeFpEEnBGAbw7ZlzEmQNbAbYwxxidrszDGGOOTJQtjjDE+WbIwxhjjkyULY4wxPlmyMMYY45MlC2OMMT5ZsjDGGOPT/wN2FLK973sBgwAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "# plotting the points  \nplt.plot(K, k_scores) \n# naming the x axis \nplt.xlabel('K - axis') \n# naming the y axis \nplt.ylabel('Scores - axis') \n# giving a title to my graph \nplt.title('Accuracy of each KNN model for k [1-19]') \n# function to show the plot \nplt.show() "
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# We can see that starting from K=13 the accuracy converges into 0.75, then we take K=13 as the best KNN."
        },
        {
            "cell_type": "code",
            "execution_count": 54,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF']\n[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1]\n"
                }
            ],
            "source": "k= 13\nneigh = KNeighborsClassifier(n_neighbors = k).fit(X,y)\n#predict the test set\nyhat_KNN = neigh.predict(X_test)\nprint(yhat_KNN)\nle_status = preprocessing.LabelEncoder()\nle_status.fit(['PAIDOFF','COLLECTION'])\nyhat_KNN_binary = le_status.transform(yhat_KNN)\nprint(yhat_KNN_binary)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Decision Tree"
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=4,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    },
                    "execution_count": 41,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn.tree import DecisionTreeClassifier\nloanTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 4)\nloanTree.fit(X,y)"
        },
        {
            "cell_type": "code",
            "execution_count": 58,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF'\n 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF'\n 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\n['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION'\n 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION'\n 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION'\n 'COLLECTION' 'COLLECTION' 'COLLECTION']\n[1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 0 1 0 1 1 0 1 1 0 1 1 1 1 1 1]\n"
                }
            ],
            "source": "yhat_predTree = loanTree.predict(X_test)\nprint (yhat_predTree)\nprint (y_test )\nle_status = preprocessing.LabelEncoder()\nle_status.fit(['PAIDOFF','COLLECTION'])\nyhat_predTree_binary = le_status.transform(yhat_predTree)\nprint(yhat_predTree_binary)"
        },
        {
            "cell_type": "code",
            "execution_count": 43,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "DecisionTrees's Accuracy:  0.7592592592592593\n"
                }
            ],
            "source": "from sklearn import metrics\nimport matplotlib.pyplot as plt\nprint(\"DecisionTrees's Accuracy: \", metrics.accuracy_score(y_test, yhat_predTree))"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Support Vector Machine"
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n  kernel='sigmoid', max_iter=-1, probability=False, random_state=None,\n  shrinking=True, tol=0.001, verbose=False)"
                    },
                    "execution_count": 44,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn import svm\nclf = svm.SVC(kernel='sigmoid')\nclf.fit(X, y) "
        },
        {
            "cell_type": "code",
            "execution_count": 61,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\n[1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]\n"
                }
            ],
            "source": "yhat_SVM = clf.predict(X_test)\nprint(yhat_SVM)\nle_status = preprocessing.LabelEncoder()\nle_status.fit(['PAIDOFF','COLLECTION'])\nyhat_SVM_binary = le_status.transform(yhat_SVM)\nprint(yhat_SVM_binary)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Logistic Regression"
        },
        {
            "cell_type": "code",
            "execution_count": 46,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0]\n"
                }
            ],
            "source": "from sklearn import preprocessing \nle_status = preprocessing.LabelEncoder()\nle_status.fit(['PAIDOFF','COLLECTION'])\ny_binary = le_status.transform(y)\nprint(y_binary)"
        },
        {
            "cell_type": "code",
            "execution_count": 47,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "LogisticRegression(C=0.001, class_weight=None, dual=False, fit_intercept=True,\n          intercept_scaling=1, max_iter=100, multi_class='warn',\n          n_jobs=None, penalty='l2', random_state=None, solver='liblinear',\n          tol=0.0001, verbose=0, warm_start=False)"
                    },
                    "execution_count": 47,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import confusion_matrix\nLR = LogisticRegression(C=0.001, solver='liblinear').fit(X,y_binary)\nLR"
        },
        {
            "cell_type": "code",
            "execution_count": 48,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "[[0.41364941 0.58635059]\n [0.40698623 0.59301377]\n [0.41589046 0.58410954]\n [0.48897409 0.51102591]\n [0.48218301 0.51781699]\n [0.48138407 0.51861593]\n [0.48364184 0.51635816]\n [0.49010954 0.50989046]\n [0.47122524 0.52877476]\n [0.48159834 0.51840166]\n [0.48417412 0.51582588]\n [0.46929108 0.53070892]\n [0.49026686 0.50973314]\n [0.47803865 0.52196135]\n [0.4813199  0.5186801 ]\n [0.487873   0.512127  ]\n [0.47998282 0.52001718]\n [0.47225522 0.52774478]\n [0.48661113 0.51338887]\n [0.48312828 0.51687172]\n [0.48834488 0.51165512]\n [0.4834427  0.5165573 ]\n [0.47089218 0.52910782]\n [0.48285581 0.51714419]\n [0.48540562 0.51459438]\n [0.48590741 0.51409259]\n [0.48708658 0.51291342]\n [0.48582845 0.51417155]\n [0.48995222 0.51004778]\n [0.48850218 0.51149782]\n [0.48850218 0.51149782]\n [0.48503857 0.51496143]\n [0.48850218 0.51149782]\n [0.48834488 0.51165512]\n [0.48503857 0.51496143]\n [0.48850218 0.51149782]\n [0.49194493 0.50805507]\n [0.48104861 0.51895139]\n [0.48116274 0.51883726]\n [0.47913831 0.52086169]\n [0.49115824 0.50884176]\n [0.47662149 0.52337851]\n [0.47822043 0.52177957]\n [0.48834488 0.51165512]\n [0.47465921 0.52534079]\n [0.49320371 0.50679629]\n [0.48661113 0.51338887]\n [0.48336332 0.51663668]\n [0.48042001 0.51957999]\n [0.49131557 0.50868443]\n [0.48010574 0.51989426]\n [0.49052892 0.50947108]\n [0.48228766 0.51771234]\n [0.49115824 0.50884176]]\n"
                }
            ],
            "source": "yhat_LR = LR.predict(X_test)\n\nyhat_prob = LR.predict_proba(X_test)\nprint(yhat_prob)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Model Evaluation using Test set"
        },
        {
            "cell_type": "code",
            "execution_count": 49,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        },
        {
            "cell_type": "code",
            "execution_count": 56,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "0.7592592592592593\n0.857142857142857\n8.31506830190743\n"
                }
            ],
            "source": "print(jaccard_similarity_score(y_test, yhat_KNN))\nprint(f1_score(y_test_binary, yhat_KNN_binary))\nprint(log_loss(y_test_binary, yhat_KNN_binary))"
        },
        {
            "cell_type": "code",
            "execution_count": 74,
            "metadata": {},
            "outputs": [],
            "source": "j_KNN = jaccard_similarity_score(y_test, yhat_predTree)\nf_KNN = f1_score(y_test_binary, yhat_predTree_binary)\nl_KNN = log_loss(y_test_binary, yhat_predTree_binary)"
        },
        {
            "cell_type": "code",
            "execution_count": 70,
            "metadata": {},
            "outputs": [],
            "source": "j_SVM = jaccard_similarity_score(y_test, yhat_SVM)\nf_SVM = f1_score(y_test_binary, yhat_SVM_binary)\nl_SVM = log_loss(y_test_binary, yhat_SVM_binary)"
        },
        {
            "cell_type": "code",
            "execution_count": 69,
            "metadata": {},
            "outputs": [],
            "source": "j_LR = jaccard_similarity_score(y_test_binary, yhat_LR)\nf_LR = f1_score(y_test_binary, yhat_LR)\nl_LR = log_loss(y_test_binary, yhat_LR)"
        },
        {
            "cell_type": "code",
            "execution_count": 72,
            "metadata": {},
            "outputs": [],
            "source": "j_DT = jaccard_similarity_score(y_test_binary, yhat_predTree_binary)\nf_DT = f1_score(y_test_binary, yhat_predTree_binary)\nl_DT = log_loss(y_test_binary, yhat_predTree_binary)"
        },
        {
            "cell_type": "code",
            "execution_count": 77,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Algorithm</th>\n      <th>Jaccard</th>\n      <th>F1-score</th>\n      <th>LogLoss</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>KNN</td>\n      <td>0.759259</td>\n      <td>0.853933</td>\n      <td>8.315053</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Decision Tree</td>\n      <td>0.759259</td>\n      <td>0.853933</td>\n      <td>8.315053</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>SVM</td>\n      <td>0.685185</td>\n      <td>0.813187</td>\n      <td>10.873526</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>Logistic Regression</td>\n      <td>0.740741</td>\n      <td>0.851064</td>\n      <td>8.954705</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "             Algorithm   Jaccard  F1-score    LogLoss\n0                  KNN  0.759259  0.853933   8.315053\n1        Decision Tree  0.759259  0.853933   8.315053\n2                  SVM  0.685185  0.813187  10.873526\n3  Logistic Regression  0.740741  0.851064   8.954705"
                    },
                    "execution_count": 77,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "report = {\n    'Algorithm' : ['KNN', 'Decision Tree', 'SVM', 'Logistic Regression'],\n    'Jaccard'   : [j_KNN, j_DT, j_SVM, j_LR],\n    'F1-score'  : [f_KNN, f_DT, f_SVM, f_LR],\n    'LogLoss'   : [l_KNN, l_DT, l_SVM, l_LR],\n}\nreport_df = pd.DataFrame (report, columns = ['Algorithm', 'Jaccard', 'F1-score', 'LogLoss'])\n\nreport_df"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | ?       | ?        | NA      |\n| Decision Tree      | ?       | ?        | NA      |\n| SVM                | ?       | ?        | NA      |\n| LogisticRegression | ?       | ?        | ?       |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.6",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 2
}