day-3-data-mining/05 - Structured Data Analysis Pt2.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T12:58:10.237420Z",
     "start_time": "2019-07-03T12:58:10.223157Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "    @font-face {\n",
       "        font-family: \"Computer Modern\";\n",
       "        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunss.otf');\n",
       "    }\n",
       "    @font-face {\n",
       "        font-family: \"Computer Modern\";\n",
       "        font-weight: bold;\n",
       "        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsx.otf');\n",
       "    }\n",
       "    @font-face {\n",
       "        font-family: \"Computer Modern\";\n",
       "        font-style: oblique;\n",
       "        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunsi.otf');\n",
       "    }\n",
       "    @font-face {\n",
       "        font-family: \"Computer Modern\";\n",
       "        font-weight: bold;\n",
       "        font-style: oblique;\n",
       "        src: url('http://9dbb143991406a7c655e-aa5fcb0a5a4ec34cff238a2d56ca4144.r56.cf5.rackcdn.com/cmunso.otf');\n",
       "    }\n",
       "    h1 {\n",
       "        font-family: Helvetica, serif;\n",
       "    }\n",
       "    h4{\n",
       "        margin-top:12px;\n",
       "        margin-bottom: 3px;\n",
       "       }\n",
       "    div.text_cell_render{\n",
       "        font-family: Computer Modern, \"Helvetica Neue\", Arial, Helvetica, Geneva, sans-serif;\n",
       "        line-height: 145%;\n",
       "        font-size: 130%;\n",
       "        margin-left:auto;\n",
       "        margin-right:auto;\n",
       "    }\n",
       "    .CodeMirror{\n",
       "            font-family: \"Source Code Pro\", source-code-pro,Consolas, monospace;\n",
       "    }\n",
       "    .text_cell_render h5 {\n",
       "        font-weight: 300;\n",
       "        font-size: 22pt;\n",
       "        color: #4057A1;\n",
       "        font-style: italic;\n",
       "        margin-bottom: .5em;\n",
       "        margin-top: 0.5em;\n",
       "        display: block;\n",
       "    }\n",
       "    \n",
       "    .warning{\n",
       "        color: rgb( 240, 20, 20 )\n",
       "        }  \n",
       "</style>\n",
       "<script>\n",
       "    MathJax.Hub.Config({\n",
       "                        TeX: {\n",
       "                           extensions: [\"AMSmath.js\"]\n",
       "                           },\n",
       "                tex2jax: {\n",
       "                    inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
       "                    displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
       "                },\n",
       "                displayAlign: 'center', // Change this to 'center' to center equations.\n",
       "                \"HTML-CSS\": {\n",
       "                    styles: {'.MathJax_Display': {\"margin\": 4}}\n",
       "                }\n",
       "        });\n",
       "</script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML\n",
    "def css_styling():\n",
    "    styles = open(\"../Data/www/styles/custom.css\", \"r\").read()\n",
    "    return HTML(styles)\n",
    "css_styling()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Synopsis\n",
    "\n",
    "In this unit we will learn:\n",
    "\n",
    "* The basic concept of a linear regression\n",
    "* How to run a linear regression with statsmodels\n",
    "* The different capabilities of statsmodels and scikit-learn to run regressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression Analysis\n",
    "\n",
    "So far we've covered how you can perform basic data analysis on structured data using Pandas. However, Pandas does not have one of the most common forms of analysis for this type of data built into it --- regression analysis.\n",
    "\n",
    "Many of you have created a linear regression before, although not all of you may know that you did it. The basic idea is pretty simple, we want to determine the linear relationship between some variable $x$ and another variable $y$. \n",
    "\n",
    "Sounds familiar right? All I'm describing is the equation $y = mx + b$ that we learned back in high school. This linear model describes the relationship between some independent $x$ variable and its dependent $y$.\n",
    "\n",
    "Fitting a regression model is common in many disciplines, so I'm going to show you how to do that in Python now. I want to explain enough of the mathematical background such that everyone can follow along, but this isn't meant as a proper explanation of the underlying statistics or when you should use a regression model. If you're not familiar with this model or the underlying statistics and want to learn more I recommend this [statistics textbook](http://www.amazon.com/Statistics-4th-Edition-David-Freedman/dp/0393929728)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# But...math!\n",
    "\n",
    "I know, so let's start with a simple, concrete example. Let's say that someone's High-School Senior GPA is almost exactly the same as their first year College GPA. This means that if we wanted to predict a student's first year gpa ($y$) we would guess based on their High school GPA ($x$). More simply, $y = x$.\n",
    "\n",
    "I'll create that data now with Numpy where I apply some small amount of noise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T12:58:15.329927Z",
     "start_time": "2019-07-03T12:58:14.185636Z"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib as mpl\n",
    "#We turn off the latex usage in matplotlib because LaTeX doesn't know\n",
    "#how to handle a '_' character without it being escaped with a backslash\n",
    "#Since we use '_' in column names typically this can be a bit of a problem\n",
    "#If we don't turn this off\n",
    "mpl.rc('text', usetex=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T12:58:21.444798Z",
     "start_time": "2019-07-03T12:58:21.438066Z"
    }
   },
   "outputs": [],
   "source": [
    "#I will create 100 gpas, with a random probability\n",
    "high_school_gpas = np.random.uniform(0, 4, 100)\n",
    "#The college gpas are created by adding some noise\n",
    "college_gpas = high_school_gpas + np.random.normal(0, 0.2, 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the relationship just so it's clear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T12:58:23.062344Z",
     "start_time": "2019-07-03T12:58:22.880418Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'High School GPA')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAH81JREFUeJzt3X+UXGWd5/H3N51OKE3SzkpnZANtXHDZI4SfLYvDrGvaHzAEYXb1rODIDBxnsuKZCTkj+Itl3dXZ4yizHhZngA0zMJpx0F0ExIBEnA66zgxgB5K0EZmNIEoPZxN1SAI2STr57h/3VlKprnvrVvX9VXU/r3P6WH3r1r0PZfpbT32f5/k+5u6IiEj/m1d0A0REJB8K+CIiFaGALyJSEQr4IiIVoYAvIlIRCvgiIhWhgC8iUhEK+CIiFaGALyJSEfOLbkCjY4891pcvX150M0REesbmzZt/5u7DSc4tVcBfvnw5ExMTRTdDRKRnmNmzSc9VSkdEpCIU8EVEKkIBX0SkIhTwRUQqQgFfRKQiFPBFRCqiVNMyRUSqYnxyijs2PcWu3dMMD9W4cuXJjK1Yluk9FfBFRHI2PjnFjfdPsu/AQQB27p7mxvsnATIN+krpiIjk7I5NTx0O9nX7Dhzkjk1PZXrfzAO+mQ2Y2RNmtiHre4mI9IJdu6c7Op6WPHr4VwNP5nAfEZGeMDxU6+h4WjIN+GZ2PLAK+PMs7yMi0kuuXHkyCwcHjjq2cHCAK1eenOl9sx60vRH4MLA44/uIiPSM+sBs38zSMbOLgJ3uvtnM3hJz3mpgNcDIyEhWzRERKZWxFcsyD/DNsuzhnwdcbGYXAscAS8zsr9z9fY0nufs6YB3A6OioZ9geEZHSKGIefmY5fHf/mLsf7+7LgUuB8eZgLyJSRfV5+Dt3T+McmYc/PjmV6X01D19EJGdFzcPPZaWtuz8MPJzHvUREyq6f5+GLiEiDvpyHLyIisxU1D18BX0SkAAvnHwm/i2uDrF21onfn4YuIyGzNlTIB9s8cyuXeCvgiUllFzIWPm6GjHr6ISAaKqklf1AwdUA5fRCqqqLnwi2qDHR1PkwK+iFRSUT1t6/B4mpTSEZFKGh6qsbNFcE86F77b/P/e6QMdHU+TevgiUklzmQs/l1o4RS26AgV8EamosRXLWLtqBUuHahiwdKiWeC78XPL/RS26AqV0RKTCuq1JP5f8f1Gbn4ACvohIIo05ezPDffb2HUnTMkVsfgIK+CJScUkGX5vn7LcK9nmlZeZCAV9EKivp4qtWOXuAeWFPP8+0zFwo4ItIZUUNvt7wta3AkaAflZt3dx68flW2jUyRAr6IZKaIWjWdiArkh9yP6unPdc5+WWhapohkoqh9WzsRF7DrPf3xyalCp1KmSQFfRDJRVK2aTrQK5I0ae/rdztkvE6V0RCQTRVaFTKoesG/42lYOtZh5A0c+pNavGeu5AN9MAV9EMtFN3ruInH/9+s2bkjQq04fUXCjgi0gmrlx58qwg2pz3bgzwi2uDvLRvhoOHgp52XvXpG68f1dPvtcHZKMrhi0gm2tWqaR7U3TN94HCwr8sz5z+2YhnXXnJ6XwzORlEPX0QyE1dCIGoxU7M80ylF1rnJgwK+iBQiaSDPIp3SPFZwzknDPLZjV18G+UYK+CJSiKhB3UZpplPqQb75njt3T7Nh80+O+j2vsYO8KYcvIoVoNQd+/oCxuDaY+lz3xvGCJMq2XiAt6uGLSCHyzJcnHS9o1C9TMRsp4ItIYfKqC99N8O6XqZiNFPBFpCfMZVFWkvGCRv00FbORAr6IpK6b4Bz3mqR166O0WgQWZalm6YiIJNNNcG73mrhCbEkCc/N4gZm1XFG7dKjG+jVjCf9Le49m6YhIqrqpktnuNWkUYhtbsYz1a8Z48PpVfb+iNop6+CKSqm6Cc7vXpL0BSb+vqI2igC8iqeomOLd7TZJCbJ3Ka4ZQmSilIyKp6mZ3qHavaVeITZLJrIdvZscA3wEWhve5y90/kdX9RKQcukmXJHlNFXvkaTOP2OVlzhc2M+CV7v6imQ0C3wWudvdHol4zOjrqExMTmbRHRLJT9s3K+5mZbXb30STnZtbD9+CT5MXw18HwJ5tPFxEpzFznyEt+Ms3hm9mAmW0BdgIPufujWd5PRPLXC5uVSyDTWTrufhA4w8xeBdxjZqe6+/cbzzGz1cBqgJGRkSybIyIZKGKzcqWQupPLLB13fwF4GLigxXPr3H3U3UeHh4fzaI6IpChquqWZMT45lfr9mrdGrKeQsrhXv8ks4JvZcNizx8xqwNuAH2Z1PxEpRqsplQCH3DMJxEohdS/LHv5xwCYz2wZ8jyCHvyHD+4lIAepz5OeZzXoui0BcRAqpX2QW8N19m7uf6e6nufup7v7JrO4lIsUaW7GMqCneaQfiqBRSP9avT5tW2opIKvIKxN2s5JWAaumIyFG6nQGTRb2bVqpa+CwNCvgicthcFlHlGYhVZqE7CvgiclgaG40oEJeXAr5IiSRNp2S18EgzYPqbAr5ISSRNp2RZuybtjUakXDRLR6Qkki4oynLhkWbA9Df18EVKImk6Jcu0i2bA9DcFfJGSSJpO6Sbt0knOXwOv/UspHZGSSJpO6TTt0qrY2Gfu3cL5n7qfy28aT1TrZnxyistvGueCDl4j5aMevkhJ1HvVt2zczp7pAwAsmD+7T9Zp2qVVzr8uyYCvNjjpHwr4IiWzb+bQ4cd7pw+0DK6dpF3a5fbbzbOf69x8KQ8FfJESiQquN3xtK5+9d0tXg6hROf9GcR8KmpvfPxTwRUokKogeCitRJk3BNKZ7zjlpmIe2TUWmdSB+wFdz8/uHBm1FSiRJEI2bc99qgPahbVO8/bRlLevV18XNs9fc/P7RUcA3s1ea2fvM7P6sGiRSZeeclGybz6hvAlEpocd27IqsVw/xg6/1DU6WDtUwYOlQjbWrVih/34PapnTMbAFwIfBegj1pvwrcmnG7RCrpsR27Ep0X9U0gLt8elZpZmuBbhebm94fIHr6Zvd3MbgeeAd4NrAd+4e5XuvvX82qgSJUkGQidP2BM759pOSc+bhMSpWYkLqWzETgR+HV3f18Y5A/FnC8icxQVsOeZYcCS2iDuwXTNeo6+caPwuKCu1IzEpXTOBi4FvmVmTwNfBmZvTS8iqYnaNaoemC+/afzwoqy6xjnx7RZlKTVTbZEB392fAJ4APmJm5wGXAQvM7BvAPe6+Lqc2ilRGu4CdZE68grpEsbiR+1knm80D3g5c6u5Xpt2Y0dFRn5iYSPuyIj2reU799P4Z9jb18CFIz6xfM1ZAC6VoZrbZ3UeTnBvZwzezpcDHgZOASeDT7r6HILe/MY2Giki0VjVs5g8YA/OMg4eOdNQ08CpJxQ3afhF4Cfg8sAi4KZcWiQjQek79zEHnlQvna+BVuhI3aPsad78ufLzRzB7Po0EiEojK1++dPsD/vuYdObdG+kFcwDcz+xWgvh57oPF3d/9F1o0TqTLVsJG0xaV0hoDNDT9LgMfDxxpZFcmYFkpJ2uKmZS7PsR0i0qSel7954/bDM3MWttgQRSSpuFk6A0DN3V8Mfz8XWBA+/YS7782hfSKVt79hQ5Q9ERuiiCQR1134DPDBht/vBK4Frgf+U5aNEpFA3G5TIp2KG7R9K/DGht9fcPd3mpkB/yfbZokIaLcpSVdcD3+eu880/P4RAA+W5i7KtFUiAsRXvxTpVFzAX2Bmi+u/uPs3AcxsCDgm64aJiGbqSLriAv5twFfMbKR+wMxeS5DLvy3rhomIdpuSdMVNy/ycmf0S+K6ZvRJwglILf+zut+TVQJGqU/VLSUvsFofufitwq5ktIqisqamYUhnNlSobyxT3w/2ketruaQtQn4vfCTM7gaAA22sIdspa5+7/o9PriBShVaXKLOe/530/qaZEAb9LM8CH3P3xcPB3s5k95O4/yPCeIsDce8tx89/HVixLvTfe7n4iacgs4Lv788Dz4eO9ZvYksAxQwJdMddtbbgziUdsC7do9nUlvXPPtJQ9tA76ZvQL4EDDi7r9nZq8HTnb3DUlvYmbLgTOBR7tsp0hi3fSWP//AJBs2/6TttYeHal1dv903AlXGlDwkqcR0B7APeFP4+3PAHyW9QTjg+1VgbbhjVvPzq81swswmdu3alfSyIpE67S2PT04lCvb1+e/dXP/G+yfZGX5zqH8jGJ+cOnyO5ttLHpIE/BPd/bPAAQB3n+ZIjfxYZjZIEOy/5O53tzrH3de5+6i7jw4PDydstki0TlentqtL0zz/vZvrt6uHo/n2kockOfz9ZlYjmIePmZ1I0OOPFdbc+QvgSXf/3JxaKdKBK1eefFSOHeJ7y3F58labg6d1/ebjmm8vWUsS8D8BPAicYGZfAs4DrkjwuvOAy4FJM9sSHvu4uz/QTUNFkqoHzaSzaKLy50DLIJ7W9ZWfl7xZUAutzUlmrwbOJfh2+4i7/yyLxoyOjvrEhDbTknw1z7qpu+jsEf7gwhWZXH/h4IBSNpIKM9vs7qNJzk0yS+es8OHz4f+OhAXUnm2qpinSkzrtsZft+iJJte3hm9kjwFnANoIe/qnh41cDH6hX0UyDevgiIp1JtYcP/Bh4v7tvDy/+BoKdrz4F3A2kFvBFmqW1olV1akSSBfx/VQ/2AO7+AzM7092fDibiSBXlEUDTWtGqOjUigSTz8J8ys1vM7N+GPzcD/2BmCwnn5ku1JFlIlIa09nPVvrAigSQ9/CsINjNfS5DD/y5wDUGwX5lZy6S08ir0lVZ9majzd+6e5oJP3d/RNxSlhqSXtQ347j4d9uo3uHtzl6jjssnS+/Iq9JXW/PW4efaN31CgfXE1pYakl7VN6ZjZxcAWgsVXmNkZZnZf1g2T8sprY+206su0uk6zJCkepYak1yXJ4X8COAd4AcDdtwDLM2yTlFxehb7Sqi/TfJ0o7b6hqISx9LokOfwZd9+tGTlSl+dCoqj6Mp3m0huvc/lN412lilQiQXpdkoD/fTN7LzAQ1sJfA/xdts2Sssur0FerwA7MKZfeafGzub5OpCySBPw/AK4jqJB5J7CRYNGVSKaiBkkXzJ83p1lC3X5DUYkE6XWJiqflRaUVpFFU6iWKAQ9evyq7BomUUCqlFczs6xC5tSfufnEXbRNJrNPBUOXSReLFpXT+JLdWSF9Ie1FS1CDpktog+2YOKZcu0qHIgO/u386zIVJu7YJ5kkVJnX4gRA2SXnX+KYBy6SKdikvpTNI6pWOAu/tpmbVKSiVJMG9XbqHbVaqNA7RLaoNcdf4ph89XgBfpTFxK56LcWiGllqR2TrtFSZ3W32m1S9S+mUNz+u8QqbrIlbbu/mz9B3gZWBH+TIfHpCKSrDBdVBtseU79eKerVFXGQCR9SbY4/A/ADcDDBOmcz5vZte5+V8Ztk5JIssJ05mDr3nf9eLtrNOf3o6ZjplnGQJUvpWqSLLy6Dniju+8EMLNh4FuAAn5Oig5MSVaYTu8/2OqlTO8/yPjkFC/vn739cf0arfL7UdKaeqnKl1JFSYqnzasH+9DPE75OUpDXZiNx5lrE7DP3bmHP9NF75SyuDR6+Rqv0TStpTr1UykiqKEkP/0Ez20hQVgHgPcA3smuSNMprs5F22tXOWVwbZG9TUI9TWzC/7YAvBB8uWXyzUeVLqaIkG6Bca2b/Hvh1ghz+One/J/OWCVCOwJQkpfTB80/hT+7bysFDyUp1NLY/Kme/dKjG+jVjc2t8BFW+lCqKm4d/EvCr7v637n43cHd4/M1mdqK7/yivRlZZ0YEpLtcNRy9++o0zT+CBx3/KoQT1mRrbX0QVSlW+lCqKy8XfCOxtcfyX4XOSg7w2G4kSlVK6eeP2WWMLD22bShTsgaPan9ZGJ50o4p4iRYtL6Sx3923NB919wsyWZ9YiOUrRJXmjUket8vX7DhxknlnboH/R2SOz2p9Xff2i7ylSpLiAf0zMc0p05qjIwBQ3J76VQ+4sHBxoOetmqea6ixQqLqXzPTP7veaDZvZ+YHN2TZIyiUopLYlYWbt0qMbbT1vGvHBLzHlmXHT2CBuvX8X6NWMK9iIFiuvhrwXuMbPf4kiAHwUWAP8u64ZJOUSllIBZg57zB4w9v9zHhs0/OXzskDsPbZvilBP+mYK9SMHiyiP/P+DXzGwlcGp4+H53H8+lZVIacSml+gfB4togL+2b4eUDs0ssFLFuQERmSzIPfxOwKYe2SI9p/CC4/KbxWatpG2lBk0jxVCJBUtEuoGtBk0jxkpRWkIIVXTwtibjZPFrQJFIOCvglN9eqjnl9WLRauQpBjZ0PNuxSJSLFUcAvubkUT8uzBHDRC8REpL3MAr6Z3U6wTeJOdz+13fnS2lyKp+VdaVMrV0XKLctB278ELsjw+pUQNdiZZBC0DJU2RaQ8Muvhu/t3VHNn7uZS1bGbSpu9MEAsIt1RDr/k5pIb7/TDol3Of3xyils2bj88337h4DwWzh9g7/QBfTiI9IDCA76ZrQZWA4yMjBTcmnLqNjfe6YdFu23//vvXtzJz0BueO8S+cGWt9oQVKb/CA767rwPWAYyOjiYrpi5tNadmPvybZ7QNxHE5/zs2PXVUsG9FJRREyq3wgC/pak67QPLed1zOP+lArwaERcors1k6ZnYn8PfAyWb2XFhWWTJUz8G3qmmz78BBbvjaVsYnpyJfH7e7VtLSCCqhIFJeWc7SuSyra0trt2zc3nLjkbpD7rE9/XY5/+YcfjOVUBApN6V0+sT45FRstcq65jx7q2mY69eMzXpd/XzN0hHpXQr4KSpyDnt9Jk0S9Tx7p6UXtJJWpLcp4Kckz7o19fs1frh0su9sPc+ed+kFESmW6uGnpN0c9jTVP1x27p7GoaNgDxzOs6v0gki1KOCnJM/g2erDpRP13vtc6vSISO9RwE9JnsEz7kNk6VANA+aZRT5fFzcNU0T6jwJ+SvIMnlEfIkuHaqxfM8aD16/i2ktOb9uesRXLWLtqxeEPiaVDNdauWqH8vUif0qBtSvLcACRJUbSk7dHMG5HqMPfylK8ZHR31iYmJopvRE1TGWEQAzGyzu48mOVc9/DbSDqxpXU89cxHplAJ+jLTm1teDfPP0SZUUFpE8VTrgt+ttJ12YFHed5g+NZlroJCJ5qWzAT9J7TzK3vt11ksyZ10InEclDZadlJlkZm2RufbvrJAnmWugkInmobMBP0ntPMre+3XWSBPOX98/E1qkXEUlDZQN+kt57koVJ7a7T6kOj2Z7pA9x4/6SCvohkqrI5/CSLl6D99Md212m1AOrl/TOzatfvO3CQmzdu1+CtiGSmsgE/rZWxSa7TfE7UUre90wcYn5xS0BeRTGilbcbGJ6e4eeN29ibYjQpgcW2Qu655R8atEpF+oZW2JdFuDn4rST8YREQ6VdlB2zzMtW69iEiaFPAz1M2CqiW1wQxaIiKigJ+puDn4A/OMgaZNSuYPGFedf0rWzRKRilLAz1DUHPwltUGuufh0rrnk9KPm+H/onadrho6IZKZyg7Z51pHvZMpmvV2fvXeL6tuLSCYqFfDTKnfciSR164tol4hUT6VSOkkKphWhrO0Skf5SqYCfpGBaEcraLhHpL5VK6QwP1WbtOlU/nrW4sYMi2yUi1VGpHn6ScsdZqOfod4Z1dOo5+np1zKLaJSLVUqmAn6TccRba5eiLapeIVEulUjqQbNZM2pLk6Itol4hUS6V6+EVJstmKiEjWKtfDb9bJQqxuF20l3WxFRCRLlQ74nSx4msviqLQ2WxERmYu+Cvid9sDjBlObX9fJua0oRy8iRcs0h29mF5jZU2a2w8w+muW92k19bKWTBU9aHCUivS6zgG9mA8CfAb8BvAG4zMzekNX9uilP0MlgqgZeRaTXZdnDPwfY4e5Pu/t+4MvAJVndrJseeCcLnrQ4SkR6XZY5/GXATxt+fw7411ndrJvyBJ0MpmrgVUR6XZYB31oc81knma0GVgOMjIx0fbNupz52MpiqgVcR6WVZpnSeA05o+P144B+bT3L3de4+6u6jw8PDXd9M5QlEROJl2cP/HvB6M3sdMAVcCrw3w/sV3gPPczctEZFOZRbw3X3GzH4f2AgMALe7+/as7lc07VolImWX6Tx8d3/A3f+lu5/o7v8ty3sVTbtWiUjZqXhaSrQwS0TKTgE/JVqYJSJl1/O1dMoyUKqKmCJSdj0d8Ms0UKqFWSJSdj0d8LutYJnVt4Kip4WKiMTp6YDfzUBpmb4ViIjkqacHbbsZKNX0SRGpqp4O+N1UsNT0SRGpqp5O6XQzUNpNVU0RkX7Q0wEfOh8o1fRJEamqng/4ndL0SRGpqsoFfND0SRGppp4etBURkeQU8EVEKkIBX0SkIhTwRUQqQgFfRKQiFPBFRCrC3L3oNhxmZruAZ7t8+bHAz1JsTlrUrs6oXZ1RuzrTj+16rbsPJzmxVAF/Lsxswt1Hi25HM7WrM2pXZ9SuzlS9XUrpiIhUhAK+iEhF9FPAX1d0AyKoXZ1RuzqjdnWm0u3qmxy+iIjE66cevoiIxOi5gG9mF5jZU2a2w8w+2uL5hWb2lfD5R81seUnadYWZ7TKzLeHP7+bQptvNbKeZfT/ieTOzm8I2bzOzs7JuU8J2vcXMdje8V/85p3adYGabzOxJM9tuZle3OCf39yxhu3J/z8zsGDN7zMy2hu36ry3Oyf3vMWG7cv97bLj3gJk9YWYbWjyX7fvl7j3zAwwAPwL+BbAA2Aq8oemcDwK3ho8vBb5SknZdAfxpzu/Xm4GzgO9HPH8h8A3AgHOBR0vSrrcAGwr493UccFb4eDHwDy3+f8z9PUvYrtzfs/A9WBQ+HgQeBc5tOqeIv8ck7cr977Hh3n8I/HWr/7+yfr96rYd/DrDD3Z929/3Al4FLms65BPhC+Pgu4K1mZiVoV+7c/TvAL2JOuQT4ogceAV5lZseVoF2FcPfn3f3x8PFe4EmgeeOE3N+zhO3KXfgevBj+Ohj+NA8K5v73mLBdhTCz44FVwJ9HnJLp+9VrAX8Z8NOG359j9j/8w+e4+wywG3h1CdoF8K4wDXCXmZ2QcZuSSNruIrwp/Er+DTM7Je+bh1+lzyToHTYq9D2LaRcU8J6F6YktwE7gIXePfL9y/HtM0i4o5u/xRuDDwKGI5zN9v3ot4Lf6pGv+5E5yTtqS3PPrwHJ3Pw34Fkc+xYtUxHuVxOMEy8VPBz4P3Jvnzc1sEfBVYK2772l+usVLcnnP2rSrkPfM3Q+6+xnA8cA5ZnZq0ymFvF8J2pX736OZXQTsdPfNcae1OJba+9VrAf85oPGT+HjgH6POMbP5wBDZpw/atsvdf+7u+8JfbwPOzrhNSSR5P3Pn7nvqX8nd/QFg0MyOzePeZjZIEFS/5O53tzilkPesXbuKfM/Ce74APAxc0PRUEX+PbdtV0N/jecDFZvZjgrTvmJn9VdM5mb5fvRbwvwe83sxeZ2YLCAY17ms65z7gd8LH7wbGPRwBKbJdTXneiwnysEW7D/jtcObJucBud3++6EaZ2WvqeUszO4fg3+nPc7ivAX8BPOnun4s4Lff3LEm7injPzGzYzF4VPq4BbwN+2HRa7n+PSdpVxN+ju3/M3Y939+UEMWLc3d/XdFqm71dPbWLu7jNm9vvARoKZMbe7+3Yz+yQw4e73EfxhrDezHQSfjJeWpF1rzOxiYCZs1xVZt8vM7iSYvXGsmT0HfIJgAAt3vxV4gGDWyQ7gl8CVWbcpYbveDVxlZjPANHBpDh/aEPTALgcmw/wvwMeBkYa2FfGeJWlXEe/ZccAXzGyA4APmf7n7hqL/HhO2K/e/xyh5vl9aaSsiUhG9ltIREZEuKeCLiFSEAr6ISEUo4IuIVIQCvohIRSjgS+HM7MWm368wsz8NH3/AzH67zesPn9/mvIvCKoVbzewHZvYfY85dbhHVPDtlZn9pZu+OeO4PzeyHZjYZtutz4SIrzOzHDce/aWavaXjdmWbmZnZ+Gm2UalDAl1Jz91vd/YtzvU4YRNcB7wzLD5xJsAKzMGb2AeAdBJUcVwBvJKj9Ums4bWXY3gmCufd1lwHfDf9XJBEFfCk1M/svZnZN+PiNYbGrvzezG5p64P/czB40s/9rZp9tcanFBAsNfw7g7vvc/anwur9qZveEPemtZvZr4WsGzOw2C2qqfzNctYmZnWFmj4RtucfMfiXueIzrgKvC5f+4+353/+MWdXIAvgOcFN7HCBZaXQG8w8yOaXMfEUABX8qhZkc2otgCfDLivDuAD7j7m4CDTc+dAbwHWAG8x5qqH7r7LwiWrT9rZnea2W+ZWf3f/03At8Oe9FnA9vD464E/c/dTgBeAd4XHvwh8JCy8NUmwUjju+CxmtpigZvszUec0uSi8JgQrb59x9x8RfEu5MOE1pOIU8KUMpt39jPoPMGu3prA2ymJ3/7vw0F83nfI37r7b3V8GfgC8tvka7v67wFuBx4BrgNvDp8aAW8JzDrr77vD4M+5eL2WwGVhuZkPAq9z92+HxLwBvjjoe899sNFRBNLPzww+8Hzd8wwDYFH4ILgE+HR67jKD4FuH/Kq0jifRULR2ptHabQOxreHyQiH/b7j5JUJNmPfAM8TVUmq9ZizqxU+6+x8xeMrPXufsz7r4R2GjBtncLGk5d6e4/q/8S1od5F0HVxesI3pdXm9nicHMUkUjq4UtPcPd/AvaGFSqhw6JSZrbIzN7ScOgM4Nnw8d8AV4XnDZjZkph27Ab+ycz+TXjocoJ0UMvjbZr1aeCWhsqOBrTLx78N2OruJ7j7cnd/LUHZ5N9s8zoR9fClp7wfuM3MXiLIXe+OP/0oBnzYzP4nQTXJlzjSu78aWGdm7yfoyV8FxJU8/h3gVjN7BfA0RypmRh2PcgvwCuBRM9sHvAj8LfBEzGsuA+5pOvbVsM3r29xPKk7VMqVnmNmi+iYfZvZR4Dh3v7rgZon0DPXwpZesMrOPEfy7fZYCa5iL9CL18EVEKkKDtiIiFaGALyJSEQr4IiIVoYAvIlIRCvgiIhWhgC8iUhH/H9/yLooh+l9KAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(high_school_gpas, college_gpas, color='steelblue')\n",
    "plt.ylabel('College GPA')\n",
    "plt.xlabel('High School GPA')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's say that we didn't create this data and we wanted to identify the relationship between High School and College GPA. We can do that by fitting a linear model to the data.\n",
    "\n",
    "With Python we can fit a Linear Model using the `statsmodels` library. Fitting a linear model involves using [Ordinary Least Squares](https://en.wikipedia.org/wiki/Ordinary_least_squares) to identify the parameters. In this lecture I won't go over how this works, but you should look into it later if you plan on fitting linear models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T12:59:36.281990Z",
     "start_time": "2019-07-03T12:59:34.558616Z"
    }
   },
   "outputs": [],
   "source": [
    "import statsmodels.api as sm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First what I do is add a constant to the independent dataset. Without this we are just modelling $y=mx$, instead of $y=mx+b$. Our example doesn't need this constant, but I want to show you how to do it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T12:59:38.291693Z",
     "start_time": "2019-07-03T12:59:38.287817Z"
    }
   },
   "outputs": [],
   "source": [
    "X = sm.add_constant(high_school_gpas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we create the ordinary least squares regression, this is our `model`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:00:04.917286Z",
     "start_time": "2019-07-03T13:00:04.913877Z"
    }
   },
   "outputs": [],
   "source": [
    "simdata_model = sm.OLS(college_gpas, X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we fit the model, this becomes our result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:00:32.766957Z",
     "start_time": "2019-07-03T13:00:32.762906Z"
    }
   },
   "outputs": [],
   "source": [
    "simdata_result = simdata_model.fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The fitted result has a number of features, but the one that we care about most is the `summary()`. This tells us about our fitted model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:00:35.436265Z",
     "start_time": "2019-07-03T13:00:35.412110Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>            <td>y</td>        <th>  R-squared:         </th> <td>   0.964</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.964</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   2651.</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Wed, 03 Jul 2019</td> <th>  Prob (F-statistic):</th> <td>9.22e-73</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>10:00:35</td>     <th>  Log-Likelihood:    </th> <td>  11.791</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>   100</td>      <th>  AIC:               </th> <td>  -19.58</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>    98</td>      <th>  BIC:               </th> <td>  -14.37</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "    <td></td>       <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th> <td>    0.0056</td> <td>    0.041</td> <td>    0.137</td> <td> 0.892</td> <td>   -0.076</td> <td>    0.087</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>x1</th>    <td>    0.9800</td> <td>    0.019</td> <td>   51.486</td> <td> 0.000</td> <td>    0.942</td> <td>    1.018</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 2.329</td> <th>  Durbin-Watson:     </th> <td>   2.239</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.312</td> <th>  Jarque-Bera (JB):  </th> <td>   1.721</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td> 0.270</td> <th>  Prob(JB):          </th> <td>   0.423</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 3.350</td> <th>  Cond. No.          </th> <td>    4.78</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                      y   R-squared:                       0.964\n",
       "Model:                            OLS   Adj. R-squared:                  0.964\n",
       "Method:                 Least Squares   F-statistic:                     2651.\n",
       "Date:                Wed, 03 Jul 2019   Prob (F-statistic):           9.22e-73\n",
       "Time:                        10:00:35   Log-Likelihood:                 11.791\n",
       "No. Observations:                 100   AIC:                            -19.58\n",
       "Df Residuals:                      98   BIC:                            -14.37\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const          0.0056      0.041      0.137      0.892      -0.076       0.087\n",
       "x1             0.9800      0.019     51.486      0.000       0.942       1.018\n",
       "==============================================================================\n",
       "Omnibus:                        2.329   Durbin-Watson:                   2.239\n",
       "Prob(Omnibus):                  0.312   Jarque-Bera (JB):                1.721\n",
       "Skew:                           0.270   Prob(JB):                        0.423\n",
       "Kurtosis:                       3.350   Cond. No.                         4.78\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "\"\"\""
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simdata_result.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Does anyone know what all of these features mean?\n",
    "\n",
    "**That's insane if you do right now!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can plot the best-fit line along with the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:00:57.152938Z",
     "start_time": "2019-07-03T13:00:56.983344Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'High School GPA')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xd8lFX2+PHPSQGCQKgBpCtCAKOIAQuiFDtg+aoruupio4S+Iojuqks1FEXAREFlwbX9VFBEEcGACosoaCSU0KQGXEAgIRBS7++PmcRkMi3JtGTO+/Xi5eSZO89zGMmZO/e591wxxqCUUqrqC/F3AEoppXxDE75SSgUJTfhKKRUkNOErpVSQ0ISvlFJBQhO+UkoFCU34SikVJDThK6VUkNCEr5RSQSLM3wEU17BhQ9O6dWt/h6GUUpXG5s2bTxhjGrnTNqASfuvWrdm0aZO/w1BKqUpDRA6421aHdJRSKkhowldKqSChCV8ppYKEJnyllAoSmvCVUipIaMJXSqkgEVDTMpVSKlgkpaSxcM1Ojqdn0Sgygkd7tad3TDOvXlMTvlJK+VhSShqzv0ghOzcfgGPpWcz+IgXAq0lfh3SUUsrHFq7ZWZTsC2Xn5rNwzU6vXtfrCV9EQkXkFxFZ7u1rKaVUZXA8PavEz7lZmXaPe5ovevijgB0+uI5SSlUKjSIjAMg+c5ItH8/gv68NIy/7XNFxb/FqwheR5kBf4E1vXkcppSqTR3pcTNqPy1k/Zwi/b/mW3HNnOHd0N4/2au/V63r7pu1sYBxQ28vXUUqpSmHDhg2Mi4tjW3IyAI3adeXa+0cy8r5elXeWjoj0A44ZYzaLSE8n7QYBgwBatmzprXCUUsqvTpw4wTPPPMNbb70FQKtWrXj11Ve54447EBGfxCDGGO+cWGQa8DCQB9QA6gBLjDEPOXpNbGys0fLISqmqpKCggDfffJMJEyZw8uRJwsPDefrpp+l+56O8/8PBCs/DF5HNxphYd9p6rYdvjJkATLAG1BMY6yzZK6VUVfPzzz8TFxfHxo0bAejTpw/z5s3jSG5tnYevlFJVwenTpxkxYgRdu3Zl48aNNG3alA8++IBVq1YRHR1ddefhAxhj1hpj+vniWkop5S/GGP7zn/8QHR3NvHnzEBHGjBlDamoq999/f9FYvaP59t6eh6+lFZRSygO2bdvGsGHD+PbbbwHo3r07CQkJXHbZZaXaNoqM4Jid5F6p5+ErpVRVl5mZybhx4+jcuTPffvstjRo1YuHChXz33Xd2kz3Ao73aUz08tMSx6uGhlX4evlJKVUnGGJYsWcLo0aM5fPgwIsKQIUOYOnUq9erVc/n66mEhReP4tSPCibulU+Wdh6+UUlXV7t27GTFiBCtXrgQgNjaWhIQEunbt6vK1tpUyAXLyCrwWa3Ga8JVSQausNemzsrKYNm0a8fHx5OTkULduXaZNm8aTTz5JaGiow9cV52yGjvbwlVLKC8pak3758uWMHDmSffv2ATBw4EDi4+OJiooq03X9NUMH9KatUipIuTsX/sCBA9x1113079+fffv2ERMTw/fff8/ChQvLnOwBakWEl+m4J2nCV0oFJVc97ZycHF566SU6dOjAZ599Rq1atZg1axabN2/muuuuK/d1HVXN8UU1HR3SUUoFJWdz4ZOSkhg2bBipqakA3H///cyaNYtmzf4c6invnrRnsnLLdNyTtIevlApK9ubCm6x0jnw1hz59+pCamkq7du1YtWoVH3zwQalkP/uLFI6lZ2H4c/w/KSXN5XUdLa7y9qIr0ISvlApSvWOaMbpvDFGREZj8fE7+8iXr5wwmacVn1KhRg0mTJrFlyxZuvPHGUq+tSC0cfy26Ah3SUUoFsd4xzaiesZ+4uDFs2bIFgP79+/Pqq6/Spk0bh6+ryEybwmGf8gwHVZQmfKVUUDp+/DjPPPMMb7/9NmDZkGTOnDnccccdpdrajtfXjggnw86Yu7vDMr1jmvkkwdvShK+UCioFBQUsWLCACRMmcOrUKcLDq9G+119o0u0uPtpfi1opaSWSsb35+qEhQliokJf/5wZSvhqWqQhN+EqpoLF582bi4uL48ccfAehydQ8ir32E8LpNAfuLr+yN1+cXGGpHhBNRLcznwzIVoQlfKVXlnTp1in/84x8kJiZijOHCCy9k9uzZfHakPsczzpdom52bT+LKbUXJ29G4fGZWLh+PvdnrsXuSztJRSnlNUkoaD89J4tZJX/DwnCS3pi16kjGGxYsXEx0dTUJCAiEhITz11FOkpqZy3333ccIm2RfKyMotitWf0yg9TXv4SimvKGutGk/bunUrcXFxfP/99wD06NGD1157jZiYmKI2jhZfAcR/mszCNTvp1rYRq7aklRjWqQzj9fZoD18p5RX+2rf1zJkzjB07ls6dO/P9998TFRXFokWL+Pbbb0ske8Bl0j6WnsWqLWncdFkzoiIjECAqMoLRfWMCfrzeHu3hK6W8wtdVIY0xfPTRR4wZM4YjR44gIgwbNozJkydTt25du6/pHdOMhJXbnJY1yM7N58c9x3lnZG+vxO1LmvCVUl5Rnn1by1ufZteuXQwfPpxVq1YB0LVrVxISEoiNjXX52rhbOpXakMSWL0oX+4ImfKWUVzzaq32pRFp87Ns2uduOlbsz5n/u3DmmTZvG9OnTycnJoV69ekybNo0nnnjC7Q1Jik+/dDSeXxlv0NqjCV8p5RXOSgjYu6G7fPPBUudwthPU8uXLGTFiBPv37wfg0UcfJT4+nkaNGpUrVntxQeW9QWuPJnyllNc4KiFg74auI7bDKfv372fUqFEsW7YMgMsuu4yEhAS6d+/ukXgL46tMC6rcpQlfKeVzZRkTLxxOyc7OZtasWUyePJmsrCxq167NxIkTGT58OGFhZUtl9oaTftxzvEom+eI04SulfM7Z/PfiCodTVq9ezbBhw9i1axcAAwYMYNasWVx44YVuX7Mwydte13Y4ydfrBXxJ5+ErpXzOUU34fle2LDHf/a+xDZk/5Sluuukmdu3aRfv2luT//vvvlznZF25Y4g5frBfwB+3hK6V8ztVYeV5eHnPnzuWJu58nMzOTiIgI/vnPf/LUU09RrVq1Ml+vLPcMClWVqZjFacJXSvmFoxu669atIy4ujpQUy7DKXXfdxezZs2nVqlW5r1We5F1VpmIWpwlfKRUQjh07xvjx4/n3v/8NQJs2bZgzZw79+vUDyr8oC9y/Z1CoKk3FLE4TvlLK48qSnPPz85k/fz7jxk8g80w6EhrGpTcOIH7SC9zWtW3R+SpSiM3eIjBHonSWjlJKuacsyfmnn34iLi6OTZs2AdCgbRei+w7mggbNSPxmD9VrRNA7ppnTQmzuJGbbewYiQoExpdpFRUZUiZo5jmjCV0p5lDvJ+eTJkzz33HO88cYbGGOoGdmQtrc+QeOO3RGRUq/xRCG24vcMqvqKWkc04SulPMpZci4oKGDx4sWMGzeO48ePExYWxujRo/ml+lWEVi99k7TwXOUpxOZMVV9R64gmfKWURzlKzmGZR7j++utZv349ANdffz0JCQl06tSJh+ckOU3orgqxlYejWUJVmS68Ukp5lO2iqrzz59iz8i1WzBrK+vXriYqKYvHixaxdu5ZOnTrZfQ2UTOi9Y5oxum9MldiExJ+81sMXkRrAd0B163U+Nsa84K3rKaUCQ2ESfjsplZT1X7N75VtkZfxBSEiIww1J3BliCcYeuad5c0gnG+htjMkUkXBgnYisMMb84MVrKqX8wHYa5o2tw/jf8pfYsno1AN26dSMxMZEuXbo4PIcmdO/zWsI3xhgg0/pjuPVP6XlQSqlKrfiMl/yc86z/5B3eXf8JJj+PevXqER8fz+OPP05IiI4g+5tXb9qKSCiwGWgLvGaM2ejN6ymlfK9wGuax1I2kfvkG508fA+Diq27lh+Xv0LBhQz9HqAp5NeEbY/KBziJSF1gqIpcaY7YWbyMig4BBAC1btvRmOEopLziwfx+pX87n+M4fAajdpA0d+sVRr2UHryT7ipRYCHY+mZZpjDktImuBW4GtNs/NB+YDxMbG6pCPUpVEdnY2M2fO5L/zJpKfm0NY9Zpc3OchWnTtS0hoKCJCUkqaR5NxRUssBDtvztJpBORak30EcCMQ763rKaV8Z9WqVQwfPrxoQ5Jml/ek7c2PUb12/aI2BcZ4PBlXtMRCsPPmXZSmwBoR2QL8BKwyxiz34vWUUl6WlpbG/fffz80338yuXbvo0KEDSUlJLH7nP0TUaVCqvac3EvFEiYVg5s1ZOluAK7x1fqWU7+Tm5jJnzhxefPFFMjMzqVmzJs8//zxjxowp2pBk+qfJdl/ryWTs6RILwUbnSSmlnPr+++/p0qULY8eOJTMzk7vvvpsdO3Ywfvz4ErtPOUq6nkzGrlbkKue0lo5SqoTCWTCH045wcM1ifvtpFQAXXXQRc+fO5fbbb7f7Om/Uu7EVrEXPPEUTvlKqSFJKGq98nsyeDV+wZ/Vi8s6fJSQsnIeeGM7rL08hIsJxb91XyVhX5JafJnylVJEZ//6M/37wChlH9gDQoO2VdOg7mIKLLnaa7AtpMg5smvCVCiDuLiry9OKjkydP8uyzz/LV/PlgDNXrNCT69kFEdbgGEdFZMFWEJnylAoS7i4o8ufiooKCARYsWMW7cOE6cOIGEhNKq+11cdMMAwoptSKKzYKoGnaWjVIBwtqioPO1c+fXXX+nRowePPfYYJ06coGfPnrz5ySpi+j5RItnrLJiqQ3v4SgUIdxcVVXTxUUZGBs8//zxz586loKCAxo0bM2vWLB588EFEhNZaq6bK0oSvVIBwd1FReRYfJaWkWTYkWbeS3V+/XbQhyciRI5k4cSKRkZFFbfXGa9WlCV+pAOHuPPayzndPSklj0sIv2fLpPE7u2wJAZItout4zkjsfu6NEsndEK1RWDZrwlQoQhQk0ceU2MrJyAagWVvo2W1nmu587d46Rf3+a7Ws+wuTnEV6zDpfc9DeaXXETEhLi1s1erVBZdWjCVyrAZOcVFD0+k5VrN7m6GnYxxrBs2TJGjRrFgQMHAGh25S1cctPfqFazzp/XcqPSpFaorDo04SsVQBwl1xmf/cr0T5PdGk757bffGDVqFMuXW4rT1mvelktuG0LdFtF227u62asVKqsOnZapVABxlEQLjMHw53BKUkpaqTbnz59n4sSJdOjYkeXLlxNW4wK63jOCFxI+ovFFnRxes1ZEuNOYfFEUTfmGJnylAog7SdTenPuVK1cSExPDCy+8QE52Nk0v70X3Ea9T7/Jb+Gbb79x0WTNE7J/PweEiWqGy6ijTkI6IXADcDTxgjOnrnZCUCl7d2jZi+eaDLtsVfhM4dOgQY8aM4ZNPPgEgskkrLrltMPXbXFbUNjs3nx/3HAcHG4iesd4gdkQrVFYdLhO+iFQDbgcexLIn7SfA616OS6mg9OOe4261a1ArnBkzZvCvf/2Ls2fPUrNmTV544QVWn22HhJUeoilM1OXdPETn5lcNDod0ROQmEXkb2AfcC7wDnDTGPGqM+dxXASoVTNy5EZp+cCvL4wcxbtw4zp49S48+t5Gamsq4ceNo3KCO3dcU9sp1aCa4OevhrwS+B64zxuwDEJFXfRKVUkHKUS88RITzGSf57Zt/c/DnbwCIqN+UDn2HULdjN3afDqFFC+eLsnRoRjlL+FcCA4DVIvIb8AEQ6qS9UqqC7CXsaqHQJvNnFs+fSXp6OiFh4bTpcR+tr7uX0PBqJebEu0rqOjQT3BwmfGPML8AvwHgR6Q48AFQTkRXAUmPMfB/FqFTQsE3YnNpH6oo3WJ66FYCGl8QS3XcwNes3LfG64kNBmtSVI27N0jHGrAfWi8hI4CYsPX9N+Ep5Qe+YZlx+YQ0mTJjAggULAKgR2Yhu9wynYfRVZJ7PK/UanROv3OEw4YtIFPAs0BZIAaYZYzKwjO2v9E14SgWXgoICFi5cyPjx4/njjz+Q0DBaX3sXbW4YQFi1GpzPzSc0RMgv+HOOpd54Ve5y1sNfDGwG5gL9gDnAQB/EpFRQSk5OJi4ujg0bNgDQ+JLOXHzrYGo1alHUJi/fUCcinBrVwvTGqyozZwm/iTHmOevjlSLysy8CUirYpKen8/zzzzNv3jwKCgpo0qQJr7zyCgt31cLe8tgzWbl8NPZmP0SqKjtnpRVEROqJSH0RqQ+E2vyslKoAYwzvvvsu0dHRzJkzB4BRo0aRmprKgAEDiKpb0+7rdLxelZezHn4kliGd4l2Mwl6+AS7yVlBKVXU7duwgLi6OtWvXAnDttdeSkJDA5ZdfXtSmrBudKOWKs2mZrX0Yh1JB4ezZs0yaNIlZs2aRl5dHw4YNiY+PZ+DAgYSElPzCXTgun7ByW1G9m+p2NkRRyl3OZumEAhHGmEzrz1cD1axP/2KMOeOD+JSqEowxLF26lNGjR3Po0CFEhMGDBzN16lTq13c+QppTbEOUDAcboijlDmdDOvHAMWC69ef3ga1ADSxDO+O9G5pSVcPevXsZMWIEK1asAKBLly4kJibSrVs3l6/V3aaUJzlL+H2ArsV+Pm2M6S8igqXGjlLKifPnz/PSSy/x0ksvkZ2dTWRkJFOnTmXw4MGEhrpXpUR3m1Ke5Czhhxhjii/pGw9gjDEiUsu7YSlVua1YsYIRI0awd+9eAB555BGmT59O48aNy3SeipQ0VsqWsztA1USkduEPxpivAUQkEsuwjlLKxqFDh7jnnnu4/fbb2bt3Lx07dmTt2rUsWrSozMkedLcp5VnOEv4C4EMRaVl4QERaYRnLX+DtwJSqTHJycoiPjyc6OpolS5ZwwQUXMGPGDJKTk7nhhhvKfd7eMc0Y3TeGqMgIBIiKjGB03xgdv1fl4mxa5ssicg5YZ93a0ABngZeMMYm+ClCpQLd27Vri4uLYsWMHAPfeey+vvPIKzZs398j5tfql8hSn1TKNMa8Dr1vH7EWnYqpgkpSS5nSzkKNHjzJ27Fjee+89ANq2bcu8efO45ZZbvHI9pSrK3fLImWU9sYi0wFKArQlQAMw3xuiOWapSSEpJK7HK9Vh6VtH89+s7NCYhIYF//vOfZGRkUKNGDZ599lmefvppatQo3+0tZ9fTpK88xa2EX055wFPGmJ+tN383i8gqY8x2L15TKaDivWVH89+nL1zKU2veIjk5GYBG7bpy7f0j6X5Xr3Ine2fX0/n2ypO8lvCNMUeBo9bHZ0RkB9AM0ISvvKq8veXiHxLG5rmcs+nsXr2ItM1fAxBRN4r2tw2iUfRVZIlUuDeu8+2VL7hM+CJSE3gKaGmMeVJELgHaG2OWu3sREWkNXAFsLGecSrmtPL3luV+msHzzwVLHTUEBaT9/ze5Vi8jNOkNIaBgde/+Fxlf9H6HV/uzRuzq/q28cOt9e+YI7lZgWAtnANdafDwOT3b2A9YbvJ8Bo645Zts8PEpFNIrLp+PHj7p5WKYfK2ltOSkmzm+wzjuzhxzefZvuyeeRmnaHhxZ15a8lqmvV4sESyd+f8s79I4Zj1m0PhN46klLSiNjrfXvmCOwn/YmPMdCAXwBiTRcmSyQ6JSDiWZP+uMWaJvTbGmPnGmFhjTGyjRo3cDFspxxz1ih0dX7hmZ4mfc7My2bE8kR/e+Dvph3dSvXZ9rvvbP/hgyecMvOOGcp3f0TeOQjrfXvmCO2P4OSISgWUePiJyMZYev1PWmjtvATuMMS9XKEqlyqCsdeQLe+bGGI5uWcuulW+Rk3kaCQmh/Q33sPGzt6lTp06Fz+/quM63V97mTsJ/AfgKaCEi7wLdcW9v2+7Aw0CKiCRbjz1rjPmyPIEq5a7CpOnuLJ1GkRH8tjuVHcsTObV/KwB1W3akQ7+hTB5yd4lkX97z6/i8CgQuE74xZpV1P9ursQzljDLGnHDjdetwc+hHKU9zt7ecmZlJzi8fsWHxAkxBPuEXRNLu5ke58PLe9O/a2uE5ytIb152rVKBwZ5ZOF+vDo9b/trQWUDtgU01TqUrDGMOSJUsYPXo0hw8fRkS4pHt/mvd4kGZNozy6yrWs3wiU8hZ3hnQSgC7AFiw99kutjxuIyJDCKppKVRZ79uxhxIgRfPXVVwBceeWVJCYm0rVrVxevLD8dn1eBwJ2Evx943BizDUBEOgJPA5OAJYAmfOU1nqovk5SSxoKVW/jpi3fYt+5jCvJyqVu3LlOnTmXQoEFub0iiVGXmTsKPLkz2AMaY7SJyhTHmN8tEHBWMfFHoy1P1ZZJS0pgw6y1SliWSdep3AJp3uZHZs2ZyT8/LPRqzUoHMnYS/U0QSgQ+sP98P7BKR6ljn5qvg4qtCX56oL3Pw4EEG/vVBDqWsB6BWVCs69BtKvdaX8umWP7inp8fCVSrguZPwBwJxwGgsY/jrgLFYkn0vr0WmApavCn1VpL5MTk4OL7/8MhMnTiQrK4vQahFc3OtBWl7dn5BQyz/7Y+lZ3DrpizJ9Q9ESxqoyc2daZpaIJADLjTE7bZ4uc9lkVfn5qtBXeeevJyUlMWzYMFJTUwFodUVPWvUZSI06DUu1LV7qAFwXV9MSxqoyc1laQUTuAJKxLL5CRDqLyDJvB6YCV1lLC5RXWevLHD16lAcffJA+ffqQmppKu3btWLVqFW8v+g+RDZzvJ2tb6sAed0okKBXI3Kml8wLQDTgNYIxJBlp7MSYV4HxV6Mvd+jJ5eXnMnj2b9u3b8/7771OjRg0mT57Mli1buPHGG0udxxFX31C0hLGq7NwZw88zxqTrjBxVyJcLiRzNXy8cS9+1ZTO7VrzO6SO/AdC/f39effVV2rRp4/A8D89JKtdQkZZIUJWdOwl/q4g8CIRaa+GPBP7r3bBUoPPVQiJ7N0kB4j/8nq1fvsWRX1YDEFGvMc/+6yX+MWKgy3OWt9SBlkhQlZ07CX8E8ByWCpnvAyuxLLpSyqvs3SR95fNk0jZ/TcqKt8nLykRCw2hz3T206XEfO009t85b3m8oWiJBVXZijO1mbv4TGxtrNm3a5O8wVICwHXrJOLKH7Z8nkJG2C4AGF19BdN8hXNDQknAF+Oqfff0RqlJ+IyKbjTGx7rR12MMXkc+h1NaeRYwxd5QjNqXcVngzNDcrkz3fvMOhn74EY6heuz7tbxtE407dKX5vScfSlXLO2ZDOTJ9FoaoETy9KalinBr9+9wU7V75N7tl0JCSEltfcRefbHqEgrIaOpStVRg4TvjHmW18GogKbq2TuzqKksnwgbN26lZR3nmPrZsu+93VbdaJDv6E0bH4xI/vGADqWrlRZORvSScH+kI4AxhhzmdeiUgHFnWTuqtyCu6tUz5w5w7/+9S9mz55Nfn4+1WvV5ZKbH6Xp5b2JrFmNobd0KmqvCV6psnE2pNPPZ1GogOZO7RxXi5JcncMYw8cff8yYMWNIS0tDRGh9dT/a9HqI8IhalvZ5BZ7+qykVVJwN6RwofCwijYHC3SF+NMYc83ZgKnC4s8K0VkQ4Z7JKF0+tFRHu8hy7d+9m+PDhfP21ZWuF2NhYGt3wGPmRLUu09UaBNqWCiTtbHP4FmAGsxTKcM1dEnjbGfOzl2FSAcGeFaV6+/d534XF758jPzeZ/G5dy6eT/R05ODtVq1qZtn0do1/tOTpzJsXs+T5Ux0KqXKhi5s/DqOaBrYa9eRBoBqwFN+D7i7+TkzgrTrJx8ey8lKyefpJQ0zueU3P74+M4f2fnlfM5ZNyRpceVNXHzjQKpdEOkw2YNnpl5q1UsVrNxJ+CE2Qzh/4F7RNeUBgZCcKrrCNP7T5KLHWaf+R+qK+RxPtcy+iYmJoWmvx6FhW5fn8dTUS1/V81cq0LiT8L8SkZVYyiqAZcerFd4LSRUXKMnJVe2c2g7G8AsV5OWyf/1SfvvuQwpyswmrHsH0aVMYMWIE/aatdLjCLyoywuPfbLTqpQpW7myA8rSI/B9wHZYx/PnGmKVej0wB/k9O7g4nxd3SiZnLfiW/oHTq/mNvMju+eJ1zJw4D0CTmetrf8jhjxjwCOL5HEBUZwTsje3v4b6RVL1XwcjYPvy3Q2Biz3hizBFhiPX69iFxsjNnrqyCDmT+Tk6vhJNsPg9uuaMGXPx+iwFqf6XzGCXZ99Ra/b/0egJoNm9Oh7xAaXNyZqGLx+7oKpVa9VMHKWQ9/NvCsnePnrM/190pEqgR/JidXOzzZfhis2pJGgTEU5OdzaOPn7El6l/ycLELCq3PRDffT+tq7CQkLL/p7FfJ1FUqteqmClbOE39oYs8X2oDFmk4i09lpEqgR/Jidnw0mOPgzSD25n2+cJZP5vPwCNoq8m+rYniaj35xaD/a5sWSp+X9XX99f1lAoEzhJ+DSfP6WCnD/krOTkbTrL9MMjOPM3uVf8usSFJ9O1DaNS+a1GbKO1JK+VXzhL+TyLypDFmQfGDIvI4sNm7YalA4Gw4aeGanRxLz8IU5HN400p2r15E3vmzhISGc9N9j1HQ7hYkrBohItzepQUjbo/x499EKQXOE/5oYKmI/JU/E3wsUA2429uBKf9zNZz04huf8OvSeWQc2Q1Aw7ZduOzOoZjIpkWbhRcYw6otaXRqUV979kr5mbNaOv8DrhWRXsCl1sNfGGOSfBKZCgj2hpNOnTrFx4lTWPf66xhjqF6nAVfcOZS67a7GzqxMXdSkVIBwuWLWGLPGGDPX+keTfRAzxrBo0SLat29PYmIioaGhjB07luOH99E2tpfdZF9IFzUp5X/urLRVipSUFOLi4li3bh0A119/PQkJCXTq1AlwndB1UZNS/qcJvxLwZ/G0M2fO8OKLL/Lqq6+Sn59PVFQUM2fO5KGHHiq1n6y9GT2gi5qUChSa8ANcRYunlffDwhjDRx99xJgxYzhy5AghISEMHz6cSZMmUbdu3VLt7c3oAUuNnbhiu1QppfxHE36Aq0jxtPJ+WOzcuZPhw4ezerVlTn23bt1ITEykS5cuDl+jq1eVCnxeS/gi8jaWbRKPGWMuddVe2VeR4mll/bA4d+4cU6dOZfr06eTm5lKvXj3i4+N5/PHHCQlxXRFbV68qFdi8Wdf+38CtXjx/UHB0s9Odm6Bl+bBYtmwZHTt2ZMqUKeTm5vL444+za9cunnzySbeSvVIq8HnKZzesAAAS8klEQVTtN9kY8x1w0lvnDxaP9mpP9fDQEsfcvQnqzofFvn376N+/P3feeScHDhzg4vYduWX0HA63upsx720hKSWtYn8BpVTA0DH8AFeRsXFnpRGys7OZMWMGU6ZM4fz589SuXZuHhz7FgTqx5Fq3p7VXDjlx5TYyrBudVA8PoXpYKGeycnXMXqlKwO8JX0QGAYMAWrZs6edoAlN5x8YdfVjkHd1GzD3D2b3bUhLhgQceYObMmYz/OJVc26Joxcohz/r8V/LyTbHnCsi2fjrovrBKBT6/J3xjzHxgPkBsbKyTtZqqLGynY467qzPt68OYMWP46KOPAIiOjua1116jd2/LrlLH03+xe67CcsjFk709WkJBqcDm94SvPMt22AXg95NnGPHMi+xb+z5Z585Ss2ZNnn/+ecaMGUO1atWK2pWlHLIjWkJBqcDltZu2IvI+sAFoLyKHrWWVlRcVzrsvnuxP7d/KD4mj2P7lm2SdO8t1fW5lx44djB8/vkSyB+c3iN0tjaAlFJQKXF7r4RtjHvDWuZV9iSu3Fd2gzc48xa6vF3I02VLvLqJeE6L7DqZep6vZkx6Kvbslrm4Q247h29ISCkoFNh3SqSKSUtLIyMrFFORzaNNX7Fm92LIhSVg4ra+7lzY97iU0vHqJcXZHZRfsjcEXHtNZOkpVXprwPcSfBc7A0itPT9vFjs8TyDiyB4AGba+kQ9/B1GxwYYm2x9OzylV2QVfSKlW5acL3gIoWOCvvNQs/YOqE5bJhyRsc3rwSjKFGZCPa3/YkUR2uKVHRslCjyIgK1ehRSlVOmvA9wNfJs/AD5nx2LkeSv2HX1wvJPZeBhITSqvvdXNRzAGHVHO9B/2iv9kz/NNnuczrLRqmqSxO+B1SkwFl5LFyzkxOH9rBjeSKnD24HoF7rS+nQL45aUa4Xr/WOaVa0CbktnWWjVNWlCd8DnM1f97SMjAy+/2AOBzd+jikooFqturS75XGaXtYTESHKOmdeRCgwpWfURFljclZ2QSlVNWnC9wBfJE9jDB9++CF///vfOXr0KEgILa/qz8W9/0p4RC3AkszfGWlZNWt7X8E2Jq1fr1Tw0YTvAd5OnqmpqQwbNoykJMuc+g4xV9Co52PUaNSmqI3tB4w7MemsG6WCixg7X/v9JTY21mzatMnfYQSMc+fOMXnyZGbOnElubi7169cnPj6exx57jLXbjmrvXCmFiGw2xsS601Z7+C54en69O+czxrBs2TJGjhzJwYMHAXjiiSeYNm0aDRs2BLR3rpQqO034Tnhqfn1hkre9sWvvfL/99hsjR47kiy++AKBz584kJCRwzTXXVPjvo5QKbkGd8F31tt2dX+/sPPZunto737WXNGDGjBlMnTqV8+fPU6dOHSZPnszQoUMJCwvq/01KKQ8J2kziTu/dnfn1rs5j70PD1o5N64iZO5g9eywlEf76178yc+ZMmjRpUoG/oVJKlRS0Cd+d3rs78+tdncfZ4qvz6SfY+dUC/rdtPQAdOnQgISGBnj17lvevpZRSDnmtHn6gc6f37s4G4q7OY2/xVUF+HvvWfcL6uUP437b1hIZX58nRz5KcnKzJXinlNUGb8B2tgi1+vHdMM0b3jSEqMgLBsrBpdN+YEuP3rs5j+6Fxcv9WNiSOZPfXC8nPOU9Ux2vpPvJ1fm/cg3U7j3vgb6aUUvYF7ZCOu6tjXU1/dHWewtcmfLaB/36cwNFf1wAQUb8pHfoOoeElVwKWYaCEldt0qqVSymuCNuF7anWsq/Pk5+ez/dulfDX9Oc6eySAkLJw2Pe6j9XX3EhpecovBM1m5JKWkadJXSnmFrrT1oh9++IGHH32SPalbAWh4SSzRfQdTs35Th6+pHRHOx2Nv9lWISqlKTlfa+tkff/zBhAkTWLBgAYDLDUmKO1NsA3KllPIkTfgeVFBQwMKFCxk/fjx//PEHIaFhtLr2Ltrc4HxDEqWU8gVN+B6SnJxMXFwcGzZsAKBXr17kXPoXLmjUokznqRMR7o3wlFIqeKdlekp6ejqjRo3iyiuvZMOGDTRp0oT33nuPb775hjZt2zl8XWiIEGozvBMWKgy9pZO3Q1ZKBSlN+OVkjOG9994jOjqaOXPmADBq1ChSU1N54IEHEBG7C7fA0osfe8fljL3z8hJz/J/qf7nO0FFKeU3QzdLxRLnjHTt2MGzYMNasscypv+aaa0hISKBz587lup6nSzArpYKHztJxoKLljs+ePcukSZN4+eWXyc3NpUGDBkyfPp2BAwcSEmL/y5KrhVueKsGslFKuBNWQjrNCZ84YY1i6dCkdO3YkPj6evLw8Bg0axM6dO3nsscccJntvxqSUUmUVVD18dwqm2dq7dy8jRoxgxYoVAHTp0oXExES6devmt5iUUqo8girhu1PuuND58+eJj49n2rRpZGdnExkZyZQpUxgyZAihoaVvxLriaJy+LDEppVRFBNWQjjvljgFWrFjBpZdeyosvvkh2djYPP/wwO3fuZNiwYeVO9rO/SOFYehaGP8fpk1LS3I5JKaUqKqh6+K4KnR06dIjRo0ezZMkSADp16sRrr73GDTfcUKHrOhunf2dkb6cxKaWUpwRVwgf7s2ZycnKYPXs2EydO5OzZs1xwwQW8+OKLjBo1ivDwiq98dTVO72omj1JKeULQJXxba9euJS4ujh07dgBw77338sorr9C8eXOPXUPH6ZVSgSBoE/7vv//O2LFjeffddwGoWb8pV/9lFEPjHqJ5c/u97fIukHJ3sxWllPKmoEv4eXl5JCYm8o9//IOMjAxCwqrR5vr7aN39HkLDqzlc9FSRBVKe2mxFKaUqokolfFc98B9++IGhQ4eSnJwMQLOOV9Hm5iepWb9JUZvCm6m2ydjZjVd3EreO0yul/M2rCV9EbgVeBUKBN40xL3nrWs564JdfWINnnnmGN998E4CWLVsyZ84cEn4NBTsbkti7yaoLpJRSlZ3X5uGLSCjwGnAb0BF4QEQ6eut69nrg57NzeXbqy7Rr144333yT8PBwJkyYwPbt27nzzjuJqlvT7rns3Ux1dINVb7wqpSoLby686gbsMcb8ZozJAT4A7vTWxWx72hlH9/Ljm0+z8YOXOXnyJL1792bLli1MnTqVCy64AHB/IVZZ2yqlVCDy5pBOM+BQsZ8PA1d562KFUx9zz59l7zf/4eCPX4ApIKJOA956fS4DBgwotZ9sWW6m6o1XpVRl582Eb2+37lLF90VkEDAILGPr5TWwZzuenjKHbV8uICfzNBISQptr72b2jKn0vyba4evKcjNVb7wqpSozbyb8w0DxDV2bA0dsGxlj5gPzwbIBSnkutH37diaPGsYva9cCULdFB7o/MIa/P3S7JmillLLyZsL/CbhERNoAacAA4EFPX+T06dNcddVVZGZm0rBhQ2bMmMEjjzxSoRr15aU7VymlApnXEr4xJk9EhgMrsUzLfNsYs83T16lbty5jx47l999/Z8qUKdSvX9/Tl3CL7lyllAp0VWJPW2NMqRuyvvbwnCS79XKiIiOKKmIqpZSnlWVP2ypRD9/fyR50YZZSKvBViYQfCHRhllIq0FX6WjqBcqNUK2IqpQJdpU74gXSjVBdmKaUCXaVO+OWtYOmtbwW6MEspFcgqdcIvz43SQPpWoJRSvlSpb9qW50aps28FSilVlVXqhF+eCpY6fVIpFawq9ZBOeW6U6obiSqlgVakTPpT9RqlOn1RKBatKn/DLSqdPKqWCVdAlfNDpk0qp4FSpb9oqpZRynyZ8pZQKEprwlVIqSGjCV0qpIKEJXymlgoQmfKWUChIBtcWhiBwHDpTz5Q2BEx4Mx1M0rrLRuMpG4yqbqhhXK2NMI3caBlTCrwgR2eTuvo6+pHGVjcZVNhpX2QR7XDqko5RSQUITvlJKBYmqlPDn+zsABzSustG4ykbjKpugjqvKjOErpZRyrir18JVSSjlR6RK+iNwqIjtFZI+IPGPn+eoi8qH1+Y0i0jpA4hooIsdFJNn65wkfxPS2iBwTka0OnhcRmWONeYuIdPF2TG7G1VNE0ou9V8/7KK4WIrJGRHaIyDYRGWWnjc/fMzfj8vl7JiI1RORHEfnVGte/7LTx+e+jm3H5/Pex2LVDReQXEVlu5znvvl/GmErzBwgF9gIXAdWAX4GONm3igNetjwcAHwZIXAOBeT5+v64HugBbHTx/O7ACEOBqYGOAxNUTWO6Hf19NgS7Wx7WBXXb+P/r8PXMzLp+/Z9b3oJb1cTiwEbjapo0/fh/dicvnv4/Frv134D17/7+8/X5Vth5+N2CPMeY3Y0wO8AFwp02bO4FF1scfA31ERAIgLp8zxnwHnHTS5E5gsbH4AagrIk0DIC6/MMYcNcb8bH18BtgB2G6c4PP3zM24fM76HmRafwy3/rG9Kejz30c34/ILEWkO9AXedNDEq+9XZUv4zYBDxX4+TOl/+EVtjDF5QDrQIADiArjHOgzwsYi08HJM7nA3bn+4xvqVfIWIdPL1xa1fpa/A0jsszq/vmZO4wA/vmXV4Ihk4Bqwyxjh8v3z4++hOXOCf38fZwDigwMHzXn2/KlvCt/dJZ/vJ7U4bT3Pnmp8DrY0xlwGr+fNT3J/88V6542csy8UvB+YCn/ry4iJSC/gEGG2MybB92s5LfPKeuYjLL++ZMSbfGNMZaA50E5FLbZr45f1yIy6f/z6KSD/gmDFms7Nmdo557P2qbAn/MFD8k7g5cMRRGxEJAyLx/vCBy7iMMX8YY7KtPy4ArvRyTO5w5/30OWNMRuFXcmPMl0C4iDT0xbVFJBxLUn3XGLPEThO/vGeu4vLne2a95mlgLXCrzVP++H10GZeffh+7A3eIyH4sw769ReQ/Nm28+n5VtoT/E3CJiLQRkWpYbmoss2mzDPib9fG9QJKx3gHxZ1w247x3YBmH9bdlwCPWmSdXA+nGmKP+DkpEmhSOW4pINyz/Tv/wwXUFeAvYYYx52UEzn79n7sTlj/dMRBqJSF3r4wjgRiDVppnPfx/dicsfv4/GmAnGmObGmNZYckSSMeYhm2Zefb8q1Sbmxpg8ERkOrMQyM+ZtY8w2EZkIbDLGLMPyi/GOiOzB8sk4IEDiGikidwB51rgGejsuEXkfy+yNhiJyGHgByw0sjDGvA19imXWyBzgHPOrtmNyM615gqIjkAVnAAB98aIOlB/YwkGId/wV4FmhZLDZ/vGfuxOWP96wpsEhEQrF8wPw/Y8xyf/8+uhmXz38fHfHl+6UrbZVSKkhUtiEdpZRS5aQJXymlgoQmfKWUChKa8JVSKkhowldKqSChCV/5nYhk2vw8UETmWR8PEZFHXLy+qL2Ldv2sVQp/FZHtIjLYSdvW4qCaZ1mJyL9F5F4Hz/1dRFJFJMUa18vWRVaIyP5ix78WkSbFXneFiBgRucUTMargoAlfBTRjzOvGmMUVPY81ic4H+lvLD1yBZQWm34jIEOBmLJUcY4CuWGq/RBRr1ssa7yYsc+8LPQCss/5XKbdowlcBTUReFJGx1sddrcWuNojIDJse+IUi8pWI7BaR6XZOVRvLQsM/AIwx2caYndbzNhaRpdae9K8icq31NaEiskAsNdW/tq7aREQ6i8gP1liWikg9Z8edeA4Yal3+jzEmxxjzkp06OQDfAW2t1xEsC60GAjeLSA0X11EK0ISvAkOE/LkRRTIw0UG7hcAQY8w1QL7Nc52B+4EY4H6xqX5ojDmJZdn6ARF5X0T+KiKF//7nAN9ae9JdgG3W45cArxljOgGngXusxxcD462Ft1KwrBR2drwUEamNpWb7PkdtbPSznhMsK2/3GWP2YvmWcrub51BBThO+CgRZxpjOhX+AUrs1WWuj1DbG/Nd66D2bJt8YY9KNMeeB7UAr23MYY54A+gA/AmOBt61P9QYSrW3yjTHp1uP7jDGFpQw2A61FJBKoa4z51np8EXC9o+NO/s5CsSqIInKL9QNvf7FvGABrrB+CdYBp1mMPYCm+hfW/Oqyj3FKpaumooOZqE4jsYo/zcfBv2xiTgqUmzTvAPpzXULE9Z4SjhmVljMkQkbMi0sYYs88YsxJYKZZt76oVa9rLGHOi8AdrfZh7sFRdfA7L+9JARGpbN0dRyiHt4atKwRhzCjhjrVAJZSwqJSK1RKRnsUOdgQPWx98AQ63tQkWkjpM40oFTItLDeuhhLMNBdo+7CGsakFissqMArsbjbwR+Nca0MMa0Nsa0wlI2+S4Xr1NKe/iqUnkcWCAiZ7GMXac7b16CAONE5A0s1STP8mfvfhQwX0Qex9KTHwo4K3n8N+B1EakJ/MafFTMdHXckEagJbBSRbCATWA/84uQ1DwBLbY59Yo35HRfXU0FOq2WqSkNEahVu8iEizwBNjTGj/ByWUpWG9vBVZdJXRCZg+Xd7AD/WMFeqMtIevlJKBQm9aauUUkFCE75SSgUJTfhKKRUkNOErpVSQ0ISvlFJBQhO+UkoFif8PIjbZY18SlYcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Here is the data again\n",
    "plt.scatter(high_school_gpas, college_gpas, color='steelblue')\n",
    "#Now for our equation\n",
    "sim_x = np.linspace(0, 4, 100)\n",
    "sim_y = 1.0262 * sim_x - 0.0608\n",
    "plt.plot(sim_x, sim_y, color='black', linewidth=2)\n",
    "#Our labelled axes\n",
    "plt.ylabel('College GPA')\n",
    "plt.xlabel('High School GPA')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pretty darn good!\n",
    "\n",
    "But this isn't actually real data. I've downloaded some real data that was a part of an introductory statistics textbook. Let's load that csv with Pandas right now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:01:09.191673Z",
     "start_time": "2019-07-03T13:01:09.156785Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>GPA</th>\n",
       "      <th>HSGPA</th>\n",
       "      <th>SATV</th>\n",
       "      <th>SATM</th>\n",
       "      <th>Male</th>\n",
       "      <th>HU</th>\n",
       "      <th>SS</th>\n",
       "      <th>FirstGen</th>\n",
       "      <th>White</th>\n",
       "      <th>CollegeBound</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.06</td>\n",
       "      <td>3.83</td>\n",
       "      <td>680</td>\n",
       "      <td>770</td>\n",
       "      <td>1</td>\n",
       "      <td>3.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.15</td>\n",
       "      <td>4.00</td>\n",
       "      <td>740</td>\n",
       "      <td>720</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.41</td>\n",
       "      <td>3.70</td>\n",
       "      <td>640</td>\n",
       "      <td>570</td>\n",
       "      <td>0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.21</td>\n",
       "      <td>3.51</td>\n",
       "      <td>740</td>\n",
       "      <td>700</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3.48</td>\n",
       "      <td>3.83</td>\n",
       "      <td>610</td>\n",
       "      <td>610</td>\n",
       "      <td>0</td>\n",
       "      <td>30.5</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    GPA  HSGPA  SATV  SATM  Male    HU    SS  FirstGen  White  CollegeBound\n",
       "0  3.06   3.83   680   770     1   3.0   9.0         1      1             1\n",
       "1  4.15   4.00   740   720     0   9.0   3.0         0      1             1\n",
       "2  3.41   3.70   640   570     0  16.0  13.0         0      0             1\n",
       "3  3.21   3.51   740   700     0  22.0   0.0         0      1             1\n",
       "4  3.48   3.83   610   610     0  30.5   1.5         0      1             1"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('../Data/Day7-Structured-Data-Analysis/student_gpa_data.csv')\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "In this dataframe we have 10 columns:\n",
    "\n",
    "    GPA - 1st Year College GPA\n",
    "    HSGPA - High School GPA\n",
    "    SATV - SAT Verbal Score\n",
    "    SATM - SAT Math Score\n",
    "    Male - Male (1) or Female (0)\n",
    "    HU - Number of credit hours earned in humanities courses in high school.\n",
    "    SS - Number of credit hours earned in social science courses in high school.\n",
    "    FirstGen - First Generation College Student\n",
    "    White - White (1) or Other Ethnicity (0) \n",
    "    CollegeBound - 1 = attended a high school where >=50% students intended to go on to college, 0 = otherwise\n",
    "    \n",
    "But right now, let's just look at the relationship between High School GPA and College GPA like we did previously.\n",
    "\n",
    "Like always we should take a look at the raw data before we try to model it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:01:35.864168Z",
     "start_time": "2019-07-03T13:01:35.685133Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1110dd710>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXuUXFWd77+/6u40IYY0kjYJgTZIGDAgEOiLD5AlGYeHSONdogRU5C5n4CJcFHxcde5Fh1kzSxcMQS4qDx+XiCMgKCYoQ3QCI9Eh3k5MICQgGWECCWgAO4QQOl1Vv/tHnaqcOn1O1Xntc/ap+n7Wykr1qV2nfmfXOfu39++1RVVBCCGEAEApbwEIIYTYA5UCIYSQBlQKhBBCGlApEEIIaUClQAghpAGVAiGEkAZUCoQQQhpQKRBCCGlApUAIIaRBb94CRGXmzJk6b968vMUghJBCsWbNmhdVdbBdu8IphXnz5mF0dDRvMQghpFCIyH+GaUfzESGEkAZUCoQQQhpQKRBCCGlApUAIIaQBlQIhhJAGVAqEEEIaUCkQQojljO0ax5PbxjC2a9z4dxUuT4EQQrqJlRu24vrlj6Knp4RKpYorzjoapxw119j3caVACCGWMrZrHNcvfxTj5SpeGy9jvFzFkuWPGl0xUCkQQoil/HHHbvT0NA/TPT0l/HHHbmPfSaVACCGWMmvGVFQq1aZjlUoVs2ZMNfadVAqEEGIpA9P6ccVZR6O/t4R9+3vR31vCFWcdjYFp/ca+k45mQgixmFOOmouFh8zEH3fsxqwZU40qBIBKgRBCrGdgWr9xZVCH5iNCCCENqBQIIYQ0oFIghBDSgEqBEEJIAyoFQgghDYwrBRHpEZHfich9Pu/1i8idIrJZRFaLyDzT8hBCSNHotIJ4nwKwCcB+Pu99AsCfVXW+iCwG8DUA52YgEyGEFIKOKognIgcBOBPAtwOanA3gNuf13QD+UkTEpEyEEFIUOrEg3vUAPg+gGvD+XADPAoCqlgHsAHCAt5GIXCQioyIyun37dlOyEkI6nCzNMGmQR0E8Y+YjEXk/gD+p6hoReU9QM59jOumA6i0AbgGA4eHhSe8TQkg7sjbDpEGnFcQ7EcCIiDwD4A4Ai0Tkdk+b5wAcDAAi0gtgBoCXDcpECOlC8jDDpEFHFcRT1S8C+CIAOCuFz6rqRz3NlgH4OIB/B3AOgJWqypUAISRVGmaY8t5Zd90Mk1VNobh0fEE8EbkawKiqLgPwHQDfF5HNqK0QFmctDyGk88nDDJMmWRbEy0QpqOpDAB5yXl/lOv46gA9lIQMhpHupm2GWeHwKtq8S8oClswkhXUHWZpiiQqVACOkasjTDpMnYrvHO9SkQQoibLAe8IpJ1KC2VAiEkN4qYO5Al7lDaeuTUkuWPYuEhM40pUFZJJYTkQlFzB7Ikj4xmKgVCSC7kMeBFwYaSGHmE0tJ8RAjJBZtzB2wxa+URSkulQEiHUDSHra25A3nY8VvR8RnNhJD0sWVmGxUbcwdsLInRcRnNhBBz2DazjYptuQM2m7WygI5mQgqO7Q7bopFWZdI0HdWdth0nIcQg3T6zNUFSs1ZSc57bP7T26ReZvEYICY+tDtui42fWCuPMT2rOcyuUcrmCKoByRTMzDVIpEJIiUSKA0owWstFh22mEnf0ncVT7KRQvpp3eVAqEpEQUk4GJaCHbHLadRJTZfxJznp9C8VLk7TgJ6RqilGxgeQc7aeXMjeLMT+Ko9lMo3o3sTzv2ICavEWI7UUwGNsbBdzvtVm7tZv9eU2Bcc57XP9TkU3B4YN1z+MjJf0HzESE2E8VkwGghuwhjGmrlzA9SKHHNeW6FsnP3HvzDPb9DuVJuvE+fAiEFIEoEEKOF7CLsys1v9m8qcbCuUMZ2jWNPudL03p5yhQXxCEkD07WBopgMGC1kD1FWbt7ZfxamQK1qy7/ThkqBdAVBS/wgRRFXgUQxGZiKFsqzMJ5tRfnCyJNk5WbaFPjHHbvRP6UXr43vNR/1T+ml+YiQJAQt8XeNl3HLio2TFEVRi8sB7R2mJgftvPvNe21R5Fl4yExc9eHjAQjmz94vtmM4bVPgrBlTsdulEABg93iZ5iNCkuC3xC+VBDc98DgmPJmih87ar7DF5drZt00O2nkX5fNe28WnLsDNKzaGkidpv8Q1BYZR0K+8tgdeY5E6x7lSICQmfkv8ckXR21PCRGWvE6+np4Qnto0VNly0lX0bgNFBO88wWz+F9K0HHkdvQF6B10yYRr9ENQWGVURPbBvz/fwT28YwNDg99PdFgclrpOPxSya65LQFqHocdpVKFUccOFDYcNFW9m3TlVTTsq3HqQbqd229PaWm2P4gefKoMBslefGIAwd8zxF0PA24UiBdgd8Sf9/+3km24KHB6YUNF21n3zap7NKwrcc14/gppGpVcclpNRNSK3nyyBmJsqoaGpyOkeEhLBvd0jg2MjxkbJUAAKJqNrwpbYaHh3V0dDRvMUiHkHb0kQ0Eyf7ghq2TBu1Wg26cPojbb2O7xnHBDStrZhyH/t4Sll6+KNR5gq4tjDxR+yUpca51y/adeGLbGI44cCC2QhCRNao63LYdlQIh3UPYQTvrSKInt43hC7evbgq93Le/F1/96NtxeEhTSRJFPrZrHJtf2AF39JHJiUHWiggIrxRoPiKkiwjjEM0jkigNM06SvA/vRjanHnsQVqx7LvKgHVaR2Jy8SKVAupYim4hMkkckUd0ncd3yR1ESQVU1M1+OnxJcXrfhR1CKUVdXtpY6p1IgXUma5pFOUy5ZO1/r/ffa62VAFSoCZGjWDrOHQTulmHeeRppQKZCuI80HOO8sXhNkWbCv3n+lkmD3HidnxMkdSXNQbaW4/ZSgl3ZKsZPKoVMpkK4jrQe4k2aHXrKweTf1nw9pDartFLefEjzt2IPwgMen0EqOTiqHTqVAuo60HuBOmh36Ydrm3c5s02oTm7CEVdx+SvAjJ/9F6O80tbrKwzRJpUA6CtNVMd3EUS6d5n9IQpDZZuqUHlSrex3NKzdsxZJl6yGlErRaxZUjx4Q20SVR3FGVYtqrq7xMk8aUgojsA+BXAPqd77lbVb/saXMhgGsAbHUO3aiq3zYlE+lsojxEUR9gv8E8qnLpRP9DEvz67+JTF2D+nBmNft6yfSeuuXcdqoqGr+Gae9eFNtGFVdzu36ZcqWLxSfNx5nFDkQf2qIqkVfJkXqZJkyuFcQCLVPVVEekDsEpE7lfVRzzt7lTVywzKQbqAOA9R2Ae41WAeVrl0sv/BTdSVUKv+W7lhK/7pp+vh3VOmosDmF17B8KGDbb87jOL2+22WPvR73PHwU5FWJVFpdV/5mtZEfK87bYwpBa2lSr/q/Nnn/CtW+jQpDKbs+2H37233Hbb4H2zcT8Gv/+r9Xg7cZaz5eBLFHeTb2FNR/NOy9Th01n4YGpyeat+1u6/8VjjjExX83V2juNLwCtOoT0FEegCsATAfwDdUdbVPsw+KyMkAfg/gClV91uc8FwG4CACGhoYMSkyKiqnoj7QGcxuiU4q0n0IrJ3Rvj2D+7BmRvruV4m4VkjpRUXzy1lU4fWG8DOdW1+e3T0L9vmok8y1bjz2uaq97nIqqJleYRktnq2pFVY8FcBCAE0TkKE+T5QDmqerRAH4J4LaA89yiqsOqOjw4aHbpRIqJX3nsNKI/0hrMTckXlijlmuOQpAS1X7nsoIG6r6eEz44c09RvSctf13+bKb3+w+FEpYrlo1tS7bupfT3Y412ZlKuY2tfT+PuUo+biy+cOYx/XMcB8ae9Moo9UdUxEHgJwOoANruMvuZrdCuBrWchDOhMTsfVphhrmWe/GtPkqrvIMWr14+71cruC8dx+G9/k4f9NQ3PXf5udrt+AHv/p9q+RmAMn7bvdEBX09Utv5z6GvR7B7otLUbv7sGfAWLTW9wjQZfTQIYMJRCFMBvBeeQV9E5qjq886fIwA2mZKHdAdpxda77cdpDuZ51bsxbb6KozzbmX3C9rvb1FIPW42juAem9eP8dx+Gk46YjU/eugoTLbKck/bdrBlTURKB2zdSEpl0ziyzy+uYXCnMAXCb41coAbhLVe8TkasBjKrqMgCXi8gIgDKAlwFcaFAeQkLhN3tdeMjMvMVKRBaDS1TlGWb1ElaJKgCIQABUATz/59cwtms81vUNDU7HZ0aSZTi3I8rvkfUKk/spEOLCbwOU3h5BCUBvb0/h8wvS2KzFj7gb8iTZWKfVeQBgSm8pMFInjLzeNib6LstkRu6nQEgM/Gav9b1+91RqG8BEif6wKYPZVPRRklDUNFYvfpE8QHCkTlh53asUU33XbiXEMheE5EyYiplhnYw2ZTCbSp5Let40TCN+kTx1vL9VHHnzSjzM6/4xGpJKSNHwho5O6S2hR5rbhHEymg4BjUrSsE0/xnaN47eb/4RSqbmDop53YFo/Dj9wIHEkjx/e3ypOP5jou3bkef9wpUCIB+/s9XdPv9i2TIJ3pmtLBnOdtKOPfPdBSOG8cfCL5AFqPgXvbxWnH/JIPOzIMheEFBm3rbddfR6/Jb4NGcxu0ow+CtoHwVvdNCui5DTE6Yc8wkLzLHPB6CNCYtIueubBDVsnDSR5Ry2l4bh8ctsYvnD7arw2Xm4cmzqlB588/UicMP9NuTnUo1xb3GipLJ2+D27YOqnMBRAvQgtg9BEhsYjy4LczEUWpoJrmYNPqfGHj/qNuX1mtaq4KIQ3a/Q5REg/T+E1POWoupk/tw9//aC1ed2U6mzZDUikQ4hA12iOMiajdQJJ2hEka54uzfWUSc0oaA2iU6/Zrq0CifnNfw9qnX0ztN82jzAXNR4QgfiJVEhNRGslb7sEIQCrnC3uOVoN52IE+DSUWVWZv2yk9Aog0hbVG6TfvBj2VShVui09cc0+dtMyQNB8REoG40UJJ4uyTRCiN7RrHz9ZswR2/3oxeZ7BYfNL8xBFPUWQKWgUFlQnx9lFa8f9RZPZrK6USvAGtUX4H7zV4SWruybrMBZUCIUgWdhi3yF2SyqJLXA7I+gz3hw8/BUjz8BbV1JA0aspvkLx22XrfMiF57FXh11arVWjMfmu170PUc7Uiy0KKTF4jBPnsdxDnO+uDrjciBagNuotPmp/oGpL2w+YXdkA8A2y5othT0UlJWHnsVeHX9sqRY3BlzGv2u4aS5/pPO/YgDEzr9903wkboUyDERR61ZqJ8p184aJ267RpA4muI0w/eFUwQ+/b34qsffTsOP3Ag1bDdpCGpcX979zUE+RQu/qsFuPkXG1NxZMf9TelTsAibiqJ1A0n6O8tlepzvDKrN5M3eTXoNUfshaAXT11NCtdo8SLpXA3ntVeFtm+SecV/Dzt0T+Id71jYp7VJJ8K0VG2v7M8TwnWRdA4lKwTA2FUXrBtLub9sUepTs3Szxs6339ZTwubOPQVW1ZfhqHorYTRr3TP0axnaNT1La5UoVvT0luDdVS+LINl2Mj0rBIHlVV+xW0u5vWxV6ntt6BjFrxlTsKTfXQJqoVHHtsvW48qyjsfTyRVbJWyfpPeOdNPjlcFx86gLcvGJj0+eSOLKZvFZgbCuK1umk2d+2K/S8Z9d+aHWyL6G+p8HSyxfh8AMHcpCqNUnumaBJg5/S3re/N9Z2oXnU0GL0kUFsK4rWSfhFcqTZ33mUSy4i9d9h8ws70D/Ff45pc7/FvWfalbYemNZcDty9Xag3bLgVeUTFcaVgkDyqK3YDQTO0NPs7S4Ue5LewzZ/hpSmTt1xBUKS+TROhMOaeMPdMlBVGwwnvapv1RkRRoFIwjI323yLTzqyTZn8vPmk+fvjwU01JV2n/fkEKzlZ/Rh2/36FHalFQqoqJimJKby1T2JaJUBRzTzuiTBrSMGtmaS6kUsgAG+2/RSXMA5a0v92DB0RwzjvfgjMNRPcEKbhDZ+1npT/DPcv2+x36p/Tibz94HKZP7cPUvh7snqhYMxFqN5mIes9EWWEUzYxMpUAKhekHzG/wuHPVZpx53FAq53cTpOCe2DZmXYBCPTGt7ij976cd6fs7zJ+93yQT2JPbxkIliZk0l6UxW/fKF3aFUTQzMpUCKRSmH7AsI8aCFNwRBw60VXxZ+hvGdo3j2nvX1RLQKrWw02/cvwGXnnEUbl6xMfB3iFKi2rS5LOlkopUfy0a/QBKoFIhRTAxeJh+wLJf6QQpuaHB6S8WXtb9h8wuvwFu5oqLArIF9A/MP/FZc1y1bv7dEdcbmsiSTibTCk4tiRqZSIMYwOXiZesDSXom0U4pBCi7oeFoDVDRlHVTLSAN/hyglqrMyl8WdTHRbvhGVAjGC7clfrUhrJZJUKfoNuGkMUFHlmj97Bnp7BGXXcqG3RzB/9ozAz0QpUR3GXJYWcSYTRXMUJ4XJa8QIRU/+8iYfhaXuWN2yfWfL5KY6KzdsxQU3rMQXbl+NC25YiQc3bG15/jT3O2gll5uBaf347MgxmNJbwj59PZjSW8JnR45p2Td+SVdBJarr5rKoJcSzKkNtKoFsy/adWLH+WWzZvjMlSdMh1kpBRPYBcJaq/ihleUiHYOPsyrRz1j0DnyhX4TW7eGf0cVZTSc1bWe4wF/SZKOYyP/LI4Ujbj3Xj/Y9h+eiWxt8jw0O49Iy3JRUzFUIrBRHpAXAqgPMAnAbgYQBUCsQX28LwTA8kYbZl9CrFPLYAzXqHOb/PBJ0nzPnzNEum5cfasn1nk0IAgGWjW3DW8DwMDU5PfP6ktFUKInIygPMBnAngtwBOBHCIqr5mWDZScGwJw8tiIPEb4OvZvX0BGdFZDdDeFZJNyjoqUctL5H3v+fHEtrHA49YrBRF5DsAWAN8C8DlV3SkiT1MhkLDYEIZnInHJi98ALwD+1znH4cltO3D8W2biyKEDmt7PItIpaIUURlnbOKiGVaQmVoZp9ccRAdVig45nTbuVwj0APgDgXAAVEfkpguPTCLESU4lLbvwG+CMP3h9X3bkGAPCDhzf72o1NRjotPGRm29IOSa45D8IoUhMrwzT7Y2hwOkaGh7DM41OwYZUAtFEKqvopEfk0gFNQ8yVcA2A/EfkwgJ+r6qsZyEhIIrJKXHIP8OVyBVfe9kjT+0F246SrqSAZr/rw8bFWSGkMqiZXGe0Uadp5BSaUzKVnvA1nDc/DE9vGcMSBA9YoBCCET0FVFcBKACtFpA/A6agpiG8CmGlWPELSIavEpfoAv2L9s77nM2E3DpIRkFgrpKSDqslVhlvZBG3ak3bkm6nktaHB6VYpgzpt8xREZKGInCMib1XVCVVdrqrnAzg4A/kISY04uQdxB5gs7cZBMs6fvV+s+Pokg2qcPIiwhM3pSDuvwMbwapO0czRfBeCjANYAuEZE/lFVbwUAVW2ZheTkMvwKQL/zPXer6pc9bfoBLAVwPICXAJyrqs/EuxTSqeTp8IxresrSbtxKxjgrpCTmNlOz6qgmnDQj34oesRWVduajcwEcq6qvicgBAP4FwK0hzz0OYJGqvuqYnVaJyP2q6ja0fgLAn1V1vogsBvA15ztJAchisPaWbL5y5JjMHZ5xB5gs7catZIzjs4h7zaZm1WGUjd+uamndl7aEV2dBO6Xwej38VFVfEpHQZTEcX0TdEd3n/PNGLp0N4CvO67sB3Cgi4nyWWEwW0Sl+JZuvuXddLvWT4g4wWdqN0w7/jZusZmJW3U7ZZHE/2hBenQXtlMKhIrLMeS2uvwW1cX+k1YedLOg1AOYD+IaqrvY0mQvgWdROVhaRHQAOAPBitMsgWZJVVmlQyebNL7yC4UMHU/seN1u277QyIiQNsjLDmZhVt1I2RS6+aCPtlMLZnr+vxd7ZvrcK7iRUtQLgWBEZAPATETlKVTe4mvidY9IqQUQuAnARAAwNpb8DFolGdqWEg0s2m8DmejRJyTrvwMSsOkjZdFtpa9O0MwcNADhKVf9NVf8NtTyF2wD8XwBvCvslqjoG4CHUwlndPAcniklEegHMAPCyz+dvUdVhVR0eHDQzQyThiWI3TlLNsl6y2U27ks1xCapHk0UFS9MVP01GBCWVK+p1+0WQdVt0kGnarRQ+D2Cx6+8pAIYBTAPwPbQoiCcigwAmVHVMRKYCeC9qjmQ3ywB8HMC/AzgHwEr6E+wnrN04jf0EPjtyDK5b/ihKIqiq4kpDUR951aPJYgZv40w6zevutugg07RTClNU1Z2Fs0pVXwLwkohMa/PZOQBuc/wKJQB3qep9InI1gFFVXQbgOwC+LyKbUVshLA4+HbGJdnbjtOy8WUV95FGPJitbeBp7MKTZ/yauu5uig0zTTins7/5DVS9z/dnSjqOqjwJY6HP8Ktfr1wF8qL2YxEZa2Y3TnJ1mEfWRRz2arGbwSWbSJlYypq67W6KDTNNOKawWkb+pJ6zVEZGLUSujTbqQMDPHItp508wrCOoj9/Es+yjOTNrUSiaPe8PEaifp+WysQgu0VwpXALhXRM4HsNY5djxqWcofMCkYsZOwM8ei2nnTyCsISrjz67ss+yjqTNrkjD7L6w57z4YdpNNYPdlahRYAJIxfV0QWATjS+fNxVV1pVKoWDA8P6+joaF5f39WM7RrHBTesrM0cHfp7S1h6+aJC1eQPQ1y5x3aN4/wlv2zKr+gR4KaLT8Zl317l23cArOyjOL932PP+ccduTO3rwe6JitHrDnsNURRH0j4x1a/tEJE1qjrcrl2o7TgdJZCbIiD54R4c48wci2jnTTKLC0q4W/v0i4F9F7VIX1aYmNH79e2sGVPx5LYxI8ohbHmMsGayNFZPNkaDuQm9RzPpPrwP8MWnLiicnyAqye3o/ivv/af1F7Lv0ozq8evba5etRwlAr2vL0jTNKGH8F1EG6TT8Ibb720LXMiLdhV/C080rNuLiUxekVpLYRhoDhIv6ABGGoIS7Y+YdkGo55yyJU3LcD7++LVcUeypqLKkuTBntKIN0GmW56+eY0lvCPn09mGLZvcCVAvElaPY0f84MLL18kZU28DRIOotrlXDX7bH0fn3rJY4ZpZ3/p12/RzWTpbHHtQKAKlQEsCxfl0qB+NJqcCyinyAs9QHiOlf0UNRZXNplrDsF7+BbrlRRqVSbfDCtyqX49WeUaLhW/e7+zeoO8LFd47F9Za3kqq/C91S0Uf3XpgJ+VArEl6KGlKaBAoBIrRSwtK376Es3D/6t8A6+q554AT98+Kkmn0LYcilp51EMTOvH2qdfTBwq2k4uOppJYelGc0djFud6YG2axeVJWuHF3sEXIjjnnW/BmccNRSqXkvbgmpaSaScXHc2k0KTlZCwKSR3NnUrY/ZHD4A1i2FOu4s5Vm33btvo90h5c0/rt28mVhrPaJFwpEILmhCqbZ3F5kLaZJq0Q0LRNnGkpmTBy2bwKp1IgXY/XZn3asQfhgXXPdZ0vJYigQXzzC69g+tS+yINanBDQoAE2zcE1TSUTRi5b/U6hylzYBMtcdB55lsIIKjlw41+fZLwEQ1Hw66MeqSmG3pgO2Qc3bJ00+Lb6fJb3SFFLs7Qj1TIXhJgi78JgQbPg3RMVHG5wL4UiMSmUtFxBFcCecrXhkI9qToo6w89yVm3rDD4rqBRIbpjeZKZTS3wnIews2NvOPYjv3L0HX7mzebVeVY0c9WPr4NupK4WwUCmQ3DAZr93pJb7jEKZPxnaN42drtuCOX2+eZBqqD+Jbtu/EhKfq30RFMbWvJ8vLMULeK1cboFIguWFqlh51BWJzJEhahOmT+j4Qe5wBP8g0tHuigim9paZcjim9NZNbkclqe1TbYZ4CyQ1T8dpx4s3zzMcY2zWOJ7eNpVoIzku7PmkqveDB23ezZkyFN89bnONxyOL6w8AclRpcKZBcMTFLL5KfICtzRbs+8TPl+bUD0jW52WSusfm+ydLPQaVAcidth2NR/ARZmiva9UlQBdOgss5pKPM8zDWtdn2z9b7JWnFSKZCOxBY/QasZXtaF0U45ai4OnbUfntg2hiMOHGjai9ov7PS8dx+G9/nUI3J/JomcWV9/fXCtqmKiopjSW4IATYOsLfdNnTwUJ5UC6VhMhzy2W9K3m+Flba5oJ0/WA2KW1980uDoEOdJtCpXNo6IqHc3EF1ucf7bSrkCc38513l3FsiyMFkaeukxZOdyzvH4/J3Idm53Jefg5uFIgk7DJ+ReVLBxyW7bvxHXLHsVEJXhJH3aGl9Xs3NYa/lldf6td32xxJvuRh5+DSoE0kWesdtIBPQtltnLDVly3bP2k5C3vABu16FsnmWqiksX1uwdXP5+CLeYiP7I261EpkCbymlEmHdCzUGb17/AqBMBs2GYa2CZPHvhtuWmDMzkMWfo5qBRIE3nMKNMY0LNQZkGx/H095sI2vSRZTdkWWZMHNjmRbYVKgTSRx4wyjQE9C2Xm9x19PYJv/s1JTeGdbtIchNIwj9kwKLbKFSD5Q6VAJpF0Rhl1NpvGgJ6FMgv6jiCFkCZJVlNbtu/0zU3Ig7piU9RCQvt6BCWRVPw/3V7dNC2oFIgvcWeUcWazaQ3oWZhH8jLBxF1N3Xj/Y1g+uqXx98jwEC49422pyxdmQPbLFaj5ZzSx/6fIEXOAXQqNSoGkRpLZbFqDbVaRLFk/uHFWU1u272xSCACwbHQLzhqel+qKIeyA3Kq+UhL/T9Grm9qm0Ji8RlIjaZXJPCuV2k6cRK8nto1FOh6HsElxgLlcgSJXN43Sf1nBlQJJDRti4W1ahqdN1NXUEQHbiQYdj0MUs5bbTOjnU4j7e9lw38XFxqRCKgXSkiiDbN6x8DYtw00ppyimq6HB6RgZHsIyj08hTdNRuwG51baeaUUf5X3fJcFGhSaqkxNxUjmxyMEAlgKYDaAK4BZV/bqnzXsA/BTA086hH6vq1a3OOzw8rKOjo62akJSIO8jmMVsf2zWOC25Y2eTE7O8tYenlizIfHGxSToC56KP677z5+R24ecXGSdebdT8UdZX44IatkxSaiX4SkTWqOtyuncmVQhnAZ1R1rYhMB7BGRH6hqhs97R5W1fcblKOJot44WZPEeZeHI9aWZbiNTs+hwemph6J6B/yLT12A+XNmNJ6rPPrBhhyMONiWVGhMKajq8wCed17vFJFNAOYC8CqFzLBtBmfe4OhvAAAOXElEQVQztgyyYbFlGV60fouD34B/84qNTauybuiHNLFJoWUSfSQi8wAsBLDa5+13ish6EblfRI40JYONXn6bsWWQDUuWZZhbUbR+i0OYaJ9u6IdOxbijWUTeAOAeAJ9W1Vc8b68F8GZVfVVE3gfgXgCH+ZzjIgAXAcDQ0FAsOThziUYRnXdxluFpmxOL2G9RCTPgd0M/dCrGHM0AICJ9AO4D8ICqXhei/TMAhlX1xaA2cR3NNjkii0Qn+GCCrsGkObET+q0VYZ2jnd4PRSJ3R7OICIDvANgUpBBEZDaAP6qqisgJqJmzXjIhD2cu8bDJ1hmHoIHftCO06P3WjrCrsk7vh07EpPnoRAAfA/CYiKxzjn0JwBAAqOpNAM4BcImIlAHsBrBYDS5dbPPyE7O0GvhpTkwOB/zOxGT00SoA0qbNjQBuNCWDH7yRu4dWAz8doSRrimJKY0Yz6VhaDfxhzYlFeZCJ3RQpHJ5KgbSkyINiu4G/nTmxSA8ysRcbExpbQaVAAumEQbHdwB9kTizag0zsncAUzX9FpUB86aRBMY4fqWgPcrdj8wSmaP4r7qdAfClyjfo0KNqD3M3YXq3Almz7sHClQHwp2qDIzOTupQiruiKFw1MpEF+KNCiaMh0U6UHuZooygSlKODyVAgmkCIMiM5NJkSYwRYBKgbTE9kHRdtOBrRExnUYRJjBFgUqBFBqbTQc2R8R0IrZPYIoCo49IobE1siPtiJixXeN4ctuYFRE1NslC0ocrBVJ4bDQdpGnWsmnFYZMsxAxcKZCOYGBaPw4/cCA1hZB0NpyWWcumGHybZCHm4EqBEA9pzIbTioixyZFukyzEHFQKhLhIM8Q1DbOWTY50m2Qh5qD5qMOhUzAaaZf3SGrWSuJIT/u3t9WpHwU+D+3hSqGDabUVpQmnbCfE5Ns4G46z4mCW92ToJA8HlUKHEmQGee31Mm7+xcbUH4xOeeBszY6NEoPPLO/JdFLVX9NQKXQofk7BUknwrRUbMVFJ98HotAeuyLNhgA5hP9gn4aFPoUPxM4OUK1X09jRvm51GOew4dnjbbbtph7hmiY0msLxhn4SHSqFD8XMKXnLakahWtaldGg9G1Adu5YatuOCGlfjC7atxwQ0r8eCGrYm+nzTTCQ7htGGfhEdUtX0rixgeHtbR0dG8xSgMXufvgxu2TrKXp2H7D3vesV3juOCGlTVTk0N/bwlLL1/EBzRlOsHxnzbd3CciskZVh9u1o0+hQwi62b1OQVP28rDnpW03O4roEDYN+6Q9VAodQNTIH1MPRpjz0rZLiN3Qp1BwilaPhrZdQuyGK4WCU0RzTNFDPgnpZKgUCk5RzTG07RJiJzQfFRyaY0hW5JlbYnteSyfBlUIHQHMMMU2eZUw6pYRKUeBKoUMocgYusZs8gxmKFkjRCVApEEJaknY58aJ8d7dCpUAIaUmewQxFDaQoMlQKhJCW5BnMwECK7KGjOSe6uQYLKR55BjMwkCJbqBRygNEUpIjkmVvCvJbsMGY+EpGDReRBEdkkIo+LyKd82oiI3CAim0XkURE5zpQ8tsBoCkKIzZj0KZQBfEZV3wrgHQAuFZEFnjZnADjM+XcRgG8ZlMcKGE1BCLEZY0pBVZ9X1bXO650ANgHw2kjOBrBUazwCYEBE5piSyQYYTUEIsZlMoo9EZB6AhQBWe96aC+BZ19/PYbLi6CjSiKZgyj8hxBTGHc0i8gYA9wD4tKq+4n3b5yOTtoITkYtQMy9haGgodRmzJkk0BZ3UhBCTGF0piEgfagrhB6r6Y58mzwE42PX3QQC2eRup6i2qOqyqw4ODg2aEzZg4ZSnopCaEmMZk9JEA+A6ATap6XUCzZQAucKKQ3gFgh6o+b0qmokMnNSHENCbNRycC+BiAx0RknXPsSwCGAEBVbwLwcwDvA7AZwGsA/ptBeQoPndThYGIgIfExphRUdRX8fQbuNgrgUlMydBp1J/USj0+BA99e6HMhJBnMaC4YTPkPxu1zqW9PumT5o1h4yEz2EyEhoVIoIEz596eI+1UTYhuskko6BvpcCEkOlQLpGFhmmZDk0HxEOgr6XAhJBpUC6TjocyEkPjQfEUIIaUClQAghpAGVAiGEkAZUCoQQQhpQKRBCCGlApUAIIaSB1GrSFQcR2QngybzliMFMAC/mLUREiigzUEy5iygzUEy5iygzkFzuN6tq2w1pipin8KSqDuctRFREZLRochdRZqCYchdRZqCYchdRZiA7uWk+IoQQ0oBKgRBCSIMiKoVb8hYgJkWUu4gyA8WUu4gyA8WUu4gyAxnJXThHMyGEEHMUcaVACCHEENYoBRE5WEQeFJFNIvK4iHzKp81HRORR599vROQY13vPiMhjIrJOREYtkvk9IrLDkWudiFzleu90EXlSRDaLyBeykDmC3J9zybxBRCoi8kbnvTz6eh8R+a2IrHdk/jufNv0icqfTn6tFZJ7rvS86x58UkdOykDmC3FeKyEbnvv5XEXmz672K63dYZpHMF4rIdpdsf+167+Mi8pTz7+NZyBxB7iUumX8vImOu9zLva9d394jI70TkPp/3sr2vVdWKfwDmADjOeT0dwO8BLPC0eReA/Z3XZwBY7XrvGQAzLZT5PQDu8/lsD4D/APAWAFMArPd+Nk+5Pe3PArAy574WAG9wXvcBWA3gHZ42nwRwk/N6MYA7ndcLnP7tB3CI0+89Fsl9CoB9ndeX1OV2/n41y36OIPOFAG70+ewbAfzB+X9/5/X+tsjtaf8/AHw3z752ffeVAP45YKzI9L62ZqWgqs+r6lrn9U4AmwDM9bT5jar+2fnzEQAHZStlM2FkbsEJADar6h9UdQ+AOwCcbUbSZmLIfR6AH2YhWxBa41Xnzz7nn9chdjaA25zXdwP4SxER5/gdqjquqk8D2Ixa/xsnjNyq+qCqvub8acN9HaavgzgNwC9U9WXnWf0FgNMNiDmJGHLnfl8DgIgcBOBMAN8OaJLpfW2NUnDjLI8Woqbpg/gEgPtdfyuAFSKyRkQuMiedP21kfqezpL1fRI50js0F8KyrzXMIr1BSo11fi8i+qD3U97gO59LXzhJ7HYA/oTbweGVu9KmqlgHsAHAAcu7rEHK78d7X+4jIqIg8IiIfMCqoi5Ayf9Axed0tIgc7xwrR146J7hAAK12Hc+lrANcD+DyAasD7md7X1ikFEXkDagPQp1X1lYA2p6D28PxP1+ETVfU41MxKl4rIycaF3StPK5nXopZefgyA/wPg3vrHfE6VaShYmL5GzXT0a1V92XUsl75W1YqqHovaTPoEETnK0ySoT3Pt6xByAwBE5KMAhgFc4zo8pLUs1vMBXC8ihxoXGKFkXg5gnqoeDeCX2DuTLURfo2aGuVtVK65jmfe1iLwfwJ9UdU2rZj7HjN3XVikFEelDbZD6gar+OKDN0agts85W1Zfqx1V1m/P/nwD8BBmZB9rJrKqv1Je0qvpzAH0iMhM1rX6wq+lBALZlIDKAcH3tsBieJXZefe36/jEAD2GyWaLRpyLSC2AGgJeRc1/XaSE3ROS9AP4WwIiqjrs+U+/rPzifXZiFrK7v95VZVV9yyXkrgOOd19b3tUOr+zrLvj4RwIiIPIOaCXmRiNzuaZPtfZ3UKZHWP9S03lIA17doM4Sa3exdnuPTAEx3vf4NgNMtkXk29uaDnABgi/O5XtSccIdgr6P5SFv62mlXv/mmWdDXgwAGnNdTATwM4P2eNpei2SF3l/P6SDQ75P6A7BzNYeReiJqT8DDP8f0B9DuvZwJ4ChkEI4SUeY7r9X8F8Ijz+o0AnnZk3995/UZb+tp573DUgiUk7772yPUe+DuaM72vbSqIdyKAjwF4zLEJAsCXUFMEUNWbAFyFmi3tmzU/C8paW+7NAvAT51gvgH9W1X+xROZzAFwiImUAuwEs1tovWhaRywA8gFok0ndV9fEMZA4rN1B72Feo6i7XZ/Pq6zkAbhORHtRWuHep6n0icjWAUVVdBuA7AL4vIptRU2aLnet5XETuArARQBnApdpsNshb7msAvAHAj5x+3aKqIwDeCuBmEak6n/2qqm60RObLRWQEtf58GbVoJKjqyyLy9wD+n3Ouq7XZ9Ji33EDNwXyH8xzWyauvfcnzvmZGMyGEkAZW+RQIIYTkC5UCIYSQBlQKhBBCGlApEEIIaUClQAghpAGVAiEOIvKq5+8LReRG5/XhIvKQU0Fzk4jc4mp3gvPeUyKyVkR+JiJvc977iohslb3VZkc837FeRHKvv0NIHZvyFAixmRsALFHVnwKAa9CfBeAuAOer6m+cYycBOBTAY85nl6jqtSLyVgAPi8ibVLXq/F0CcLKITPPkgxCSC1wpEBKOOaiVFQAAqGp9wL8MwG11heC8t0pV7/V8Hqq6CbUko5nOofMBfB/ACgAj3vaE5AFXCoTsZaorwxuolWyoZ8EuAbBSRH6D2iD+Pa3V1zkSe4vBtURE3o5aJcztzqFzAfwVamUXLoMFZZwJ4UqBkL3sVtVj6/9QK6sCAFDV76FWCuFHqNWoeURE+r0ncHbG2iQiX3cdvsJRNtcCOFdVVUT+C4DtqvqfAP4VwHEisr+5SyMkHFQKhIREVbep6ndV9WzUzEBHAXgcwHGuNm8H8L9RKyZYZ4mjaN6tqg87x84DcIRTHfM/AOwH4IMZXAYhLaFSICQEUttPu895PRu1woxbAXwDwIUi8i5X833bnKsE4EMAjlbVeao6D7VdtM4zITshUaBPgZBwnArg6yLyuvP351T1BQAQkXMBfE1E5qK249eLAK5uca6TAWxV1a2uY78CsEBE5qjq8+mLT0g4WCWVEEJIA5qPCCGENKBSIIQQ0oBKgRBCSAMqBUIIIQ2oFAghhDSgUiCEENKASoEQQkgDKgVCCCEN/j9b74E6nbrpRAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.plot(kind='scatter', x = 'HSGPA', y = 'GPA', color='steelblue')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yikes! Looks like my initially generated data was a bit optimistic....\n",
    "\n",
    "Let's try fitting a linear model to this dataset and see what it looks like."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Now here's a cool thing, there are actually two ways to run regressions with statsmodels. The first one is by importing the vanilla `statsmodels` from `statsmodels.api`. The other way is by calling `statsmodels` formulas, which are from `statmodels.formula.api`. Why is there this crazy naming scheme? ...I can't really tell you actually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:01:47.689383Z",
     "start_time": "2019-07-03T13:01:47.685853Z"
    }
   },
   "outputs": [],
   "source": [
    "import statsmodels.formula.api as smf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The great thing about the formula is that it works directly with a Dataframe! We just have to learn a little bit of syntax. I'll show you how it works first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:02:24.786608Z",
     "start_time": "2019-07-03T13:02:24.764837Z"
    }
   },
   "outputs": [],
   "source": [
    "realdata_model = smf.ols(formula = 'GPA ~ HSGPA', data = df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can type out a linear formula and it will automatically add the constant that we previously had to add manually. When we write a formula it is set up as:\n",
    "\n",
    "    dependent variable = independent variable\n",
    "    \n",
    "but the equals sign is replaced by a `~`. All we need to do is tell it the name of the columns in the dataframe that we want it to use in the model.\n",
    "\n",
    "We also tell the model that the data is coming from the dataframe.\n",
    "\n",
    "Now we can fit the model just like before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:02:27.227811Z",
     "start_time": "2019-07-03T13:02:27.209776Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>           <td>GPA</td>       <th>  R-squared:         </th> <td>   0.200</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.196</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   54.15</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Wed, 03 Jul 2019</td> <th>  Prob (F-statistic):</th> <td>3.78e-12</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>10:02:27</td>     <th>  Log-Likelihood:    </th> <td> -118.38</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>   219</td>      <th>  AIC:               </th> <td>   240.8</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>   217</td>      <th>  BIC:               </th> <td>   247.5</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>    1.1799</td> <td>    0.262</td> <td>    4.504</td> <td> 0.000</td> <td>    0.664</td> <td>    1.696</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>HSGPA</th>     <td>    0.5550</td> <td>    0.075</td> <td>    7.359</td> <td> 0.000</td> <td>    0.406</td> <td>    0.704</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 5.915</td> <th>  Durbin-Watson:     </th> <td>   2.175</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.052</td> <th>  Jarque-Bera (JB):  </th> <td>   4.952</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.279</td> <th>  Prob(JB):          </th> <td>  0.0841</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 2.518</td> <th>  Cond. No.          </th> <td>    34.9</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                    GPA   R-squared:                       0.200\n",
       "Model:                            OLS   Adj. R-squared:                  0.196\n",
       "Method:                 Least Squares   F-statistic:                     54.15\n",
       "Date:                Wed, 03 Jul 2019   Prob (F-statistic):           3.78e-12\n",
       "Time:                        10:02:27   Log-Likelihood:                -118.38\n",
       "No. Observations:                 219   AIC:                             240.8\n",
       "Df Residuals:                     217   BIC:                             247.5\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "Intercept      1.1799      0.262      4.504      0.000       0.664       1.696\n",
       "HSGPA          0.5550      0.075      7.359      0.000       0.406       0.704\n",
       "==============================================================================\n",
       "Omnibus:                        5.915   Durbin-Watson:                   2.175\n",
       "Prob(Omnibus):                  0.052   Jarque-Bera (JB):                4.952\n",
       "Skew:                          -0.279   Prob(JB):                       0.0841\n",
       "Kurtosis:                       2.518   Cond. No.                         34.9\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "\"\"\""
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "realdata_result = realdata_model.fit()\n",
    "realdata_result.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### So what does this all mean?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "First, we might notice the difference in the slope between the simulated data model line and the real data model line. We can show that here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember that the slope of a line is defined as the rise over the run (or $\\frac{\\Delta y}{\\Delta x}$)?\n",
    "\n",
    "This plot shows that in the fake data for every point increase in High School GPA there was an **equal** point increase in College GPA.\n",
    "\n",
    "In the real data, the slope of the line is not as steep - meaning that the College GPA *rises* less than the High School GPA *runs*. So if a student had a High School GPA of 4.0, we would predict a College GPA of 3.4. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:02:40.945196Z",
     "start_time": "2019-07-03T13:02:40.929470Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>           <td>GPA</td>       <th>  R-squared:         </th> <td>   0.200</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.196</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   54.15</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Wed, 03 Jul 2019</td> <th>  Prob (F-statistic):</th> <td>3.78e-12</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>10:02:40</td>     <th>  Log-Likelihood:    </th> <td> -118.38</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>   219</td>      <th>  AIC:               </th> <td>   240.8</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>   217</td>      <th>  BIC:               </th> <td>   247.5</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>    1.1799</td> <td>    0.262</td> <td>    4.504</td> <td> 0.000</td> <td>    0.664</td> <td>    1.696</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>HSGPA</th>     <td>    0.5550</td> <td>    0.075</td> <td>    7.359</td> <td> 0.000</td> <td>    0.406</td> <td>    0.704</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 5.915</td> <th>  Durbin-Watson:     </th> <td>   2.175</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.052</td> <th>  Jarque-Bera (JB):  </th> <td>   4.952</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.279</td> <th>  Prob(JB):          </th> <td>  0.0841</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 2.518</td> <th>  Cond. No.          </th> <td>    34.9</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                    GPA   R-squared:                       0.200\n",
       "Model:                            OLS   Adj. R-squared:                  0.196\n",
       "Method:                 Least Squares   F-statistic:                     54.15\n",
       "Date:                Wed, 03 Jul 2019   Prob (F-statistic):           3.78e-12\n",
       "Time:                        10:02:40   Log-Likelihood:                -118.38\n",
       "No. Observations:                 219   AIC:                             240.8\n",
       "Df Residuals:                     217   BIC:                             247.5\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "Intercept      1.1799      0.262      4.504      0.000       0.664       1.696\n",
       "HSGPA          0.5550      0.075      7.359      0.000       0.406       0.704\n",
       "==============================================================================\n",
       "Omnibus:                        5.915   Durbin-Watson:                   2.175\n",
       "Prob(Omnibus):                  0.052   Jarque-Bera (JB):                4.952\n",
       "Skew:                          -0.279   Prob(JB):                       0.0841\n",
       "Kurtosis:                       2.518   Cond. No.                         34.9\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "\"\"\""
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "realdata_result.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we should notice that this model doesn't explain nearly as much of the real data as the synthetic data. We can tell this by the R-squared value. \n",
    "\n",
    "With real data, high school GPA only predicts 20% of the variance in college GPA.\n",
    "\n",
    "This becomes a bit more intuitive when we plot the regression line against the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:04:08.544964Z",
     "start_time": "2019-07-03T13:04:08.525440Z"
    }
   },
   "outputs": [],
   "source": [
    "#We need to generate our predicted line\n",
    "\n",
    "real_x = np.arange(2.20, 4.00, 0.1)\n",
    "real_y = []\n",
    "for x in real_x:\n",
    "    real_y.append( realdata_result.params['HSGPA'] * x + realdata_result.params['Intercept'] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:04:15.480176Z",
     "start_time": "2019-07-03T13:04:15.300106Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'High School GPA')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztvXucFOWV+P09PTcGBEYFQVQEBUExAjp4CYqggDCjuNlkV5PdGPeT/Znbrsm6ySa7ed9kf9nP+2425hezrhsv2WzemGQ1iTEJynAVUFBBB7mKgsQLKqDcBmTAYabnvH90d01PT1+qu6uqq3vO9/PhQ3dVddWpZ6qe85zLcx5RVQzDMAwDIFJqAQzDMIzwYErBMAzDcDClYBiGYTiYUjAMwzAcTCkYhmEYDqYUDMMwDAdTCoZhGIaDKQXDMAzDwZSCYRiG4VBdagHyZdiwYTpmzJhSi2EYhlFWbNiw4YCqDs91XNkphTFjxtDa2lpqMQzDMMoKEXnLzXHmPjIMwzAcTCkYhmEYDqYUDMMwDAdTCoZhGIaDKQXDMAzDwZSCYRiG4WBKwTAMI+S0tXewY08bbe0dvl+r7OYpGIZh9CdWbnuXHz6xhaqqCNFoN3930yXMuvgs365nloJhGEZIaWvv4IdPbKGjq5vjHV10dHVzzxNbfLUYTCkYhmGElPeOnKCqqnc3XVUV4b0jJ3y7pikFwzCMkDJiaD3RaHevbdFoNyOG1vt2TVMKhmEYIaVhUB1/d9Ml1FVHGFhXTV11hL+76RIaBtX5dk0LNBuGYYSYWRefxdSxw3jvyAlGDK33VSGAKQXDMIzQ0zCozndlkMDcR4ZhGIaDKQXDMAzDwZSCYRiG4WBKwTAMw3AwpWAYhmE4+K4URKRKRDaKyJNp9tWJyK9EZJeIrBeRMX7LYxiGUW5UWkG8LwOvAEPS7PsscFhVx4nIrcC/AbcEIJNhGEZZUFEF8UTkbKAZ+K8Mh9wM/Cz++THgehERP2UyDMMoFyqxIN4PgX8AujPsPwt4G0BVu4AjwOmpB4nIHSLSKiKt+/fv90tWwzD6GUG6ZQqhFAXxfHMficiNwPuqukFEZmY6LM027bNB9SHgIYDGxsY++w3DMPIlaLdMIVRaQbzpwAIReRN4FLhORH6Rcsw7wDkAIlINDAUO+SiTYRhGSdwyhVBRBfFU9R+BfwSIWwpfVdW/TDlsIfAZ4HngE8BKVTVLwDAMX3HcMl09o/CEWyaoGkNuqfiCeCLyHaBVVRcCPwF+LiK7iFkItwYtj2EY/Y9SuGWKIciCeIEoBVVdDayOf/5W0vYPgT8LQgbDMIwECbfMPSkxhbBZCaXASmcbhtEvCdotUy6YUjAMo98SpFumGNraOyo3pmAYhpGLIDvBsBN06qwpBcMwQkU5zB8IiuTU2USm1D1PbGHq2GG+KUurkmoYRmgol/kDQVGKGc2mFAzDCA2l6ATdUKpyGKVInTX3kWEYoSGM8wdK6c4qReqsKQXD6EeEPYAbtvkDpfDpp1LxM5oNwygN5RLADdP8gbCUw6i4Gc2GYZSWMIx48yEs8wfC6M7yGws0G0Y/IKwB3LDjRZVSL4LUlbYcp2EYJaY/jni9ohh3VqEuu+TYz0tvHLDJa4ZheEvYArjlRjp3Vq6gfaEuu2RF0tUVpRvoimpgbj9TCoZRQtxmA3mRNRSmAG6548YCKCRInU6RpOJ3oNuUgmGUCLeuBS+zhsISwC1n3FoAhbjs0imSVMp5OU7DMDLgtpyDlX0oDdkCu26D9oUEqdMpktSF7G+YcrZNXjOMSsOtayEsefL9iVyWWTYLINXNl6/LLjX20yumEGfppnf4ixkXmPvIMCoJt64FyxoKFjeuoUxB+0xZQvm67JIVyQcnTvL//HYjXdEuZ7/FFAyjAnGbDWRZQ8Hi1jJLtQAAbrt3pWeTAxOKpK29g5Nd0V77TnZFrSCeYZQKP2sFuXUtWNZQcORjmSVbADv2tPnm5tNuzfrda0wpGEYGMvmW0ymKQpWHW9eC11lDQRfGC0shvlxyFGqZ+eXme+/ICepqqzne0eM+qqutNveRYQRNJt9ye0cXDy3b3qvDUCiLQnMJcgVSve7AS1WIL/U+3MoxdewwvvXnlwHCuJFDXCttP9x8I4bWcyJJIQCc6Ogy95FhBE0633IkIjyw9GU6k2aX/mDhZhDhZJkUmssVSPW6Ay9VIb7U+/jc3It4cNn2nHIUc/+FuPlyKeCjx0+S6izS+HazFAwjQNK5A7qiSnVVhM5oT+BPIpE+eeRhThnNFkgFPO/AS5FSm04R3b/0ZaozzC1IdgEWe//5uPncKKBX97Sl/e2re9oYPXywq+vki01eM4w0pJt49IUbLqK7T9Cvm27tvS3MKaPZfN9+VFL1yteeT5XQdPdRXRXpleufTo4gK8m6nZQ4cVRD2t9n2u4FZikYRgbSuQMG1lX38RsDZZMymsv37XWw1Atfe74unXSKqLtb+cINMRdSJjmCnBPi1oIaPXwwCxpHs7B1t7NtQeNo36wEAFH1N73JaxobG7W1tbXUYhj9GC+zj0pFJnlXbXu3TweeqQPO554LbZ+29o6e/P84ddURHr7zuqznyXQfueTI5/6LId/72r3/A17d08bEUQ0FKwQR2aCqjTmPM6VgGEYybjrwoDKKduxp4xu/WN8rJXNgXTXf/csrmJDDhVKMItq17wjJ2Ud+KP2gFFACt0rB3EeGYfQiV7A0yIyiYlw6hc7tSC1XMXfK2Szb9I7rztutAgnrpERTCobhknJzEflFkBlFiZjED57YQkSEblVfYzbpFN4TCX++CwWYrwUVxlLmphQMwwVeuUsqQbEEEZBNbicFUEVFwGd3t5v1DDIpwFLNyfAaUwqGkQOvXvZSzez1Gr+L9GVcjjI+P8SLjjaTck6n8FLJpAArpcy5KQXDyIEXL3uljCIT+OUPD2I5ymzKOZ3Cu2HK2SxNiSmku3allDk3pWAYOfDiZa+UUWQyfvjD81mOshBXnBvlnE7h/cWMC3Jeyw8LqhTuRlMKRr/Hr8qZyeSrWCoh9lAI6dqpSmIKtDplQZt7Fm5GIhG0u5u7Fkx25YorVDm7VYBeWlClcjf6phREZADwDFAXv85jqvrtlGNuB+4G3o1vuk9V/8svmQwjFbcvXj4ve7oOPR/FUimxh0LI1E7JbX/0+Ek+9+AzdCtOnOHu329y5Ypzo5x7xTSi3dx69TiaLx3tS/0jyDwZslTuRj8thQ7gOlU9JiI1wFoRWayq61KO+5Wq/o2PchhGWvJ98dy87Nk6dDeKpdJiDwnysXwytVOiiuv/+cNmUteZiSrs2neUxvOHZ71mLuWcrv0fXr2TR9e85toayYdMz0taN5pIn3v0A9+UgsamSh+Lf62J/yuv6dNGReO1n9/t+r7Zzh2G2EMY1lNI106J9u3KuPJYz/ZClXOmmMbJqPJ/Fm7m/BFDGDKw1pP2yfa8pLNoOjqj/O9ft3KXz5ajrzEFEakCNgDjgP9U1fVpDvu4iMwAdgJ/p6pvpznPHcAdAKNHj/ZRYqM/4XW2iBcdeqkzWMK8nkK2IHR1lTBu5FDX18yknLOlpHZGlc8/tAZUqautLrp93jtyIu1aCe8dOcGEUQ2xSXsLN3MyqbrryXg1VT8tR19LZ6tqVFWnAGcDl4vIxSmHPAGMUdVLgBXAzzKc5yFVbVTVxuHD/TWdjP5DuvLYxWSLeNGhey1TPrgt55wPhZajTlcqO1OHXVMV4asLJjttVEwJ7ET711an7xqj3UpU8aR96muqYoszJXGyq5v6miogZtF8+5ZGBsS/53svhRJI9pGqtonIamAesC1p+8Gkw34M/FsQ8hhGAi+zRbxKSSxVTRw/XFeFKMpM1kpq+3Z1RfnkNeNpSgkCF6ucE+3f8tJufvnMzmzZsUW1z4nOKDVVElvJL05NlXCis2cRp3Ejh5JatNRvy9HP7KPhQGdcIdQDs0np9EXkTFXdG/+6AHjFL3kMIxNe5Nsn/PBTxw7j4TuvK7pDL0VNHD9cV/kqylyuHzcK06mXlJSymq9ybhhUx6euGc/VE0fyxR+vpTODS6mY9hkxtJ6ICMmxkIhIr/P5PXs8HX5aCmcCP4vHFSLAr1X1SRH5DtCqqguBO0VkAdAFHAJu91Eew/CFdOsBjztzaKnFyhu/OqB8LB831oobhakAIgjQDew9fJy29o6872X08MH8/YKeNjnZFUW7e8cUilH8bto7aMvR1lMwjCJIt1gKQH1tFd3dWpZzDLxY0CWZfBfjKWRRnVznAKitjqTN3HEjX/IxR4+fLFn7FIOtp2AYAZApI+bEyfyLt4VhFrPX2UeFlJIu1lpJl9UD6TN33MqXsE78mFiYzfKxMheGUWbkqqrpNhAZhlnMXk+cK/R8xbpL0mX1JEj+e+QrX9ATC0v1TPiakmoYlU5yCml9bVWf/W4CkX6kghZCMamcySTSSXftO1pUauiEUQ1FZfWkI/nvke/9etU+bijlM2GWgmEUSfLIdtfeIzy4bHtG10c6d0AYZjGDN9lHqXWDSjERL11WD8RiCsl/j3zvN8iJhRVZ5sIw+hMJv/CEUQ1Mnzgyresjkzug1LOYk++hGH9+OvdKdZVQC1RXVwWSTgl97yPTfIZ87zfI9NBSlrmw7CPDCIBcWTWrtr3bp7MpVdZSocHNHXva+MYv1nO8o8vZNrCumm9+fCqD62sDD6C7vY987zeo4O+qbe/2KXMB+WdjJfAl+0hEBgEfAz6pqs15SWQYFYjbDiKXi8htBVWvOqNs58o1DyCfpSyj0W7GjRxadhVei2mfdOcYOrAWkfRxjkzMuvgsBtfX8C+/eYkPk2Y5++1azKkURKQWaAI+RaxMxW+BB3yRxjDKiHyyQ9y4iLJ1Nl5mohRzrnyXsizUvVKsAnR7j+mOUyi4fRJy79zTxj2PLGf/zlb2vfoC0z4ygZbHH837PkpR5iKj+0hE5gCfBG4AVgG/Av5DVcf4Jo0LzH1khIFCJlkV6iIqdkJXcgcLFHwut3Jk69DddPbFKsB85Ew9rrZKQKRXSqvb9lmy4Y/8832PsH9nK3tffYETh/Y6+2oHDWXv3r2cNjj/ztwr16IX7qOlwBrgalV9I37Sf89bEsOoQArJGCo0/77Q7KS29g4WbdjNo8/ucpayvPXqcQVnOrmVI5PFk66zT20PL+YCuJUz3XESiZDq5MnWPrt376alpYU/PPEky5evINrZkzJaUz+Y08dfyvDxjZw96XL2f9BRkFIIusxFNqVwGXArsEJEXgceBfomYhtGP6TQjKFCCt0VWmn0nqQgZWLk+8ia1yDFt+3WHVFMllS6zv77CzcToXdm0qjTBgW2JkW647S7G83SPp2dnTz//PMsWrSIlpYWtm3b1uvYwSPPY9gF0xh+QSNDz74AicS6zOrqSNGFBYOKy2ScvKaqG1X166p6PvDPwFSgVkQWxxe9MYx+S5DrHuR7rUQHnJq1ArEO+NarxxUkdzH3vGvfkT6B1q6ocjKqvSZn1ddUBbYmRbrj7lowmbtStn3mqjP5w2OPcssttzB8+HCuvfZavve977Ft2zZOOeUUPvaxj/Hv9/2I2V9/mKu+eC/jZ3+ahtEXOgoB4IYpZzuWUOo6EWEjr5RUEYkAc4BbVfWvfJMqCxZTMMJEkLVp3F4rXWpogoR/HChY7nzvOdVqycTAumq++5dXsOdQuyc+9EJTUru7u1m99nke/8MTPP/0U7y0oXd/M2HCBJqbm2lqauKaa66htrYW6PH9RyLi1L5KUFcd4XNzLuLB5duLCmAX85wVHVMQkTOAfyK2lOZW4F9V9SixWMPSgqQyPCEMhdMqkULaNUiz3u21MtVjSp3RW0zJ53wntKUqhJqqCN3d3SRvTlgEE0Y1eOJDdytnw6A66DzBskV/oKWlhZaWxezf/76zv66ujlmzZtHU1ERTUxPnn39+2vMkfP8v7HqfHy15uZdiiESE+5dtj63LkGesJOgaSNliCg8TW1/5P4AbgXux9Q5KThgKp1UiXrVrGBS22xm9QZAumFtTFeFrN0+mWzVj+qrfylZVefnll53YwLPPPks02tOJ1zecwRkTpnHHp/+ML9/+CQYNGuTqvA2D6rh83Bnc19071tAV7aa6KkLSdAPXyQJBFuGD7EphpKp+M/55qYi85IsEhmtK8YD0B7xq1zAp7FIt6ZnKiKH1nOzq7UrpjHbz/YWbueumSzxZpc4t7e3trFy5Mm4NtLB7925nX1VVFdOvvoZDp4zn1HGXMWj4aESEdUcjfNHFHN/UwUDqfI3Pzb2IB5dt7/UbN7GSUtTFyna3IiKngpOhVZX8XVUP+SKRkZGwFE6rNLxo1zAq7CBdW9nQ7r6xhMTaBg/feR0TRjX4du0//vGPjjWwevVqOjp6ArwjRoxg/vz5NDc3M2fOHPa1a59YjJvnINNgIFUpD6yrznuJ0FLUxcqmFIYScx8lpwwkrAUFzvNLKCM9YSmcVs6kc+940a6msPvS1t7BC7vep7amqk/gFfxpn46ODtasWUNLSwuLFi1i586dzj4R4YorrqCpqYnm5mamTp1KJNKTgKnVHXk/B7kGA8n3lrxEaGraayaCLMKXIKNSKPXMZaMvpXhAKolMIzov2jUIhZ1OoYUhhpGORFuny8RJ4FX7vPPOOyxevJiWlhZWrFjBsWPHnH0NDQ3MmzePpqYmbrjhBs4444xev83l9sn1HLgdDDgB96Tj3FqSoZm8JiJVQL2qHot/vxKoje/eqKof+CqZkZaw+IrLjVwjOi/a9darx/HImtd8KRPtdY0eP+nV1knUVAmdUaW2OjZruND26erqYv369Y5baPPmzb32X3LJJY41cOWVV1Jdnb6bc+v2yYbbwUCxlmSQrsBs7qN/A94Hvhf//giwDRhAzI30dX9FMzIRFl9xOeHmpSy0XZM7F0T4xFXn0exhpk86hfaDhZt7avSEIIaRPOJO19b1tVV8cd4kJo5q4ERnNG/Fu3//fpYuXcqiRYtYunQphw8fdvYNGjSI2bNn09TUxPz58znnnHNcyevW7ZMNt9ZFObl+symF64FpSd/bVPUmiU1LXOOvWIbhLX69lOk6l1+t3UXzpaOLOm8yXtTo8ZPE5LREAPXzN0zq09bd3crl487o5fbasactoytsSH0NGzdu5PHfL2RRy2K2bGztVS10/PjxjjUwY8YM6uryu+diRu6pLic31kU5uX6zKYWIqiZPifw6gKqqiJzir1iG4S1+vZRBBJgLqdEDwcQb2to7+P7vN8UmocXz/P9z8Ta+NP/ijMuSZnKF3f2b5zn4+mb2vbKe9rc2c+hAzwSySFU1l17+UT59y5/S1NTE+PHji5K70EFCtrhU2GIDhZJNKdSKyOBE7EBVlwGIyFBiLiTD8B0vOzY/Xsog3AKZFBqQUckFNWdi176jpFaviCqMaBiYdg5CsmWlnVHa97/Nl772GO/vbOXQmy+j3T1B6QFDTmfYBdMYNr6R086bzKBBg/jMX+e/4lg6ChkkeJF2XA6u32xK4cfAr0Tk86q6G0BEzgXuj+8zDF/xo2Pz+qX00gLJpgAzKbR027zovNwr40z1jDRtW7+57yD7d77Iu9tfYP/OVj5se8/ZJ5EIDedOYvj4RkZddAWDR47pla3jtQWW7yChv6QdZ0tJ/YGIHAfWxpfhVKAd+K6q3h+UgEb/JIyTwTLhhQVSqAJM1/EW23nlI8u4kUOprhK6ksyF6iph3Mihzvc33njDmTewatUqPvzwQ2dfzcAhnHFBIyMmTmPo2KnU1Mc807XVEQhgxbF8BgnlFCwuhqzzt1X1AeCBeAxBLA3VCIpyG5UVaoG0tXewa9+RniqiGRSg10t/ZpMnH2XcMKiOry6YzA+e2EJEhG5V/nbeRDa+8JyTMvrKK6/0+s2ESZPREZM486IrGDzyfO66eYpzHbfusXRy++2r9yMutXv/B7y6p42JoxoYPXywh9IWTu6iHkBiroJhBEVYRmV+djaJjl5E+lQRTVaAhXTUhXZeha4od2Z9J4//4UnWPf0Un/jXp/jgg57x45AhQ7jhhhuclNERI0akbVe37rFM7RjEfA0v41L3Ld7KE6099ZcWNI7mS/M/4oWYReFKKRhG0IQhhc/PzibTBK8EyQowyKU/3SrjaDTKCy+84LiFNm7c2Gv/pEmTnDUHPvrRj1JTU9NrfzrLyu22ZErhZvQiLrV7/we9FALAwtbd3NQ4puQWgykFI7SUMoXP784mXUcPUFdTBaq9FKDfS3+6LfVw8OBBli5dSktLC0uWLOHgwYPOOerr67n++uudNQfOPfdct01RFPmUmQhTKuire9oybg+9UhCRgcDfA6NV9X+JyHhggqo+6bt0Rr+nVCl8Xk5uSke6jr46Itx42WimTxjBpNGnO9v9zHDKVuphX9tx9u9+jTULf8b/9blFrFu3ju7uHpnHjB3L1bPmcPNNN3LjvDkMGBB8probhem1xeeFgpmYoTJspu1B4sZS+CmxaqlXxb+/A/wGMKVgVCxeT25KJbWjP3Gyi65u5bfr3uC3697o41/2I8MpUeM/2Rq6+7F1vLs1ytMrl9PS0sKePXuc39fU1DBr1iyam5s5bdxlPLb1GMeqq3j4lW5OH3ewJHWXcilMry0+rxTM6OGDWdA4moUpMYVSWwngYo1mEWlV1UYR2aiqU+PbNqvq5EAkTMHWaDaCIrHmrtsOoK29g9vuXdkrTpBYEzlb1szmNw/w/z6+qc++H39+hmedRDrZaqqEqohwcO9uDuxsZf/OFzn81stotKeQwahRoxyX0OzZsxk8eHBB95kshx9unEznTbdedWIt6HzXcSjmvjMRZPZR0Ws0J3FSROpxyoHL+UBH9p8YRvkTxOSmhkF1GYPNXvqXk2WLdp7k8JtbOfzHDbz36oscP7S350CJcPmVV3LzTTfR1NTE5MmTkZRyGoW61vwI3Ccrg3SdvJdZbH6kSY8ePjgU1kEybpTCt4ElwDki8ktgOrZWs9FPCGJyUxD+5ZNH9/PH555g7ysvcOiNLXR39ozragYOYcSEyzjjgmn833/zaf7k6ouznquQ+/QjcO9GyXgZjwlLmrTf5FQKqro8vj7zlcRWYfuyqh7I9TsRGQA8A9TFr/OYqn475Zg64GHgMuAgcIuqvpnvTRhGOoLOOCm0A/LDv9zZ2clzzz3nrEe8bVvvheSHjhrHTTc288Xbb+GCSZM5cOyk63Yq5D69HmXno2S8ymILQ5p0ELjJPro0/jFhY46OF8V7K6WKaiodwHWqekxEaoiVy1isquuSjvkscFhVx4nIrcTWcLgl/9swyhG/J4Yll3O+a8HkQAKhhXZAX5r/EW5qHFOUf/m9995jyZIlLFq0iGXLlnHkyBFn3+DBg5kzZw6zZs/lkitmcMmE83rJdvqQgXldy6/FaNziRsmkPl9ePGPlUum0GNy4j34EXApsIWYpXBz/fHq8WN6ydD/SWAQ7MRO6Jv4vNap9M/DP8c+PAfeJiGiu6LdR9vg9MSy1nPPdv98UWN2kQjugfP3L3d3dtLa2OhPIUhMwLrzwQmfNgenTp1NbW5vhTIWRz316PcrOpWT8fL7KodJpMbhRCm8Cn1XVlwFE5CLga8C/AI8DaZVC/NgqYums44D/VNX1KYecBbwNoKpdInIEOB3I6Z4yyhe/J4ZlKue8a99RGs8fXvT5kwm6ds3hw4dZtmwZLS0tLF68mP379zv7BgwYwHXXXcf8+fNpbm5m7NixeZ3bb3ebl6PsbEqmnIophhE3SmFiQiEAqOp2EZmqqq+nZiWkoqpRYIqINAC/E5GLVTXZuZnuBH2sBBG5A7gDYPRo71a0MkqD/8XuMpdz9pIgateoKtu2bXOKyz333HNEoz1rDpx77rk0NzfT3NzMzJkzGTgwPzdQgqDqB3k5ys6kZMqtmGLYcKMUdojI/cCj8e+3ADvjQeJONxdR1TYRWQ3MI7bOc4J3gHOAd0SkGhgKHErz+4eAhyA2T8HNNY3wko9/uZDRq5tyzsXiZ+2aY8eOsXLlSidI/Pbbbzv7qqurufqaGdy8IJYyeuGFF/ZJGc2XsI2s8/mbp1My/SVLyC/cKIXbgS8CXyE2sl8LfJWYQpiV6UciMhzojCuEemA2sUByMguBzwDPA58AVlo8ofJx618uZo2B1HLOd3mcJeJ17Zpdu3Y51sDq1as5efKks2/EiBFMvWom++vPZ+SEy4jU1nPZvEu46CJvRvJhGll7YbH0lywhv3CTknpCRH4EPKmqO1J2ZyupfSbws3hcIQL8WlWfFJHvAK2quhD4CfBzEdlFzEK4taC7MMqOXP7lYkevfmeJFDu3oKOjg2eeecZRBK+99pqzT0S44oornCqjYy+4iNvvW83pXd0x07yr29ORfLHrL3jVxl5aLP0hS8gv3KSkLgDuBmqBsSIyBfiOqi7I9jtV3QJMTbP9W0mfPwT+LF+hjcogm3/Zi9Grn1kihcwteOeddxyX0IoVK2hvb++RtaGBefPm0dTUxLx58xg+vCcgvmNPm68j+UJH1l7HIby2WCo9S8gv3M5ovhxYDaCqm0RkjH8iGf2BXCPMcvAL55pb0NXVxbp16xxrYMuWLb32X3LJJTQ3N3PNrNmMnnAJZ50+OG2OfRBtke/I2o84RJB/c68snGLPE7aS3uBOKXSp6pFig1mGkSDo8gR+kjq3YP/+/SxZsoSWlhaWLl3K4cOHnX2DBg1i9uzZNDc3M3/+fM4+++yeSXYbWp1Jdgp92ieItshnZO1HHCKov7mb589NZ12spRTkinH54KZK6k+Ap4BvAB8H7gRqVPXz/ovXF6uSWt7kW2kyjCOpZLq7u9m4cSO//d0faGlZzJZNG0h+py644AKnyuiMGTOoq+u98Mun7lnRa05FlUBVpPfynIn2AULTFn5UDE38retrqjjRGfXlPt3I7VZpFHP/frRfLryskvq3wDeJla14BFhKbOKaYbgiuWPPd4QZRr/wkSNHWL58uTOBbN++fc6+SFU1l10xnU/f8qc0NTUxbty4jOfJNMmuSoTkORWJ9pkwqiE0beH1qD7dWg/J1/KKXM+fW7dYsZb/lWl7AAAdt0lEQVRSmDK+UnGTfXScmFL4pv/iGJVGupc97LGCVFSV7du3O0HitWvX0tXVU/ZrwNDhDBt/GcMumMZp501m0MCB3PZZNyO+9FZ68upmEN728SrDJ11HfG/LNuprq+juVk/dKrniFm4762LjH2GOmWVUCiLyBFmmgObKPjKMdC/7g8u2Oyt+hTlWcPz4cVatWuUEid966y1nX1VVFTNmzIhNHmu8mv9u/YATJ6NJ+92N+DJNsvviDZNC3z4JvLDkMq1XnWhTL9Nvc1k4bjvrYi2lxO+T59KE5e+czVL4fmBSGBVJplHXuDOH8vCd14XGP57g9ddfd4rLrVq1io6OnjUHhg8f7tQUmjNnDqeeeioQU3z/9cLKXudxO+LLNMlu1sVnMX3iyNC1j1+k64iTydetkisOlc3Cyaezd2spZZJHAVRREQjRnN2MSkFVnw5SEKPyyDbqCkOs4OTJk6xdu9axBl599dVe+6dNm+ZUGb3sssuIRCJ9zuGM+JLKdOcz4svUsYShfYIiuSOORKSX1QX5lUDJZ43sTO2b/DdJBL3b2jsKinllkidhRZ+MqlPJNyxF+7K5j7aS3n0kxCpjX+KbVEZFEMa00j179rB48WIWLVrE8uXLOXasZ1L+0KFDmTt3Ls3NzcybN48RI0a4Omd8ndrYi1FA6nZ/UgCZSO6It7x5kJ+t3kF1VcSJKbgpgTJ17DDP5k40DKrjpTcOFJUymi1oXa6B5hsDk8KoWEpdbiAajbJ+/XonSLxx48Ze+y+++GLHGrjqqquoqanJ6/zOiC/p5Q7LiC9IvEgdTnTEP396J9VVEbqiyhduuChtOmi6zvZbf97oWUfrxeS8bB1/WQaaVdWJrInICGBa/OsLqvq+34IZlUPQI+GDBw+ydOlSFi1axJIlSzh0qKfwbn19Pddff70zgezcc88t6lphHvEFhVeTsHp1xHEeXLad6RNHukoHBfWso/Xi75rLfRo2KzqBm9pHf06s9tFqYq6j/xCRr6nqYz7LZhiuUFU2bdrkWAPr1q3rldZ53nnnOWsOXHvttQwYMMCT67a1d/DBiU66QjriCwIvy10Umw46buRQzzpaL0byuTr+UlvRmXAzee2bwLSEdRAvib2C2PKZhlESPvjgA1asWMGiRYtYvHgxe/bscfbV1NQwa9Ysp8roBRdcUPSaA6mkjo6rq4Ta6qpQjfiCIF1HHokIL+x6n8vHnZFXO3iRDupVR+vVSD6XPGGMJ7kpc7FVVT+S9D0CbE7eFiRW5qJ/oqrs2LHDSRlds2YNnZ09azydeeYomptj5SRmz57N4MH+LY2ZrkRBbZXw7VsaGTdyaOhecj9J1xZAwRPPVm17t09HnOn3QZRACXuZlXzwsszFEhFZSqzEBcRWXltcjHCG4YYTJ06wevVqxy30+uuvO/sikQiTpjQSPWMSIydO45QRY/nUgsmBFBRLNzqurq5icH1t2Xcc+ZIpnbTQiWf5jPSDGGWHcSTvN27KXHxNRP4UuJpYTOEhVf2d75IZ/ZK33nrLmTewcuVKTpw44ew7/fTTmTdvHs3NzVwx/Vq+8svNzgj1ZFQ9y/qphLLexZDr/lP3JzryF3a9z72LttKZNEO7WzXvoHuYOuJKshTckm2ewjhghKo+q6qPA4/Ht88QkfNV9Y9BCWlULp2dnTz77LOOW2j79u299l966aVOyui0adOoqqoC/Ft4ppLKehdCtvtva+9g0YbdPPrsLqpT9jcMqmPiqIZeCgGgM6rU11SV4laKJqylrf0mm6XwQ+Cf0mw/Ht93ky8SGRXPvn37WLx4MS0tLSxbtoyjR486+wYPHszcuXOdFchGjRqV9hx+jNbzyaQJa+ZIMWS7/5feOMA9Czc7Jb1PpmmfE51RaqsjveZs1FZHONEZ7XuxkOPHIkLlQjalMCa+pGYvVLXVVl4z8iEajdLa2uq4hTZs2NBr/4UXXuhkCk2fPp3a2tqc5/RjtB72st5+uzIy3f+ufUd7SjKkkNw+I4bWk5rjJVCQoi6126Y/zz/JphSyJXNXhvPU8I3Dhw+zdOlSZ82BAwcOOPsGDBjAdddd5yw+M3bs2IKu4fVoPcyxgiBcGZnuHzRtFdPE/kT7eKWow+C2CduzEKSSzKYUXhSR/6WqP07eKCKfBTZk+I3RT1FVtm7d6lgDzz33XK8JZGPGjHGsgVmzZlFf783L5eVoPayxgqBcGZnuf9zIoWmrmNZWR/q0T7GKOmi3TVt7B7v2HQW0VzpxmJ6FoJVkNqXwFeB3IvIX9CiBRqAW+JhvEhllw7Fjx3jqqaeclNF33nnH2VddXc21117rBIknTpzo+QQyPyhlrCDTaDBIV8asi8/i/BFDeHVPGxNHNTjrTyd3kF1dUT55zXiaLh3tuVstyHtdue1dvv/7Tc7qd9VVwleT0prDEDcqRWwjW+2j94CPisgs4OL45kWqujLTb4zK57XXXnOsgaeffpqTJ086+0aOHOm4hObMmcOQIUNKKGnh+BEryGX+ZxsNBunKyCRHUB1kUPfa1t7BPQs391oOtSuq/CClwy11emwpYhtu5imsAlb5cnUj9Hz44Yc888wzjiLYtWuXs09EuPLKKx1rYMqUKb3WHCh1sDAs5DL/c40Gg3JluJEjiMliQdzre0dOIJGIs5ZBgohIqILJpYhtuJnRbPQz3n77bccltGLFCo4fP+7sO/XUU5k3b56TMjps2LC05whDsNAtfiqvtvYO7kmU1s5g/rsZDQYxUg9Lxk0Q9zpiaD3a3TdO0q0aisSCBKWIbZhSMOjq6uL55593rIGtW7f22j958mQnSHzFFVdQXZ39sSmFH7TQjt1v5bXopd298vahb0ebTyE4PzuDMGXc+H2vDYPquGvBZO5OiSncFYLEglSCjm2YUuinvP/++yxZsoRFixaxbNky2tranH2DBg1izpw5TnzgrLPy6ySDHnEW2rH7rbza2jt4dM1rfbZ3pXS0Ycl0CYscQZHobNNlH4WNIGMbphT6Cd3d3WzYsMFxC7344oskV8idMGGCowSuueYa6uoKfwCDHHEW07H7rbzeO3KC6uoqTka7em2/9epxvs6QLsYdFoaMmyBpGFRH4/nDSy1GqDClUMG0tbWxfPlyZ82B99/vWTCvrq6OmTNnOiuQjRs3zrPrBjniLKZj91t5pTt/bZXQfOnotMd7MRr0wh1WqoybhDKrr6niRGe0XyilMGJKoYJQVbZv3+7EBtauXUs0KbvinHPOcVYgmzVrFoMGDfJNlmJGnPmMdIvp2P1WXkG7Ywq1mnbv/6DPvISgSSgzJVZXqaZKiIgUFeOx7LfCMKVQ5rS3t7Nq1SpHEezevdvZV1VVxYwZM5wg8aRJkwKdQFbIiDPfkW6xHa/f7pIg3TGFWE33Ld7KE609z8yCxtF8ab6362e5KcWdujZzrNpq4eXQyyn7LUFYlJgphTLk9ddfd5TAqlWr6OjocPadccYZzJ8/n6amJubOnUtDQ0MJJc2PQke6xXa8QWS6BPGS52s17d7/QS+FALCwdTc3NY7xzGJw0zmnU2YJConxlGOF0zApMVMKZcDJkydZs2aNowh27NjRa/+0adMca+Cyyy7rNYGsnCgmPlDqmadhIF+r6dU9bRm3e6EU3HbO6ZRZgkJiPGGZb+GWsCkxUwoh5d1332Xx4sUsWrSIFStWcOzYMWff0KFDmTt3Ls3NzcybN48RI0aUUFLvKFWefFjMdi/Ix2qaOCq9FZlpe7647ZyTlVm6mEK+f5MwzbdwQ9iUmCmFkBCNRlm3bp2zAtnmzZt77b/44osda+Cqq66ipqamRJLmj9tOtxR58mEw271WSm6tptHDB7OgcTQLU2IKXrmOsnXOmZb09CL7qNzmW4RNiUlyrrqnJxY5B3gYGAl0E1vb+d9TjpkJ/AF4I77pcVX9TrbzNjY2amtrq/cCl4ADBw6wZMkSWlpaWLp0KYcOHXL2DRw4kOuvv95JGR09On0aY9gppNMNauTe1t7Bbfeu7BXgrKuO8PCd1wXWgYRBKXmdfZT899v4xoE+nbNCIPdcThbgqm3v9mknr9tERDaoamOu4/y0FLqAv1fVl0RkMLBBRJar6vaU49ao6o0+yhEIbh7A7u5uNm3a5FgD69ev7zWB7Pzzz3dSRmfMmMGAAdnWOQo/hfpKg4oPlNpsD4svefTwwb4Glh++8zrn3QB6FLHP91xOcaYwTRr0TSmo6l5gb/zzByLyCnAWkKoUyp5so72jR4+yfPlyZybxvn37nN/V1tb2WnNg/PjxpboFXyh1p5uLUpvtYW+ffMmk5B6+8zomxOMUO/a0VdQ9e0lYlFggMYX4ms5TgfVpdl8lIpuBPcBXVfXlIGTyitQXQVX5zk+e5Nkz2li5fClr1qyhq6unzMFZZ53lKIHrr7+eU045pYTS+0upO91clNr3HPb2yRc3Sq7S7rkS8V0piMgpwG+Br6jq0ZTdLwHnquoxEWkCfg/0GS6LyB3AHUDofOvvHTkB3Z3s3/kSB3a2sn9nKx+2vcfq+P5IJMLVV1/tKIKPfOQjZbECmReUutN1QyFmu1e+6nJon3xw0+FX2j1XIr4FmgFEpAZ4Eliqqj9wcfybQKOqHsh0TFgCzW+++SYtLS38fuETPPXUSrq7elYgqx04hI/dfCN/suAm5s6dy2mnnVZCSUtPOQX8EmSS2Y/AcDm2TybcBkwr6Z7LhZIHmiU2HP4J8EomhSAiI4H3VFVF5HIgAhz0S6Zi6OzsZO3atU6Q+JVXXum1f+hZ4xg58XKGX9DItz/3cWZPDpdFU0rC4it1S6aO36/AcLm1TzbcWl6VdM+Vhp/uo+nAp4GtIrIpvu2fgNEAqvoA8AngCyLSBZwAblU/TZc82bt3r7PmwPLlyzl6tMf7NWTIEObOneusQFY/5DQb+VQA2Tr+SgsM+4V1+OWNn9lHa4GsznNVvQ+4zy8Z8iUajfLiiy861sBLL73Ua/9FF13kxAamT5/eZwKZvQjlT7aO34KkRrGUg9us389oPnToEEuXLqWlpYUlS5Zw4EBPOGPAgAFcf/31NDU1MX/+fMaOHVtCSY0gyNbxuwmSlsNLb5SGMExUdEO/UwqqypYtWxxr4Pnnn6c7aQHvMWPGOBPIZs6cSX29jQKLpZw6ylwdfzafebm89EbwhGWiohv6jVJYvnw5v/nNb2hpaeHdd991tldXVzNz5kzHLTRhwoR+kzIaBOXYUeYKlqbzmZfTS99fCNNgpJziUf1GKTzyyCP89Kc/BWDkyJGOEpg9ezZDhgwpsXSVSTl3lPkGS8vppe8PhG0wUk7xqH6jFG677TbOO+88mpqamDJlStmuOVBO9KeOspxe+konjIORcpq012+UwsyZM5k5c2apxehXlENHabOTK4+wDkbCVPQuG/1GKRjBE/aO0msXQ7m89JVOmAcj5TCHw5SC4Sth7ShtdnLlEvbBSNgxpWD4Thg7yjC6GMKULVPuhHUwUg6YUjD6JWFzMYQtW6YSCONgpBywFByjX5JwMdRVRxhYV01ddaRkLoZkV9bxji46urq554kttLV3FHSuHXvaCvqtF5T6+kbxmKVg9FvC4mLwypVVamuj1Nc3vMEsBaNf0zCojgmjGopWCMWMkL1wZXlpbRRCqa9veIdZCoZRJMWOkL3Ilil14LzU1ze8w5SCYRSBV6mtxbqySh04L/X1De8w95HhCgsgpscZISeRGCHnSzGurEID5179XcMUuM8He677YpaCkZNsy1N6GaQtxzz9MI2Q87U2+vuMbguMp8eUgpGVTO6R4x928eDy7Z69UOX6goZt9qzb3Pz+PqM7jEXzwoIpBSMr6QKIkYhw/7LtdEa9eaHK/QUttxEyWGC4v99/NiymYGQlnXukK9pNdVXvhYgK9aND/n75MPqBvUptDYowub1KQX+//2yYUjCyki6A+IUbJtHdrb2OK+aFyucFXbntXW67dyXf+MV6brt3Jau2vdvnGCM35RoY9or+fv/ZEFXNfVSIaGxs1NbW1lKL0e9IDQKv2vZuHz96MTEAN+dra+/gtntXxtxMceqqIzx853X2MhdIOQb3vaQ/3b+IbFDVxlzHWUzB6EO6FyU1gOi1H93N+cwP7D3lEhj2i/5+/+kwpWD0Ip8sIK9fqFznMz+wYfiPxRQMh7DXrzE/sGH4j1kKhkM5uGfKMf3TMMoJUwqGQ7m4Z8wPbBj+Ye4jw8HcM0ahlGLuSBjnq1QCZikYvTD3jJEvpShRUq5lUcoBsxSMPpTb7FyjdJQiOSHsCRHljikFwzAKxsvS4WG+Zn/ClIJhGAVTiuSEckmIKFdMKRiGUTClSE6whAh/sUBzhdCfargY4aIUyQmWEOEfphQqAMvEMEpNKeaO2HwVf/DNfSQi54jIKhF5RUReFpEvpzlGROReEdklIltE5FK/5KlULBPDMAwv8TOm0AX8vapeCFwJfElELko5Zj4wPv7vDuB+H+WpSCwTwzAML/FNKajqXlV9Kf75A+AVINWncTPwsMZYBzSIyJl+yVSJWCaGYRheEkj2kYiMAaYC61N2nQW8nfT9HfoqDiMLxWRiWJkAwzBS8T3QLCKnAL8FvqKqR1N3p/lJn6XgROQOYu4lRo8e7bmM5U4hmRgWnDYMIx2+WgoiUkNMIfxSVR9Pc8g7wDlJ388G9qQepKoPqWqjqjYOHz7cH2HLnHxKU1hw2jCMTPiZfSTAT4BXVPUHGQ5bCNwWz0K6Ejiiqnv9ksmIYcFpwzAy4af7aDrwaWCriGyKb/snYDSAqj4AtABNwC7gOPBXPspjxLHgdHpsAqBh+KgUVHUt6WMGycco8CW/ZDDSkwhO35MSU+jPHaHFWAwjhs1o7qdYmYAekmMsiaVI73liC1PHDuvX7WL0T0wp9GOsTECMclib2jCCwqqkGv0ei7EYRg+mFIx+j5ViNowezH1kGFiMxTASmFIwjDgWYzEMcx8ZhmEYSZhSMAzDMBxMKRiGYRgOphQMwzAMB1MKhmEYhoMpBcMwDMNBYjXpygcR2Q+85eMlhgEHfDy/V5ic3mJyeku5yAnlI2uxcp6rqjkXpCk7peA3ItKqqo2lliMXJqe3mJzeUi5yQvnIGpSc5j4yDMMwHEwpGIZhGA6mFPryUKkFcInJ6S0mp7eUi5xQPrIGIqfFFAzDMAwHsxQMwzAMh36hFETkHBFZJSKviMjLIvLlNMf8hYhsif97TkQmJ+17U0S2isgmEWkNgawzReRIXJ5NIvKtpH3zRGSHiOwSkW+UWM6vJcm4TUSiInJafF8gbSoiA0TkBRHZHJfzf6c5pk5EfhVvs/UiMiZp3z/Gt+8QkRtKLOddIrI9/ow+JSLnJu2LJrX1whLLebuI7E+S56+T9n1GRF6L//tMieW8J0nGnSLSlrQvkPZMul6ViGwUkSfT7Av2+VTViv8HnAlcGv88GNgJXJRyzEeBU+Of5wPrk/a9CQwLkawzgSfT/LYK+CNwHlALbE79bZByphx/E7Ay6DYFBDgl/rkGWA9cmXLMF4EH4p9vBX4V/3xRvA3rgLHxtq0qoZyzgIHxz19IyBn/fiyg59ONnLcD96X57WnA6/H/T41/PrVUcqYc/7fAfwfdnknXuwv4nwzvdaDPZ7+wFFR1r6q+FP/8AfAKcFbKMc+p6uH413XA2cFK6ciRU9YsXA7sUtXXVfUk8Chwc0jk/CTwiB+yZENjHIt/rYn/Sw2k3Qz8LP75MeB6EZH49kdVtUNV3wB2EWvjksipqqtU9Xj8a0meUZftmYkbgOWqeij+ri0H5vkgZiFyluT5BBCRs4Fm4L8yHBLo89kvlEIycdNrKrGRQyY+CyxO+q7AMhHZICJ3+Cddb3LIelXcNF4sIpPi284C3k465h3cK5SCydWmIjKQ2Mv/26TNgbVp3DTfBLxPrFNKldNpN1XtAo4ApxNwe7qQM5nUZ3SAiLSKyDoR+RO/ZMxDzo/H3VyPicg58W2hbM+4G24ssDJpc2DtCfwQ+AegO8P+QJ/PfqUUROQUYh3TV1T1aIZjZhF74b6etHm6ql5KzK30JRGZUWJZXyI2ZX0y8B/A7xM/S3MqX9PL3LQpMdfRs6p6KGlbYG2qqlFVnUJsZH25iFycckimdgu0PV3ICYCI/CXQCNydtHm0xma7fgr4oYicX0I5nwDGqOolwAp6RrmhbE9iLpnHVDWatC2Q9hSRG4H3VXVDtsPSbPPt+ew3SkFEaoh1Xr9U1cczHHMJMRPuZlU9mNiuqnvi/78P/A6fXAhuZVXVownTWFVbgBoRGUZspHBO0qFnA3tKJWcSt5JimgfdpvFrtQGr6euycNpNRKqBocAhAm5PF3IiIrOBbwILVLUj6TeJ9nw9/tuppZJTVQ8myfZj4LL459C1Z5xsz6ff7TkdWCAibxJz914nIr9IOSbY57PYoEQ5/COmUR8GfpjlmNHEfHIfTdk+CBic9Pk5YF6JZR1JzxyTy4Hd8d9VEwvejaUn0DypVHLGj0s8wINK0abAcKAh/rkeWAPcmHLMl+gdyPt1/PMkegfyXse/QLMbOacSCyaOT9l+KlAX/zwMeA3/EgzcyHlm0uePAevin08D3ojLe2r882mlkjO+bwKxpAcpRXumyDKT9IHmQJ/PavoH04FPA1vjPkaAfyKmCFDVB4BvEfPT/SgWw6FLY+bjCOB38W3VwP+o6pISy/oJ4Asi0gWcAG7V2FPSJSJ/Aywllon036r6cgnlhFinsExV25N+G2Sbngn8TESqiFnGv1bVJ0XkO0Crqi4EfgL8XER2EVNgt8bv4WUR+TWwHegCvqS9XQxBy3k3cArwm3jb7VbVBcCFwIMi0h3/7XdVdXsJ5bxTRBYQa7NDxLKRUNVDIvIvwIvxc31He7sUg5YTYgHmR+PvT4Ig2zMtpXw+bUazYRiG4dBvYgqGYRhGbkwpGIZhGA6mFAzDMAwHUwqGYRiGgykFwzAMw8GUglE2iMixlO+3i8h98c+fF5HbcvzeOT7HcTfGK1ZullhV0s9lOXaMiGxzew85rvv/icgnMuy7S0RelVhl2c0i8oP45MHkirObRWSZiIxM+t1UEVHPKmgaFY8pBaMiUNUHVPXhYs8T72gfAm7SWBmRqcRmtJYMEfk8MJdYlc+PANOI1fOpTzpsVlzeVmLzRRJ8Elgb/98wcmJKwagIROSfReSr8c/T4sXYnheRu1NG8qNEZInE6vl/L82pBhObUHcQQGMVKHfEzztCRH4XH5FvFpGPxn9TJSI/lljd/mUiUh8/fkq8oNqW+O9OzbY9C98EvqCxcg2o6klV/a6mrzX1DDAufh0hNtHxdmCuiAzIcR3DMKVglBX10rPwySbgOxmO+ynweVW9Ckid4TkFuAX4CHBLUgVPIDbrFlgIvCUij0hs8aXEe3Iv8HR8RH4pkJgtPh74T1WdBLQBH49vfxj4usYKw20Fvp1jex9EZDCxdQHeyHRMCjfGzwmxWedvqOofiVk7TS7PYfRjTCkY5cQJVZ2S+EesNEkvRKSBWF2l5+Kb/iflkKdU9YiqfkisPMC5KftR1b8GrgdeAL4K/Hd813XA/fFjoqp6JL79DVVNlPrYAIwRkaHEau88Hd/+M2BGpu1Z7llIqnwpIjfEleKbSZYKwKq4ohwC/Gt82yeJFVkj/r+5kIyc9JfaR0b/IV054WQ6kj5HyfAOqOpWYnWdfk6scNvteZyzPtOB+aKqR0WkXUTGquobqroUWCqxZRtrkw6dpaoHEl/iNX8+TqwC5zeJtcvpIjJYY4siGUZazFIwKgqNrej1gYhcGd90az6/F5FTRGRm0qYpwFvxz08RWwYzsYDLkCxyHAEOi8g18U2fJuZ6Srs9h1j/Ctwft4ISsYJc8YHZwGZVPUdVx6jqucTKnPu9YIxR5pilYFQinwV+LCLtxHzpR7If3gsB/kFEHiRWgbadHivhy8BDIvJZYhbBF4C9Wc71GeABia089zrwVzm2Z+J+YCCwXkQ6gGPAs8DGLL/5JLF1KpL5bVzmn+e4ntGPsSqpRsUhIqdofBEiEfkGsfr+Xy6xWIZRFpilYFQizSLyj8Se77fIHg8wDCMJsxQMwzAMBws0G4ZhGA6mFAzDMAwHUwqGYRiGgykFwzAMw8GUgmEYhuFgSsEwDMNw+P8BSDd1LO+NLIQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Here is the data again\n",
    "ax = df.plot(kind='scatter', x = 'HSGPA', y = 'GPA', color='steelblue')\n",
    "#Now for our equation\n",
    "plt.plot(real_x, real_y, color='black', linewidth=2)\n",
    "#Our labelled axes\n",
    "plt.ylabel('College GPA')\n",
    "plt.xlabel('High School GPA')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We can tell by looking at the data, that there is much more 'spread' or *variability* in the vertical axis than the line accounts for. This makes sense given the definition of the [$R^2$ coefficient](https://en.wikipedia.org/wiki/Coefficient_of_determination). This measure compares the distance from each point to the fitted line against the total variability in the dependent data. We would have $R^2=1.0$ if all of the datapoints sat directly on the predicted line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This means that in the real world, High School GPA is not solely predictive of a student's College GPA.\n",
    "\n",
    "I wonder why?\n",
    "\n",
    "Maybe it's because some students may be bored with regular instruction in High School and don't pay attention or turn in assignments. Maybe if we used both the High School GPA and the SAT scores we would be able to predict the College GPA better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:06:29.944702Z",
     "start_time": "2019-07-03T13:06:29.906009Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>           <td>GPA</td>       <th>  R-squared:         </th> <td>   0.246</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.236</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   23.43</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Wed, 03 Jul 2019</td> <th>  Prob (F-statistic):</th> <td>3.67e-13</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>10:06:29</td>     <th>  Log-Likelihood:    </th> <td> -111.80</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>   219</td>      <th>  AIC:               </th> <td>   231.6</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>   215</td>      <th>  BIC:               </th> <td>   245.2</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>    0.5878</td> <td>    0.327</td> <td>    1.798</td> <td> 0.074</td> <td>   -0.057</td> <td>    1.232</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>HSGPA</th>     <td>    0.4962</td> <td>    0.075</td> <td>    6.589</td> <td> 0.000</td> <td>    0.348</td> <td>    0.645</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>SATV</th>      <td>    0.0012</td> <td>    0.000</td> <td>    2.946</td> <td> 0.004</td> <td>    0.000</td> <td>    0.002</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>SATM</th>      <td>    0.0001</td> <td>    0.000</td> <td>    0.341</td> <td> 0.733</td> <td>   -0.001</td> <td>    0.001</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 4.495</td> <th>  Durbin-Watson:     </th> <td>   2.163</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.106</td> <th>  Jarque-Bera (JB):  </th> <td>   3.477</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.188</td> <th>  Prob(JB):          </th> <td>   0.176</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 2.511</td> <th>  Cond. No.          </th> <td>1.06e+04</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.06e+04. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                    GPA   R-squared:                       0.246\n",
       "Model:                            OLS   Adj. R-squared:                  0.236\n",
       "Method:                 Least Squares   F-statistic:                     23.43\n",
       "Date:                Wed, 03 Jul 2019   Prob (F-statistic):           3.67e-13\n",
       "Time:                        10:06:29   Log-Likelihood:                -111.80\n",
       "No. Observations:                 219   AIC:                             231.6\n",
       "Df Residuals:                     215   BIC:                             245.2\n",
       "Df Model:                           3                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "Intercept      0.5878      0.327      1.798      0.074      -0.057       1.232\n",
       "HSGPA          0.4962      0.075      6.589      0.000       0.348       0.645\n",
       "SATV           0.0012      0.000      2.946      0.004       0.000       0.002\n",
       "SATM           0.0001      0.000      0.341      0.733      -0.001       0.001\n",
       "==============================================================================\n",
       "Omnibus:                        4.495   Durbin-Watson:                   2.163\n",
       "Prob(Omnibus):                  0.106   Jarque-Bera (JB):                3.477\n",
       "Skew:                          -0.188   Prob(JB):                        0.176\n",
       "Kurtosis:                       2.511   Cond. No.                     1.06e+04\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.06e+04. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_scores_model = smf.ols(formula='GPA ~ HSGPA + SATV + SATM', data=df)\n",
    "test_scores_result = test_scores_model.fit()\n",
    "test_scores_result.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interesting! Judging by the $R^2$, there was a minor increase in how well this new model fits the data. However, the results weren't quite what we expected!\n",
    "\n",
    "There is a *small* but *significant* positive relationship between the SAT Verbal score and a student's First Year College GPA. However, there is *no significant* relationship with the SAT Math score.\n",
    "\n",
    "Guess being good at math only goes so far, huh?\n",
    "\n",
    "Well, if our SAT test scores only have a very minor effect on our College GPA I wonder what being a first generation college student will do. It's possible that people who are the first in their family to go to college work harder than those of us who had parents go."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:07:24.570776Z",
     "start_time": "2019-07-03T13:07:24.527098Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>OLS Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>           <td>GPA</td>       <th>  R-squared:         </th> <td>   0.263</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.253</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   25.60</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Wed, 03 Jul 2019</td> <th>  Prob (F-statistic):</th> <td>3.35e-14</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>10:07:24</td>     <th>  Log-Likelihood:    </th> <td> -109.33</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>No. Observations:</th>      <td>   219</td>      <th>  AIC:               </th> <td>   226.7</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Residuals:</th>          <td>   215</td>      <th>  BIC:               </th> <td>   240.2</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Intercept</th> <td>    0.7160</td> <td>    0.295</td> <td>    2.427</td> <td> 0.016</td> <td>    0.135</td> <td>    1.297</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>HSGPA</th>     <td>    0.5185</td> <td>    0.075</td> <td>    6.918</td> <td> 0.000</td> <td>    0.371</td> <td>    0.666</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>SATV</th>      <td>    0.0010</td> <td>    0.000</td> <td>    2.911</td> <td> 0.004</td> <td>    0.000</td> <td>    0.002</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>FirstGen</th>  <td>   -0.1998</td> <td>    0.089</td> <td>   -2.241</td> <td> 0.026</td> <td>   -0.376</td> <td>   -0.024</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "  <th>Omnibus:</th>       <td> 6.028</td> <th>  Durbin-Watson:     </th> <td>   2.168</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Prob(Omnibus):</th> <td> 0.049</td> <th>  Jarque-Bera (JB):  </th> <td>   4.446</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Skew:</th>          <td>-0.224</td> <th>  Prob(JB):          </th> <td>   0.108</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Kurtosis:</th>      <td> 2.464</td> <th>  Cond. No.          </th> <td>6.74e+03</td>\n",
       "</tr>\n",
       "</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 6.74e+03. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                    GPA   R-squared:                       0.263\n",
       "Model:                            OLS   Adj. R-squared:                  0.253\n",
       "Method:                 Least Squares   F-statistic:                     25.60\n",
       "Date:                Wed, 03 Jul 2019   Prob (F-statistic):           3.35e-14\n",
       "Time:                        10:07:24   Log-Likelihood:                -109.33\n",
       "No. Observations:                 219   AIC:                             226.7\n",
       "Df Residuals:                     215   BIC:                             240.2\n",
       "Df Model:                           3                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "Intercept      0.7160      0.295      2.427      0.016       0.135       1.297\n",
       "HSGPA          0.5185      0.075      6.918      0.000       0.371       0.666\n",
       "SATV           0.0010      0.000      2.911      0.004       0.000       0.002\n",
       "FirstGen      -0.1998      0.089     -2.241      0.026      -0.376      -0.024\n",
       "==============================================================================\n",
       "Omnibus:                        6.028   Durbin-Watson:                   2.168\n",
       "Prob(Omnibus):                  0.049   Jarque-Bera (JB):                4.446\n",
       "Skew:                          -0.224   Prob(JB):                        0.108\n",
       "Kurtosis:                       2.464   Cond. No.                     6.74e+03\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 6.74e+03. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "firstgen_model = smf.ols(formula='GPA ~ HSGPA + SATV + FirstGen', data=df)\n",
    "firstgen_result = firstgen_model.fit()\n",
    "firstgen_result.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Guess I was wrong! Being a first generation college student has a *significant, negative* relationship with College GPA."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot this model. \n",
    "\n",
    "But wait, how do we plot it now that the x-axis has three components and isn't just the High School GPA?\n",
    "\n",
    "What we do is calculate the x position for each data point we have and then plot that x-value against the College GPA.\n",
    "\n",
    "Let's write that function now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#YOur code here\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we will `apply` it to the dataframe. I haven't covered how to do this yet, but I will now!\n",
    "\n",
    "In Pandas we can `apply` our own functions to a dataframe. We can apply them to a single column (in which case the input value to the function is a single variable) or to the entire dataframe (in which case the input value to the function is a `Series` and is either a row or column in the dataframe).\n",
    "\n",
    "Here I need to access multiple columns of data for each student so I will `apply` the `calculate_firstgen_x_value()` function to the entire dataframe. I also want to calculate each student's value so I need to apply this to the rows. To pick `rows` instead of columns we need to change the `axis` argument to `1`.\n",
    "\n",
    "Since I want to keep this value I'll set the output equal to a new column name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-07-03T13:07:55.900951Z",
     "start_time": "2019-07-03T13:07:55.897293Z"
    }
   },
   "outputs": [],
   "source": [
    "df['firstgen_model'] = df.apply(calculate_firstgen_x_value, axis = 1)\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot(kind='scatter', x='firstgen_model', y='GPA', color='steelblue')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that when we transform the x data values the spread in the y-axis is much smaller than when we first modeled the relationship between College GPA and the High School GPA."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, we could also use the built-in plotting from `statsmodels` too. It will only show a single independent variable at a time, but it's a really quick way to plot!\n",
    "\n",
    "The first plot we can use is with `plot_fit()`. The `plot_fit()` funciton shows the real data points and the predicted data point (with confidence intervals)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sm.graphics.plot_fit(firstgen_result, 'HSGPA');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The other plotting function is more of a diagnostic one for users to evaluate how good their model is. It's the `plot_regress_exog` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize = (8,8))\n",
    "sm.graphics.plot_regress_exog(firstgen_result, 'HSGPA', fig=fig);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Regression methods in Python\n",
    "\n",
    "Statsmodels has more built into than basic linear regression though. It also has:\n",
    "\n",
    "    * Generalized Linear Models (Regression with a defined distribution link - logit and probit are in this category)\n",
    "    * Generalized Estimating Equations (GLM with correlation)\n",
    "    * Linear Mixed Effects models)\n",
    "    * Hazard Regression\n",
    "    * Kernel Regression\n",
    "    * Generalized Method of Moments\n",
    "    * Time Series analysis (ARMA, ARIMA, VARMA, VAR)\n",
    "    \n",
    "However, if you're looking for some of the newer machine-learning based regression techniques such as:\n",
    "\n",
    "    * LASSO (Least Absolute Shrinkage and Selection Operator)\n",
    "    * Ridge Regression\n",
    "    * Random Forest Regression\n",
    "    * Support Vector Regression\n",
    "    \n",
    "Then you'll need to use `scikit-learn` instead. Scikit Learn also duplicates some of the functions that are contained within `statsmodels` (linear regression, logistic regression, ...., [full list here](http://scikit-learn.org/stable/modules/linear_model.html). One downside of using scikit-learn is that it isn't integrated with Pandas as well as statsmodels (for now, that will likely change in the future).\n",
    "\n",
    "As an example, we'll show a LASSO regression with scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import linear_model\n",
    "\n",
    "#First we instantiate the mdoel\n",
    "clf = linear_model.Lasso(alpha = 0.1)\n",
    "\n",
    "#Create the X and Y variable sets - must be numpy arrays!\n",
    "y = df.GPA.values\n",
    "x = df.loc[:, 'HSGPA':'CollegeBound'].values\n",
    "#Fit the model\n",
    "clf.fit(x, y)\n",
    "#Get the parameters\n",
    "for param_name, param_value in zip(df.columns[1:-1], clf.coef_):\n",
    "    print(param_name, param_value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}