class10_partyhubs_python3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Class Session 10 - Date Hubs and Party Hubs\n",
    "\n",
    "## Comparing the histograms of local clustering coefficients of date hubs and party hubs\n",
    "\n",
    "In this class, we will analyze the protein-protein interaction network for two classes of yeast proteins, \"date hubs\" and \"party hubs\" as defined by Han et al. in their 2004 study of protein-interaction networks and gene expression (Han et al., Nature, v430, p88, 2004). The authors of that study claimed that there is no difference in the local clustering density, between \"date hubs\" and \"party hubs\".  We will put this to the test. We for each of the \"date hub\" and \"party hub\" proteins, we will compute its local clustering coefficient (*C<sub>i</sub>*) in the protein-protein interaction network.  We will then histogram the *C<sub>i</sub>*) values for the two sets of hubs, so that we can compare the distributions of local clustering coefficients for \"date hubs\" and \"party hubs\". We will use a statistical test (Kolmogorov-Smirnov) to compare the two distributions of *C<sub>i</sub>* values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get started, we load the python packages that we will require:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import igraph\n",
    "import numpy\n",
    "import pandas\n",
    "import matplotlib.pyplot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `scipy` package doesn't import the `stats` module automatically. so we have to force its import using `from`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll load the file of hub types `shared/han_hub_data.txt` (which is a two-column TSV file in which the first column is the protein name and the second column contains the string `date` or `party` for each row; the first row of the file contains the column headers), using our old friend `pandas.read_csv`. This file has a header so pass `header=0` to `read_csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "hub_data = pandas.read_csv(\"shared/han_hub_data.txt\", sep=\"\\t\", header=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a peek at the structure of the `hub_data` data frame, using `head` and `shape`.  Here's what it should look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Protein</th>\n",
       "      <th>HubType</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ACT1</td>\n",
       "      <td>date</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ADA2</td>\n",
       "      <td>date</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AKR1</td>\n",
       "      <td>party</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>APC1</td>\n",
       "      <td>party</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>APC11</td>\n",
       "      <td>date</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Protein HubType\n",
       "0    ACT1    date\n",
       "1    ADA2    date\n",
       "2    AKR1   party\n",
       "3    APC1   party\n",
       "4   APC11    date"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hub_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(199, 2)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hub_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's load the file of yeat protein-protein interaction network edges `shared/han_network_edges.txt` (which is a two-column file, with first column is the first protein in the interacting pair, and the second column is the second protein in the interacting pair).This file has a header so pass `header=0` to `read_csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "edge_data = pandas.read_csv(\"shared/han_network_edges.txt\", sep=\"\\t\", header=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a peek at the data frame `edge_df`, using `head` and `shape`:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "make an undirected igraph `Graph` from the edgelist data; show summary data on the graph as a sanity-check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PROTEINA</th>\n",
       "      <th>PROTEINB</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAR2</td>\n",
       "      <td>PRP8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ABP1</td>\n",
       "      <td>SRV2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABP1</td>\n",
       "      <td>ACT1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ABP1</td>\n",
       "      <td>RVS167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ACT1</td>\n",
       "      <td>IQG1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  PROTEINA PROTEINB\n",
       "0     AAR2     PRP8\n",
       "1     ABP1     SRV2\n",
       "2     ABP1     ACT1\n",
       "3     ABP1   RVS167\n",
       "4     ACT1     IQG1"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "edge_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2523, 2)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "edge_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It will be convenient to let `igraph` compute the local clustering coefficients.  So, we'll want to make an undirected igraph `igraph.Graph` object from the edgelist data, using our old friend `igraph.Graph.TupleList`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "ppi_graph = igraph.Graph.TupleList(edge_data.values.tolist(), directed=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As always, we'll use `igraph.Graph.summary` to sanity check the `Graph` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'IGRAPH UN-- 1344 2523 -- \\n+ attr: name (v)'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ppi_graph.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generate a list of the names of the proteins in the order of the proteins' corresponding vertices in the igraph `Graph` object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph_vertices = ppi_graph.vs()[\"name\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['AAR2', 'PRP8', 'ABP1', 'SRV2', 'ACT1', 'RVS167', 'IQG1', 'AIP1', 'COF1']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_vertices[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a dataframe containing the protein names (as column \"Protein\") using `pandas.DataFrame` and `pandas.Series`. Set the column name to `Protein` using the `columns` attribute on the data frame. Define a new column `order` that will contain the list of IDs in `graph_vertices_df.index`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph_vertices_df = pandas.DataFrame(pandas.Series(graph_vertices))\n",
    "graph_vertices_df.columns = [\"Protein\"]\n",
    "graph_vertices_df[\"order\"]=graph_vertices_df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a peek at this data frame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Protein</th>\n",
       "      <th>order</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAR2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PRP8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABP1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>SRV2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ACT1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Protein  order\n",
       "0    AAR2      0\n",
       "1    PRP8      1\n",
       "2    ABP1      2\n",
       "3    SRV2      3\n",
       "4    ACT1      4"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_vertices_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use the `pandas.DataFrame.merge` method on the `graph_vertices_df` object to pull in the hub type (date or party) for vertices that are hubs, by passing `hub_data` to `merge`.  Don't forget to specify `how='outer'` and `on=\"Protein\"`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph_vertices_df_merged = graph_vertices_df.merge(hub_data, how='outer', on=\"Protein\")\n",
    "graph_vertices_df_merged = graph_vertices_df_merged.sort_values(\"order\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Having merged the hub type information into `graph_vertices_df`, let's take a peek at it using `head` and `shape`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Protein</th>\n",
       "      <th>order</th>\n",
       "      <th>HubType</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAR2</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PRP8</td>\n",
       "      <td>1</td>\n",
       "      <td>party</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ABP1</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>SRV2</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ACT1</td>\n",
       "      <td>4</td>\n",
       "      <td>date</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Protein  order HubType\n",
       "0    AAR2      0     NaN\n",
       "1    PRP8      1   party\n",
       "2    ABP1      2     NaN\n",
       "3    SRV2      3     NaN\n",
       "4    ACT1      4    date"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_vertices_df_merged.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NOTE: a `NaN` in the `HubType` column means that the protein is not a hub."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1344, 2)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_vertices_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's pull out the `HubType` column as a numpy array, using column indexing (`[\"HubType\"]`) and then `values.tolist()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "vertex_types_np = numpy.array(graph_vertices_df_merged[\"HubType\"].values.tolist())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a peek at this `numpy.array` that we have just created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['nan', 'party', 'nan', ..., 'nan', 'nan', 'nan'], dtype='<U32')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vertex_types_np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use `numpy.where` in order to find the index numbers of the proteins that are \"date hubs\" and that are \"party hubs\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_hub_inds = numpy.where(vertex_types_np == \"date\")\n",
    "party_hub_inds = numpy.where(vertex_types_np == \"party\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `igraph.Graph.transitivity_local_undirected` function in igraph to compute the local clustering coefficients for every vertex in the graph. Make a `numpy.array` from the resulting list of Ci values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "ci_values = ppi_graph.transitivity_local_undirected()\n",
    "ci_values_np = numpy.array(ci_values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a peek at the `ci_values_np` array that you have just created.  What are the `nan` values, and what do they signify?  Is this normal?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([       nan, 0.10714286, 0.66666667, ..., 0.        ,        nan,\n",
       "              nan])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ci_values_np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a `numpy.array` of the Ci values of the date hubs (`ci_values_date_hubs`) and the Ci values of the party hubs (`ci_values_party_hubs`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "ci_values_date_hubs = ci_values_np[date_hub_inds]\n",
    "ci_values_party_hubs = ci_values_np[party_hub_inds]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the histograms of the local clustering coefficients of the \"date hubs\" and the \"party hubs\". Use `matplotlib.pyplot.hist`. Use alpha blending so that the two overlapping histograms can be plotted together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAV+0lEQVR4nO3df5BV5Z3n8fc3SsIYUWegLV1BG11JYMhgoBWNxtWdMRrLwTGYkUQnI1GZgpgZs0mqYkxNLDWbsuIkVRozDq4/8sNEVyay6OA4TiBRR83QOKAgIYJxtbNuQog/YBEF/e4f90J1oBsudJ977fu8X1Vd3nPO0+d8Hxv49HPOc86JzESSVK53tLoASVJrGQSSVDiDQJIKZxBIUuEMAkkq3L6tLmBPjRo1Kjs7O1tdhiQNKUuXLv1NZnb0tW3IBUFnZyfd3d2tLkOShpSI+N/9bfPUkCQVziCQpMIZBJJUuCF3jUDalS1bttDT08PmzZtbXUpTDR8+nNGjRzNs2LBWl6IhyCBQW+np6WHEiBF0dnYSEa0upykyk/Xr19PT08PYsWNbXY6GIE8Nqa1s3ryZkSNHFhMCABHByJEjixsFafAYBGo7JYXANiX2WYPHIJCkwnmNQG3tGw/+fFD395nTxu3x91x55ZXsv//+fO5zn+tz+/z58xk3bhwTJkwYaHnSXikqCAb7H4U9sTf/gKgM8+fP56yzzjII1DKeGpIq8JWvfIVx48Zx0kknsXr1agBuvvlmjj32WCZNmsT06dPZtGkTjz76KAsWLODzn/88xxxzDGvXrmXt2rWcccYZTJkyhQ9+8IP87Gc/a3Fv1O4MAmmQLV26lDvvvJNly5axcOFClixZAsBHPvIRlixZwvLlyxk/fjy33HILH/jAB5g2bRpf+9rXWLZsGUcddRSzZs3ihhtuYOnSpVx33XXMmTOnxT1Suyvq1JDUDA8//DDnnHMO++23HwDTpk0DYMWKFXzpS1/i5ZdfZuPGjZx++uk7fe/GjRt59NFH+ehHP7p93euvv96cwlUsg0BqkgsvvJD58+czadIkbr/9dn784x/v1Oatt97ioIMOYtmyZc0vUMXy1JA0yE4++WTmz5/Pa6+9xoYNG7j33nsB2LBhA4ceeihbtmzhjjvu2N5+xIgRbNiwAYADDjiAsWPHcvfddwO1u4aXL1/e/E6oKI4I1NZaMVtr8uTJnHfeeUyaNImDDz6YY489FoCrr76aqVOn0tHRwdSpU7f/4z9jxgwuueQSrr/+eubNm8cdd9zB7Nmzueaaa9iyZQszZsxg0qRJTe+HyhGZ2eoa9khXV1fu7YtpnD7a/latWsX48eNbXUZLlNx37V5ELM3Mrr62eWpIkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEkFc77CNTeFn91cPd36uWDu78++FhqNVtlI4KIGBMRiyPi6YhYGRF/00ebUyLilYhYVv/626rqkYaCrVu3Mn/+fJ5++ulWl6KCVHlqaCvw2cycABwPfCoi+voV5+HMPKb+dVWF9UhN8dxzz/He976X888/n/Hjx3PuueeyadMmrrrqKo499lgmTpzIrFmz2HYz5ymnnMJll11GV1cX11577U6PpZ48efL2fT/zzDO/sywNhsqCIDNfzMwn6p83AKuAw6o6nvR2snr1aubMmcOqVas44IAD+Na3vsWll17KkiVLWLFiBa+99hr33Xff9vZvvPEG3d3dXHHFFTs9lvrAAw/c/hC62267jZkzZ7aqW2pTTblYHBGdwPuBn/ax+YSIWB4R90fEH/bz/bMiojsiutetW1dhpdLgGDNmDCeeeCIAF1xwAY888giLFy9m6tSpvO9972PRokWsXLlye/vzzjuv331dfPHF3Hbbbbz55pvcddddfPzjH6+8fpWl8iCIiP2BfwQuy8xXd9j8BHBEZk4CbgDm97WPzJybmV2Z2dXR0VFtwdIgiIidlufMmcO8efN46qmnuOSSS9i8efP27e9+97v73df06dO5//77ue+++5gyZQojR46srG6VqdIgiIhh1ELgjsz84Y7bM/PVzNxY/7wQGBYRo6qsSWqG559/nsceewyA73//+5x00kkAjBo1io0bNzJv3rx+v7f3Y6kBhg8fzumnn87s2bM9LaRKVDZ9NGq/Et0CrMrMr/fT5hDgV5mZEXEctWBaX1VNKlATpnv25T3veQ833ngjn/zkJ5kwYQKzZ8/mpZdeYuLEiRxyyCHbH03dlx0fS33UUUdx/vnnc8899/ChD32oib1QKaq8j+BE4C+ApyJi2+uWvggcDpCZNwHnArMjYivwGjAjh9pzsaU+7Lvvvnzve9/7nXXXXHMN11xzzU5td3xT2YknnrjT9NFHHnmEmTNnss8++wx6rVJlQZCZjwCxmzbfBL5ZVQ1SOzjnnHNYu3YtixYtanUpalPeWSwNss7OTlasWDFo+7vnnnsGbV9SX3zWkNpOiWcXS+yzBo9BoLYyfPhw1q9fX9Q/jJnJ+vXrGT58eKtL0RDlqSG1ldGjR9PT00NpNx4OHz6c0aNHt7oMDVEGgdrKsGHDGDt2bKvLkIYUTw1JUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCmcQSFLhDAJJKpxBIEmFMwgkqXA+dE6S9sA3Hvx5y479mdPGVbJfRwSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCmcQSFLhKguCiBgTEYsj4umIWBkRf9NHm4iI6yNiTUQ8GRGTq6pHktS3Kh86txX4bGY+EREjgKUR8WBmPt2rzYeBo+tfU4G/r/9XktQklY0IMvPFzHyi/nkDsAo4bIdmZwPfyZrHgYMi4tCqapIk7awp1wgiohN4P/DTHTYdBrzQa7mHncOCiJgVEd0R0b1u3bqqypSkIlUeBBGxP/CPwGWZ+ere7CMz52ZmV2Z2dXR0DG6BklS4SoMgIoZRC4E7MvOHfTT5JTCm1/Lo+jpJUpNUOWsogFuAVZn59X6aLQA+UZ89dDzwSma+WFVNkqSdVTlr6ETgL4CnImJZfd0XgcMBMvMmYCFwJrAG2ATMrLAeSRqw45+f28KjX1fJXisLgsx8BIjdtEngU1XVIEnaPe8slqTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCmcQSFLhDAJJKtxugyAiRjajEElSazQyIng8Iu6OiDPrL5uRJLWRRoJgHDCX2ktmnomI/x4R46otS5LULLsNgqx5MDM/BlwC/CXw7xHxk4g4ofIKJUmV2u0byurXCC6gNiL4FfBpau8aPga4GxhbZYGSpGo18qrKx4DvAn+WmT291ndHxE3VlCVJapZGguA99XcL7yQzrx3keiRJTdbIxeJ/iYiDti1ExO9HxAMV1iRJaqJGgqAjM1/etpCZLwEHV1eSJKmZGgmCNyPi8G0LEXEE0OepIknS0NPINYIrgEci4idAAB8EZlValSSpaXYbBJn5zxExGTi+vuqyzPxNtWVJkpqlkREBwLuA39bbT4gIMvOh6sqSJDVLIzeUXQucB6wE3qqvTsAgkKQ20MiI4M+o3UvwetXFSJKar5FZQ88Cw6ouRJLUGo2MCDYByyLiR8D2UUFm/vWuvikibgXOAn6dmRP72H4K8L+AX9RX/TAzr2qwbknSIGkkCBbUv/bU7cA3ge/sos3DmXnWXuxbkjRIGpk++u2I+D3g8Mxc3eiOM/OhiOgcQG2SpCZo5FWVfwosA/65vnxMROzNCKEvJ0TE8oi4PyL+cBc1zIqI7ojoXrdu3SAdWpIEjV0svhI4DngZIDOXAUcOwrGfAI7IzEnADcD8/hpm5tzM7MrMro6OjkE4tCRpm0aCYEtmvrLDurf6bLkHMvPVzNxY/7wQGBYRowa6X0nSnmkkCFZGxMeBfSLi6Ii4AXh0oAeOiEMiIuqfj6vXsn6g+5Uk7ZlGZg19mtqD514HfgA8AFy9u2+KiB8ApwCjIqIH+DL1+xEy8ybgXGB2RGwFXgNm9PcCHElSdRqZNbSJWhBcsSc7rr/sflfbv0lteqkkqYUaedbQYvp4/0Bm/tdKKpIkNVUjp4Y+1+vzcGA6sLWaciRJzdbIqaGlO6z6t4j494rqkYa2xV9tzXFPvbw1x1VbaOTU0B/0WnwHMAU4sLKKJElN1cipoaXUrhEEtVNCvwAuqrIoSVLzNHJqaGwzCpEktUYjp4Y+sqvtmfnDwStHktRsjZwaugj4ALCovnwqtTuL11E7ZWQQSNIQ1kgQDAMmZOaLABFxKHB7Zs6stDJJUlM08qyhMdtCoO5XwOEV1SNJarJGRgQ/iogHqD1nCOA84F+rK0mS1EyNzBq6NCLOAU6ur5qbmfdUW5YkqVkaGRFA7SUyGzLzXyNiv4gYkZkbqixMktQcjbyq8hJgHvAP9VWHsYu3iUmShpZGLhZ/CjgReBUgM58BDq6yKElS8zQSBK9n5hvbFiJiX/p4LLUkaWhq5BrBTyLii8DvRcRpwBzg3mrLaj/fePDnrS6h6T5z2rhWlyCpAY2MCL5A7S7ip4C/AhYCX6qyKElS8+xyRBAR+wDfyczzgZubU5IkqZl2OSLIzDeBIyLinU2qR5LUZI1cI3iW2lvJFgD/b9vKzPx6ZVVJkpqm3xFBRHy3/nEacF+97YheX5KkNrCrEcGUiPhPwPPADU2qR5LUZLsKgpuAHwFjge5e64PafQRHVliXJKlJ+j01lJnXZ+Z44LbMPLLX19jMNAQkqU3s9j6CzJzdjEIkSa3RyA1lkqQ2ZhBIUuEqC4KIuDUifh0RK/rZHhFxfUSsiYgnI2JyVbVIkvpX5YjgduCMXWz/MHB0/WsW8PcV1iJJ6kejbyjbY5n5UER07qLJ2dSeY5TA4xFxUEQcmpkvVlWT1LYWf7U1xz318tYcV4OqldcIDgNe6LXcU1+3k4iYFRHdEdG9bt26phQnSaUYEheLM3NuZnZlZldHR0ery5GkttLKIPglMKbX8uj6OklSE7UyCBYAn6jPHjoeeMXrA5LUfJVdLI6IHwCnAKMiogf4MjAMIDNvovamszOBNcAmYGZVtUiS+lflrKGP7WZ7Ap+q6viSpMZUFgRvR8c/P7fVJTTd44fPanUJzdeqqZTSEDUkZg1JkqpjEEhS4QwCSSqcQSBJhTMIJKlwBoEkFa6o6aMlau2U2etaeGxJjXJEIEmFMwgkqXAGgSQVziCQpMIZBJJUOINAkgrn9FFVx6eAqkr++Ro0jggkqXAGgSQVziCQpMIZBJJUOINAkgrnrCGpDTz27PqWHPeEU1tyWA0yRwSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXO+wgk7b0WPgG0VfdOtKNKRwQRcUZErI6INRHxhT62XxgR6yJiWf3r4irrkSTtrLIRQUTsA9wInAb0AEsiYkFmPr1D07sy89Kq6pAk7VqVI4LjgDWZ+WxmvgHcCZxd4fEkSXuhyiA4DHih13JPfd2OpkfEkxExLyLG9LWjiJgVEd0R0b1u3boqapWkYrV61tC9QGdm/hHwIPDtvhpl5tzM7MrMro6OjqYWKEntrsog+CXQ+zf80fV122Xm+sx8vb74P4ApFdYjSepDlUGwBDg6IsZGxDuBGcCC3g0i4tBei9OAVRXWI0nqQ2WzhjJza0RcCjwA7APcmpkrI+IqoDszFwB/HRHTgK3Ab4ELq6pHktS3Sm8oy8yFwMId1v1tr8+XA5dXWYNap8Qbfk44cmSrS5D2WKsvFkuSWswgkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYXzxTSS9lqJ94q0I0cEklQ4g0CSCmcQSFLhDAJJKpxBIEmFMwgkqXAGgSQVzvsIpEHkvHoNRY4IJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCldpEETEGRGxOiLWRMQX+tj+roi4q779pxHRWWU9kqSdVRYEEbEPcCPwYWAC8LGImLBDs4uAlzLzPwPfAK6tqh5JUt+qHBEcB6zJzGcz8w3gTuDsHdqcDXy7/nke8McRERXWJEnaQZUvpjkMeKHXcg8wtb82mbk1Il4BRgK/6d0oImYBs+qLGyNi9V7WNGrHfRfAPpfBPpfg4r8bSJ+P6G/DkHhDWWbOBeYOdD8R0Z2ZXYNQ0pBhn8tgn8tQVZ+rPDX0S2BMr+XR9XV9tomIfYEDAd/1J0lNVGUQLAGOjoixEfFOYAawYIc2C4C/rH8+F1iUmVlhTZKkHVR2aqh+zv9S4AFgH+DWzFwZEVcB3Zm5ALgF+G5ErAF+Sy0sqjTg00tDkH0ug30uQyV9Dn8Bl6SyeWexJBXOIJCkwrVlEJT4aIsG+vzfIuLpiHgyIn4UEf3OKR4qdtfnXu2mR0RGxJCfathInyPiz+s/65UR8f1m1zjYGvizfXhELI6I/6j/+T6zFXUOloi4NSJ+HREr+tkeEXF9/f/HkxExecAHzcy2+qJ2YXotcCTwTmA5MGGHNnOAm+qfZwB3tbruJvT5VGC/+ufZJfS53m4E8BDwONDV6rqb8HM+GvgP4Pfrywe3uu4m9HkuMLv+eQLwXKvrHmCfTwYmAyv62X4mcD8QwPHATwd6zHYcEZT4aIvd9jkzF2fmpvri49Tu6xjKGvk5A1xN7RlWm5tZXEUa6fMlwI2Z+RJAZv66yTUOtkb6nMAB9c8HAv+nifUNusx8iNosyv6cDXwnax4HDoqIQwdyzHYMgr4ebXFYf20ycyuw7dEWQ1Ujfe7tImq/UQxlu+1zfcg8JjP/qZmFVaiRn/M4YFxE/FtEPB4RZzStumo00ucrgQsiogdYCHy6OaW1zJ7+fd+tIfGICQ2eiLgA6AL+S6trqVJEvAP4OnBhi0tptn2pnR46hdqo76GIeF9mvtzSqqr1MeD2zPy7iDiB2r1JEzPzrVYXNlS044igxEdbNNJnIuJPgCuAaZn5epNqq8ru+jwCmAj8OCKeo3YudcEQv2DcyM+5B1iQmVsy8xfAz6kFw1DVSJ8vAv4nQGY+Bgyn9kC6dtXQ3/c90Y5BUOKjLXbb54h4P/AP1EJgqJ83ht30OTNfycxRmdmZmZ3UrotMy8zu1pQ7KBr5sz2f2miAiBhF7VTRs80scpA10ufngT8GiIjx1IJgXVOrbK4FwCfqs4eOB17JzBcHssO2OzWUb89HW1SqwT5/DdgfuLt+Xfz5zJzWsqIHqME+t5UG+/wA8KGIeBp4E/h8Zg7Z0W6Dff4scHNEfIbaheMLh/IvdhHxA2phPqp+3ePLwDCAzLyJ2nWQM4E1wCZg5oCPOYT/f0mSBkE7nhqSJO0Bg0CSCmcQSFLhDAJJKpxBIEmFMwikAYiIQyLizohYGxFLI2JhRJwcEfNaXZvUKKePSnup/qDCR4Fv1+d3ExGTgAMy8+GWFiftAUcE0t47FdiyLQQAMnM58EJ/z5KX3o4MAmnvTQSWtroIaaAMAkkqnEEg7b2VwJRWFyENlEEg7b1FwLsiYta2FRHxR/zuI4Kltz2DQNpL9SdcngP8SX366Ergq8D/bW1l0p5x+qgkFc4RgSQVziCQpMIZBJJUOINAkgpnEEhS4QwCSSqcQSBJhfv/s1l1vL7TWBsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "matplotlib.pyplot.hist(ci_values_date_hubs, density=1, alpha=0.5, label=\"date\")\n",
    "matplotlib.pyplot.hist(ci_values_party_hubs, density=1, alpha=0.5, label=\"party\")\n",
    "matplotlib.pyplot.legend(loc=\"upper center\")\n",
    "matplotlib.pyplot.xlabel(\"Ci\")\n",
    "matplotlib.pyplot.ylabel(\"frequency\")\n",
    "matplotlib.pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Do these histograms look the same to you? Let's test it with a Kolmogorov-Smirnov test, using the function `scipy.stats.ks_2samp`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ks_2sampResult(statistic=0.25895400895400894, pvalue=0.0020227281019755683)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stats.ks_2samp(ci_values_date_hubs, ci_values_party_hubs)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}