CSP-BCICIV2a.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Common Spatial Pattern BCI\n",
    "Date: 2020 Aug 12th\n",
    "Writer: Orvin Demsy\n",
    "Description:  \n",
    "\n",
    "This code is a revision of previous CSP version, revision include:\n",
    "* Applying band-pass 8-30Hz before \n",
    "* Using W = B.T * P \n",
    "* Add np.log10 in feature extraction step\n",
    "* Use sqrtm(inv(X)) when computing white matrix\n",
    "\n",
    "Same as previous version this will used BCI competition IV IIa dataset and only take left and right class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "dframe = pd.DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "IPython.notebook.set_autosave_interval(5000)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Autosaving every 5 seconds\n"
     ]
    }
   ],
   "source": [
    "%autosave 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating dict to store original data and modified data\n",
    "ori_data = dict()\n",
    "mod_data = dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create function to print keys() within dictionary\n",
    "def showKeys(dict_name):\n",
    "    print('Keys in this dict are: ')\n",
    "    for i in dict_name.keys():\n",
    "        print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of subject to be processed (n+1 for iteration purpose)\n",
    "ns = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the training data of all subjects\n",
    "\n",
    "# Iter over all data path then store them in sub0X variable\n",
    "for i in range(1, ns):\n",
    "    data_path = './datasets/A{:02d}T.npz'.format(i)\n",
    "    subject = 'subject{:02d}'.format(i)\n",
    "    \n",
    "    # Load EEG data from datapath and store into subj0X variabel then store into data dictionary\n",
    "    ori_data[subject] = np.load(data_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to count subject\n",
    "def subjectCounter(i):\n",
    "    return 'subject0{}'.format(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'numpy.lib.npyio.NpzFile'>\n",
      "['s', 'etyp', 'epos', 'edur', 'artifacts']\n"
     ]
    }
   ],
   "source": [
    "# Peek into ori_data type and its keys \n",
    "print(type(ori_data['subject01']))\n",
    "print(ori_data['subject01'].files)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 1], dtype=uint8)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ori_data['subject01']['artifacts'].ravel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explanation regarding contents of the keys inside each subject data:\n",
    "* **s**: 's' contains continuous time-series recorded EEG signals are, shape of M x N array. Size may vary between subjects but N is fixed to 25, indicates 25 electrodes with 22 first EEG and 3 last EOG\n",
    "* **etype**: 'etyp' stands for event type which indicate event occurence, event code will be describe in subsequent table\n",
    "* **epos**: 'epos' stands for event position, denoting corresponding event begins at n-th sample at **'s'**\n",
    "* **edur**: 'edur' stands for event duration, denoting duration of corresponding event\n",
    "* **artifacts**: size of 288 x 1, 288 comes form 6 x 48, 6 runs where @run has 48 trials, @class has 12 trials"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ori_data will contain original data that is the raw file\n",
    "# mod_data will contain modified original data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'subject01': {},\n",
       " 'subject02': {},\n",
       " 'subject03': {},\n",
       " 'subject04': {},\n",
       " 'subject05': {},\n",
       " 'subject06': {},\n",
       " 'subject07': {},\n",
       " 'subject08': {},\n",
       " 'subject09': {}}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Initialize 'subject0x' dict inside mod_data\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    mod_data[subj] = {}\n",
    "    \n",
    "mod_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample\tElectrodes\n",
      "(672528, 25)\n",
      "(677169, 25)\n",
      "(660530, 25)\n",
      "(600915, 25)\n",
      "(686120, 25)\n",
      "(678980, 25)\n",
      "(681071, 25)\n",
      "(675270, 25)\n",
      "(673328, 25)\n"
     ]
    }
   ],
   "source": [
    "# Peek into different shape of recorded data/'s' between subjects\n",
    "print('Sample\\tElectrodes')\n",
    "for i in range(1, ns):\n",
    "    subj_temp = subjectCounter(i)\n",
    "    print(ori_data[subj_temp]['s'].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First extract the sample data from ori_data into mod_data\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    mod_data[subj]['raw_EEG'] = ori_data[subj]['s']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "      <th>22</th>\n",
       "      <th>23</th>\n",
       "      <th>24</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-21.142578</td>\n",
       "      <td>-23.681641</td>\n",
       "      <td>-21.484375</td>\n",
       "      <td>-25.146484</td>\n",
       "      <td>-25.732422</td>\n",
       "      <td>-27.929688</td>\n",
       "      <td>-14.550781</td>\n",
       "      <td>-22.509766</td>\n",
       "      <td>-25.439453</td>\n",
       "      <td>-28.710938</td>\n",
       "      <td>...</td>\n",
       "      <td>-30.615234</td>\n",
       "      <td>-29.638672</td>\n",
       "      <td>-26.660156</td>\n",
       "      <td>-20.947266</td>\n",
       "      <td>-25.439453</td>\n",
       "      <td>-24.902344</td>\n",
       "      <td>-19.677734</td>\n",
       "      <td>31.738281</td>\n",
       "      <td>-13.671875</td>\n",
       "      <td>-43.945312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-21.923828</td>\n",
       "      <td>-23.925781</td>\n",
       "      <td>-24.316406</td>\n",
       "      <td>-25.341797</td>\n",
       "      <td>-26.074219</td>\n",
       "      <td>-25.244141</td>\n",
       "      <td>-16.064453</td>\n",
       "      <td>-22.753906</td>\n",
       "      <td>-28.320312</td>\n",
       "      <td>-29.248047</td>\n",
       "      <td>...</td>\n",
       "      <td>-31.054688</td>\n",
       "      <td>-30.712891</td>\n",
       "      <td>-26.123047</td>\n",
       "      <td>-22.998047</td>\n",
       "      <td>-27.099609</td>\n",
       "      <td>-26.367188</td>\n",
       "      <td>-23.339844</td>\n",
       "      <td>22.460938</td>\n",
       "      <td>-13.183594</td>\n",
       "      <td>-40.527344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-15.625000</td>\n",
       "      <td>-19.726562</td>\n",
       "      <td>-18.847656</td>\n",
       "      <td>-21.582031</td>\n",
       "      <td>-20.751953</td>\n",
       "      <td>-21.728516</td>\n",
       "      <td>-12.451172</td>\n",
       "      <td>-20.410156</td>\n",
       "      <td>-23.730469</td>\n",
       "      <td>-25.927734</td>\n",
       "      <td>...</td>\n",
       "      <td>-28.759766</td>\n",
       "      <td>-28.417969</td>\n",
       "      <td>-24.462891</td>\n",
       "      <td>-21.386719</td>\n",
       "      <td>-25.097656</td>\n",
       "      <td>-24.267578</td>\n",
       "      <td>-21.484375</td>\n",
       "      <td>33.691406</td>\n",
       "      <td>-8.300781</td>\n",
       "      <td>-38.574219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-16.699219</td>\n",
       "      <td>-13.720703</td>\n",
       "      <td>-20.361328</td>\n",
       "      <td>-20.849609</td>\n",
       "      <td>-24.169922</td>\n",
       "      <td>-22.021484</td>\n",
       "      <td>-10.986328</td>\n",
       "      <td>-16.357422</td>\n",
       "      <td>-25.585938</td>\n",
       "      <td>-24.951172</td>\n",
       "      <td>...</td>\n",
       "      <td>-27.783203</td>\n",
       "      <td>-28.857422</td>\n",
       "      <td>-25.537109</td>\n",
       "      <td>-19.335938</td>\n",
       "      <td>-24.316406</td>\n",
       "      <td>-24.511719</td>\n",
       "      <td>-17.919922</td>\n",
       "      <td>41.992188</td>\n",
       "      <td>-7.324219</td>\n",
       "      <td>-38.574219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-19.335938</td>\n",
       "      <td>-17.626953</td>\n",
       "      <td>-20.410156</td>\n",
       "      <td>-26.464844</td>\n",
       "      <td>-27.392578</td>\n",
       "      <td>-29.882812</td>\n",
       "      <td>-9.277344</td>\n",
       "      <td>-18.359375</td>\n",
       "      <td>-24.609375</td>\n",
       "      <td>-30.517578</td>\n",
       "      <td>...</td>\n",
       "      <td>-31.884766</td>\n",
       "      <td>-33.740234</td>\n",
       "      <td>-32.666016</td>\n",
       "      <td>-18.554688</td>\n",
       "      <td>-26.611328</td>\n",
       "      <td>-28.466797</td>\n",
       "      <td>-18.408203</td>\n",
       "      <td>37.109375</td>\n",
       "      <td>-15.136719</td>\n",
       "      <td>-49.804688</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          0          1          2          3          4          5   \\\n",
       "0 -21.142578 -23.681641 -21.484375 -25.146484 -25.732422 -27.929688   \n",
       "1 -21.923828 -23.925781 -24.316406 -25.341797 -26.074219 -25.244141   \n",
       "2 -15.625000 -19.726562 -18.847656 -21.582031 -20.751953 -21.728516   \n",
       "3 -16.699219 -13.720703 -20.361328 -20.849609 -24.169922 -22.021484   \n",
       "4 -19.335938 -17.626953 -20.410156 -26.464844 -27.392578 -29.882812   \n",
       "\n",
       "          6          7          8          9   ...         15         16  \\\n",
       "0 -14.550781 -22.509766 -25.439453 -28.710938  ... -30.615234 -29.638672   \n",
       "1 -16.064453 -22.753906 -28.320312 -29.248047  ... -31.054688 -30.712891   \n",
       "2 -12.451172 -20.410156 -23.730469 -25.927734  ... -28.759766 -28.417969   \n",
       "3 -10.986328 -16.357422 -25.585938 -24.951172  ... -27.783203 -28.857422   \n",
       "4  -9.277344 -18.359375 -24.609375 -30.517578  ... -31.884766 -33.740234   \n",
       "\n",
       "          17         18         19         20         21         22  \\\n",
       "0 -26.660156 -20.947266 -25.439453 -24.902344 -19.677734  31.738281   \n",
       "1 -26.123047 -22.998047 -27.099609 -26.367188 -23.339844  22.460938   \n",
       "2 -24.462891 -21.386719 -25.097656 -24.267578 -21.484375  33.691406   \n",
       "3 -25.537109 -19.335938 -24.316406 -24.511719 -17.919922  41.992188   \n",
       "4 -32.666016 -18.554688 -26.611328 -28.466797 -18.408203  37.109375   \n",
       "\n",
       "          23         24  \n",
       "0 -13.671875 -43.945312  \n",
       "1 -13.183594 -40.527344  \n",
       "2  -8.300781 -38.574219  \n",
       "3  -7.324219 -38.574219  \n",
       "4 -15.136719 -49.804688  \n",
       "\n",
       "[5 rows x 25 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Visualizing sample_data inside subject01\n",
    "# You can see that it has 25 electrodes stored in columns\n",
    "pd.DataFrame(mod_data['subject01']['raw_EEG']).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Removing last three EOG electrode using np.delete, store data as EEG_only to each subject dict\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    mod_data[subj]['raw_EEG'] = np.delete(mod_data[subj]['raw_EEG'], np.s_[22:], 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['raw_EEG'])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mod_data['subject01'].keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bandpass filtering 8-30 Hz\n",
    "Using FIR [8 30] Hz band-pass filter to remove muscle artifacts, powerline noise, and DC drift  \n",
    "The command fir1 in used in MATLAB which is equivalent to scipy.signal.firwin in python  \n",
    "Next EEG signals between [0.5, 3.5] seconds after cue onset are extracted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.signal import firwin, freqz, lfilter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# From document freqeuncy sampling at 250 Hz\n",
    "fs = 250"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fir_bandpass(numtaps, low, high, fs):\n",
    "    fnyq = fs/2\n",
    "    b = firwin(numtaps, np.array([low, high])/(fnyq), pass_zero = 'bandpass')\n",
    "    \n",
    "    return b "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function will apply bandpass filter to raw EEG data\n",
    "def apply_bandpass(raw_EEG, b):\n",
    "    '''\n",
    "    INPUT:\n",
    "    raw_EEG : EEG data in the shape of S x N\n",
    "    b : coefficient of band-pass filter\n",
    "    \n",
    "    OUTPUT:\n",
    "    EEG_filtered : filtered EEG data shape S x N\n",
    "    \n",
    "    N : number of channel\n",
    "    S : number of sample\n",
    "    '''\n",
    "    raw_EEG = lfilter(b, 1, raw_EEG, axis=0)\n",
    "    \n",
    "    return raw_EEG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating band-pass filter\n",
    "b = fir_bandpass(51, low=8, high=30, fs=fs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Applying bandpass\n",
    "\n",
    "# Key to access data\n",
    "raw_EEG = 'raw_EEG'\n",
    "\n",
    "# Key to store result\n",
    "EEG_filtered = 'EEG_filtered'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    temp_raw_EEG = mod_data[subj][raw_EEG]\n",
    "    \n",
    "    mod_data[subj][EEG_filtered] = apply_bandpass(temp_raw_EEG, b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.020422</td>\n",
       "      <td>-0.022875</td>\n",
       "      <td>-0.020752</td>\n",
       "      <td>-0.024290</td>\n",
       "      <td>-0.024856</td>\n",
       "      <td>-0.026978</td>\n",
       "      <td>-0.014055</td>\n",
       "      <td>-0.021743</td>\n",
       "      <td>-0.024573</td>\n",
       "      <td>-0.027733</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.025186</td>\n",
       "      <td>-0.022356</td>\n",
       "      <td>-0.023960</td>\n",
       "      <td>-0.029572</td>\n",
       "      <td>-0.028629</td>\n",
       "      <td>-0.025752</td>\n",
       "      <td>-0.020234</td>\n",
       "      <td>-0.024573</td>\n",
       "      <td>-0.024054</td>\n",
       "      <td>-0.019007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.028404</td>\n",
       "      <td>-0.031205</td>\n",
       "      <td>-0.030831</td>\n",
       "      <td>-0.033074</td>\n",
       "      <td>-0.033981</td>\n",
       "      <td>-0.033931</td>\n",
       "      <td>-0.020491</td>\n",
       "      <td>-0.029673</td>\n",
       "      <td>-0.036051</td>\n",
       "      <td>-0.038065</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.031740</td>\n",
       "      <td>-0.030597</td>\n",
       "      <td>-0.034277</td>\n",
       "      <td>-0.040461</td>\n",
       "      <td>-0.039797</td>\n",
       "      <td>-0.034346</td>\n",
       "      <td>-0.029374</td>\n",
       "      <td>-0.034872</td>\n",
       "      <td>-0.033981</td>\n",
       "      <td>-0.029271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.022534</td>\n",
       "      <td>-0.027174</td>\n",
       "      <td>-0.026464</td>\n",
       "      <td>-0.029447</td>\n",
       "      <td>-0.028894</td>\n",
       "      <td>-0.029548</td>\n",
       "      <td>-0.017482</td>\n",
       "      <td>-0.027437</td>\n",
       "      <td>-0.032539</td>\n",
       "      <td>-0.034971</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.024285</td>\n",
       "      <td>-0.029289</td>\n",
       "      <td>-0.032084</td>\n",
       "      <td>-0.038319</td>\n",
       "      <td>-0.037874</td>\n",
       "      <td>-0.032493</td>\n",
       "      <td>-0.028467</td>\n",
       "      <td>-0.033443</td>\n",
       "      <td>-0.032392</td>\n",
       "      <td>-0.028682</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.027857</td>\n",
       "      <td>-0.027150</td>\n",
       "      <td>-0.032594</td>\n",
       "      <td>-0.035113</td>\n",
       "      <td>-0.038213</td>\n",
       "      <td>-0.037143</td>\n",
       "      <td>-0.019260</td>\n",
       "      <td>-0.029577</td>\n",
       "      <td>-0.040505</td>\n",
       "      <td>-0.041637</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.035856</td>\n",
       "      <td>-0.031609</td>\n",
       "      <td>-0.039735</td>\n",
       "      <td>-0.045916</td>\n",
       "      <td>-0.046540</td>\n",
       "      <td>-0.041085</td>\n",
       "      <td>-0.032311</td>\n",
       "      <td>-0.039749</td>\n",
       "      <td>-0.039492</td>\n",
       "      <td>-0.030589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.064138</td>\n",
       "      <td>-0.066045</td>\n",
       "      <td>-0.067683</td>\n",
       "      <td>-0.079740</td>\n",
       "      <td>-0.082914</td>\n",
       "      <td>-0.087771</td>\n",
       "      <td>-0.040368</td>\n",
       "      <td>-0.065460</td>\n",
       "      <td>-0.080927</td>\n",
       "      <td>-0.091818</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.087361</td>\n",
       "      <td>-0.067454</td>\n",
       "      <td>-0.079225</td>\n",
       "      <td>-0.097632</td>\n",
       "      <td>-0.098159</td>\n",
       "      <td>-0.089933</td>\n",
       "      <td>-0.064292</td>\n",
       "      <td>-0.082052</td>\n",
       "      <td>-0.082849</td>\n",
       "      <td>-0.061782</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1         2         3         4         5         6   \\\n",
       "0 -0.020422 -0.022875 -0.020752 -0.024290 -0.024856 -0.026978 -0.014055   \n",
       "1 -0.028404 -0.031205 -0.030831 -0.033074 -0.033981 -0.033931 -0.020491   \n",
       "2 -0.022534 -0.027174 -0.026464 -0.029447 -0.028894 -0.029548 -0.017482   \n",
       "3 -0.027857 -0.027150 -0.032594 -0.035113 -0.038213 -0.037143 -0.019260   \n",
       "4 -0.064138 -0.066045 -0.067683 -0.079740 -0.082914 -0.087771 -0.040368   \n",
       "\n",
       "         7         8         9   ...        12        13        14        15  \\\n",
       "0 -0.021743 -0.024573 -0.027733  ... -0.025186 -0.022356 -0.023960 -0.029572   \n",
       "1 -0.029673 -0.036051 -0.038065  ... -0.031740 -0.030597 -0.034277 -0.040461   \n",
       "2 -0.027437 -0.032539 -0.034971  ... -0.024285 -0.029289 -0.032084 -0.038319   \n",
       "3 -0.029577 -0.040505 -0.041637  ... -0.035856 -0.031609 -0.039735 -0.045916   \n",
       "4 -0.065460 -0.080927 -0.091818  ... -0.087361 -0.067454 -0.079225 -0.097632   \n",
       "\n",
       "         16        17        18        19        20        21  \n",
       "0 -0.028629 -0.025752 -0.020234 -0.024573 -0.024054 -0.019007  \n",
       "1 -0.039797 -0.034346 -0.029374 -0.034872 -0.033981 -0.029271  \n",
       "2 -0.037874 -0.032493 -0.028467 -0.033443 -0.032392 -0.028682  \n",
       "3 -0.046540 -0.041085 -0.032311 -0.039749 -0.039492 -0.030589  \n",
       "4 -0.098159 -0.089933 -0.064292 -0.082052 -0.082849 -0.061782  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(mod_data['subject01']['EEG_filtered']).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Taking EEG data corresponds  to left and right class\n",
    "Current method of CSP calculation requires EEG left and EEG right data,  \n",
    "thus the goal here is to make EEG left and EEG right class each has dimension T x C x N, where  \n",
    "T = trial, C = channel, N = sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First take the position of left and right class\n",
    "left_pos = 'left_pos'\n",
    "right_pos = 'right_pos'\n",
    "\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    mod_data[subj][left_pos] = ori_data[subj]['epos'][ori_data[subj]['etyp'] == 769]\n",
    "    mod_data[subj][right_pos] = ori_data[subj]['epos'][ori_data[subj]['etyp'] == 770]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now take EEG data within [0.5 3.5] seconds after cue onset position of each class\n",
    "start = 0.5\n",
    "end = 3.5\n",
    "\n",
    "# Key to access data\n",
    "EEG_filtered = 'EEG_filtered'\n",
    "\n",
    "# Key to store result\n",
    "EEG_left = 'EEG_left'\n",
    "EEG_right = 'EEG_right'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "        \n",
    "    # Temporary variable of left and right pos    \n",
    "    temp_pos_left = mod_data[subj][left_pos]\n",
    "    temp_pos_right = mod_data[subj][right_pos]\n",
    "\n",
    "    temp_EEG_left = []\n",
    "    temp_EEG_right = []\n",
    "    \n",
    "    # LEFT\n",
    "    for j in range(len(temp_pos_left)):\n",
    "        temp_EEG_left.append(mod_data[subj][EEG_filtered]\\\n",
    "                             [temp_pos_left[j] + int(start*fs) : temp_pos_left[j] + int(end*fs)].T)\n",
    "    \n",
    "    mod_data[subj][EEG_left] = np.array(temp_EEG_left)\n",
    "    \n",
    "    # RIGHT\n",
    "    for j in range(len(temp_pos_right)):\n",
    "        temp_EEG_right.append(mod_data[subj][EEG_filtered]\\\n",
    "                              [temp_pos_right[j] + int(start*fs) : temp_pos_right[j] + int(end*fs)].T)\n",
    "        \n",
    "    mod_data[subj][EEG_right] = np.array(temp_EEG_right)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>740</th>\n",
       "      <th>741</th>\n",
       "      <th>742</th>\n",
       "      <th>743</th>\n",
       "      <th>744</th>\n",
       "      <th>745</th>\n",
       "      <th>746</th>\n",
       "      <th>747</th>\n",
       "      <th>748</th>\n",
       "      <th>749</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-3.280087</td>\n",
       "      <td>-5.109051</td>\n",
       "      <td>-5.672537</td>\n",
       "      <td>-5.111599</td>\n",
       "      <td>-3.899460</td>\n",
       "      <td>-2.593373</td>\n",
       "      <td>-1.591841</td>\n",
       "      <td>-0.985517</td>\n",
       "      <td>-0.578749</td>\n",
       "      <td>-0.090275</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.772163</td>\n",
       "      <td>-0.085597</td>\n",
       "      <td>1.080908</td>\n",
       "      <td>1.711972</td>\n",
       "      <td>1.949453</td>\n",
       "      <td>1.976977</td>\n",
       "      <td>1.964539</td>\n",
       "      <td>2.014536</td>\n",
       "      <td>2.125415</td>\n",
       "      <td>2.254110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-2.547175</td>\n",
       "      <td>-4.302690</td>\n",
       "      <td>-5.309319</td>\n",
       "      <td>-5.470119</td>\n",
       "      <td>-4.832418</td>\n",
       "      <td>-3.556131</td>\n",
       "      <td>-1.878648</td>\n",
       "      <td>-0.117910</td>\n",
       "      <td>1.424508</td>\n",
       "      <td>2.437497</td>\n",
       "      <td>...</td>\n",
       "      <td>-3.521003</td>\n",
       "      <td>-1.265150</td>\n",
       "      <td>0.136607</td>\n",
       "      <td>0.545482</td>\n",
       "      <td>0.184084</td>\n",
       "      <td>-0.502538</td>\n",
       "      <td>-1.020506</td>\n",
       "      <td>-0.986947</td>\n",
       "      <td>-0.281741</td>\n",
       "      <td>0.981587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-2.991825</td>\n",
       "      <td>-4.763764</td>\n",
       "      <td>-5.588473</td>\n",
       "      <td>-5.527517</td>\n",
       "      <td>-4.789856</td>\n",
       "      <td>-3.621242</td>\n",
       "      <td>-2.252209</td>\n",
       "      <td>-0.868619</td>\n",
       "      <td>0.400097</td>\n",
       "      <td>1.423731</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.760539</td>\n",
       "      <td>-0.133423</td>\n",
       "      <td>0.715726</td>\n",
       "      <td>0.875251</td>\n",
       "      <td>0.646276</td>\n",
       "      <td>0.361736</td>\n",
       "      <td>0.278000</td>\n",
       "      <td>0.499216</td>\n",
       "      <td>0.961176</td>\n",
       "      <td>1.526529</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-3.141083</td>\n",
       "      <td>-4.660437</td>\n",
       "      <td>-5.053033</td>\n",
       "      <td>-4.502334</td>\n",
       "      <td>-3.423306</td>\n",
       "      <td>-2.255246</td>\n",
       "      <td>-1.293399</td>\n",
       "      <td>-0.622783</td>\n",
       "      <td>-0.114256</td>\n",
       "      <td>0.372676</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.897646</td>\n",
       "      <td>0.340081</td>\n",
       "      <td>0.821234</td>\n",
       "      <td>0.741936</td>\n",
       "      <td>0.492841</td>\n",
       "      <td>0.444858</td>\n",
       "      <td>0.780600</td>\n",
       "      <td>1.439134</td>\n",
       "      <td>2.130652</td>\n",
       "      <td>2.550930</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-2.631041</td>\n",
       "      <td>-3.825343</td>\n",
       "      <td>-3.846291</td>\n",
       "      <td>-3.100074</td>\n",
       "      <td>-2.217474</td>\n",
       "      <td>-1.704968</td>\n",
       "      <td>-1.736631</td>\n",
       "      <td>-2.096243</td>\n",
       "      <td>-2.316440</td>\n",
       "      <td>-1.978562</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.552320</td>\n",
       "      <td>0.527479</td>\n",
       "      <td>1.061591</td>\n",
       "      <td>1.233409</td>\n",
       "      <td>1.301396</td>\n",
       "      <td>1.451247</td>\n",
       "      <td>1.730466</td>\n",
       "      <td>2.047716</td>\n",
       "      <td>2.241499</td>\n",
       "      <td>2.211129</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 750 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        0         1         2         3         4         5         6    \\\n",
       "0 -3.280087 -5.109051 -5.672537 -5.111599 -3.899460 -2.593373 -1.591841   \n",
       "1 -2.547175 -4.302690 -5.309319 -5.470119 -4.832418 -3.556131 -1.878648   \n",
       "2 -2.991825 -4.763764 -5.588473 -5.527517 -4.789856 -3.621242 -2.252209   \n",
       "3 -3.141083 -4.660437 -5.053033 -4.502334 -3.423306 -2.255246 -1.293399   \n",
       "4 -2.631041 -3.825343 -3.846291 -3.100074 -2.217474 -1.704968 -1.736631   \n",
       "\n",
       "        7         8         9    ...       740       741       742       743  \\\n",
       "0 -0.985517 -0.578749 -0.090275  ... -1.772163 -0.085597  1.080908  1.711972   \n",
       "1 -0.117910  1.424508  2.437497  ... -3.521003 -1.265150  0.136607  0.545482   \n",
       "2 -0.868619  0.400097  1.423731  ... -1.760539 -0.133423  0.715726  0.875251   \n",
       "3 -0.622783 -0.114256  0.372676  ... -0.897646  0.340081  0.821234  0.741936   \n",
       "4 -2.096243 -2.316440 -1.978562  ... -0.552320  0.527479  1.061591  1.233409   \n",
       "\n",
       "        744       745       746       747       748       749  \n",
       "0  1.949453  1.976977  1.964539  2.014536  2.125415  2.254110  \n",
       "1  0.184084 -0.502538 -1.020506 -0.986947 -0.281741  0.981587  \n",
       "2  0.646276  0.361736  0.278000  0.499216  0.961176  1.526529  \n",
       "3  0.492841  0.444858  0.780600  1.439134  2.130652  2.550930  \n",
       "4  1.301396  1.451247  1.730466  2.047716  2.241499  2.211129  \n",
       "\n",
       "[5 rows x 750 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(mod_data['subject01']['EEG_right'][-1]).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 0, 0, 0, 0, 0, 0, 0, 0]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking if raw EEG contains NaN\n",
    "isnan = []\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    temp = np.isnan(mod_data[subj]['raw_EEG'])\n",
    "    \n",
    "    isnan.append(np.sum(temp))\n",
    "    \n",
    "isnan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t\tLeft \t\t Right\n",
      "subject01:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject02:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject03:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject04:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject05:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject06:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject07:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject08:\t(72, 22, 750)\t(72, 22, 750)\n",
      "subject09:\t(72, 22, 750)\t(72, 22, 750)\n"
     ]
    }
   ],
   "source": [
    "# Checking current size of EEG left and right data\n",
    "print('\\t\\tLeft \\t\\t Right')\n",
    "\n",
    "for i in range(1, ns):\n",
    "    \n",
    "    subj = subjectCounter(i)\n",
    "    temp = mod_data[subj]\n",
    "    \n",
    "    print('subject{:02d}:\\t{}\\t{}'.format(i, temp['EEG_left'].shape, temp['EEG_right'].shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "temp = dframe(mod_data['subject01']['EEG_right'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Split into test and train data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def split_EEG_one_class(EEG_one_class, percent_train=0.8):\n",
    "    '''\n",
    "    split_EEG_one_class will receive EEG data of one class, with size of T x N x M, where\n",
    "    T = number of trial\n",
    "    N = number of electrodes\n",
    "    M = sample number\n",
    "    \n",
    "    INPUT:\n",
    "    EEG_data_one_class: the data of one class of EEG data\n",
    "    \n",
    "    percent_train: allocation percentage of training data, default is 0.8\n",
    "    \n",
    "    OUTPUT:\n",
    "    EEG_train: EEG data for training\n",
    "    \n",
    "    EEG_test: EEG data for test\n",
    "    \n",
    "    Both have type of np.arrray dimension of T x M x N\n",
    "    '''\n",
    "\n",
    "    # Number of all trials\n",
    "    n = EEG_one_class.shape[0]\n",
    "    \n",
    "    n_tr = round(n*percent_train)\n",
    "    n_te = n - n_tr\n",
    "    \n",
    "    EEG_train = EEG_one_class[:n_tr]\n",
    "    EEG_test = EEG_one_class[n_tr:n_tr+n_te]\n",
    "        \n",
    "    return EEG_train, EEG_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Keys to access data\n",
    "EEG_left = 'EEG_left'\n",
    "EEG_right = 'EEG_right'\n",
    "\n",
    "# Keys to store result\n",
    "EEG_left_train = 'EEG_left_train'\n",
    "EEG_right_train = 'EEG_right_train'\n",
    "\n",
    "EEG_left_test = 'EEG_left_test'\n",
    "EEG_right_test = 'EEG_right_test'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    \n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    mod_data[subj][EEG_left_train], mod_data[subj][EEG_left_test] = split_EEG_one_class(mod_data[subj][EEG_left], 0.8)\n",
    "    mod_data[subj][EEG_right_train], mod_data[subj][EEG_right_test] = split_EEG_one_class(mod_data[subj][EEG_right], 0.8)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t\tLeft \t\t Right\n",
      "TRAINING\n",
      "subject01:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject02:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject03:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject04:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject05:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject06:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject07:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject08:\t(58, 22, 750)\t(58, 22, 750)\n",
      "subject09:\t(58, 22, 750)\t(58, 22, 750)\n",
      "TEST\n",
      "subject01:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject02:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject03:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject04:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject05:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject06:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject07:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject08:\t(14, 22, 750)\t(14, 22, 750)\n",
      "subject09:\t(14, 22, 750)\t(14, 22, 750)\n"
     ]
    }
   ],
   "source": [
    "# Checking size of EEG left and right data training and test data\n",
    "print('\\t\\tLeft \\t\\t Right')\n",
    "print('TRAINING')\n",
    "for i in range(1, ns):\n",
    "    \n",
    "    subj = subjectCounter(i)\n",
    "    temp = mod_data[subj]\n",
    "    \n",
    "    print('subject{:02d}:\\t{}\\t{}'.format(i, temp[EEG_left_train].shape, temp[EEG_right_train].shape))\n",
    "\n",
    "print('TEST')\n",
    "for i in range(1, ns):\n",
    "    \n",
    "    subj = subjectCounter(i)\n",
    "    temp = mod_data[subj]\n",
    "    \n",
    "    print('subject{:02d}:\\t{}\\t{}'.format(i, temp[EEG_left_test].shape, temp[EEG_right_test].shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CSP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For all subject create new keys to store all result in CSP step\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    mod_data[subj]['CSP'] = {}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Covariance and Composite Covariance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_cov(EEG_data):\n",
    "    '''\n",
    "    INPUT:\n",
    "    EEG_data : EEG_data in shape T x N x S\n",
    "    \n",
    "    OUTPUT:\n",
    "    avg_cov : covariance matrix of averaged over all trials\n",
    "    '''\n",
    "    cov = []\n",
    "    for i in range(EEG_data.shape[0]):\n",
    "        cov.append(EEG_data[i]@EEG_data[i].T/np.trace(EEG_data[i]@EEG_data[i].T))\n",
    "        \n",
    "    cov = np.mean(np.array(cov), 0)\n",
    "    \n",
    "    return cov"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Key to access data\n",
    "EEG_left_train = 'EEG_left_train'\n",
    "EEG_right_train = 'EEG_right_train'\n",
    "\n",
    "# Key to store data\n",
    "CSP = 'CSP'\n",
    "cov_left = 'cov_left'\n",
    "cov_right = 'cov_right'\n",
    "cov_comp = 'cov_comp'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    # New key to store result\n",
    "    temp_CSP = mod_data[subj][CSP]\n",
    "    \n",
    "    # LEFT\n",
    "    temp_CSP[cov_left] = compute_cov(mod_data[subj][EEG_left_train])\n",
    "\n",
    "    # RIGHT\n",
    "    temp_CSP[cov_right] = compute_cov(mod_data[subj][EEG_right_train])\n",
    "    \n",
    "    # Add covariance of left and right class as composite covariance\n",
    "    temp_CSP[cov_comp] = temp_CSP[cov_left] + temp_CSP[cov_right]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "temp1 = mod_data['subject03'][CSP][cov_left]\n",
    "temp2 = mod_data['subject03'][CSP][cov_right]\n",
    "temp3 = mod_data['subject03'][CSP][cov_comp]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## White Matrix (P)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create new keys for result in whitening step\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    mod_data[subj]['CSP']['whitening'] = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.linalg import sqrtm\n",
    "from scipy.linalg import inv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def decompose_cov(avg_cov):\n",
    "    '''\n",
    "    This function will decompose average covariance matrix of one class of each subject into \n",
    "    eigenvalues denoted by lambda and eigenvector denoted by V\n",
    "    Both will be in descending order\n",
    "    \n",
    "    Parameter:\n",
    "    avgCov = the averaged covariance of one class\n",
    "    \n",
    "    Return:\n",
    "    λ_dsc and V_dsc, i.e. eigenvalues and eigenvector in descending order\n",
    "    \n",
    "    '''\n",
    "    λ, V = np.linalg.eig(avg_cov)\n",
    "    λ_dsc = np.sort(λ)[::-1] # Sort eigenvalue descending order, default is ascending order sort\n",
    "    idx_dsc = np.argsort(λ)[::-1] # Find index in descending order\n",
    "    V_dsc = V[:, idx_dsc] # Sort eigenvectors descending order\n",
    "    λ_dsc = np.diag(λ_dsc) # Diagonalize λ_dsc\n",
    "    \n",
    "    return λ_dsc, V_dsc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "def white_matrix(λ_dsc, V_dsc):\n",
    "    '''\n",
    "    '''\n",
    "    λ_dsc_sqr = sqrtm(inv(λ_dsc))\n",
    "    P = (λ_dsc_sqr)@(V_dsc.T)\n",
    "    \n",
    "    return P"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Keys to access data\n",
    "whitening = 'whitening'\n",
    "cov_comp = 'cov_comp'\n",
    "\n",
    "# Keys to store result\n",
    "eigval = 'eigval'\n",
    "eigvec = 'eigvec'\n",
    "P = 'P'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    temp_whitening = mod_data[subj][CSP][whitening]\n",
    "    \n",
    "    temp_cov = mod_data[subj][CSP][cov_comp]\n",
    "    \n",
    "    # Decomposing composite covariance into eigenvector and eigenvalue\n",
    "    temp_whitening[eigval], temp_whitening[eigvec] = decompose_cov(temp_cov)\n",
    "    \n",
    "    # White matrix\n",
    "    temp_whitening[P] = white_matrix(temp_whitening[eigval], temp_whitening[eigvec])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "tempP = mod_data['subject01'][CSP][whitening][P]\n",
    "tempeigval = mod_data['subject01'][CSP][whitening][eigval]\n",
    "tempeigvec = mod_data['subject01'][CSP][whitening][eigvec]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(22, 22)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tempP.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Common Eigenvec from Sl and Sr\n",
    "In this step the Sl and Sr will not be stored, will only be used to compute each eigenvector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create new keys for result in decomposing Sl and Sr\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    mod_data[subj]['CSP']['S_left'] = {}\n",
    "    mod_data[subj]['CSP']['S_right'] = {}        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_S(avg_Cov, white):\n",
    "    '''\n",
    "    This function will compute S matrix, S = P * C * P.T\n",
    "\n",
    "    INPUT:\n",
    "    avg_Cov: averaged covariance of one class, dimension N x N, where N is number of electrodes\n",
    "    white: the whitening transformation matrix\n",
    "    \n",
    "    OUTPUT:\n",
    "    S\n",
    "    '''\n",
    "    S = white@avg_Cov@white.T\n",
    "    \n",
    "    return S"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "def decompose_S(S_one_class, order='descending'):\n",
    "    '''\n",
    "    This function will decompose the S matrix of one class to get the eigen vector\n",
    "    Both eigenvector will be the same but in opposite order\n",
    "    \n",
    "    i.e the highest eigenvector in S left will be equal to lowest eigenvector in S right matrix \n",
    "    '''\n",
    "    # Decompose S\n",
    "    λ, B = np.linalg.eig(S_one_class)\n",
    "    \n",
    "    # Sort eigenvalues either descending or ascending\n",
    "    if order == 'ascending':\n",
    "        idx = λ.argsort() # Use this index to sort eigenvector smallest -> largest\n",
    "    elif order == 'descending':\n",
    "        idx = λ.argsort()[::-1] # Use this index to sort eigenvector largest -> smallest\n",
    "    else:\n",
    "        print('Wrong order input')\n",
    "    \n",
    "    λ = λ[idx]\n",
    "    B = B[:, idx]\n",
    "    \n",
    "    return B, λ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Keys to access data\n",
    "P = 'P'\n",
    "cov_left = 'cov_left'\n",
    "cov_right = 'cov_right'\n",
    "CSP = 'CSP'\n",
    "\n",
    "# Keys to store data\n",
    "S_left = 'S_left'\n",
    "S_right = 'S_right'\n",
    "eigvec = 'eigvec'\n",
    "eigval = 'eigval'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    # Where to access data\n",
    "    temp_P = mod_data[subj][CSP][whitening][P]\n",
    "    Cl = mod_data[subj][CSP][cov_left]\n",
    "    Cr = mod_data[subj][CSP][cov_right]\n",
    "    \n",
    "    # Where to store result\n",
    "    temp_Sl = mod_data[subj][CSP][S_left]\n",
    "    temp_Sr = mod_data[subj][CSP][S_right]\n",
    "\n",
    "    # LEFT\n",
    "    Sl = compute_S(Cl, temp_P)\n",
    "    temp_Sl[eigvec], temp_Sl[eigval] = decompose_S(Sl, 'descending')\n",
    "    \n",
    "    # RIGHT\n",
    "    Sr = compute_S(Cr, temp_P)\n",
    "    temp_Sr[eigvec], temp_Sr[eigval] = decompose_S(Sr, 'ascending')   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Checking eigenvalues\n",
    "temp1 = mod_data['subject05'][CSP][S_left]['eigval']\n",
    "temp2 = mod_data['subject05'][CSP][S_right]['eigval']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
       "       1., 1., 1., 1., 1.])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temp1 + temp2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Checking eigenvector\n",
    "temp1 = mod_data['subject05'][CSP][S_left]['eigvec']\n",
    "temp2 = mod_data['subject05'][CSP][S_right]['eigvec']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.034337</td>\n",
       "      <td>-0.274925</td>\n",
       "      <td>0.018084</td>\n",
       "      <td>0.113816</td>\n",
       "      <td>0.039164</td>\n",
       "      <td>-0.195245</td>\n",
       "      <td>0.276451</td>\n",
       "      <td>-0.131420</td>\n",
       "      <td>0.095024</td>\n",
       "      <td>-0.365595</td>\n",
       "      <td>...</td>\n",
       "      <td>0.004114</td>\n",
       "      <td>-0.316165</td>\n",
       "      <td>-0.160454</td>\n",
       "      <td>-0.056422</td>\n",
       "      <td>-0.235143</td>\n",
       "      <td>0.025086</td>\n",
       "      <td>0.519633</td>\n",
       "      <td>-0.030565</td>\n",
       "      <td>0.070294</td>\n",
       "      <td>-0.069894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.104875</td>\n",
       "      <td>-0.032112</td>\n",
       "      <td>0.104122</td>\n",
       "      <td>0.080557</td>\n",
       "      <td>-0.208968</td>\n",
       "      <td>0.229181</td>\n",
       "      <td>0.269978</td>\n",
       "      <td>0.124068</td>\n",
       "      <td>-0.015836</td>\n",
       "      <td>-0.057588</td>\n",
       "      <td>...</td>\n",
       "      <td>0.107342</td>\n",
       "      <td>-0.047098</td>\n",
       "      <td>-0.115680</td>\n",
       "      <td>-0.090030</td>\n",
       "      <td>0.518725</td>\n",
       "      <td>-0.369035</td>\n",
       "      <td>-0.244437</td>\n",
       "      <td>-0.242531</td>\n",
       "      <td>-0.151809</td>\n",
       "      <td>-0.080475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.120334</td>\n",
       "      <td>-0.142576</td>\n",
       "      <td>-0.123378</td>\n",
       "      <td>-0.119650</td>\n",
       "      <td>0.037941</td>\n",
       "      <td>0.269308</td>\n",
       "      <td>0.092294</td>\n",
       "      <td>-0.222557</td>\n",
       "      <td>-0.203056</td>\n",
       "      <td>0.152577</td>\n",
       "      <td>...</td>\n",
       "      <td>0.159274</td>\n",
       "      <td>0.244363</td>\n",
       "      <td>0.205827</td>\n",
       "      <td>-0.032077</td>\n",
       "      <td>0.205215</td>\n",
       "      <td>0.032955</td>\n",
       "      <td>0.264689</td>\n",
       "      <td>0.253524</td>\n",
       "      <td>0.295630</td>\n",
       "      <td>-0.546305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.127787</td>\n",
       "      <td>0.559799</td>\n",
       "      <td>-0.045335</td>\n",
       "      <td>-0.087937</td>\n",
       "      <td>-0.241207</td>\n",
       "      <td>0.152786</td>\n",
       "      <td>0.093145</td>\n",
       "      <td>0.239154</td>\n",
       "      <td>-0.016404</td>\n",
       "      <td>0.023806</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.016499</td>\n",
       "      <td>0.225648</td>\n",
       "      <td>0.083725</td>\n",
       "      <td>0.119577</td>\n",
       "      <td>-0.364421</td>\n",
       "      <td>0.105362</td>\n",
       "      <td>0.127887</td>\n",
       "      <td>-0.373712</td>\n",
       "      <td>0.211826</td>\n",
       "      <td>-0.070774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.583685</td>\n",
       "      <td>-0.079217</td>\n",
       "      <td>0.054737</td>\n",
       "      <td>0.059975</td>\n",
       "      <td>-0.010330</td>\n",
       "      <td>0.337588</td>\n",
       "      <td>0.021809</td>\n",
       "      <td>0.260570</td>\n",
       "      <td>0.311055</td>\n",
       "      <td>0.110553</td>\n",
       "      <td>...</td>\n",
       "      <td>0.285020</td>\n",
       "      <td>-0.064470</td>\n",
       "      <td>0.004414</td>\n",
       "      <td>-0.108865</td>\n",
       "      <td>-0.219486</td>\n",
       "      <td>0.073666</td>\n",
       "      <td>0.223155</td>\n",
       "      <td>-0.064999</td>\n",
       "      <td>-0.214659</td>\n",
       "      <td>-0.167897</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1         2         3         4         5         6   \\\n",
       "0  0.034337 -0.274925  0.018084  0.113816  0.039164 -0.195245  0.276451   \n",
       "1 -0.104875 -0.032112  0.104122  0.080557 -0.208968  0.229181  0.269978   \n",
       "2  0.120334 -0.142576 -0.123378 -0.119650  0.037941  0.269308  0.092294   \n",
       "3  0.127787  0.559799 -0.045335 -0.087937 -0.241207  0.152786  0.093145   \n",
       "4 -0.583685 -0.079217  0.054737  0.059975 -0.010330  0.337588  0.021809   \n",
       "\n",
       "         7         8         9   ...        12        13        14        15  \\\n",
       "0 -0.131420  0.095024 -0.365595  ...  0.004114 -0.316165 -0.160454 -0.056422   \n",
       "1  0.124068 -0.015836 -0.057588  ...  0.107342 -0.047098 -0.115680 -0.090030   \n",
       "2 -0.222557 -0.203056  0.152577  ...  0.159274  0.244363  0.205827 -0.032077   \n",
       "3  0.239154 -0.016404  0.023806  ... -0.016499  0.225648  0.083725  0.119577   \n",
       "4  0.260570  0.311055  0.110553  ...  0.285020 -0.064470  0.004414 -0.108865   \n",
       "\n",
       "         16        17        18        19        20        21  \n",
       "0 -0.235143  0.025086  0.519633 -0.030565  0.070294 -0.069894  \n",
       "1  0.518725 -0.369035 -0.244437 -0.242531 -0.151809 -0.080475  \n",
       "2  0.205215  0.032955  0.264689  0.253524  0.295630 -0.546305  \n",
       "3 -0.364421  0.105362  0.127887 -0.373712  0.211826 -0.070774  \n",
       "4 -0.219486  0.073666  0.223155 -0.064999 -0.214659 -0.167897  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(temp1).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.034337</td>\n",
       "      <td>-0.274925</td>\n",
       "      <td>0.018084</td>\n",
       "      <td>0.113816</td>\n",
       "      <td>0.039164</td>\n",
       "      <td>-0.195245</td>\n",
       "      <td>0.276451</td>\n",
       "      <td>-0.131420</td>\n",
       "      <td>0.095024</td>\n",
       "      <td>-0.365595</td>\n",
       "      <td>...</td>\n",
       "      <td>0.004114</td>\n",
       "      <td>-0.316165</td>\n",
       "      <td>-0.160454</td>\n",
       "      <td>-0.056422</td>\n",
       "      <td>-0.235143</td>\n",
       "      <td>0.025086</td>\n",
       "      <td>0.519633</td>\n",
       "      <td>-0.030565</td>\n",
       "      <td>0.070294</td>\n",
       "      <td>0.069894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.104875</td>\n",
       "      <td>-0.032112</td>\n",
       "      <td>0.104122</td>\n",
       "      <td>0.080557</td>\n",
       "      <td>-0.208968</td>\n",
       "      <td>0.229181</td>\n",
       "      <td>0.269978</td>\n",
       "      <td>0.124068</td>\n",
       "      <td>-0.015836</td>\n",
       "      <td>-0.057588</td>\n",
       "      <td>...</td>\n",
       "      <td>0.107342</td>\n",
       "      <td>-0.047098</td>\n",
       "      <td>-0.115680</td>\n",
       "      <td>-0.090030</td>\n",
       "      <td>0.518725</td>\n",
       "      <td>-0.369035</td>\n",
       "      <td>-0.244437</td>\n",
       "      <td>-0.242531</td>\n",
       "      <td>-0.151809</td>\n",
       "      <td>0.080475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.120334</td>\n",
       "      <td>-0.142576</td>\n",
       "      <td>-0.123378</td>\n",
       "      <td>-0.119650</td>\n",
       "      <td>0.037941</td>\n",
       "      <td>0.269308</td>\n",
       "      <td>0.092294</td>\n",
       "      <td>-0.222557</td>\n",
       "      <td>-0.203056</td>\n",
       "      <td>0.152577</td>\n",
       "      <td>...</td>\n",
       "      <td>0.159274</td>\n",
       "      <td>0.244363</td>\n",
       "      <td>0.205827</td>\n",
       "      <td>-0.032077</td>\n",
       "      <td>0.205215</td>\n",
       "      <td>0.032955</td>\n",
       "      <td>0.264689</td>\n",
       "      <td>0.253524</td>\n",
       "      <td>0.295630</td>\n",
       "      <td>0.546305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.127787</td>\n",
       "      <td>0.559799</td>\n",
       "      <td>-0.045335</td>\n",
       "      <td>-0.087937</td>\n",
       "      <td>-0.241207</td>\n",
       "      <td>0.152786</td>\n",
       "      <td>0.093145</td>\n",
       "      <td>0.239154</td>\n",
       "      <td>-0.016404</td>\n",
       "      <td>0.023806</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.016499</td>\n",
       "      <td>0.225648</td>\n",
       "      <td>0.083725</td>\n",
       "      <td>0.119577</td>\n",
       "      <td>-0.364421</td>\n",
       "      <td>0.105362</td>\n",
       "      <td>0.127887</td>\n",
       "      <td>-0.373712</td>\n",
       "      <td>0.211826</td>\n",
       "      <td>0.070774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.583685</td>\n",
       "      <td>-0.079217</td>\n",
       "      <td>0.054737</td>\n",
       "      <td>0.059975</td>\n",
       "      <td>-0.010330</td>\n",
       "      <td>0.337588</td>\n",
       "      <td>0.021809</td>\n",
       "      <td>0.260570</td>\n",
       "      <td>0.311055</td>\n",
       "      <td>0.110553</td>\n",
       "      <td>...</td>\n",
       "      <td>0.285020</td>\n",
       "      <td>-0.064470</td>\n",
       "      <td>0.004414</td>\n",
       "      <td>-0.108865</td>\n",
       "      <td>-0.219486</td>\n",
       "      <td>0.073666</td>\n",
       "      <td>0.223155</td>\n",
       "      <td>-0.064999</td>\n",
       "      <td>-0.214659</td>\n",
       "      <td>0.167897</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1         2         3         4         5         6   \\\n",
       "0 -0.034337 -0.274925  0.018084  0.113816  0.039164 -0.195245  0.276451   \n",
       "1  0.104875 -0.032112  0.104122  0.080557 -0.208968  0.229181  0.269978   \n",
       "2 -0.120334 -0.142576 -0.123378 -0.119650  0.037941  0.269308  0.092294   \n",
       "3 -0.127787  0.559799 -0.045335 -0.087937 -0.241207  0.152786  0.093145   \n",
       "4  0.583685 -0.079217  0.054737  0.059975 -0.010330  0.337588  0.021809   \n",
       "\n",
       "         7         8         9   ...        12        13        14        15  \\\n",
       "0 -0.131420  0.095024 -0.365595  ...  0.004114 -0.316165 -0.160454 -0.056422   \n",
       "1  0.124068 -0.015836 -0.057588  ...  0.107342 -0.047098 -0.115680 -0.090030   \n",
       "2 -0.222557 -0.203056  0.152577  ...  0.159274  0.244363  0.205827 -0.032077   \n",
       "3  0.239154 -0.016404  0.023806  ... -0.016499  0.225648  0.083725  0.119577   \n",
       "4  0.260570  0.311055  0.110553  ...  0.285020 -0.064470  0.004414 -0.108865   \n",
       "\n",
       "         16        17        18        19        20        21  \n",
       "0 -0.235143  0.025086  0.519633 -0.030565  0.070294  0.069894  \n",
       "1  0.518725 -0.369035 -0.244437 -0.242531 -0.151809  0.080475  \n",
       "2  0.205215  0.032955  0.264689  0.253524  0.295630  0.546305  \n",
       "3 -0.364421  0.105362  0.127887 -0.373712  0.211826  0.070774  \n",
       "4 -0.219486  0.073666  0.223155 -0.064999 -0.214659  0.167897  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(temp2).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spatial Filter (W)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "def spatial_filter(B, P):\n",
    "    '''\n",
    "    Will compute projection matrix using the following equation:\n",
    "    W = B' @ P\n",
    "    \n",
    "    INPUT:\n",
    "    B: the eigenvector either left or right class, choose one, size N x N, N is number of electrodes\n",
    "    P: white matrix in size of N x N \n",
    "    \n",
    "    OUTPUT:\n",
    "    W spatial filter to filter EEG\n",
    "    '''\n",
    "    \n",
    "    return (B.T@P)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Key to access data\n",
    "CSP = 'CSP'\n",
    "eigvec = 'eigvec'\n",
    "S_left = 'S_right'\n",
    "P = 'P'\n",
    "\n",
    "# Key to store variable\n",
    "W = 'W'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    temp_eigvec = mod_data[subj][CSP][S_left][eigvec]\n",
    "    temp_P = mod_data[subj][CSP][whitening][P]\n",
    "    \n",
    "    mod_data[subj][CSP][W] = spatial_filter(temp_eigvec, temp_P)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "temp = mod_data['subject01'][CSP][W]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.239296</td>\n",
       "      <td>0.462506</td>\n",
       "      <td>-1.043150</td>\n",
       "      <td>-2.249543</td>\n",
       "      <td>0.371913</td>\n",
       "      <td>0.205224</td>\n",
       "      <td>1.268590</td>\n",
       "      <td>3.171638</td>\n",
       "      <td>13.385910</td>\n",
       "      <td>1.462306</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.028542</td>\n",
       "      <td>-8.645908</td>\n",
       "      <td>-18.374046</td>\n",
       "      <td>2.587676</td>\n",
       "      <td>2.333038</td>\n",
       "      <td>1.267433</td>\n",
       "      <td>7.946352</td>\n",
       "      <td>1.410562</td>\n",
       "      <td>-1.748744</td>\n",
       "      <td>-0.990233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.598296</td>\n",
       "      <td>-8.562549</td>\n",
       "      <td>0.527477</td>\n",
       "      <td>-2.396045</td>\n",
       "      <td>-0.126731</td>\n",
       "      <td>0.333898</td>\n",
       "      <td>-6.560346</td>\n",
       "      <td>23.591195</td>\n",
       "      <td>5.045960</td>\n",
       "      <td>-6.706237</td>\n",
       "      <td>...</td>\n",
       "      <td>0.664940</td>\n",
       "      <td>-0.725840</td>\n",
       "      <td>-9.853490</td>\n",
       "      <td>-0.195212</td>\n",
       "      <td>0.676024</td>\n",
       "      <td>1.916427</td>\n",
       "      <td>-4.936273</td>\n",
       "      <td>5.639671</td>\n",
       "      <td>-2.293612</td>\n",
       "      <td>0.420784</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1.507999</td>\n",
       "      <td>-2.137315</td>\n",
       "      <td>12.414533</td>\n",
       "      <td>-7.762040</td>\n",
       "      <td>6.652715</td>\n",
       "      <td>-1.903582</td>\n",
       "      <td>2.985691</td>\n",
       "      <td>9.558595</td>\n",
       "      <td>-25.506480</td>\n",
       "      <td>9.277136</td>\n",
       "      <td>...</td>\n",
       "      <td>0.923863</td>\n",
       "      <td>2.334684</td>\n",
       "      <td>-4.772428</td>\n",
       "      <td>-3.249354</td>\n",
       "      <td>1.749295</td>\n",
       "      <td>6.002303</td>\n",
       "      <td>-5.682335</td>\n",
       "      <td>16.738734</td>\n",
       "      <td>-11.851948</td>\n",
       "      <td>0.627884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2.148876</td>\n",
       "      <td>-1.324807</td>\n",
       "      <td>6.214181</td>\n",
       "      <td>-5.832079</td>\n",
       "      <td>-3.060560</td>\n",
       "      <td>5.267160</td>\n",
       "      <td>-2.556745</td>\n",
       "      <td>-0.916345</td>\n",
       "      <td>-11.749503</td>\n",
       "      <td>2.648002</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.058330</td>\n",
       "      <td>5.699912</td>\n",
       "      <td>15.540547</td>\n",
       "      <td>0.484365</td>\n",
       "      <td>-1.584218</td>\n",
       "      <td>3.963987</td>\n",
       "      <td>-18.757887</td>\n",
       "      <td>3.231591</td>\n",
       "      <td>-3.681772</td>\n",
       "      <td>5.861262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.390341</td>\n",
       "      <td>4.904491</td>\n",
       "      <td>-8.471209</td>\n",
       "      <td>6.075540</td>\n",
       "      <td>3.375504</td>\n",
       "      <td>-3.852752</td>\n",
       "      <td>6.461842</td>\n",
       "      <td>-3.808187</td>\n",
       "      <td>-4.553312</td>\n",
       "      <td>-5.072799</td>\n",
       "      <td>...</td>\n",
       "      <td>-1.680322</td>\n",
       "      <td>-6.517700</td>\n",
       "      <td>21.676512</td>\n",
       "      <td>3.659052</td>\n",
       "      <td>2.432170</td>\n",
       "      <td>-6.942282</td>\n",
       "      <td>-5.498651</td>\n",
       "      <td>-13.744784</td>\n",
       "      <td>10.434427</td>\n",
       "      <td>1.164731</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         0         1          2         3         4         5         6   \\\n",
       "0 -0.239296  0.462506  -1.043150 -2.249543  0.371913  0.205224  1.268590   \n",
       "1  1.598296 -8.562549   0.527477 -2.396045 -0.126731  0.333898 -6.560346   \n",
       "2 -1.507999 -2.137315  12.414533 -7.762040  6.652715 -1.903582  2.985691   \n",
       "3  2.148876 -1.324807   6.214181 -5.832079 -3.060560  5.267160 -2.556745   \n",
       "4  0.390341  4.904491  -8.471209  6.075540  3.375504 -3.852752  6.461842   \n",
       "\n",
       "          7          8         9   ...        12        13         14  \\\n",
       "0   3.171638  13.385910  1.462306  ... -0.028542 -8.645908 -18.374046   \n",
       "1  23.591195   5.045960 -6.706237  ...  0.664940 -0.725840  -9.853490   \n",
       "2   9.558595 -25.506480  9.277136  ...  0.923863  2.334684  -4.772428   \n",
       "3  -0.916345 -11.749503  2.648002  ... -0.058330  5.699912  15.540547   \n",
       "4  -3.808187  -4.553312 -5.072799  ... -1.680322 -6.517700  21.676512   \n",
       "\n",
       "         15        16        17         18         19         20        21  \n",
       "0  2.587676  2.333038  1.267433   7.946352   1.410562  -1.748744 -0.990233  \n",
       "1 -0.195212  0.676024  1.916427  -4.936273   5.639671  -2.293612  0.420784  \n",
       "2 -3.249354  1.749295  6.002303  -5.682335  16.738734 -11.851948  0.627884  \n",
       "3  0.484365 -1.584218  3.963987 -18.757887   3.231591  -3.681772  5.861262  \n",
       "4  3.659052  2.432170 -6.942282  -5.498651 -13.744784  10.434427  1.164731  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(temp).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Vector Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create new keys for trainning and test feature vector\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    mod_data[subj]['train']= {}\n",
    "    mod_data[subj]['test']= {}       "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select number of filter\n",
    "m = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_Z(W, E, m):\n",
    "    '''\n",
    "    Will compute the Z\n",
    "    Z = W @ E, \n",
    "    \n",
    "    E is in the shape of N x M, N is number of electrodes, M is sample\n",
    "    In application, E has nth trial, so there will be n numbers of Z\n",
    "    \n",
    "    Z, in each trial will have dimension of m x M, \n",
    "    where m is the first and last m rows of W, corresponds to smallest and largest eigenvalues\n",
    "    '''\n",
    "    Z = []\n",
    "    \n",
    "    W = np.delete(W, np.s_[m:-m:], 0)\n",
    "    \n",
    "    for i in range(E.shape[0]):\n",
    "        Z.append(W @ E[i])\n",
    "    \n",
    "    return np.array(Z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "def feat_vector(Z):\n",
    "    '''\n",
    "    Will compute the feature vector of Z matrix\n",
    "    \n",
    "    INPUT:\n",
    "    Z : projected EEG shape of T x N x S\n",
    "    \n",
    "    OUTPUT:\n",
    "    feat : feature vector shape of T x m\n",
    "    \n",
    "    T = trial\n",
    "    N = channel\n",
    "    S = sample\n",
    "    m = number of filter\n",
    "    '''\n",
    "    \n",
    "    feat = []\n",
    "    \n",
    "    for i in range(Z.shape[0]):\n",
    "        var = np.var(Z[i], ddof=1, axis=1)\n",
    "        varsum = np.sum(var)\n",
    "        \n",
    "        feat.append(np.log10(var/varsum))\n",
    "        \n",
    "    return np.array(feat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Where to access data\n",
    "EEG_left_train = 'EEG_left_train'\n",
    "EEG_right_train = 'EEG_right_train'\n",
    "W = 'W'\n",
    "\n",
    "# Where to store data\n",
    "train = 'train'\n",
    "Z_left = 'Z_left'\n",
    "Z_right = 'Z_right'\n",
    "feat_left = 'feat_left'\n",
    "feat_right = 'feat_right'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    temp_W = mod_data[subj][CSP][W]\n",
    "    temp_EEG_left = mod_data[subj][EEG_left_train]\n",
    "    temp_EEG_right = mod_data[subj][EEG_right_train]\n",
    "    \n",
    "    # LEFT\n",
    "    mod_data[subj][train][Z_left] = compute_Z(temp_W, temp_EEG_left, m)\n",
    "    mod_data[subj][train][feat_left] = feat_vector(mod_data[subj][train][Z_left])\n",
    "\n",
    "    \n",
    "    # RIGHT\n",
    "    mod_data[subj][train][Z_right] = compute_Z(temp_W, temp_EEG_right, m)\n",
    "    mod_data[subj][train][feat_right] = feat_vector(mod_data[subj][train][Z_right])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## True label for each class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Key to access data\n",
    "train = 'train'\n",
    "feat_left = 'feat_left'\n",
    "feat_right = 'feat_right'\n",
    "\n",
    "# Key to store data\n",
    "feat_train = 'feat_train'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    temp_feat_left = mod_data[subj][train][feat_left]\n",
    "    temp_feat_right = mod_data[subj][train][feat_right]\n",
    "    \n",
    "    left_label = np.ones([len(temp_feat_left), 1])*-1\n",
    "    right_label = np.ones([len(temp_feat_right), 1])    \n",
    "    \n",
    "    left  = np.concatenate([temp_feat_left, left_label], axis=1)\n",
    "    right  = np.concatenate([temp_feat_right, right_label], axis=1)    \n",
    "    \n",
    "    mod_data[subj][train][feat_train] = np.vstack([left, right])\n",
    "    \n",
    "    np.random.shuffle(mod_data[subj][train][feat_train])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.487668</td>\n",
       "      <td>-0.344725</td>\n",
       "      <td>-0.788350</td>\n",
       "      <td>-1.223858</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.703536</td>\n",
       "      <td>-0.776090</td>\n",
       "      <td>-0.575813</td>\n",
       "      <td>-0.432906</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.618249</td>\n",
       "      <td>-0.656897</td>\n",
       "      <td>-0.696321</td>\n",
       "      <td>-0.471624</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.894126</td>\n",
       "      <td>-0.949117</td>\n",
       "      <td>-0.537532</td>\n",
       "      <td>-0.327980</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.523900</td>\n",
       "      <td>-0.833503</td>\n",
       "      <td>-0.760054</td>\n",
       "      <td>-0.419961</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0         1         2         3    4\n",
       "0 -0.487668 -0.344725 -0.788350 -1.223858 -1.0\n",
       "1 -0.703536 -0.776090 -0.575813 -0.432906  1.0\n",
       "2 -0.618249 -0.656897 -0.696321 -0.471624  1.0\n",
       "3 -0.894126 -0.949117 -0.537532 -0.327980  1.0\n",
       "4 -0.523900 -0.833503 -0.760054 -0.419961  1.0"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(mod_data[subj][train][feat_train]).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Vector Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Where to access data\n",
    "EEG_left_test = 'EEG_left_test'\n",
    "EEG_right_test = 'EEG_right_test'\n",
    "W = 'W'\n",
    "\n",
    "# Where to store data\n",
    "test = 'test'\n",
    "feat_test = 'feat_test'\n",
    "Z_left = 'Z_left'\n",
    "Z_right = 'Z_right'\n",
    "feat_left = 'feat_left'\n",
    "feat_right = 'feat_right'\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    temp_W = mod_data[subj][CSP][W]\n",
    "    temp_EEG_left = mod_data[subj][EEG_left_test]\n",
    "    temp_EEG_right = mod_data[subj][EEG_right_test]\n",
    "    \n",
    "    # LEFT\n",
    "    mod_data[subj][test][Z_left] = compute_Z(temp_W, temp_EEG_left, m)\n",
    "    mod_data[subj][test][feat_left] = feat_vector(mod_data[subj][test][Z_left])\n",
    "    temp_feat_left = mod_data[subj][test][feat_left]\n",
    "    \n",
    "    # RIGHT\n",
    "    mod_data[subj][test][Z_right] = compute_Z(temp_W, temp_EEG_right, m)\n",
    "    mod_data[subj][test][feat_right] = feat_vector(mod_data[subj][test][Z_right])\n",
    "    temp_feat_right = mod_data[subj][test][feat_right]\n",
    "    \n",
    "    \n",
    "    # True label for test data\n",
    "    left_label = np.ones([len(temp_feat_left), 1])*-1\n",
    "    right_label = np.ones([len(temp_feat_left), 1])\n",
    "    \n",
    "    \n",
    "    # LEFT + RIGHT\n",
    "    mod_data[subj][test][feat_test] = np.vstack([np.c_[temp_feat_left, left_label], np.c_[temp_feat_right, right_label]])\n",
    "    \n",
    "    np.random.shuffle(mod_data[subj][test][feat_test])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.768477</td>\n",
       "      <td>-0.674261</td>\n",
       "      <td>-0.514459</td>\n",
       "      <td>-0.505850</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.498002</td>\n",
       "      <td>-0.425992</td>\n",
       "      <td>-0.727665</td>\n",
       "      <td>-0.920377</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.416546</td>\n",
       "      <td>-0.374797</td>\n",
       "      <td>-0.886235</td>\n",
       "      <td>-1.187523</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.771616</td>\n",
       "      <td>-0.767140</td>\n",
       "      <td>-0.489748</td>\n",
       "      <td>-0.473559</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.467475</td>\n",
       "      <td>-0.584639</td>\n",
       "      <td>-0.668011</td>\n",
       "      <td>-0.734780</td>\n",
       "      <td>-1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0         1         2         3    4\n",
       "0 -0.768477 -0.674261 -0.514459 -0.505850  1.0\n",
       "1 -0.498002 -0.425992 -0.727665 -0.920377 -1.0\n",
       "2 -0.416546 -0.374797 -0.886235 -1.187523 -1.0\n",
       "3 -0.771616 -0.767140 -0.489748 -0.473559  1.0\n",
       "4 -0.467475 -0.584639 -0.668011 -0.734780 -1.0"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe(mod_data[subj][test][feat_test]).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classifying"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using SVM as Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.metrics import classification_report, accuracy_score\n",
    "from sklearn.model_selection import cross_val_score, GridSearchCV, KFold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define model\n",
    "model = SVC()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 1.0,\n",
       " 'break_ties': False,\n",
       " 'cache_size': 200,\n",
       " 'class_weight': None,\n",
       " 'coef0': 0.0,\n",
       " 'decision_function_shape': 'ovr',\n",
       " 'degree': 3,\n",
       " 'gamma': 'scale',\n",
       " 'kernel': 'rbf',\n",
       " 'max_iter': -1,\n",
       " 'probability': False,\n",
       " 'random_state': None,\n",
       " 'shrinking': True,\n",
       " 'tol': 0.001,\n",
       " 'verbose': False}"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate Model Performace on Train Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing for subject01\n",
      "Processing for subject02\n",
      "Processing for subject03\n",
      "Processing for subject04\n",
      "Processing for subject05\n",
      "Processing for subject06\n",
      "Processing for subject07\n",
      "Processing for subject08\n",
      "Processing for subject09\n"
     ]
    }
   ],
   "source": [
    "# Iterate over each subject\n",
    "model_acc = []\n",
    "model_std = []\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    print('Processing for {}'.format(subj))\n",
    "    X_train = mod_data[subj]['train']['feat_train'][:, :-1]\n",
    "    y_train = mod_data[subj]['train']['feat_train'][:, -1]\n",
    "    \n",
    "    model_acc.append(cross_val_score(model, X_train, y_train).mean()*100)\n",
    "    model_std.append(cross_val_score(model, X_train, y_train).std()*100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Necessary variable to label x axis\n",
    "subject=[]\n",
    "for i in range(1, ns):\n",
    "    subject.append(subjectCounter(i))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcMAAAFUCAYAAABY5VclAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAApY0lEQVR4nO3de5QcZZ3/8feHBB1JGCFGUQfICAYjiQZhFEURdqMiuiISdYMg4Crxpz9YUbzwO8tluB0vxwsooERREBRDNFxEV1wUXJEVCesmOjBCsgwGDBIkhNwmXPz+/nhqtNOZme7QPV01XZ/XOX2Srqqu+nT1TH/neeqpKkUEZmZmZbZd3gHMzMzy5mJoZmal52JoZmal52JoZmal52JoZmal52JoZmal52JouZJ0sKTfS3pC0s11vuY4SU/WWKZbUkh6XVOCthFJA5JOrXh+s6Rv1HhNr6TlTdh2KT+XZu0/Gzsuhtaw7Bc9hnm8uI6XfxX4b2AP4IixTbo1STtIOlXSMkkbJT0i6TZJJ0raodV5RiLpo5I2SZoywvwfSrrlaa7+COBjTz/d8CQtl9RbNXkl8ALgtmZvb5jtV/5c/lXSKklXS3rpWG+7jmynShrIO4f9nYuhNcsA6Uuu8nFvHa+bDvxHRKyMiEfGLt7WJHUCvwJOBC4EDgD2Az4PvBt40wiv216SWpUzc1n273uHydMFHAoseDorjohHIuKxBrJty7aeiogHI+KJVmyPv/9cdgFvB3YGfizpGS3avo0TLobWLENfcpWPp0ZaOOseDWAC8O3sr/fjsnmvlvSfWUtojaTvSnreaBuX9O6sJTIo6Vbg5XVkPheYAbw6Ii6OiP+JiHsjYhHweuDmbN2XSroxay0OAJuBSZJeIulHktZnjx9WtoYldUr6lqQHJW2WtFLSFyvmv07SryStyx5LJR0yXNDsD4XvA8cPM/tfgHXAIklvzLo9H5G0VtIvJL2qxr7boptU0jMlfTV7/RpJXwWeWfWafSX9u6SHsvd+u6Q3V64T2BM4o6J11j1cN2kd+/E4SU9Keq2k/85a8LdL2m+095UZ+rlcFRG/Ab4AdAMvqVj/iyX9QNKj2fv9qaSXVcyv9Tlu1c08Wssv+zk/G5hWsW9663gvNoZcDK1ZdpV0f/b4d0kH1Fj+VtJf7AAnZP9fKOn5wE+B+4FXAW8DZgE/GGlFkl4BfA9YBMwmtezOH23jkrYD3gN8JyK2asFG8mjFpFcB/wgcnm1DWc4O4KDsMRn4SUWr4xxgX1KLZDrwz8Bd2fYnANeRugv3zR69wMZRYl8MzJT0mqr38X7g8ojYlGW4EHg1qaV7T5bpOaPtjyqfAeYCxwCvATYA/7dqmU7SPj84y34DcJ2kvbL5R5BaZV/g7z0FK6s3JOlZ1N6PkL6rPg18JNveGuAqSRPrfVNKXcxHZ08fz6btAtwCPAQcSNpvfwBulvTcbNkRP8enaSHwWdLP+NC++XwD67NmiAg//GjoQeqiezepNXYg8F3gKeCNdbw2gKMrnp9N+pJ4RsW02dlyr8+eHwc8WTH/CuDWqvWekL3mdSNs93nZ/I/VkfFS4FFgcsW095MK19SKabsAm4BjsufXApeOsM6ds+0fvI37ug/4ZsXzQ7L1zBph+e1IheOoimkDwKkVz28GvpH9fxIwCBxftZ4lwPIa2ZYC/1bxfDnQW7VMd+XnUud+PC57zb4Vy7w6m/aSUfL0An8F1pMKemSP71ct8+uq1wlYAZxU63Os3n8V004FBqq2s3yk+X7k/3DL0BoWEf8eEVdFxLKI+GVEvIf01/YnACTtXtEFtl7S10ZZ3UzSl9PjFetfCqzN5g1nb9Kxv0q1BpMMHfOr90r1d0XE+qqcd0bEwxU5/0xqVQzlvAh4p9Jo2fMlHZq15IiINcA3gBuylvQpkl5CbQuAf1Y63gmp2/S/IuL3AJJeJOnyrMv4MeAx4NnAtDrf556kLtFbq6ZvsT8lPVfSRZL6s+7F9dn7rnc7Q+rZj5A+p6UVzx/I/t2lxvpXAvsAPcC/Av3AhyrmvxLYr/Lnk9Tl3E1qBcIon6O1j7q7GMy20X/x99GhfyJ9IQ2pNVhjpAI10nSNMm8kq0ktppEKbLUNdeb5W5aIuEHS7qTW28GkFuzvJM2JNJDkeEnnkwbqvBE4W9IJEXHxKDm+TeouPErSYuAwYH7F/OuBh0ndmitJ3YG3APUOGKn3j4RLgd2BT5IGSm0idZs+nYEpo+7HzF9jy2PQQ/NqFaUnImLolIa7lAYbLSR1eQ+9/meknoRqa6H250hqfVYPqNq+Ri4rGP91Y2PlFWTHiCLiyYhYXvF4aJTX9QGvqTxeJGk2qXXTN8prXls1rfr5FiLir6Tu3KMkvah6vpJn18g5U9LUitfsAuxVmTPSSM0rI+KDwFtJx8T2rpj/+4j4YkQcClzCloVtuNxrSMdGjyd1H24Ersq2/5xs3Z+JiBsi4k5Sl+eog4+qLCcV0Or9V30M+PXARRFxXUT8DlhFOj2m0uOkAVKjqWs/NtHngFdJmps9X0L6g+iBqp/R5RGxeuhFNT7Hh4AXVm1n3xo56tk31kIuhtYwSV+U9I+S9pC0j6QLSS2d857G6i4gDc64VNKsbNTh5cAtEfHLEV7zJVIBPVfSXpLeAZxcx7b+jTTA5NeS5kuanXUzvgP4BfAPo7z2u6TW5UKlkZX7kVpGD5BaHmR5jshGS04HjiIdv/pjNoLxs0ojSqdlg2IOBO6sI/fFpD82PgVcERFDg27WZJmOz/bDa4ArSa22ukTEBuBrwDmSDsuyf4406rbSH0h/SLxM0j7Zdqq/3O8FXpt1k08doWux5n5spkijci8hvb8JpJ+3CcA1kg5UGu36uuyzOwBG/xyz1d4IvEFpRPOLJZ1C+ixHcy/wfEmvyfZNYc5pLSsXQ2uGF5C67+4ijQx8CfCGiPjhtq4oO170JmBX4HZSt9/vSaMbR3rNHaSRofOA3wGnAB+tY1trSaMlLyIdT/o16QIAp5C+iG8Y5bWbspybgf8kFc8NwJsrjncOAmcBd5BaIC8HDs22u4F0TOp7wN2k0bK3Mnx3XfW2byXtk52pOLcwa+2+i3TcbxmpK/M8UqttW5wCXEP6I+Q3wE6kEaqV3kf6/vhNtuxPSJ9XpTNILfo/kAre7sO8l3r2Y7N9EXgxcFz28/YaUtfy4izrd0jHPof222ifI6RzQC8kFdYlwG7Al2tkuIbUwv8Rad98sgnvyxqgCN/p3szMys0tQzMzKz0XQzMzKz0XQzMzKz0XQzMzKz0XQzMzK722vQLN1KlTo7u7O+8YZmZWIHfcccfDEfHc6ultWwy7u7tZsmRJ3jHMzKxAJN033HR3k5qZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZWem5GJqZjSO9vb1Iqvno7e3NO+q40rJiKOkESUskbZZ0adW8OZL6JW2UdJOkaRXzJOmzkv6SPT4nSa3KbWZWJL29vUTE3x4HHXQQBx100BbTIsLFcBu1smX4J+Ac4JuVEyVNBRYDpwFTgCXAwopF5gOHA7OBlwP/BHxw7ONaHvxXr5nloWXFMCIWR8Q1wF+qZh0B9EXEoogYBHqB2ZJmZPOPBb4QEfdHxAPAF4DjWpPaWs1/9ZpZHopwP8OZwNKhJxGxQdKKbHp/9fzs/zOHW5Gk+aSWJF1dXQwMDIxRZGuVwcFBAH+WZiPw70hzFKEYTgZWV01bC+xYMX9t1bzJkhQRUfmiiFgALADo6enxne7bQEdHB5Bu1mxmW/PvSHMUYTTpeqCzalonsG6E+Z3A+upCaGZm9nQVoRj2kQbHACBpErBnNn2r+dn/+zAzM2uSVp5aMVFSBzABmCCpQ9JE4GpglqS52fzTgWUR0Z+99NvAxyR1SXohcDJwaatym5lZ+2vlMcNTgTMqnh8NnBkRvZLmAhcAVwC3AfMqlrsY2AP4Xfb8G9k0M7P2cN3cp//av/Q1vg6Aw37Q2OvHuZYVw4joJZ02Mdy8G4EZI8wL4JPZw8zMrOmKMJrU2l3ef/WW/C9eM6utCANozMzMcuViaGZmpediaGZmpediaGZmpediaGZmpediaGZmpedTK8zMxpHeK+/kzO/1bzVdb1+8xfMz5s2g98i9WxVr3HMxtELxL7rZ6HqP3Ns/+2PAxbCJent7OfPMM2sud8YZZ/jmtCPwL7qZ5cHFsIl6e3u3KHIHH3wwADfffHMueczMrD4eQGNmZqXnYmhmLdXb24ukmg8fSrBWcjdpLb7ItFlT+XCCFZFbhmZmVnqFKYaSXirp55LWSlou6R0V8+ZI6pe0UdJNkqblmdXMzNpLIYqhpInAtcD1wBRgPnCFpL0kTQUWA6dl85YAC/PKamZm7acoxwxnAC8EvpTd2f7nkn4FvBdYCfRFxCIASb3Aw5JmRMTWZ2ebmZlto6IUQ40wbRbQCSwdmhgRGyStAGYChSqGvnqKmdn4VJRi2A88BHxC0peAfwAOAm4CJgOrq5ZfC+xYvRJJ80ldrHR1dTEwMNB4sphe96LHzZvOcfPeXteyA1HnSpvxHvK2DftwTLTDPmxjg4ODAM35fW2C8847j/PPP7/mch/5yEc46aSTmrPRvH9HoPS/J4UohhHxhKTDga8AnyIdF7wK2AysJ7UOK3UC64ZZzwJgAUBPT090d3c3Hm7ZPY2voxHNeA95a6N96EvuNV9HRwcATfl9bYLzzjuP884772/PW3LqR96/I9Ae3zUNKEQxBIiIZaTWIACSbgUuAwI4tmL6JGBPoK/VGc18jpxZeyrEaFIASS+X1CFpB0kfB14AXApcDcySNFdSB3A6sMyDZ8zMrFkKUwxJI0dXkY4dzgHeGBGbI2I1MBc4F1gD7A/Myy2lmZm1nSJ1k34C+MQI824knX5hZmbWdEVqGZqZmeXCxdDMzErPxdDMzErPxdDMzErPxdDMzJpqPN7AuTCjSc3MrD2Mx4tTuGVoZmal52JoZmal52JoZmal52JoZmal5wE0Zta46+Y+/df+pa/xdRz2g9HnFz2f5c4tQzMzKz0XQzMzKz0XQ7M2Mh5PdjYrAh8zNGsj4/FkZxsn2vy4a2FahpK6Jf1Y0hpJD0q6QNLEbN4cSf2SNkq6SdK0vPOOR241mJkNr0gtw4tId7l/AbAT8B/AhyV9F1gMfAD4IXA2sBB4dT4xxy+3GkbQ5n/xmllthWkZAi8CroqIwYh4EPgJMBM4AuiLiEURMQj0ArMlzcgvqpmZtZMiFcPzgXmSdpDUBRzK3wvi0qGFImIDsCKbbmZm1rAidZP+AjgeeAyYAFwGXAP8E7C6atm1wI7VK5A0H5gP0NXVxcDAQOOpYnrj62hEM97DCAYHB7NNjN02gOLvwwbyDcbtaRONvMeSf8Yt2YdtnK9pxvs+bFAhiqGk7YAbgIuBA4DJwDeBzwLrgc6ql3QC66rXExELgAUAPT090d3d3Xi4Zfc0vo5GNOM9jKCjoyPbxNhtAyj+PmwgX4c2pU2ogfdY8s+4JfuwjfM1zXjfhw0qRDEEpgC7ARdExGZgs6RvAecAXwaOHVpQ0iRgT6Avj6BmLZf3AB/wIB9re4U4ZhgRDwP3Ah+SNFHSTqQCuBS4Gpglaa6kDuB0YFlE9OcW2MzM2kohimHmCODNpOODy4EngY9GxGpgLnAusAbYH5iXV0gzM2s/RekmJSL+Bzh4hHk3Aj6VwszMxkRhiqGZmbWH3ivv5MzvbX0kS29fvMXzM+bNoPfIvVsVa1QuhmZm1lS9R+5dmCJXLxdDM7MK47FVY41zMTRrI/4ib9x4bNVY41wMzdqIv8jNnp4inVphZmaWCxdDMzMrPRdDMzMrPR8zNLOW8iAfKyIXw/Eu74s4+wLOto08yMeKyN2kZmZWei6GZmZWei6GZmZWej5maLYNPPjDrD25GJptAw/+MGtP7iY1M7PSK0QxlLS+6vGUpK9UzJ8jqV/SRkk3SZqWZ14zM2svhSiGETF56AHsAmwCFgFImgosBk4DpgBLgIV5ZTUzs/ZTiGJY5Z3AQ8Avs+dHAH0RsSgiBoFeYLakGTnlMzOzNlPEYngs8O2IiOz5TGDp0MyI2ACsyKabmZk1rFCjSSXtDhwEvL9i8mRgddWia4Edh3n9fGA+QFdXFwMDA42HiumNr6MRtd5DA/kG4/a0iUbeYz37uI33YVMUPR8UP6PzNa7oGZvxfT6KQhVD4Bjgloi4t2LaeqCzarlOYF31iyNiAbAAoKenJ7q7uxtPtOyextfRiFrvoYF8HdqUNqEG3mM9+7iN92FTFD0fFD+j8zWu6Bmb8X0+iqJ1kx4DXFY1rQ+YPfRE0iRgz2y6mZlZwwpTDCUdAHSRjSKtcDUwS9JcSR3A6cCyiNj6MiBmZmZPQ2GKIWngzOKI2KL7MyJWA3OBc4E1wP7AvNbHMzOzdlWYY4YR8cFR5t0I+FQKMzMbE0VqGZqZmeXCxdDMzErPxdDMzEqvMMcMbez5XnxmZsNzMSwR34vPzGx47iY1M7PSczE0M7PSczE0M7PSczE0M7PSczE0M7PSq6sYSnr5WAcxMzPLS70tw59JWirp45JeMKaJzMzMWqzeYvgC0q2T9gfukfRTSUdL2mHsopmZmbVGXcUwIp6MiGsj4l2kew5eBXwS+LOkb0t67ViGNDMzG0vbNIBG0mTgcNL9BHcFvgfcA3xH0oVNT2dmZtYC9Q6geauk7wEPAP8MfAN4YUQcHxFnA/uSbs7bEEnzJN0laYOkFZIOzKbPkdQvaaOkmyRNa3RbZmZmQ+ptGX4GuAOYERFviYjvRcTg0MyIeAQ4qZEgkt4IfBZ4H7Aj8HrgfyVNBRYDpwFTgCXAwka2ZWZmVqmuC3VHxMvqWOYbDWY5EzgrIn6dPX8AQNJ8oC8iFmXPe4GHJc2IiK1vwWBmZraN6u0mXTzUZVkx7UBJ329GCEkTgB7guZKWS7pf0gWSngXMBJYOLRsRG4AV2XQzM7OG1XsLp4OAd1VN+y/gmibl2AXYHngncCDwBHAtcCowGVhdtfxaUlfqFrJW5HyArq4uBgYGGk8W0xtfRyNqvYei54PiZ3S+2oqe0fkaV/SMzfg+H0W9xXAQmAQ8VjFtMqloNcOm7N+vRMQqAElfJBXD/wQ6q5bvBNZVryQiFgALAHp6eqK7u7vxZMvuaXwdjaj1HoqeD4qf0flqK3pG52tc0TM24/t8FPUOoLkBuFhSJ0D27wXAT5oRIiLWAPcDMczsPmD20BNJk4A9s+lmZmYNq7cYnkxqjT0i6SHgEeDZNDiCtMq3gBMlPU/Sztm6rweuBmZJmiupg3QlnGUePGNmZs1S72jSNcBbs+uS7gqsjIgHm5zlbGAqcDepW/Yq4NyIGJQ0l9QSvQK4jXTSv5mZWVPUe8wQgIhYJelBQJK2y6b9tRlBIuIJ4MPZo3rejcCMZmzHzMysWr2nVrxQ0tWS/gI8SRo4M/QwMzMb1+o9Zngx8DgwB1hPuvzadcD/GaNcZmZmLVNvN+kBwO4RsUFSRMRSSe8HbgW+PnbxzMzMxl69LcOnSN2jAI9Kei6wgXQ7JzMzs3Gt3mJ4G/CW7P83kC6UvZh00WwzM7Nxrd5u0vfy98J5Eum8wx2B85ofyczMrLVqFsPsItrnk13zMyI2AeeMcS4zM7OWqdlNGhFPAW8CmnI+oZmZWdHUe8zwS8CZkrYfyzBmZmZ5qPeY4YnA84GPSVpNxQW1I2L3sQhmZmbWKvUWw6PHNIWZmVmO6r1Q9y/GOoiZmVle6iqGks4aaV5EnN68OGZmZq1XbzfpblXPnw8cRLrXoJmZ2bhWbzfp+6qnSXozcGTTE5mZmbVYvadWDOenwOFNymFmZpabeu9nuEfVYxbpKjQrmxVE0s2SBiWtzx5/qJg3R1K/pI2SbpI0rVnbNTMzq7dluBy4J/t3OfBr4EDg2CbnOSEiJmePlwBImkq6KPhpwBTSxcEXNnm7ZmZWYvUeM2ykO7VRRwB9EbEIQFIv8LCkGRHRn2MuMzNrE/V2k+4jabeqabtJmt3kPJ+W9LCkX0k6OJs2E1g6tEBEbABWZNPNzMwaVu+pFVcAh1VNewZwOfDyJmX5FHAn8DgwD/ihpH2AycDqqmXXkm4htQVJ88nurtHV1cXAwEDjqWJ64+toRK33UPR8UPyMzldb0TM6X+OKnrEZ3+ejqLcY7h4R/1s5ISJWSOpuVpCIuK3i6WWSjiTdUHg90Fm1eCewbph1LAAWAPT09ER3dxPiLbun8XU0otZ7KHo+KH5G56ut6Bmdr3FFz9iM7/NR1Hss8H5J+1ZOyJ7/qfmR/iYAAX3A37pjJU0C9symm5mZNWxbbuF0raQTJb1F0omkq898sRkhJO0k6RBJHZImSjoKeD1wQ7adWZLmSuoATgeWefCMmZk1S72jSb8u6VHg/aRLs60ETo6I7zcpx/ak8xZnAE8B/cDhEfEHAElzgQtIxy5vIx1TNDMza4p6jxmSndqwaCxCRMRq4JWjzL+RVCjNzMyart5TK74s6YCqaQdIOm9MUpmZmbVQvccMjyRd+aXSHcB7mhvHzMys9eothjHMshO24fVmZmaFVW8x+yVwjqTtALJ/z8ymm5mZjWv1DqD5CHA9sErSfcA00jmGbxurYGZmZq1S76kVQyfdv4p0asWfSfcy/A3wwjFLZ2Zm1gJ1n1oBPAfYHziOdD3SX5JajGZmZuPaqMVQ0vakC3QfBxxCupfhlcDuwLsj4qGxDmhmZjbWag2g+TNwMfAH4NURsXdEnE26s4SZmVlbqFUMlwE7kbpHXylp5zFPZGZm1mKjFsOIOJh0h4ifAh8HHpT0Q2AS6XqiZmZm417N8wwj4r6IODsipgNzgFXAX4Glkj431gHNzMzG2jZdQSYibomI+cDzgROBl41JKjMzsxZ6WpdTi4jBiLgyIg5tdiAzM7NW87VFzcys9ApVDCVNlzQo6YqKaXMk9UvaKOkmSdPyzGhmZu2nUMUQuBC4feiJpKnAYuA0YArpNlIL84lmZmbtqjDFUNI84FHgZxWTjwD6ImJRRAwCvcBsSb7rvZmZNU0hiqGkTuAs4OSqWTOBpUNPImIDsCKbbmZm1hTbcqHusXQ2cElErJRUOX0ysLpq2bXAjsOtRNJ8YD5AV1cXAwMDjSeL6Y2voxG13kPR80HxMzpfbUXP6HyNK3rGZnyfjyL3YihpH+ANwCuGmb0e6Kya1gmsG25dEbEAWADQ09MT3d3djQdcdk/j62hErfdQ9HxQ/IzOV1vRMzpf44qesRnf56PIvRgCBwPdwB+zVuFkYIKkvYGvAccOLShpEunycH0tT2lmZm2rCMcMF5AK3D7Z42vAj0i3jLoamCVprqQO4HRgWUT05xPVzMzaUe4tw4jYCGwcei5pPTAYEauz53OBC4ArgNuAeXnkNDOz9pV7MawWEb1Vz28EfCqFmZmNmSJ0k5qZmeXKxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzErPxdDMzEqvMMVQ0hWSVkl6TNLdkj5QMW+OpH5JGyXdJGlanlnNzKy9FKYYAp8GuiOiEzgMOEfSfpKmAouB04ApwBJgYX4xzcys3UzMO8CQiOirfJo99gT2A/oiYhGApF7gYUkzIqK/5UHNzKztFKYYAki6CDgOeBbwW+DHwLnA0qFlImKDpBXATKC/6vXzgfkAXV1dDAwMNB4qpje+jkbUeg9FzwfFz+h8tRU9o/M1rugZm/F9PopCFcOI+LCkE4HXAAcDm4HJwOqqRdcCOw7z+gXAAoCenp7o7u5uPNSyexpfRyNqvYei54PiZ3S+2oqe0fkaV/SMzfg+H0WRjhkCEBFPRcQtwK7Ah4D1QGfVYp3AulZnMzOz9lS4YlhhIumYYR8we2iipEkV083MzBpWiGIo6XmS5kmaLGmCpEOAI4GfA1cDsyTNldQBnA4s8+AZMzNrlkIUQ9LI0Q8B9wNrgM8DJ0XEtRGxGphLGkizBtgfmJdXUDMzaz+FGECTFbyDRpl/IzCjdYnMzKxMitIyNDMzy42LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlZ6LoZmZlV4hiqGkZ0q6RNJ9ktZJ+q2kQyvmz5HUL2mjpJskTcszr5mZtZdCFEPSTYZXkm7w+2zgNOAqSd2SpgKLs2lTgCXAwryCmplZ+ynKne43AL0Vk66XdC+wH/AcoC8iFgFI6gUeljQjIvpbndXMzNpPUVqGW5C0C7AX0AfMBJYOzcsK54psupmZWcMK0TKsJGl74DvAZRHRL2kysLpqsbXAjsO8dj4wH6Crq4uBgYHGA8X0xtfRiFrvoej5oPgZna+2omd0vsYVPWMzvs9HUahiKGk74HLgceCEbPJ6oLNq0U5gXfXrI2IBsACgp6cnuru7Gw+17J7G19GIWu+h6Pmg+Bmdr7aiZ3S+xhU9YzO+z0dRmG5SSQIuAXYB5kbEE9msPmB2xXKTgD2z6WZmZg0rTDEEvgq8FHhbRGyqmH41MEvSXEkdwOnAMg+eMTOzZilEMczOG/wgsA/woKT12eOoiFgNzAXOBdYA+wPzcgtrZmZtpxDHDCPiPkCjzL8RmNG6RGZmViaFaBmamZnlycXQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKz8XQzMxKrxDFUNIJkpZI2izp0qp5cyT1S9oo6SZJ03KKaWZmbaoQxRD4E3AO8M3KiZKmAouB04ApwBJgYcvTmZlZW5uYdwCAiFgMIKkH2LVi1hFAX0Qsyub3Ag9LmhER/S0PamZmbakQxXAUM4GlQ08iYoOkFdn0rYqhpPnAfICuri4GBgYaTxDTG19HI2q9h6Lng+JndL7aip7R+RpX9IzN+D4fRdGL4WRgddW0tcCOwy0cEQuABQA9PT3R3d3deIJl9zS+jkbUeg9FzwfFz+h8tRU9o/M1rugZm/F9PoqiHDMcyXqgs2paJ7AuhyxmZtamil4M+4DZQ08kTQL2zKabmZk1RSGKoaSJkjqACcAESR2SJgJXA7Mkzc3mnw4s8+AZMzNrpkIUQ+BUYBNwCnB09v9TI2I1MBc4F1gD7A/MyyukmZm1p0IMoImIXqB3hHk3AjNamcfMzMqlKC1DMzOz3LgYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6Y2LYihpiqSrJW2QdJ+k9+SdyczM2kchbu5bhwuBx4FdgH2AH0laGhF9uaYyM7O2UPiWoaRJwFzgtIhYHxG3ANcB7803mZmZtYvCF0NgL+CpiLi7YtpSYGZOeczMrM2Mh27SycDaqmlrgR2rF5Q0H5ifPV0v6Q9jnK0eU4GHn/7L1bQgIyh6Pih+xjbPB8XP6Hz4M67XtOEmjodiuB7orJrWCayrXjAiFgALWhGqXpKWRERP3jlGUvR8UPyMzte4omd0vsYVPeN46Ca9G5goaXrFtNmAB8+YmVlTFL4YRsQGYDFwlqRJkl4LvB24PN9kZmbWLgpfDDMfBp4FPARcCXxoHJ1WUahu22EUPR8UP6PzNa7oGZ2vcYXOqIjIO4OZmVmuxkvL0MzMbMy4GJqZWem5GI4BSSdIWiJps6RL885TTdIzJV2SXed1naTfSjo071zVJF0haZWkxyTdLekDeWcajqTpkgYlXZF3lkqSbs5yrc8eRTjvdiuS5km6K7v28ApJB+adCaBivw09npL0lbxzVZLULenHktZIelDSBZIKdcqcpJdK+rmktZKWS3pH3pmG42I4Nv4EnAN8M+8gI5gIrAQOAp4NnAZcJak7z1DD+DTQHRGdwGHAOZL2yznTcC4Ebs87xAhOiIjJ2eMleYepJumNwGeB95EupPF64H9zDZWp2G+TSddF3gQsyjlWtYtIAwtfQLpu80GkAYeFkBXma4HrgSmki6JcIWmvXIMNw8VwDETE4oi4BvhL3lmGExEbIqI3IgYi4q8RcT1wL1CoQhMRfRGxeehp9tgzx0hbkTQPeBT4Wc5RxqszgbMi4tfZz+IDEfFA3qGG8U5S0fll3kGqvAi4KiIGI+JB4CcU61KVM4AXAl+KiKci4ufAryjgtaVdDA1Ju5CuAVu401UkXSRpI9APrAJ+nHOkv5HUCZwFnJx3llF8WtLDkn4l6eC8w1SSNAHoAZ6bdZ/dn3XzPSvvbMM4Fvh2FG/4/fnAPEk7SOoCDiUVxKIY7hpqAma1OkgtLoYlJ2l74DvAZRHRn3eeahHxYVL32YGkiy9sHv0VLXU2cElErMw7yAg+BewBdJHO8fqhpCK1rHcBtie1ug4kdfO9Ajg1x0xbkbQ7qfvxsryzDOMXpJbgY8D9wBLgmjwDVekntag/IWl7SW8i7csd8o21NRfDEpO0HelKPo8DJ+QcZ0RZ98otwK7Ah/LOAyBpH+ANwJdyjjKiiLgtItZFxOaIuIzUPfWWvHNV2JT9+5WIWBURDwNfpFgZAY4BbomIe/MOUin7/b2B9EfiJNKFsHcmHYMthIh4AjgceCvwIKkX5SpS4S6UQo06staRJOAS0l/nb8l+aItuIsU5Zngw0A38Me1KJgMTJO0dEfvmmGs0QYtuM1KPiFgj6X5SriI7BvhM3iGGMQXYDbggO7a+WdK3SIP3PplrsgoRsYzUGgRA0q0UsJXtluEYkDRRUgcwgfQF2VG04c7AV4GXAm+LiE21Fm41Sc/LhtxPljRB0iHAkcDP886WWUAqzPtkj68BPwIOyS/S30naSdIhQz97ko4ijdS8Ie9sVb4FnJh93jsDJ5FGHhaCpANI3cxFG0VK1pK+F/hQ9hnvRDq2uTTXYFUkvTz7OdxB0sdJI18vzTnWVlwMx8appC6gU4Cjs/8X5jiIpGnAB0lf4g9WnEd1VL7JthCkLtH7gTXA54GTIuLaXFNlImJjRDw49CDdamwwIlbnnS2zPamFsJp0D7kTgcMjomjnGp5NOi3lbuAu4LfAubkm2tKxwOKI2OqWcQVxBPBm0ue8HHgS+Giuibb2XtLgt4eAOcAbK0aJF4avTWpmZqXnlqGZmZWei6GZmZWei6GZmZWei6GZmZWei6GZmZWei6GZmZWei6FZm8juXzjsPR8l7Z6dSzqh1bnMxgMXQ7OCkfQ6SbdmN0N9JLvjxCsbWWdE/DG7N99TDWYbkPSGRtZhVkRFu0SYWallt4W6nnT1nauAZ5Du6FC4K3aYtRO3DM2KZS+AiLgyu1vHpoj4aUQsk9Qr6YqhBSV1S4qq697uKek3WavyWklThltW0rMlXSJplaQHJJ1T2YUq6XhJd0laJ+lOSftKuhzYnXQrqPWSCnMxaLNGuRiaFcvdwFOSLpN0aHbx6m1xDPAvpLuLPwl8eYTlLsvmv5h0D8E3AR8AkPQuoDdbVydwGPCXiHgv8EfSxd0nR8TntjGbWWG5GJoVSEQ8BryOdKHyrwOrJV0naZc6V3F5RPw+IjYApwHvrh40k63rUNKFzzdExEOk+zLOyxb5APC5iLg9kuURcV8T3p5ZYfmYoVnBRMRdwHEAkmYAVwDnAfXccWJlxf/vI929YmrVMtOy6auyezFC+sN46LW7ASu2PbnZ+OWWoVmBRUQ/6d5vs4ANwA4Vs58/zEt2q/j/7sATpFs4VVpJGpAzNSJ2yh6dETGzYv5IN1H2bW6sLbkYmhWIpBmSTpa0a/Z8N9JNjX8N/A/w+uycwWcD/2+YVRwtaW9JOwBnAd+vPp0iIlYBPwW+IKlT0naS9pQ0dDfybwAfl7Sfkhdn98AE+DOwR5PftlnuXAzNimUdsD9wm6QNpCL4e+DkiPgPYCGwDLiD4e8IfzmpJfkg0AH86wjbOYZ02sadpJsnf590B3IiYhHpBrvfzfJcA0zJXvdp4FRJj2Z3LTdrC765r1kJSNoDuAeYGP6lN9uKW4Zm5TALGHAhNBuei6FZm5P0MWABcEreWcyKyt2kZmZWem4ZmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6bkYmplZ6f1/BGtLlnaOytQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(7, 5))\n",
    "plt.rcParams.update({'font.size': 12})\n",
    "ax.set_title('Accuracy (%)')\n",
    "ax.bar(np.arange(1, 10), model_acc, color=\"#ffb152\", yerr=model_std, capsize=5)\n",
    "ax.set(xticks=np.arange(1, 10), xlabel='Subject', \n",
    "       yticks=np.arange(0, 101, step=10), ylabel='Accuracy',\n",
    "       title='5-fold Cross Validation Result')\n",
    "ax.grid(axis='y', alpha=0.5)\n",
    "plt.savefig('5fold_train_result.jpg')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Result on Test Data\n",
    "20200930 Currently working under progress, need to know how to evaluate on test data correctly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Key to access data\n",
    "train = 'train'\n",
    "feat_train = 'feat_train'\n",
    "feat_test = 'feat_test'\n",
    "\n",
    "# Blank list to store accuracy values\n",
    "acc_train = []\n",
    "acc_test = []\n",
    "\n",
    "for i in range(1, ns):\n",
    "    subj = subjectCounter(i)\n",
    "    \n",
    "    data_train = mod_data[subj][train][feat_train]\n",
    "    X_train = data_train[:, :-1]\n",
    "    y_train = data_train[:, -1]\n",
    "    \n",
    "    data_test = mod_data[subj][test][feat_test]\n",
    "    X_test = data_test[:, :-1]\n",
    "    y_test = data_test[:, -1]\n",
    "    \n",
    "    # Fit into model\n",
    "    model.fit(X_train, y_train)\n",
    "    pred_train = model.predict(X_train)\n",
    "    pred_test = model.predict(X_test)\n",
    "    \n",
    "    acc_train.append(accuracy_score(y_true=y_train, y_pred=pred_train)*100)\n",
    "    acc_test.append(accuracy_score(y_true=y_test, y_pred=pred_test)*100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "258.097px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "toc-autonumbering": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}