7th_Semester/Data Warehousing and Data Mining/Lab4-Attribute_Transformation_and_Dimensionality_Reduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 4: Attribute Transformation an Dimensionality Reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EHYqQxpQ6hJE"
   },
   "source": [
    "## Attribute Transformation\n",
    "\n",
    "An attribute transform is a function that maps the entire set of values of a given attribute to a new set of replacement values such that each old value can be identified with one of the new values\n",
    "\n",
    "Dataset contains features with different metrics and scales. For example: \"pregnant\" and \"insulin\" values are based on different scales of measurement. The magnitude of \"insulin\" value is higher than \"pregnant\" in the diabetes dataset. Hence, many algorithm that are sensitive to varying scales of value will be biased towards the one with higher magnitdue.For example neural netwroks are highly sensitive to scaling of the data attributes. Hence we need to convert the dataset into suitable format before it is fed into the neurons.\n",
    "\n",
    "#### The solution to varying scale values\n",
    "\n",
    "We need a mechanism that scales all the attribute values into a given range typically between 0 to +1 or between a certain specified range. This approach is called feature scaling.\n",
    "\n",
    "Below are two approaches taht converts each feature into same scale\n",
    "\n",
    "1. Min-Max Scaler (Normalization)\n",
    "2. Standardization\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gVKKhofF6hJH"
   },
   "source": [
    "## Using MinMaxScaler() \n",
    "\n",
    "Rescaling X_train dataset\n",
    "\n",
    "\"minj\" and \"maxj\" represent the minimum and maximum values of attribute 'j'. The $j^{th}$ attribute value $x_{i}^{j}$ of the $i^{th}$ row is scaled as:\n",
    "\n",
    "### $y_{i}^{j} = (x_{i}^{j} - min_{j})/(max_{j}-min_{j})$\n",
    "\n",
    "We transform only the train dataset for scaling or any data tranformation tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zvFre-Sl6hJI"
   },
   "source": [
    "#### Split the cleaned data into input features $(X_{i})$ and output component (Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "gFpc8Jhe6hJI",
    "outputId": "af7d4371-18f0-4efe-92d7-c33a58b93137"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "aLImNdEt6hJK",
    "outputId": "3f2eab97-5463-464d-ef79-9491ee1da9e4"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pregnant</th>\n",
       "      <th>glucose</th>\n",
       "      <th>bp</th>\n",
       "      <th>skin</th>\n",
       "      <th>insulin</th>\n",
       "      <th>bmi</th>\n",
       "      <th>pedigree</th>\n",
       "      <th>age</th>\n",
       "      <th>Diabetic</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>85.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>125.0</td>\n",
       "      <td>26.6</td>\n",
       "      <td>0.351</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8</td>\n",
       "      <td>183.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>29.142593</td>\n",
       "      <td>125.0</td>\n",
       "      <td>23.3</td>\n",
       "      <td>0.672</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>89.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>94.0</td>\n",
       "      <td>28.1</td>\n",
       "      <td>0.167</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>168.0</td>\n",
       "      <td>43.1</td>\n",
       "      <td>2.288</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>116.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>29.142593</td>\n",
       "      <td>125.0</td>\n",
       "      <td>25.6</td>\n",
       "      <td>0.201</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pregnant  glucose    bp       skin  insulin   bmi  pedigree  age  Diabetic\n",
       "0         1     85.0  66.0  29.000000    125.0  26.6     0.351   31         0\n",
       "1         8    183.0  64.0  29.142593    125.0  23.3     0.672   32         1\n",
       "2         1     89.0  66.0  23.000000     94.0  28.1     0.167   21         0\n",
       "3         0    137.0  40.0  35.000000    168.0  43.1     2.288   33         1\n",
       "4         5    116.0  74.0  29.142593    125.0  25.6     0.201   30         0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('dataset/imputed_data_diabetes.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "f5rEkKty6hJL"
   },
   "outputs": [],
   "source": [
    "splitted_data = df.values\n",
    "\n",
    "X = splitted_data[:, 0:8]\n",
    "Y = splitted_data[:, 8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hZ4FvhEG6hJL"
   },
   "source": [
    "#### Separate the splitted dataset into training and testing dataset with training  dataset = 80% of cleaned data and test dataset  = 20% of cleaned dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "rllrcT-g6hJL"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size = 0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nr0zlfX66hJM"
   },
   "source": [
    "#### Use Sci-Kit learn MinMaxScaler () for normlization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "s7eYwvRH6hJM",
    "outputId": "e6d3af3f-f575-4f36-cc47-da8108ec7270"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.3529 0.4685 0.4898 0.413  0.2596 0.3149 0.2778 0.2167]\n",
      " [0.7647 0.6783 0.6531 0.3261 0.1514 0.4581 0.4666 0.3   ]\n",
      " [0.4118 0.972  0.4694 0.2826 0.1575 0.1411 0.0338 0.5667]\n",
      " [0.1176 0.2378 0.3673 0.2407 0.1334 0.1084 0.0458 0.0667]\n",
      " [0.7647 0.4895 0.6735 0.2407 0.1334 0.5153 0.2136 0.35  ]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "sclr = MinMaxScaler(feature_range=(0, 1))\n",
    "scaled_data_X_train = sclr.fit_transform(X_train)\n",
    "np.set_printoptions(precision=4)\n",
    "print(scaled_data_X_train[0:5, :])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wacBLwM76hJM"
   },
   "source": [
    "#### The above code converted all the feature values into the  scale between 0 and 1 using Normalization or Min-Max scaling.\n",
    "\n",
    "Some learning algorithms like Neural Networks expect input values between [0,1] hence we use normalization for scaling in such case"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wnHqmie06hJN"
   },
   "source": [
    "## Standardization\n",
    "\n",
    "It is another approach to scaling where the scaled value isn't within the [0,1] range. It is suitable where the data collection process has errors and hence has extreme values or outliers.\n",
    "\n",
    "The $j^{th}$ attribute value $x_{i}^{j}$ of the ith row is normalized by:\n",
    "\n",
    "####                         Z-score_normalization (x')=  ($x_{i}^{j}$ -$\\mu_{j}$)  /  $\\sigma_{j}$\n",
    "\n",
    " where the $j^{th}$  attribute has mean $\\mu_{j}$ and standard deviation $\\sigma_{j}$ .\n",
    "                       \n",
    "We use a function `StandardScaler()` for standardization purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "jrri4zbh6hJN",
    "outputId": "eb57189b-833c-4b91-edf0-c560625f89fa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.614  0.024 -0.052  1.826  0.98   0.133  0.769  0.059]\n",
      " [ 2.656  1.007  1.267  0.91  -0.029  1.13   2.067  0.481]\n",
      " [ 0.905  2.383 -0.217  0.451  0.027 -1.077 -0.907  1.832]\n",
      " [-0.554 -1.057 -1.041  0.01  -0.197 -1.305 -0.825 -0.701]\n",
      " [ 2.656  0.122  1.431  0.01  -0.197  1.529  0.328  0.734]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "scale_ftrs_stndrd = StandardScaler().fit(X_train)\n",
    "scaled_stndrd_X_train = scale_ftrs_stndrd.transform(X_train)\n",
    "\n",
    "# summarize transformed data\n",
    "np.set_printoptions(precision=3)\n",
    "print(scaled_stndrd_X_train[0:5, :])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dimensionality Reduction\n",
    "\n",
    "Dimensionality reduction is all about summarizing the data with most of the information preserved in compact form.Reducing the dimension of the feature space, creates fewer relationships between variables and hence the model is less likely to overfit.\n",
    "\n",
    "> One of such technique discussed here is the Principal Component Analysis (PCA)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r4ZcdKI46hJN"
   },
   "source": [
    "## Principal Component Analysis (PCA)\n",
    "\n",
    "PCA is a dimensionality-reduction technique for reducing the dimensionality of large data sets, i.e. by transforming a large set of input features into a smaller set which still contains most of the information in the original dataset .But Before applying PCA, the dataset must be rescaled, if not rescaled, the model/algorithm's accuracy may not be improved much."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "Dtnw70x56hJO",
    "outputId": "b7cbe93a-9631-4da3-e6f4-0eabdd090270"
   },
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "# use three diagonal compnents for data reduction and summarization\n",
    "principal_components = PCA(n_components=3)\n",
    "principal_summary = principal_components.fit(scaled_stndrd_X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explained Variance: [0.283 0.185 0.145]\n"
     ]
    }
   ],
   "source": [
    "# summarize the components\n",
    "print(f\"Explained Variance: {principal_summary.explained_variance_ratio_}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "9UXzIyb06hJO",
    "outputId": "3fd89790-5b6f-4e47-e679-96b7f5922849"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.303  0.414  0.379  0.408  0.305  0.4    0.167  0.382]\n",
      " [ 0.569 -0.1    0.158 -0.273 -0.236 -0.388 -0.3    0.52 ]\n",
      " [-0.024  0.451 -0.282 -0.406  0.555 -0.398  0.272  0.098]]\n"
     ]
    }
   ],
   "source": [
    "print(principal_summary.components_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KciK_0Wd6hJO"
   },
   "source": [
    "\n",
    "Above code created three principial components as denoted in three separate arrays. Each array represents the component that summarizes the overall data."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}