content based filtering.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cc9d6e5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Importing required libraries\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d926c77",
   "metadata": {},
   "source": [
    "In this notebook I am gonna apply Conteny based filtering to the famous MovieLens 20M dataset and recommend similar movies to a\n",
    "\n",
    "movie which the user is watching\n",
    "\n",
    "Lets start by exploring our datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c9f2608f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:53:47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:31:16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>32</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:33:39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:32:07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:29:40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000258</th>\n",
       "      <td>138493</td>\n",
       "      <td>68954</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2009-11-13 15:42:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000259</th>\n",
       "      <td>138493</td>\n",
       "      <td>69526</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2009-12-03 18:31:48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000260</th>\n",
       "      <td>138493</td>\n",
       "      <td>69644</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2009-12-07 18:10:57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000261</th>\n",
       "      <td>138493</td>\n",
       "      <td>70286</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2009-11-13 15:42:24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000262</th>\n",
       "      <td>138493</td>\n",
       "      <td>71619</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2009-10-17 20:25:36</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20000263 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          userId  movieId  rating            timestamp\n",
       "0              1        2     3.5  2005-04-02 23:53:47\n",
       "1              1       29     3.5  2005-04-02 23:31:16\n",
       "2              1       32     3.5  2005-04-02 23:33:39\n",
       "3              1       47     3.5  2005-04-02 23:32:07\n",
       "4              1       50     3.5  2005-04-02 23:29:40\n",
       "...          ...      ...     ...                  ...\n",
       "20000258  138493    68954     4.5  2009-11-13 15:42:00\n",
       "20000259  138493    69526     4.5  2009-12-03 18:31:48\n",
       "20000260  138493    69644     3.0  2009-12-07 18:10:57\n",
       "20000261  138493    70286     5.0  2009-11-13 15:42:24\n",
       "20000262  138493    71619     2.5  2009-10-17 20:25:36\n",
       "\n",
       "[20000263 rows x 4 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = pd.read_csv('rating.csv')\n",
    "ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f9ff573f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27273</th>\n",
       "      <td>131254</td>\n",
       "      <td>Kein Bund für's Leben (2007)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27274</th>\n",
       "      <td>131256</td>\n",
       "      <td>Feuer, Eis &amp; Dosenbier (2002)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27275</th>\n",
       "      <td>131258</td>\n",
       "      <td>The Pirates (2014)</td>\n",
       "      <td>Adventure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27276</th>\n",
       "      <td>131260</td>\n",
       "      <td>Rentun Ruusu (2001)</td>\n",
       "      <td>(no genres listed)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27277</th>\n",
       "      <td>131262</td>\n",
       "      <td>Innocence (2014)</td>\n",
       "      <td>Adventure|Fantasy|Horror</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>27278 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       movieId                               title  \\\n",
       "0            1                    Toy Story (1995)   \n",
       "1            2                      Jumanji (1995)   \n",
       "2            3             Grumpier Old Men (1995)   \n",
       "3            4            Waiting to Exhale (1995)   \n",
       "4            5  Father of the Bride Part II (1995)   \n",
       "...        ...                                 ...   \n",
       "27273   131254        Kein Bund für's Leben (2007)   \n",
       "27274   131256       Feuer, Eis & Dosenbier (2002)   \n",
       "27275   131258                  The Pirates (2014)   \n",
       "27276   131260                 Rentun Ruusu (2001)   \n",
       "27277   131262                    Innocence (2014)   \n",
       "\n",
       "                                            genres  \n",
       "0      Adventure|Animation|Children|Comedy|Fantasy  \n",
       "1                       Adventure|Children|Fantasy  \n",
       "2                                   Comedy|Romance  \n",
       "3                             Comedy|Drama|Romance  \n",
       "4                                           Comedy  \n",
       "...                                            ...  \n",
       "27273                                       Comedy  \n",
       "27274                                       Comedy  \n",
       "27275                                    Adventure  \n",
       "27276                           (no genres listed)  \n",
       "27277                     Adventure|Fantasy|Horror  \n",
       "\n",
       "[27278 rows x 3 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies = pd.read_csv('movie.csv')\n",
    "movies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0a860acb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>userId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1999-12-11 13:36:47</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>6</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1997-03-13 17:50:52</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>8</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1996-06-05 13:37:51</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>10</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1999-11-25 02:44:47</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>11</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2009-01-02 01:13:41</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000258</th>\n",
       "      <td>131254</td>\n",
       "      <td>Kein Bund für's Leben (2007)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>79570</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2015-03-30 19:32:59</td>\n",
       "      <td>20000259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000259</th>\n",
       "      <td>131256</td>\n",
       "      <td>Feuer, Eis &amp; Dosenbier (2002)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>79570</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2015-03-30 19:48:08</td>\n",
       "      <td>20000260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000260</th>\n",
       "      <td>131258</td>\n",
       "      <td>The Pirates (2014)</td>\n",
       "      <td>Adventure</td>\n",
       "      <td>28906</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2015-03-30 19:56:32</td>\n",
       "      <td>20000261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000261</th>\n",
       "      <td>131260</td>\n",
       "      <td>Rentun Ruusu (2001)</td>\n",
       "      <td>(no genres listed)</td>\n",
       "      <td>65409</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2015-03-30 19:57:46</td>\n",
       "      <td>20000262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000262</th>\n",
       "      <td>131262</td>\n",
       "      <td>Innocence (2014)</td>\n",
       "      <td>Adventure|Fantasy|Horror</td>\n",
       "      <td>133047</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2015-03-30 20:39:26</td>\n",
       "      <td>20000263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20000263 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          movieId                          title  \\\n",
       "0               1               Toy Story (1995)   \n",
       "1               1               Toy Story (1995)   \n",
       "2               1               Toy Story (1995)   \n",
       "3               1               Toy Story (1995)   \n",
       "4               1               Toy Story (1995)   \n",
       "...           ...                            ...   \n",
       "20000258   131254   Kein Bund für's Leben (2007)   \n",
       "20000259   131256  Feuer, Eis & Dosenbier (2002)   \n",
       "20000260   131258             The Pirates (2014)   \n",
       "20000261   131260            Rentun Ruusu (2001)   \n",
       "20000262   131262               Innocence (2014)   \n",
       "\n",
       "                                               genres  userId  rating  \\\n",
       "0         Adventure|Animation|Children|Comedy|Fantasy       3     4.0   \n",
       "1         Adventure|Animation|Children|Comedy|Fantasy       6     5.0   \n",
       "2         Adventure|Animation|Children|Comedy|Fantasy       8     4.0   \n",
       "3         Adventure|Animation|Children|Comedy|Fantasy      10     4.0   \n",
       "4         Adventure|Animation|Children|Comedy|Fantasy      11     4.5   \n",
       "...                                               ...     ...     ...   \n",
       "20000258                                       Comedy   79570     4.0   \n",
       "20000259                                       Comedy   79570     4.0   \n",
       "20000260                                    Adventure   28906     2.5   \n",
       "20000261                           (no genres listed)   65409     3.0   \n",
       "20000262                     Adventure|Fantasy|Horror  133047     4.0   \n",
       "\n",
       "                    timestamp     index  \n",
       "0         1999-12-11 13:36:47         1  \n",
       "1         1997-03-13 17:50:52         2  \n",
       "2         1996-06-05 13:37:51         3  \n",
       "3         1999-11-25 02:44:47         4  \n",
       "4         2009-01-02 01:13:41         5  \n",
       "...                       ...       ...  \n",
       "20000258  2015-03-30 19:32:59  20000259  \n",
       "20000259  2015-03-30 19:48:08  20000260  \n",
       "20000260  2015-03-30 19:56:32  20000261  \n",
       "20000261  2015-03-30 19:57:46  20000262  \n",
       "20000262  2015-03-30 20:39:26  20000263  \n",
       "\n",
       "[20000263 rows x 7 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = pd.merge(movies,ratings,on =\"movieId\",how =\"inner\") #Merging ratings dataframe and movies dataframe so as to get relevant information\n",
    "ratings['index'] = range(1, len(ratings) + 1)\n",
    "ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0e29fd4e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>userId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1999-12-11 13:36:47</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49695</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:53:47</td>\n",
       "      <td>49696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71938</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>2</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2000-11-21 15:34:49</td>\n",
       "      <td>71939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84673</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>41</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2000-02-27 23:29:58</td>\n",
       "      <td>84674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87429</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>12</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1997-03-22 20:49:34</td>\n",
       "      <td>87430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000258</th>\n",
       "      <td>131254</td>\n",
       "      <td>Kein Bund für's Leben (2007)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>79570</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2015-03-30 19:32:59</td>\n",
       "      <td>20000259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000259</th>\n",
       "      <td>131256</td>\n",
       "      <td>Feuer, Eis &amp; Dosenbier (2002)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>79570</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2015-03-30 19:48:08</td>\n",
       "      <td>20000260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000260</th>\n",
       "      <td>131258</td>\n",
       "      <td>The Pirates (2014)</td>\n",
       "      <td>Adventure</td>\n",
       "      <td>28906</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2015-03-30 19:56:32</td>\n",
       "      <td>20000261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000261</th>\n",
       "      <td>131260</td>\n",
       "      <td>Rentun Ruusu (2001)</td>\n",
       "      <td>(no genres listed)</td>\n",
       "      <td>65409</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2015-03-30 19:57:46</td>\n",
       "      <td>20000262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20000262</th>\n",
       "      <td>131262</td>\n",
       "      <td>Innocence (2014)</td>\n",
       "      <td>Adventure|Fantasy|Horror</td>\n",
       "      <td>133047</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2015-03-30 20:39:26</td>\n",
       "      <td>20000263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>26744 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          movieId                               title  \\\n",
       "0               1                    Toy Story (1995)   \n",
       "49695           2                      Jumanji (1995)   \n",
       "71938           3             Grumpier Old Men (1995)   \n",
       "84673           4            Waiting to Exhale (1995)   \n",
       "87429           5  Father of the Bride Part II (1995)   \n",
       "...           ...                                 ...   \n",
       "20000258   131254        Kein Bund für's Leben (2007)   \n",
       "20000259   131256       Feuer, Eis & Dosenbier (2002)   \n",
       "20000260   131258                  The Pirates (2014)   \n",
       "20000261   131260                 Rentun Ruusu (2001)   \n",
       "20000262   131262                    Innocence (2014)   \n",
       "\n",
       "                                               genres  userId  rating  \\\n",
       "0         Adventure|Animation|Children|Comedy|Fantasy       3     4.0   \n",
       "49695                      Adventure|Children|Fantasy       1     3.5   \n",
       "71938                                  Comedy|Romance       2     4.0   \n",
       "84673                            Comedy|Drama|Romance      41     2.0   \n",
       "87429                                          Comedy      12     2.0   \n",
       "...                                               ...     ...     ...   \n",
       "20000258                                       Comedy   79570     4.0   \n",
       "20000259                                       Comedy   79570     4.0   \n",
       "20000260                                    Adventure   28906     2.5   \n",
       "20000261                           (no genres listed)   65409     3.0   \n",
       "20000262                     Adventure|Fantasy|Horror  133047     4.0   \n",
       "\n",
       "                    timestamp     index  \n",
       "0         1999-12-11 13:36:47         1  \n",
       "49695     2005-04-02 23:53:47     49696  \n",
       "71938     2000-11-21 15:34:49     71939  \n",
       "84673     2000-02-27 23:29:58     84674  \n",
       "87429     1997-03-22 20:49:34     87430  \n",
       "...                       ...       ...  \n",
       "20000258  2015-03-30 19:32:59  20000259  \n",
       "20000259  2015-03-30 19:48:08  20000260  \n",
       "20000260  2015-03-30 19:56:32  20000261  \n",
       "20000261  2015-03-30 19:57:46  20000262  \n",
       "20000262  2015-03-30 20:39:26  20000263  \n",
       "\n",
       "[26744 rows x 7 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings.drop_duplicates(subset = ['title','movieId'], inplace=True)\n",
    "ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ead2aa2e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>18</td>\n",
       "      <td>4141</td>\n",
       "      <td>Mark Waters</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>65</td>\n",
       "      <td>208</td>\n",
       "      <td>dark hero</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>65</td>\n",
       "      <td>353</td>\n",
       "      <td>dark hero</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>65</td>\n",
       "      <td>521</td>\n",
       "      <td>noir thriller</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>65</td>\n",
       "      <td>592</td>\n",
       "      <td>dark hero</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>465559</th>\n",
       "      <td>138446</td>\n",
       "      <td>55999</td>\n",
       "      <td>dragged</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>465560</th>\n",
       "      <td>138446</td>\n",
       "      <td>55999</td>\n",
       "      <td>Jason Bateman</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>465561</th>\n",
       "      <td>138446</td>\n",
       "      <td>55999</td>\n",
       "      <td>quirky</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>465562</th>\n",
       "      <td>138446</td>\n",
       "      <td>55999</td>\n",
       "      <td>sad</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>465563</th>\n",
       "      <td>138472</td>\n",
       "      <td>923</td>\n",
       "      <td>rise to power</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>465564 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        userId  movieId            tag\n",
       "0           18     4141    Mark Waters\n",
       "1           65      208      dark hero\n",
       "2           65      353      dark hero\n",
       "3           65      521  noir thriller\n",
       "4           65      592      dark hero\n",
       "...        ...      ...            ...\n",
       "465559  138446    55999        dragged\n",
       "465560  138446    55999  Jason Bateman\n",
       "465561  138446    55999         quirky\n",
       "465562  138446    55999            sad\n",
       "465563  138472      923  rise to power\n",
       "\n",
       "[465564 rows x 3 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tags = pd.read_csv('tag.csv')\n",
    "tags.drop('timestamp',axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "38b0d979",
   "metadata": {},
   "outputs": [],
   "source": [
    "genome_score = pd.read_csv('genome_scores.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "18f3ec72",
   "metadata": {},
   "outputs": [],
   "source": [
    "genome_tag = pd.read_csv('genome_tags.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3b4cdd",
   "metadata": {},
   "source": [
    "We have loaded all our datasets from the folder. \n",
    "\n",
    "The first file has following features:  userid,movieid,rating,timestamp\n",
    "\n",
    "second file has movieid,title,genres\n",
    "\n",
    "third file has userid,movieid,tag,timestamp\n",
    "\n",
    "fourth file has movieid,tagid,relevance\n",
    "\n",
    "fifth file has tag id and tag \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7b8e2ffc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>userId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1999-12-11 13:36:47</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49695</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2005-04-02 23:53:47</td>\n",
       "      <td>49696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71938</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>2</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2000-11-21 15:34:49</td>\n",
       "      <td>71939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84673</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>41</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2000-02-27 23:29:58</td>\n",
       "      <td>84674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87429</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>12</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1997-03-22 20:49:34</td>\n",
       "      <td>87430</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       movieId                               title  \\\n",
       "0            1                    Toy Story (1995)   \n",
       "49695        2                      Jumanji (1995)   \n",
       "71938        3             Grumpier Old Men (1995)   \n",
       "84673        4            Waiting to Exhale (1995)   \n",
       "87429        5  Father of the Bride Part II (1995)   \n",
       "\n",
       "                                            genres  userId  rating  \\\n",
       "0      Adventure|Animation|Children|Comedy|Fantasy       3     4.0   \n",
       "49695                   Adventure|Children|Fantasy       1     3.5   \n",
       "71938                               Comedy|Romance       2     4.0   \n",
       "84673                         Comedy|Drama|Romance      41     2.0   \n",
       "87429                                       Comedy      12     2.0   \n",
       "\n",
       "                 timestamp  index  \n",
       "0      1999-12-11 13:36:47      1  \n",
       "49695  2005-04-02 23:53:47  49696  \n",
       "71938  2000-11-21 15:34:49  71939  \n",
       "84673  2000-02-27 23:29:58  84674  \n",
       "87429  1997-03-22 20:49:34  87430  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "72b2ead8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>tagId</th>\n",
       "      <th>relevance</th>\n",
       "      <th>tag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.02500</td>\n",
       "      <td>007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.03975</td>\n",
       "      <td>007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0.04350</td>\n",
       "      <td>007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0.03725</td>\n",
       "      <td>007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0.04200</td>\n",
       "      <td>007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11709763</th>\n",
       "      <td>130578</td>\n",
       "      <td>1128</td>\n",
       "      <td>0.01325</td>\n",
       "      <td>zombies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11709764</th>\n",
       "      <td>130840</td>\n",
       "      <td>1128</td>\n",
       "      <td>0.13375</td>\n",
       "      <td>zombies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11709765</th>\n",
       "      <td>131013</td>\n",
       "      <td>1128</td>\n",
       "      <td>0.01625</td>\n",
       "      <td>zombies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11709766</th>\n",
       "      <td>131168</td>\n",
       "      <td>1128</td>\n",
       "      <td>0.01725</td>\n",
       "      <td>zombies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11709767</th>\n",
       "      <td>131170</td>\n",
       "      <td>1128</td>\n",
       "      <td>0.02175</td>\n",
       "      <td>zombies</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>11709768 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          movieId  tagId  relevance      tag\n",
       "0               1      1    0.02500      007\n",
       "1               2      1    0.03975      007\n",
       "2               3      1    0.04350      007\n",
       "3               4      1    0.03725      007\n",
       "4               5      1    0.04200      007\n",
       "...           ...    ...        ...      ...\n",
       "11709763   130578   1128    0.01325  zombies\n",
       "11709764   130840   1128    0.13375  zombies\n",
       "11709765   131013   1128    0.01625  zombies\n",
       "11709766   131168   1128    0.01725  zombies\n",
       "11709767   131170   1128    0.02175  zombies\n",
       "\n",
       "[11709768 rows x 4 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tage = genome_score.merge(genome_tag,on =\"tagId\")\n",
    "tage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "b28614b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_ratings = ratings[['userId', 'movieId', 'rating','title','genres','index']]\n",
    "train_ratings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34dc0b58",
   "metadata": {},
   "source": [
    "Content Based Filtering\n",
    "\n",
    "\n",
    "In content based filtering we compute the similarity score of users from his past behaviour and recommend new movies based on his previous likings.We can calculate similarity score by using cosine similarity or euclidean similarity and then recommend top 4-5 movies for him.\n",
    "\n",
    "In this recommender system the content of the movie (overview, cast, crew, keyword, tags,genres etc) is used to find its similarity with other movies. Then the movies that are most likely to be similar are recommended.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af6c4c1b",
   "metadata": {},
   "source": [
    "Machines cannot understand characters and words. So when dealing with text data we need to represent it in numbers to be\n",
    "\n",
    "understood by the machine. Countvectorizer is a method to convert text to numerical data.\n",
    "\n",
    "If I  have 2 text inputs, what happens is that each input is preprocessed, tokenized, and represented as a sparse matrix. By\n",
    "\n",
    "default, Countvectorizer converts the text to lowercase and uses word-level tokenization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2156b8b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>action</th>\n",
       "      <th>adventure</th>\n",
       "      <th>animation</th>\n",
       "      <th>children</th>\n",
       "      <th>comedy</th>\n",
       "      <th>crime</th>\n",
       "      <th>documentary</th>\n",
       "      <th>drama</th>\n",
       "      <th>fantasy</th>\n",
       "      <th>fi</th>\n",
       "      <th>...</th>\n",
       "      <th>imax</th>\n",
       "      <th>listed</th>\n",
       "      <th>musical</th>\n",
       "      <th>mystery</th>\n",
       "      <th>noir</th>\n",
       "      <th>romance</th>\n",
       "      <th>sci</th>\n",
       "      <th>thriller</th>\n",
       "      <th>war</th>\n",
       "      <th>western</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26739</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26740</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26741</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26742</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26743</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>26744 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       action  adventure  animation  children  comedy  crime  documentary  \\\n",
       "0           0          1          1         1       1      0            0   \n",
       "1           0          1          0         1       0      0            0   \n",
       "2           0          0          0         0       1      0            0   \n",
       "3           0          0          0         0       1      0            0   \n",
       "4           0          0          0         0       1      0            0   \n",
       "...       ...        ...        ...       ...     ...    ...          ...   \n",
       "26739       0          0          0         0       1      0            0   \n",
       "26740       0          0          0         0       1      0            0   \n",
       "26741       0          1          0         0       0      0            0   \n",
       "26742       0          0          0         0       0      0            0   \n",
       "26743       0          1          0         0       0      0            0   \n",
       "\n",
       "       drama  fantasy  fi  ...  imax  listed  musical  mystery  noir  romance  \\\n",
       "0          0        1   0  ...     0       0        0        0     0        0   \n",
       "1          0        1   0  ...     0       0        0        0     0        0   \n",
       "2          0        0   0  ...     0       0        0        0     0        1   \n",
       "3          1        0   0  ...     0       0        0        0     0        1   \n",
       "4          0        0   0  ...     0       0        0        0     0        0   \n",
       "...      ...      ...  ..  ...   ...     ...      ...      ...   ...      ...   \n",
       "26739      0        0   0  ...     0       0        0        0     0        0   \n",
       "26740      0        0   0  ...     0       0        0        0     0        0   \n",
       "26741      0        0   0  ...     0       0        0        0     0        0   \n",
       "26742      0        0   0  ...     0       1        0        0     0        0   \n",
       "26743      0        1   0  ...     0       0        0        0     0        0   \n",
       "\n",
       "       sci  thriller  war  western  \n",
       "0        0         0    0        0  \n",
       "1        0         0    0        0  \n",
       "2        0         0    0        0  \n",
       "3        0         0    0        0  \n",
       "4        0         0    0        0  \n",
       "...    ...       ...  ...      ...  \n",
       "26739    0         0    0        0  \n",
       "26740    0         0    0        0  \n",
       "26741    0         0    0        0  \n",
       "26742    0         0    0        0  \n",
       "26743    0         0    0        0  \n",
       "\n",
       "[26744 rows x 23 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import CountVectorizer and create the count_matrix\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "count = CountVectorizer(stop_words='english')\n",
    "\n",
    "#Stopwords are the words in any language which does not add much meaning to a sentence.\n",
    "#They can safely be ignored without sacrificing the meaning of the sentence\n",
    "\n",
    "count_matrix = count.fit_transform(train_ratings['genres'])\n",
    "\n",
    "#Visualizing count matrix using pandas dataframe\n",
    "\n",
    "count_array = count_matrix.toarray()\n",
    "\n",
    "df = pd.DataFrame(data=count_array,columns = count.get_feature_names())\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b6f3981",
   "metadata": {},
   "source": [
    "We can see that 23 words can describe 26744 movies in our dataset. With the count_matrix in our hand we can calculate\n",
    "\n",
    "similarity score of each movies.Now let us assume we have 26774 vectors describing our movies. Now any movie similar to other\n",
    "\n",
    "movie can be calculated by using cosine similarity between their vectors.Here, we are taking one movie and calculating\n",
    "\n",
    "similarity with all movies present in our dataset. Hence our output matrix is of the shape\n",
    "\n",
    "(np.size(count_matrix),np.size(count_matrix) )  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "06b70011",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute the Cosine Similarity matrix based on the count_matrix\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "cosine_sim2 = cosine_similarity(count_matrix, count_matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "474ce6a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>level_0</th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "      <th>index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>49695</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3.5</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "      <td>49696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>71938</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "      <td>71939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84673</td>\n",
       "      <td>41</td>\n",
       "      <td>4</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "      <td>84674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>87429</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>87430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26739</th>\n",
       "      <td>20000258</td>\n",
       "      <td>79570</td>\n",
       "      <td>131254</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Kein Bund für's Leben (2007)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>20000259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26740</th>\n",
       "      <td>20000259</td>\n",
       "      <td>79570</td>\n",
       "      <td>131256</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Feuer, Eis &amp; Dosenbier (2002)</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>20000260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26741</th>\n",
       "      <td>20000260</td>\n",
       "      <td>28906</td>\n",
       "      <td>131258</td>\n",
       "      <td>2.5</td>\n",
       "      <td>The Pirates (2014)</td>\n",
       "      <td>Adventure</td>\n",
       "      <td>20000261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26742</th>\n",
       "      <td>20000261</td>\n",
       "      <td>65409</td>\n",
       "      <td>131260</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Rentun Ruusu (2001)</td>\n",
       "      <td>(no genres listed)</td>\n",
       "      <td>20000262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26743</th>\n",
       "      <td>20000262</td>\n",
       "      <td>133047</td>\n",
       "      <td>131262</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Innocence (2014)</td>\n",
       "      <td>Adventure|Fantasy|Horror</td>\n",
       "      <td>20000263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>26744 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        level_0  userId  movieId  rating                               title  \\\n",
       "0             0       3        1     4.0                    Toy Story (1995)   \n",
       "1         49695       1        2     3.5                      Jumanji (1995)   \n",
       "2         71938       2        3     4.0             Grumpier Old Men (1995)   \n",
       "3         84673      41        4     2.0            Waiting to Exhale (1995)   \n",
       "4         87429      12        5     2.0  Father of the Bride Part II (1995)   \n",
       "...         ...     ...      ...     ...                                 ...   \n",
       "26739  20000258   79570   131254     4.0        Kein Bund für's Leben (2007)   \n",
       "26740  20000259   79570   131256     4.0       Feuer, Eis & Dosenbier (2002)   \n",
       "26741  20000260   28906   131258     2.5                  The Pirates (2014)   \n",
       "26742  20000261   65409   131260     3.0                 Rentun Ruusu (2001)   \n",
       "26743  20000262  133047   131262     4.0                    Innocence (2014)   \n",
       "\n",
       "                                            genres     index  \n",
       "0      Adventure|Animation|Children|Comedy|Fantasy         1  \n",
       "1                       Adventure|Children|Fantasy     49696  \n",
       "2                                   Comedy|Romance     71939  \n",
       "3                             Comedy|Drama|Romance     84674  \n",
       "4                                           Comedy     87430  \n",
       "...                                            ...       ...  \n",
       "26739                                       Comedy  20000259  \n",
       "26740                                       Comedy  20000260  \n",
       "26741                                    Adventure  20000261  \n",
       "26742                           (no genres listed)  20000262  \n",
       "26743                     Adventure|Fantasy|Horror  20000263  \n",
       "\n",
       "[26744 rows x 7 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Reset index of our main DataFrame and construct reverse mapping\n",
    "train_ratings = train_ratings.reset_index()\n",
    "train_ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "786f48bf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "title\n",
       "Toy Story (1995)                          0\n",
       "Jumanji (1995)                            1\n",
       "Grumpier Old Men (1995)                   2\n",
       "Waiting to Exhale (1995)                  3\n",
       "Father of the Bride Part II (1995)        4\n",
       "                                      ...  \n",
       "Kein Bund für's Leben (2007)          26739\n",
       "Feuer, Eis & Dosenbier (2002)         26740\n",
       "The Pirates (2014)                    26741\n",
       "Rentun Ruusu (2001)                   26742\n",
       "Innocence (2014)                      26743\n",
       "Length: 26744, dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indices = pd.Series(train_ratings.index, index=train_ratings['title'])\n",
    "indices.to_csv('movieslist.csv')\n",
    "indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "39ecabd2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function that takes in movie title as input and outputs most similar movies\n",
    "def get_recommendations(title, cosine_sim=cosine_sim2):\n",
    "    # Get the index of the movie that matches the title\n",
    "    idx = indices[title]\n",
    "    #print(idx)\n",
    "    # Get the pairwsie similarity scores of all movies with that movie\n",
    "    sim_scores = list(enumerate(cosine_sim[idx]))\n",
    "    #for i in sim_scores:\n",
    "        #print(i)\n",
    "    # Sort the movies based on the similarity scores\n",
    "    sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)\n",
    "\n",
    "    # Get the scores of the 10 most similar movies\n",
    "    sim_scores = sim_scores[1:11]\n",
    "    #Slicing from one so that the same movie with itself having sim score is not showed\n",
    "    # Get the movie indices\n",
    "    movie_indices = [i[0] for i in sim_scores]\n",
    "\n",
    "    # Return the top 10 most similar movies\n",
    "    return train_ratings['title'].iloc[movie_indices]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "121a852e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "290     Léon: The Professional (a.k.a. The Professiona...\n",
       "345                       Clear and Present Danger (1994)\n",
       "977                              Last Man Standing (1996)\n",
       "1192            Killer, The (Die xue shuang xiong) (1989)\n",
       "1522                                      Face/Off (1997)\n",
       "1537                                      Cop Land (1997)\n",
       "2455                                Corruptor, The (1999)\n",
       "2795                               Double Jeopardy (1999)\n",
       "2873                                    Fight Club (1999)\n",
       "3178                Hard-Boiled (Lat sau san taam) (1992)\n",
       "Name: title, dtype: object"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_recommendations('The Dark Knight (2011)', cosine_sim2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "402d9621",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2209                                           Antz (1998)\n",
       "3027                                    Toy Story 2 (1999)\n",
       "3663        Adventures of Rocky and Bullwinkle, The (2000)\n",
       "3922                      Emperor's New Groove, The (2000)\n",
       "4790                                 Monsters, Inc. (2001)\n",
       "10106    DuckTales: The Movie - Treasure of the Lost La...\n",
       "10978                                     Wild, The (2006)\n",
       "11861                               Shrek the Third (2007)\n",
       "13325                       Tale of Despereaux, The (2008)\n",
       "18230    Asterix and the Vikings (Astérix et les Viking...\n",
       "Name: title, dtype: object"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_recommendations('Toy Story (1995)', cosine_sim2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bff54c99",
   "metadata": {},
   "source": [
    "DRAWBACKS OF THIS RECOMMENDER SYSTEM:\n",
    "\n",
    "It is only capable of suggesting movies belonging to some genre which the user is searching. It is not able to produce recommendations from different genres capturing user's tastes.\n",
    "\n",
    "The engine which I built is not so personal, anyone searching for a particular movie would see the same result.So, lets explore other recommender system Collaborative Filtering and get better recommendations than this."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}