token ,stemmer ,lemmization on data -notebook.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Untitled",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mV_4P2f79bRI",
        "colab_type": "text"
      },
      "source": [
        "Importing NLTK "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RWMoULwOWdI-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import nltk"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7l2C6dLeqAH7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "text= ('I must be honest with you. I was about to go to sleep before I opened my mail and read your letter. Now I am wide awake, sitting upright, because you made me question my preferences between shoes and slippers.Be patient with me. I need proper peaceful sleep and let’s sort this out.I am a shoe person. Correction: I am a person obsessed with dry, dust-free feet. Whatever helps me keep a grip on my walk and posture in spite of my profusely sweaty feet, I am that person. Whatever keeps the grainy, pokey sensation of soil or sand in parts of my skin away, I am that person. I guess, I am a shoe person because it is loyal to my whole feet. Not making parts of it lying exposed and parts of it covered, without any symmetry. I am also cotton sock person, preferably red. Sometimes, I am a clean matte (anti-slippery) floor person. Basically, I am a person with issues.I really liked your host-guest theory of travelling, how we do not just get out of home when we travel, the home expands when we step foot at a place engulfing it into our comfort zone. But the thing is I wear socks all the time in my house, and open them when I go to bed, replacing the sock with a comforter. I guess I need that comfort zone close and maybe, mine is restricted to my own skin. So much so, that it doesn’t even include my own home. When I travel, I like wearing shoes, with a firm grip on myself, dust free, protected. When I face situations where I have to open them and I don’t want to, I usually wrinkle my nose. It doesn’t get better that I can’t wear those shoes again with a dirty feet. I take a handkerchief.Imagine my situation at the beach or in a hill stream, when the water feels soothing to my sweaty feet and removes the dust, only to attract more of it when I walk on the sand. I let it irritate me those times, because I always have the ocean and the river on my side. I like my bare feet drowning with no air bubbles left. I don’t like it when droplets of water make only a part of my feet wet, only to let me realise and miss the comfort of dry skin and scowl at the linger of irritation due to a little dampness. I don’t like it when I have to keep on brushing my left leg against my right to flatten the spheres of water. And repeat it with the right. I just realised why I don’t like drizzling rains.Eventually, when I have to get out of the water, I give up my fight with the dirt and wear those damn shoes with muddy feet. Those are the times I miss my slippery slippers. Don’t let my socks or shoes hear me say that. They do not know that sometimes I keep a back-up pair of red flip-flops for such scenarios. Does this make me a slipper person? I guess I am a slipper person when my feet are ankle deep in mud, so that there is no place for me to complain. No alternative. No uneasy half-done feeling. Otherwise, not much of a slipper person.I am a person who likes symmetry. All in or none at all. It’s difficult being me in this world with its shades of grey. But then, my favourite colour is red and thankfully it covers the entire spectrum. All I need is an emotion, and the rest is taken care of.Thank you for being the listener you are. This has been a selfish post. I used your letter for my sense of clarity. You always make me reflect and dig deep. I can finally sleep now. Goodnight.'"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-MsOMLmUq5bI",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "601d39e9-cd65-43c4-8e1f-a36fc7cd2644"
      },
      "source": [
        "nltk.download('punkt')\n",
        "from nltk.tokenize import sent_tokenize"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xqIbx_kW-JLv",
        "colab_type": "text"
      },
      "source": [
        "**Tokenization** \n",
        "\n",
        "Tokenization is the first step in text analytics. The process of breaking down a text paragraph into smaller chunks such as words or sentence is called Tokenization. Token is a single entity that is building blocks for sentence or paragraph. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vi5PABi9-Syq",
        "colab_type": "text"
      },
      "source": [
        "**Sentence Tokenization** \n",
        "\n",
        "Sentence tokenizer breaks text paragraph into sentences. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z77VR18fsqde",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 768
        },
        "outputId": "0e4e4ff6-c76a-4fc5-bc96-6cd658f7c5b6"
      },
      "source": [
        "sent_tokenize(text)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['I must be honest with you.',\n",
              " 'I was about to go to sleep before I opened my mail and read your letter.',\n",
              " 'Now I am wide awake, sitting upright, because you made me question my preferences between shoes and slippers.',\n",
              " 'Be patient with me.',\n",
              " 'I need proper peaceful sleep and let’s sort this out.I am a shoe person.',\n",
              " 'Correction: I am a person obsessed with dry, dust-free feet.',\n",
              " 'Whatever helps me keep a grip on my walk and posture in spite of my profusely sweaty feet, I am that person.',\n",
              " 'Whatever keeps the grainy, pokey sensation of soil or sand in parts of my skin away, I am that person.',\n",
              " 'I guess, I am a shoe person because it is loyal to my whole feet.',\n",
              " 'Not making parts of it lying exposed and parts of it covered, without any symmetry.',\n",
              " 'I am also cotton sock person, preferably red.',\n",
              " 'Sometimes, I am a clean matte (anti-slippery) floor person.',\n",
              " 'Basically, I am a person with issues.I really liked your host-guest theory of travelling, how we do not just get out of home when we travel, the home expands when we step foot at a place engulfing it into our comfort zone.',\n",
              " 'But the thing is I wear socks all the time in my house, and open them when I go to bed, replacing the sock with a comforter.',\n",
              " 'I guess I need that comfort zone close and maybe, mine is restricted to my own skin.',\n",
              " 'So much so, that it doesn’t even include my own home.',\n",
              " 'When I travel, I like wearing shoes, with a firm grip on myself, dust free, protected.',\n",
              " 'When I face situations where I have to open them and I don’t want to, I usually wrinkle my nose.',\n",
              " 'It doesn’t get better that I can’t wear those shoes again with a dirty feet.',\n",
              " 'I take a handkerchief.Imagine my situation at the beach or in a hill stream, when the water feels soothing to my sweaty feet and removes the dust, only to attract more of it when I walk on the sand.',\n",
              " 'I let it irritate me those times, because I always have the ocean and the river on my side.',\n",
              " 'I like my bare feet drowning with no air bubbles left.',\n",
              " 'I don’t like it when droplets of water make only a part of my feet wet, only to let me realise and miss the comfort of dry skin and scowl at the linger of irritation due to a little dampness.',\n",
              " 'I don’t like it when I have to keep on brushing my left leg against my right to flatten the spheres of water.',\n",
              " 'And repeat it with the right.',\n",
              " 'I just realised why I don’t like drizzling rains.Eventually, when I have to get out of the water, I give up my fight with the dirt and wear those damn shoes with muddy feet.',\n",
              " 'Those are the times I miss my slippery slippers.',\n",
              " 'Don’t let my socks or shoes hear me say that.',\n",
              " 'They do not know that sometimes I keep a back-up pair of red flip-flops for such scenarios.',\n",
              " 'Does this make me a slipper person?',\n",
              " 'I guess I am a slipper person when my feet are ankle deep in mud, so that there is no place for me to complain.',\n",
              " 'No alternative.',\n",
              " 'No uneasy half-done feeling.',\n",
              " 'Otherwise, not much of a slipper person.I am a person who likes symmetry.',\n",
              " 'All in or none at all.',\n",
              " 'It’s difficult being me in this world with its shades of grey.',\n",
              " 'But then, my favourite colour is red and thankfully it covers the entire spectrum.',\n",
              " 'All I need is an emotion, and the rest is taken care of.Thank you for being the listener you are.',\n",
              " 'This has been a selfish post.',\n",
              " 'I used your letter for my sense of clarity.',\n",
              " 'You always make me reflect and dig deep.',\n",
              " 'I can finally sleep now.',\n",
              " 'Goodnight.']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pEAq1NdZ-YhK",
        "colab_type": "text"
      },
      "source": [
        "**Word Tokenization**\n",
        "\n",
        " Breaks sentence into words"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rLNEVkMbsqhu",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "94ff0b04-85c3-40de-c106-affdf02abe86"
      },
      "source": [
        "import regex\n",
        "regex.split(\"[\\s\\.\\,]\",text)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['I',\n",
              " 'must',\n",
              " 'be',\n",
              " 'honest',\n",
              " 'with',\n",
              " 'you',\n",
              " '',\n",
              " 'I',\n",
              " 'was',\n",
              " 'about',\n",
              " 'to',\n",
              " 'go',\n",
              " 'to',\n",
              " 'sleep',\n",
              " 'before',\n",
              " 'I',\n",
              " 'opened',\n",
              " 'my',\n",
              " 'mail',\n",
              " 'and',\n",
              " 'read',\n",
              " 'your',\n",
              " 'letter',\n",
              " '',\n",
              " 'Now',\n",
              " 'I',\n",
              " 'am',\n",
              " 'wide',\n",
              " 'awake',\n",
              " '',\n",
              " 'sitting',\n",
              " 'upright',\n",
              " '',\n",
              " 'because',\n",
              " 'you',\n",
              " 'made',\n",
              " 'me',\n",
              " 'question',\n",
              " 'my',\n",
              " 'preferences',\n",
              " 'between',\n",
              " 'shoes',\n",
              " 'and',\n",
              " 'slippers',\n",
              " '',\n",
              " 'Be',\n",
              " 'patient',\n",
              " 'with',\n",
              " 'me',\n",
              " '',\n",
              " 'I',\n",
              " 'need',\n",
              " 'proper',\n",
              " 'peaceful',\n",
              " 'sleep',\n",
              " 'and',\n",
              " 'let’s',\n",
              " 'sort',\n",
              " 'this',\n",
              " 'out',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'shoe',\n",
              " 'person',\n",
              " '',\n",
              " 'Correction:',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'person',\n",
              " 'obsessed',\n",
              " 'with',\n",
              " 'dry',\n",
              " '',\n",
              " 'dust-free',\n",
              " 'feet',\n",
              " '',\n",
              " 'Whatever',\n",
              " 'helps',\n",
              " 'me',\n",
              " 'keep',\n",
              " 'a',\n",
              " 'grip',\n",
              " 'on',\n",
              " 'my',\n",
              " 'walk',\n",
              " 'and',\n",
              " 'posture',\n",
              " 'in',\n",
              " 'spite',\n",
              " 'of',\n",
              " 'my',\n",
              " 'profusely',\n",
              " 'sweaty',\n",
              " 'feet',\n",
              " '',\n",
              " 'I',\n",
              " 'am',\n",
              " 'that',\n",
              " 'person',\n",
              " '',\n",
              " 'Whatever',\n",
              " 'keeps',\n",
              " 'the',\n",
              " 'grainy',\n",
              " '',\n",
              " 'pokey',\n",
              " 'sensation',\n",
              " 'of',\n",
              " 'soil',\n",
              " 'or',\n",
              " 'sand',\n",
              " 'in',\n",
              " 'parts',\n",
              " 'of',\n",
              " 'my',\n",
              " 'skin',\n",
              " 'away',\n",
              " '',\n",
              " 'I',\n",
              " 'am',\n",
              " 'that',\n",
              " 'person',\n",
              " '',\n",
              " 'I',\n",
              " 'guess',\n",
              " '',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'shoe',\n",
              " 'person',\n",
              " 'because',\n",
              " 'it',\n",
              " 'is',\n",
              " 'loyal',\n",
              " 'to',\n",
              " 'my',\n",
              " 'whole',\n",
              " 'feet',\n",
              " '',\n",
              " 'Not',\n",
              " 'making',\n",
              " 'parts',\n",
              " 'of',\n",
              " 'it',\n",
              " 'lying',\n",
              " 'exposed',\n",
              " 'and',\n",
              " 'parts',\n",
              " 'of',\n",
              " 'it',\n",
              " 'covered',\n",
              " '',\n",
              " 'without',\n",
              " 'any',\n",
              " 'symmetry',\n",
              " '',\n",
              " 'I',\n",
              " 'am',\n",
              " 'also',\n",
              " 'cotton',\n",
              " 'sock',\n",
              " 'person',\n",
              " '',\n",
              " 'preferably',\n",
              " 'red',\n",
              " '',\n",
              " 'Sometimes',\n",
              " '',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'clean',\n",
              " 'matte',\n",
              " '(anti-slippery)',\n",
              " 'floor',\n",
              " 'person',\n",
              " '',\n",
              " 'Basically',\n",
              " '',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'person',\n",
              " 'with',\n",
              " 'issues',\n",
              " 'I',\n",
              " 'really',\n",
              " 'liked',\n",
              " 'your',\n",
              " 'host-guest',\n",
              " 'theory',\n",
              " 'of',\n",
              " 'travelling',\n",
              " '',\n",
              " 'how',\n",
              " 'we',\n",
              " 'do',\n",
              " 'not',\n",
              " 'just',\n",
              " 'get',\n",
              " 'out',\n",
              " 'of',\n",
              " 'home',\n",
              " 'when',\n",
              " 'we',\n",
              " 'travel',\n",
              " '',\n",
              " 'the',\n",
              " 'home',\n",
              " 'expands',\n",
              " 'when',\n",
              " 'we',\n",
              " 'step',\n",
              " 'foot',\n",
              " 'at',\n",
              " 'a',\n",
              " 'place',\n",
              " 'engulfing',\n",
              " 'it',\n",
              " 'into',\n",
              " 'our',\n",
              " 'comfort',\n",
              " 'zone',\n",
              " '',\n",
              " 'But',\n",
              " 'the',\n",
              " 'thing',\n",
              " 'is',\n",
              " 'I',\n",
              " 'wear',\n",
              " 'socks',\n",
              " 'all',\n",
              " 'the',\n",
              " 'time',\n",
              " 'in',\n",
              " 'my',\n",
              " 'house',\n",
              " '',\n",
              " 'and',\n",
              " 'open',\n",
              " 'them',\n",
              " 'when',\n",
              " 'I',\n",
              " 'go',\n",
              " 'to',\n",
              " 'bed',\n",
              " '',\n",
              " 'replacing',\n",
              " 'the',\n",
              " 'sock',\n",
              " 'with',\n",
              " 'a',\n",
              " 'comforter',\n",
              " '',\n",
              " 'I',\n",
              " 'guess',\n",
              " 'I',\n",
              " 'need',\n",
              " 'that',\n",
              " 'comfort',\n",
              " 'zone',\n",
              " 'close',\n",
              " 'and',\n",
              " 'maybe',\n",
              " '',\n",
              " 'mine',\n",
              " 'is',\n",
              " 'restricted',\n",
              " 'to',\n",
              " 'my',\n",
              " 'own',\n",
              " 'skin',\n",
              " '',\n",
              " 'So',\n",
              " 'much',\n",
              " 'so',\n",
              " '',\n",
              " 'that',\n",
              " 'it',\n",
              " 'doesn’t',\n",
              " 'even',\n",
              " 'include',\n",
              " 'my',\n",
              " 'own',\n",
              " 'home',\n",
              " '',\n",
              " 'When',\n",
              " 'I',\n",
              " 'travel',\n",
              " '',\n",
              " 'I',\n",
              " 'like',\n",
              " 'wearing',\n",
              " 'shoes',\n",
              " '',\n",
              " 'with',\n",
              " 'a',\n",
              " 'firm',\n",
              " 'grip',\n",
              " 'on',\n",
              " 'myself',\n",
              " '',\n",
              " 'dust',\n",
              " 'free',\n",
              " '',\n",
              " 'protected',\n",
              " '',\n",
              " 'When',\n",
              " 'I',\n",
              " 'face',\n",
              " 'situations',\n",
              " 'where',\n",
              " 'I',\n",
              " 'have',\n",
              " 'to',\n",
              " 'open',\n",
              " 'them',\n",
              " 'and',\n",
              " 'I',\n",
              " 'don’t',\n",
              " 'want',\n",
              " 'to',\n",
              " '',\n",
              " 'I',\n",
              " 'usually',\n",
              " 'wrinkle',\n",
              " 'my',\n",
              " 'nose',\n",
              " '',\n",
              " 'It',\n",
              " 'doesn’t',\n",
              " 'get',\n",
              " 'better',\n",
              " 'that',\n",
              " 'I',\n",
              " 'can’t',\n",
              " 'wear',\n",
              " 'those',\n",
              " 'shoes',\n",
              " 'again',\n",
              " 'with',\n",
              " 'a',\n",
              " 'dirty',\n",
              " 'feet',\n",
              " '',\n",
              " 'I',\n",
              " 'take',\n",
              " 'a',\n",
              " 'handkerchief',\n",
              " 'Imagine',\n",
              " 'my',\n",
              " 'situation',\n",
              " 'at',\n",
              " 'the',\n",
              " 'beach',\n",
              " 'or',\n",
              " 'in',\n",
              " 'a',\n",
              " 'hill',\n",
              " 'stream',\n",
              " '',\n",
              " 'when',\n",
              " 'the',\n",
              " 'water',\n",
              " 'feels',\n",
              " 'soothing',\n",
              " 'to',\n",
              " 'my',\n",
              " 'sweaty',\n",
              " 'feet',\n",
              " 'and',\n",
              " 'removes',\n",
              " 'the',\n",
              " 'dust',\n",
              " '',\n",
              " 'only',\n",
              " 'to',\n",
              " 'attract',\n",
              " 'more',\n",
              " 'of',\n",
              " 'it',\n",
              " 'when',\n",
              " 'I',\n",
              " 'walk',\n",
              " 'on',\n",
              " 'the',\n",
              " 'sand',\n",
              " '',\n",
              " 'I',\n",
              " 'let',\n",
              " 'it',\n",
              " 'irritate',\n",
              " 'me',\n",
              " 'those',\n",
              " 'times',\n",
              " '',\n",
              " 'because',\n",
              " 'I',\n",
              " 'always',\n",
              " 'have',\n",
              " 'the',\n",
              " 'ocean',\n",
              " 'and',\n",
              " 'the',\n",
              " 'river',\n",
              " 'on',\n",
              " 'my',\n",
              " 'side',\n",
              " '',\n",
              " 'I',\n",
              " 'like',\n",
              " 'my',\n",
              " 'bare',\n",
              " 'feet',\n",
              " 'drowning',\n",
              " 'with',\n",
              " 'no',\n",
              " 'air',\n",
              " 'bubbles',\n",
              " 'left',\n",
              " '',\n",
              " 'I',\n",
              " 'don’t',\n",
              " 'like',\n",
              " 'it',\n",
              " 'when',\n",
              " 'droplets',\n",
              " 'of',\n",
              " 'water',\n",
              " 'make',\n",
              " 'only',\n",
              " 'a',\n",
              " 'part',\n",
              " 'of',\n",
              " 'my',\n",
              " 'feet',\n",
              " 'wet',\n",
              " '',\n",
              " 'only',\n",
              " 'to',\n",
              " 'let',\n",
              " 'me',\n",
              " 'realise',\n",
              " 'and',\n",
              " 'miss',\n",
              " 'the',\n",
              " 'comfort',\n",
              " 'of',\n",
              " 'dry',\n",
              " 'skin',\n",
              " 'and',\n",
              " 'scowl',\n",
              " 'at',\n",
              " 'the',\n",
              " 'linger',\n",
              " 'of',\n",
              " 'irritation',\n",
              " 'due',\n",
              " 'to',\n",
              " 'a',\n",
              " 'little',\n",
              " 'dampness',\n",
              " '',\n",
              " 'I',\n",
              " 'don’t',\n",
              " 'like',\n",
              " 'it',\n",
              " 'when',\n",
              " 'I',\n",
              " 'have',\n",
              " 'to',\n",
              " 'keep',\n",
              " 'on',\n",
              " 'brushing',\n",
              " 'my',\n",
              " 'left',\n",
              " 'leg',\n",
              " 'against',\n",
              " 'my',\n",
              " 'right',\n",
              " 'to',\n",
              " 'flatten',\n",
              " 'the',\n",
              " 'spheres',\n",
              " 'of',\n",
              " 'water',\n",
              " '',\n",
              " 'And',\n",
              " 'repeat',\n",
              " 'it',\n",
              " 'with',\n",
              " 'the',\n",
              " 'right',\n",
              " '',\n",
              " 'I',\n",
              " 'just',\n",
              " 'realised',\n",
              " 'why',\n",
              " 'I',\n",
              " 'don’t',\n",
              " 'like',\n",
              " 'drizzling',\n",
              " 'rains',\n",
              " 'Eventually',\n",
              " '',\n",
              " 'when',\n",
              " 'I',\n",
              " 'have',\n",
              " 'to',\n",
              " 'get',\n",
              " 'out',\n",
              " 'of',\n",
              " 'the',\n",
              " 'water',\n",
              " '',\n",
              " 'I',\n",
              " 'give',\n",
              " 'up',\n",
              " 'my',\n",
              " 'fight',\n",
              " 'with',\n",
              " 'the',\n",
              " 'dirt',\n",
              " 'and',\n",
              " 'wear',\n",
              " 'those',\n",
              " 'damn',\n",
              " 'shoes',\n",
              " 'with',\n",
              " 'muddy',\n",
              " 'feet',\n",
              " '',\n",
              " 'Those',\n",
              " 'are',\n",
              " 'the',\n",
              " 'times',\n",
              " 'I',\n",
              " 'miss',\n",
              " 'my',\n",
              " 'slippery',\n",
              " 'slippers',\n",
              " '',\n",
              " 'Don’t',\n",
              " 'let',\n",
              " 'my',\n",
              " 'socks',\n",
              " 'or',\n",
              " 'shoes',\n",
              " 'hear',\n",
              " 'me',\n",
              " 'say',\n",
              " 'that',\n",
              " '',\n",
              " 'They',\n",
              " 'do',\n",
              " 'not',\n",
              " 'know',\n",
              " 'that',\n",
              " 'sometimes',\n",
              " 'I',\n",
              " 'keep',\n",
              " 'a',\n",
              " 'back-up',\n",
              " 'pair',\n",
              " 'of',\n",
              " 'red',\n",
              " 'flip-flops',\n",
              " 'for',\n",
              " 'such',\n",
              " 'scenarios',\n",
              " '',\n",
              " 'Does',\n",
              " 'this',\n",
              " 'make',\n",
              " 'me',\n",
              " 'a',\n",
              " 'slipper',\n",
              " 'person?',\n",
              " 'I',\n",
              " 'guess',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'slipper',\n",
              " 'person',\n",
              " 'when',\n",
              " 'my',\n",
              " 'feet',\n",
              " 'are',\n",
              " 'ankle',\n",
              " 'deep',\n",
              " 'in',\n",
              " 'mud',\n",
              " '',\n",
              " 'so',\n",
              " 'that',\n",
              " 'there',\n",
              " 'is',\n",
              " 'no',\n",
              " 'place',\n",
              " 'for',\n",
              " 'me',\n",
              " 'to',\n",
              " 'complain',\n",
              " '',\n",
              " 'No',\n",
              " 'alternative',\n",
              " '',\n",
              " 'No',\n",
              " 'uneasy',\n",
              " 'half-done',\n",
              " 'feeling',\n",
              " '',\n",
              " 'Otherwise',\n",
              " '',\n",
              " 'not',\n",
              " 'much',\n",
              " 'of',\n",
              " 'a',\n",
              " 'slipper',\n",
              " 'person',\n",
              " 'I',\n",
              " 'am',\n",
              " 'a',\n",
              " 'person',\n",
              " 'who',\n",
              " 'likes',\n",
              " 'symmetry',\n",
              " '',\n",
              " 'All',\n",
              " 'in',\n",
              " 'or',\n",
              " 'none',\n",
              " 'at',\n",
              " 'all',\n",
              " '',\n",
              " 'It’s',\n",
              " 'difficult',\n",
              " 'being',\n",
              " 'me',\n",
              " 'in',\n",
              " 'this',\n",
              " 'world',\n",
              " 'with',\n",
              " 'its',\n",
              " 'shades',\n",
              " 'of',\n",
              " 'grey',\n",
              " '',\n",
              " 'But',\n",
              " 'then',\n",
              " '',\n",
              " 'my',\n",
              " 'favourite',\n",
              " 'colour',\n",
              " 'is',\n",
              " 'red',\n",
              " 'and',\n",
              " 'thankfully',\n",
              " 'it',\n",
              " 'covers',\n",
              " 'the',\n",
              " 'entire',\n",
              " 'spectrum',\n",
              " '',\n",
              " 'All',\n",
              " 'I',\n",
              " 'need',\n",
              " 'is',\n",
              " 'an',\n",
              " 'emotion',\n",
              " '',\n",
              " 'and',\n",
              " 'the',\n",
              " 'rest',\n",
              " 'is',\n",
              " 'taken',\n",
              " 'care',\n",
              " 'of',\n",
              " 'Thank',\n",
              " 'you',\n",
              " 'for',\n",
              " 'being',\n",
              " 'the',\n",
              " 'listener',\n",
              " 'you',\n",
              " 'are',\n",
              " '',\n",
              " 'This',\n",
              " 'has',\n",
              " 'been',\n",
              " 'a',\n",
              " 'selfish',\n",
              " 'post',\n",
              " '',\n",
              " 'I',\n",
              " 'used',\n",
              " 'your',\n",
              " 'letter',\n",
              " 'for',\n",
              " 'my',\n",
              " 'sense',\n",
              " 'of',\n",
              " 'clarity',\n",
              " '',\n",
              " 'You',\n",
              " 'always',\n",
              " 'make',\n",
              " 'me',\n",
              " 'reflect',\n",
              " 'and',\n",
              " 'dig',\n",
              " 'deep',\n",
              " '',\n",
              " 'I',\n",
              " 'can',\n",
              " 'finally',\n",
              " 'sleep',\n",
              " 'now',\n",
              " '',\n",
              " 'Goodnight',\n",
              " '']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hv2ht5FBsqbe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "58e89166-fe82-452d-b063-e47e0f78aeae"
      },
      "source": [
        "from nltk.tokenize import WordPunctTokenizer\n",
        "tokenizer = WordPunctTokenizer()\n",
        "u= tokenizer.tokenize(text)\n",
        "print(u)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['I', 'must', 'be', 'honest', 'with', 'you', '.', 'I', 'was', 'about', 'to', 'go', 'to', 'sleep', 'before', 'I', 'opened', 'my', 'mail', 'and', 'read', 'your', 'letter', '.', 'Now', 'I', 'am', 'wide', 'awake', ',', 'sitting', 'upright', ',', 'because', 'you', 'made', 'me', 'question', 'my', 'preferences', 'between', 'shoes', 'and', 'slippers', '.', 'Be', 'patient', 'with', 'me', '.', 'I', 'need', 'proper', 'peaceful', 'sleep', 'and', 'let', '’', 's', 'sort', 'this', 'out', '.', 'I', 'am', 'a', 'shoe', 'person', '.', 'Correction', ':', 'I', 'am', 'a', 'person', 'obsessed', 'with', 'dry', ',', 'dust', '-', 'free', 'feet', '.', 'Whatever', 'helps', 'me', 'keep', 'a', 'grip', 'on', 'my', 'walk', 'and', 'posture', 'in', 'spite', 'of', 'my', 'profusely', 'sweaty', 'feet', ',', 'I', 'am', 'that', 'person', '.', 'Whatever', 'keeps', 'the', 'grainy', ',', 'pokey', 'sensation', 'of', 'soil', 'or', 'sand', 'in', 'parts', 'of', 'my', 'skin', 'away', ',', 'I', 'am', 'that', 'person', '.', 'I', 'guess', ',', 'I', 'am', 'a', 'shoe', 'person', 'because', 'it', 'is', 'loyal', 'to', 'my', 'whole', 'feet', '.', 'Not', 'making', 'parts', 'of', 'it', 'lying', 'exposed', 'and', 'parts', 'of', 'it', 'covered', ',', 'without', 'any', 'symmetry', '.', 'I', 'am', 'also', 'cotton', 'sock', 'person', ',', 'preferably', 'red', '.', 'Sometimes', ',', 'I', 'am', 'a', 'clean', 'matte', '(', 'anti', '-', 'slippery', ')', 'floor', 'person', '.', 'Basically', ',', 'I', 'am', 'a', 'person', 'with', 'issues', '.', 'I', 'really', 'liked', 'your', 'host', '-', 'guest', 'theory', 'of', 'travelling', ',', 'how', 'we', 'do', 'not', 'just', 'get', 'out', 'of', 'home', 'when', 'we', 'travel', ',', 'the', 'home', 'expands', 'when', 'we', 'step', 'foot', 'at', 'a', 'place', 'engulfing', 'it', 'into', 'our', 'comfort', 'zone', '.', 'But', 'the', 'thing', 'is', 'I', 'wear', 'socks', 'all', 'the', 'time', 'in', 'my', 'house', ',', 'and', 'open', 'them', 'when', 'I', 'go', 'to', 'bed', ',', 'replacing', 'the', 'sock', 'with', 'a', 'comforter', '.', 'I', 'guess', 'I', 'need', 'that', 'comfort', 'zone', 'close', 'and', 'maybe', ',', 'mine', 'is', 'restricted', 'to', 'my', 'own', 'skin', '.', 'So', 'much', 'so', ',', 'that', 'it', 'doesn', '’', 't', 'even', 'include', 'my', 'own', 'home', '.', 'When', 'I', 'travel', ',', 'I', 'like', 'wearing', 'shoes', ',', 'with', 'a', 'firm', 'grip', 'on', 'myself', ',', 'dust', 'free', ',', 'protected', '.', 'When', 'I', 'face', 'situations', 'where', 'I', 'have', 'to', 'open', 'them', 'and', 'I', 'don', '’', 't', 'want', 'to', ',', 'I', 'usually', 'wrinkle', 'my', 'nose', '.', 'It', 'doesn', '’', 't', 'get', 'better', 'that', 'I', 'can', '’', 't', 'wear', 'those', 'shoes', 'again', 'with', 'a', 'dirty', 'feet', '.', 'I', 'take', 'a', 'handkerchief', '.', 'Imagine', 'my', 'situation', 'at', 'the', 'beach', 'or', 'in', 'a', 'hill', 'stream', ',', 'when', 'the', 'water', 'feels', 'soothing', 'to', 'my', 'sweaty', 'feet', 'and', 'removes', 'the', 'dust', ',', 'only', 'to', 'attract', 'more', 'of', 'it', 'when', 'I', 'walk', 'on', 'the', 'sand', '.', 'I', 'let', 'it', 'irritate', 'me', 'those', 'times', ',', 'because', 'I', 'always', 'have', 'the', 'ocean', 'and', 'the', 'river', 'on', 'my', 'side', '.', 'I', 'like', 'my', 'bare', 'feet', 'drowning', 'with', 'no', 'air', 'bubbles', 'left', '.', 'I', 'don', '’', 't', 'like', 'it', 'when', 'droplets', 'of', 'water', 'make', 'only', 'a', 'part', 'of', 'my', 'feet', 'wet', ',', 'only', 'to', 'let', 'me', 'realise', 'and', 'miss', 'the', 'comfort', 'of', 'dry', 'skin', 'and', 'scowl', 'at', 'the', 'linger', 'of', 'irritation', 'due', 'to', 'a', 'little', 'dampness', '.', 'I', 'don', '’', 't', 'like', 'it', 'when', 'I', 'have', 'to', 'keep', 'on', 'brushing', 'my', 'left', 'leg', 'against', 'my', 'right', 'to', 'flatten', 'the', 'spheres', 'of', 'water', '.', 'And', 'repeat', 'it', 'with', 'the', 'right', '.', 'I', 'just', 'realised', 'why', 'I', 'don', '’', 't', 'like', 'drizzling', 'rains', '.', 'Eventually', ',', 'when', 'I', 'have', 'to', 'get', 'out', 'of', 'the', 'water', ',', 'I', 'give', 'up', 'my', 'fight', 'with', 'the', 'dirt', 'and', 'wear', 'those', 'damn', 'shoes', 'with', 'muddy', 'feet', '.', 'Those', 'are', 'the', 'times', 'I', 'miss', 'my', 'slippery', 'slippers', '.', 'Don', '’', 't', 'let', 'my', 'socks', 'or', 'shoes', 'hear', 'me', 'say', 'that', '.', 'They', 'do', 'not', 'know', 'that', 'sometimes', 'I', 'keep', 'a', 'back', '-', 'up', 'pair', 'of', 'red', 'flip', '-', 'flops', 'for', 'such', 'scenarios', '.', 'Does', 'this', 'make', 'me', 'a', 'slipper', 'person', '?', 'I', 'guess', 'I', 'am', 'a', 'slipper', 'person', 'when', 'my', 'feet', 'are', 'ankle', 'deep', 'in', 'mud', ',', 'so', 'that', 'there', 'is', 'no', 'place', 'for', 'me', 'to', 'complain', '.', 'No', 'alternative', '.', 'No', 'uneasy', 'half', '-', 'done', 'feeling', '.', 'Otherwise', ',', 'not', 'much', 'of', 'a', 'slipper', 'person', '.', 'I', 'am', 'a', 'person', 'who', 'likes', 'symmetry', '.', 'All', 'in', 'or', 'none', 'at', 'all', '.', 'It', '’', 's', 'difficult', 'being', 'me', 'in', 'this', 'world', 'with', 'its', 'shades', 'of', 'grey', '.', 'But', 'then', ',', 'my', 'favourite', 'colour', 'is', 'red', 'and', 'thankfully', 'it', 'covers', 'the', 'entire', 'spectrum', '.', 'All', 'I', 'need', 'is', 'an', 'emotion', ',', 'and', 'the', 'rest', 'is', 'taken', 'care', 'of', '.', 'Thank', 'you', 'for', 'being', 'the', 'listener', 'you', 'are', '.', 'This', 'has', 'been', 'a', 'selfish', 'post', '.', 'I', 'used', 'your', 'letter', 'for', 'my', 'sense', 'of', 'clarity', '.', 'You', 'always', 'make', 'me', 'reflect', 'and', 'dig', 'deep', '.', 'I', 'can', 'finally', 'sleep', 'now', '.', 'Goodnight', '.']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mpLsDRuw9-E5",
        "colab_type": "text"
      },
      "source": [
        "**Stemming** \n",
        "\n",
        "is the process of reducing a word to its word stem that affixes to suffixes and prefixes or to the roots of words known as a lemma. Stemming is important in natural language understanding (NLU) and obviously in natural language processing (NLP) as well.\n",
        "It is a process of linguistic normalization, which reduces words to their word root word or chops off the derivational affixes. For example, connection, connected, connecting word reduce to a common word \"connect\"."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7SmEV8qItDLW",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "b6813713-9491-4847-a1c9-2e2838a31901"
      },
      "source": [
        "from nltk.stem import PorterStemmer\n",
        "stemmer= PorterStemmer()\n",
        "for word in u:\n",
        "  print(f\"{word}>>>{stemmer.stem(word)}\")"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "I>>>I\n",
            "must>>>must\n",
            "be>>>be\n",
            "honest>>>honest\n",
            "with>>>with\n",
            "you>>>you\n",
            ".>>>.\n",
            "I>>>I\n",
            "was>>>wa\n",
            "about>>>about\n",
            "to>>>to\n",
            "go>>>go\n",
            "to>>>to\n",
            "sleep>>>sleep\n",
            "before>>>befor\n",
            "I>>>I\n",
            "opened>>>open\n",
            "my>>>my\n",
            "mail>>>mail\n",
            "and>>>and\n",
            "read>>>read\n",
            "your>>>your\n",
            "letter>>>letter\n",
            ".>>>.\n",
            "Now>>>now\n",
            "I>>>I\n",
            "am>>>am\n",
            "wide>>>wide\n",
            "awake>>>awak\n",
            ",>>>,\n",
            "sitting>>>sit\n",
            "upright>>>upright\n",
            ",>>>,\n",
            "because>>>becaus\n",
            "you>>>you\n",
            "made>>>made\n",
            "me>>>me\n",
            "question>>>question\n",
            "my>>>my\n",
            "preferences>>>prefer\n",
            "between>>>between\n",
            "shoes>>>shoe\n",
            "and>>>and\n",
            "slippers>>>slipper\n",
            ".>>>.\n",
            "Be>>>Be\n",
            "patient>>>patient\n",
            "with>>>with\n",
            "me>>>me\n",
            ".>>>.\n",
            "I>>>I\n",
            "need>>>need\n",
            "proper>>>proper\n",
            "peaceful>>>peac\n",
            "sleep>>>sleep\n",
            "and>>>and\n",
            "let>>>let\n",
            "’>>>’\n",
            "s>>>s\n",
            "sort>>>sort\n",
            "this>>>thi\n",
            "out>>>out\n",
            ".>>>.\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "shoe>>>shoe\n",
            "person>>>person\n",
            ".>>>.\n",
            "Correction>>>correct\n",
            ":>>>:\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "person>>>person\n",
            "obsessed>>>obsess\n",
            "with>>>with\n",
            "dry>>>dri\n",
            ",>>>,\n",
            "dust>>>dust\n",
            "->>>-\n",
            "free>>>free\n",
            "feet>>>feet\n",
            ".>>>.\n",
            "Whatever>>>whatev\n",
            "helps>>>help\n",
            "me>>>me\n",
            "keep>>>keep\n",
            "a>>>a\n",
            "grip>>>grip\n",
            "on>>>on\n",
            "my>>>my\n",
            "walk>>>walk\n",
            "and>>>and\n",
            "posture>>>postur\n",
            "in>>>in\n",
            "spite>>>spite\n",
            "of>>>of\n",
            "my>>>my\n",
            "profusely>>>profus\n",
            "sweaty>>>sweati\n",
            "feet>>>feet\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "that>>>that\n",
            "person>>>person\n",
            ".>>>.\n",
            "Whatever>>>whatev\n",
            "keeps>>>keep\n",
            "the>>>the\n",
            "grainy>>>graini\n",
            ",>>>,\n",
            "pokey>>>pokey\n",
            "sensation>>>sensat\n",
            "of>>>of\n",
            "soil>>>soil\n",
            "or>>>or\n",
            "sand>>>sand\n",
            "in>>>in\n",
            "parts>>>part\n",
            "of>>>of\n",
            "my>>>my\n",
            "skin>>>skin\n",
            "away>>>away\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "that>>>that\n",
            "person>>>person\n",
            ".>>>.\n",
            "I>>>I\n",
            "guess>>>guess\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "shoe>>>shoe\n",
            "person>>>person\n",
            "because>>>becaus\n",
            "it>>>it\n",
            "is>>>is\n",
            "loyal>>>loyal\n",
            "to>>>to\n",
            "my>>>my\n",
            "whole>>>whole\n",
            "feet>>>feet\n",
            ".>>>.\n",
            "Not>>>not\n",
            "making>>>make\n",
            "parts>>>part\n",
            "of>>>of\n",
            "it>>>it\n",
            "lying>>>lie\n",
            "exposed>>>expos\n",
            "and>>>and\n",
            "parts>>>part\n",
            "of>>>of\n",
            "it>>>it\n",
            "covered>>>cover\n",
            ",>>>,\n",
            "without>>>without\n",
            "any>>>ani\n",
            "symmetry>>>symmetri\n",
            ".>>>.\n",
            "I>>>I\n",
            "am>>>am\n",
            "also>>>also\n",
            "cotton>>>cotton\n",
            "sock>>>sock\n",
            "person>>>person\n",
            ",>>>,\n",
            "preferably>>>prefer\n",
            "red>>>red\n",
            ".>>>.\n",
            "Sometimes>>>sometim\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "clean>>>clean\n",
            "matte>>>matt\n",
            "(>>>(\n",
            "anti>>>anti\n",
            "->>>-\n",
            "slippery>>>slipperi\n",
            ")>>>)\n",
            "floor>>>floor\n",
            "person>>>person\n",
            ".>>>.\n",
            "Basically>>>basic\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "person>>>person\n",
            "with>>>with\n",
            "issues>>>issu\n",
            ".>>>.\n",
            "I>>>I\n",
            "really>>>realli\n",
            "liked>>>like\n",
            "your>>>your\n",
            "host>>>host\n",
            "->>>-\n",
            "guest>>>guest\n",
            "theory>>>theori\n",
            "of>>>of\n",
            "travelling>>>travel\n",
            ",>>>,\n",
            "how>>>how\n",
            "we>>>we\n",
            "do>>>do\n",
            "not>>>not\n",
            "just>>>just\n",
            "get>>>get\n",
            "out>>>out\n",
            "of>>>of\n",
            "home>>>home\n",
            "when>>>when\n",
            "we>>>we\n",
            "travel>>>travel\n",
            ",>>>,\n",
            "the>>>the\n",
            "home>>>home\n",
            "expands>>>expand\n",
            "when>>>when\n",
            "we>>>we\n",
            "step>>>step\n",
            "foot>>>foot\n",
            "at>>>at\n",
            "a>>>a\n",
            "place>>>place\n",
            "engulfing>>>engulf\n",
            "it>>>it\n",
            "into>>>into\n",
            "our>>>our\n",
            "comfort>>>comfort\n",
            "zone>>>zone\n",
            ".>>>.\n",
            "But>>>but\n",
            "the>>>the\n",
            "thing>>>thing\n",
            "is>>>is\n",
            "I>>>I\n",
            "wear>>>wear\n",
            "socks>>>sock\n",
            "all>>>all\n",
            "the>>>the\n",
            "time>>>time\n",
            "in>>>in\n",
            "my>>>my\n",
            "house>>>hous\n",
            ",>>>,\n",
            "and>>>and\n",
            "open>>>open\n",
            "them>>>them\n",
            "when>>>when\n",
            "I>>>I\n",
            "go>>>go\n",
            "to>>>to\n",
            "bed>>>bed\n",
            ",>>>,\n",
            "replacing>>>replac\n",
            "the>>>the\n",
            "sock>>>sock\n",
            "with>>>with\n",
            "a>>>a\n",
            "comforter>>>comfort\n",
            ".>>>.\n",
            "I>>>I\n",
            "guess>>>guess\n",
            "I>>>I\n",
            "need>>>need\n",
            "that>>>that\n",
            "comfort>>>comfort\n",
            "zone>>>zone\n",
            "close>>>close\n",
            "and>>>and\n",
            "maybe>>>mayb\n",
            ",>>>,\n",
            "mine>>>mine\n",
            "is>>>is\n",
            "restricted>>>restrict\n",
            "to>>>to\n",
            "my>>>my\n",
            "own>>>own\n",
            "skin>>>skin\n",
            ".>>>.\n",
            "So>>>So\n",
            "much>>>much\n",
            "so>>>so\n",
            ",>>>,\n",
            "that>>>that\n",
            "it>>>it\n",
            "doesn>>>doesn\n",
            "’>>>’\n",
            "t>>>t\n",
            "even>>>even\n",
            "include>>>includ\n",
            "my>>>my\n",
            "own>>>own\n",
            "home>>>home\n",
            ".>>>.\n",
            "When>>>when\n",
            "I>>>I\n",
            "travel>>>travel\n",
            ",>>>,\n",
            "I>>>I\n",
            "like>>>like\n",
            "wearing>>>wear\n",
            "shoes>>>shoe\n",
            ",>>>,\n",
            "with>>>with\n",
            "a>>>a\n",
            "firm>>>firm\n",
            "grip>>>grip\n",
            "on>>>on\n",
            "myself>>>myself\n",
            ",>>>,\n",
            "dust>>>dust\n",
            "free>>>free\n",
            ",>>>,\n",
            "protected>>>protect\n",
            ".>>>.\n",
            "When>>>when\n",
            "I>>>I\n",
            "face>>>face\n",
            "situations>>>situat\n",
            "where>>>where\n",
            "I>>>I\n",
            "have>>>have\n",
            "to>>>to\n",
            "open>>>open\n",
            "them>>>them\n",
            "and>>>and\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "want>>>want\n",
            "to>>>to\n",
            ",>>>,\n",
            "I>>>I\n",
            "usually>>>usual\n",
            "wrinkle>>>wrinkl\n",
            "my>>>my\n",
            "nose>>>nose\n",
            ".>>>.\n",
            "It>>>It\n",
            "doesn>>>doesn\n",
            "’>>>’\n",
            "t>>>t\n",
            "get>>>get\n",
            "better>>>better\n",
            "that>>>that\n",
            "I>>>I\n",
            "can>>>can\n",
            "’>>>’\n",
            "t>>>t\n",
            "wear>>>wear\n",
            "those>>>those\n",
            "shoes>>>shoe\n",
            "again>>>again\n",
            "with>>>with\n",
            "a>>>a\n",
            "dirty>>>dirti\n",
            "feet>>>feet\n",
            ".>>>.\n",
            "I>>>I\n",
            "take>>>take\n",
            "a>>>a\n",
            "handkerchief>>>handkerchief\n",
            ".>>>.\n",
            "Imagine>>>imagin\n",
            "my>>>my\n",
            "situation>>>situat\n",
            "at>>>at\n",
            "the>>>the\n",
            "beach>>>beach\n",
            "or>>>or\n",
            "in>>>in\n",
            "a>>>a\n",
            "hill>>>hill\n",
            "stream>>>stream\n",
            ",>>>,\n",
            "when>>>when\n",
            "the>>>the\n",
            "water>>>water\n",
            "feels>>>feel\n",
            "soothing>>>sooth\n",
            "to>>>to\n",
            "my>>>my\n",
            "sweaty>>>sweati\n",
            "feet>>>feet\n",
            "and>>>and\n",
            "removes>>>remov\n",
            "the>>>the\n",
            "dust>>>dust\n",
            ",>>>,\n",
            "only>>>onli\n",
            "to>>>to\n",
            "attract>>>attract\n",
            "more>>>more\n",
            "of>>>of\n",
            "it>>>it\n",
            "when>>>when\n",
            "I>>>I\n",
            "walk>>>walk\n",
            "on>>>on\n",
            "the>>>the\n",
            "sand>>>sand\n",
            ".>>>.\n",
            "I>>>I\n",
            "let>>>let\n",
            "it>>>it\n",
            "irritate>>>irrit\n",
            "me>>>me\n",
            "those>>>those\n",
            "times>>>time\n",
            ",>>>,\n",
            "because>>>becaus\n",
            "I>>>I\n",
            "always>>>alway\n",
            "have>>>have\n",
            "the>>>the\n",
            "ocean>>>ocean\n",
            "and>>>and\n",
            "the>>>the\n",
            "river>>>river\n",
            "on>>>on\n",
            "my>>>my\n",
            "side>>>side\n",
            ".>>>.\n",
            "I>>>I\n",
            "like>>>like\n",
            "my>>>my\n",
            "bare>>>bare\n",
            "feet>>>feet\n",
            "drowning>>>drown\n",
            "with>>>with\n",
            "no>>>no\n",
            "air>>>air\n",
            "bubbles>>>bubbl\n",
            "left>>>left\n",
            ".>>>.\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "like>>>like\n",
            "it>>>it\n",
            "when>>>when\n",
            "droplets>>>droplet\n",
            "of>>>of\n",
            "water>>>water\n",
            "make>>>make\n",
            "only>>>onli\n",
            "a>>>a\n",
            "part>>>part\n",
            "of>>>of\n",
            "my>>>my\n",
            "feet>>>feet\n",
            "wet>>>wet\n",
            ",>>>,\n",
            "only>>>onli\n",
            "to>>>to\n",
            "let>>>let\n",
            "me>>>me\n",
            "realise>>>realis\n",
            "and>>>and\n",
            "miss>>>miss\n",
            "the>>>the\n",
            "comfort>>>comfort\n",
            "of>>>of\n",
            "dry>>>dri\n",
            "skin>>>skin\n",
            "and>>>and\n",
            "scowl>>>scowl\n",
            "at>>>at\n",
            "the>>>the\n",
            "linger>>>linger\n",
            "of>>>of\n",
            "irritation>>>irrit\n",
            "due>>>due\n",
            "to>>>to\n",
            "a>>>a\n",
            "little>>>littl\n",
            "dampness>>>damp\n",
            ".>>>.\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "like>>>like\n",
            "it>>>it\n",
            "when>>>when\n",
            "I>>>I\n",
            "have>>>have\n",
            "to>>>to\n",
            "keep>>>keep\n",
            "on>>>on\n",
            "brushing>>>brush\n",
            "my>>>my\n",
            "left>>>left\n",
            "leg>>>leg\n",
            "against>>>against\n",
            "my>>>my\n",
            "right>>>right\n",
            "to>>>to\n",
            "flatten>>>flatten\n",
            "the>>>the\n",
            "spheres>>>sphere\n",
            "of>>>of\n",
            "water>>>water\n",
            ".>>>.\n",
            "And>>>and\n",
            "repeat>>>repeat\n",
            "it>>>it\n",
            "with>>>with\n",
            "the>>>the\n",
            "right>>>right\n",
            ".>>>.\n",
            "I>>>I\n",
            "just>>>just\n",
            "realised>>>realis\n",
            "why>>>whi\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "like>>>like\n",
            "drizzling>>>drizzl\n",
            "rains>>>rain\n",
            ".>>>.\n",
            "Eventually>>>eventu\n",
            ",>>>,\n",
            "when>>>when\n",
            "I>>>I\n",
            "have>>>have\n",
            "to>>>to\n",
            "get>>>get\n",
            "out>>>out\n",
            "of>>>of\n",
            "the>>>the\n",
            "water>>>water\n",
            ",>>>,\n",
            "I>>>I\n",
            "give>>>give\n",
            "up>>>up\n",
            "my>>>my\n",
            "fight>>>fight\n",
            "with>>>with\n",
            "the>>>the\n",
            "dirt>>>dirt\n",
            "and>>>and\n",
            "wear>>>wear\n",
            "those>>>those\n",
            "damn>>>damn\n",
            "shoes>>>shoe\n",
            "with>>>with\n",
            "muddy>>>muddi\n",
            "feet>>>feet\n",
            ".>>>.\n",
            "Those>>>those\n",
            "are>>>are\n",
            "the>>>the\n",
            "times>>>time\n",
            "I>>>I\n",
            "miss>>>miss\n",
            "my>>>my\n",
            "slippery>>>slipperi\n",
            "slippers>>>slipper\n",
            ".>>>.\n",
            "Don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "let>>>let\n",
            "my>>>my\n",
            "socks>>>sock\n",
            "or>>>or\n",
            "shoes>>>shoe\n",
            "hear>>>hear\n",
            "me>>>me\n",
            "say>>>say\n",
            "that>>>that\n",
            ".>>>.\n",
            "They>>>they\n",
            "do>>>do\n",
            "not>>>not\n",
            "know>>>know\n",
            "that>>>that\n",
            "sometimes>>>sometim\n",
            "I>>>I\n",
            "keep>>>keep\n",
            "a>>>a\n",
            "back>>>back\n",
            "->>>-\n",
            "up>>>up\n",
            "pair>>>pair\n",
            "of>>>of\n",
            "red>>>red\n",
            "flip>>>flip\n",
            "->>>-\n",
            "flops>>>flop\n",
            "for>>>for\n",
            "such>>>such\n",
            "scenarios>>>scenario\n",
            ".>>>.\n",
            "Does>>>doe\n",
            "this>>>thi\n",
            "make>>>make\n",
            "me>>>me\n",
            "a>>>a\n",
            "slipper>>>slipper\n",
            "person>>>person\n",
            "?>>>?\n",
            "I>>>I\n",
            "guess>>>guess\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "slipper>>>slipper\n",
            "person>>>person\n",
            "when>>>when\n",
            "my>>>my\n",
            "feet>>>feet\n",
            "are>>>are\n",
            "ankle>>>ankl\n",
            "deep>>>deep\n",
            "in>>>in\n",
            "mud>>>mud\n",
            ",>>>,\n",
            "so>>>so\n",
            "that>>>that\n",
            "there>>>there\n",
            "is>>>is\n",
            "no>>>no\n",
            "place>>>place\n",
            "for>>>for\n",
            "me>>>me\n",
            "to>>>to\n",
            "complain>>>complain\n",
            ".>>>.\n",
            "No>>>No\n",
            "alternative>>>altern\n",
            ".>>>.\n",
            "No>>>No\n",
            "uneasy>>>uneasi\n",
            "half>>>half\n",
            "->>>-\n",
            "done>>>done\n",
            "feeling>>>feel\n",
            ".>>>.\n",
            "Otherwise>>>otherwis\n",
            ",>>>,\n",
            "not>>>not\n",
            "much>>>much\n",
            "of>>>of\n",
            "a>>>a\n",
            "slipper>>>slipper\n",
            "person>>>person\n",
            ".>>>.\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "person>>>person\n",
            "who>>>who\n",
            "likes>>>like\n",
            "symmetry>>>symmetri\n",
            ".>>>.\n",
            "All>>>all\n",
            "in>>>in\n",
            "or>>>or\n",
            "none>>>none\n",
            "at>>>at\n",
            "all>>>all\n",
            ".>>>.\n",
            "It>>>It\n",
            "’>>>’\n",
            "s>>>s\n",
            "difficult>>>difficult\n",
            "being>>>be\n",
            "me>>>me\n",
            "in>>>in\n",
            "this>>>thi\n",
            "world>>>world\n",
            "with>>>with\n",
            "its>>>it\n",
            "shades>>>shade\n",
            "of>>>of\n",
            "grey>>>grey\n",
            ".>>>.\n",
            "But>>>but\n",
            "then>>>then\n",
            ",>>>,\n",
            "my>>>my\n",
            "favourite>>>favourit\n",
            "colour>>>colour\n",
            "is>>>is\n",
            "red>>>red\n",
            "and>>>and\n",
            "thankfully>>>thank\n",
            "it>>>it\n",
            "covers>>>cover\n",
            "the>>>the\n",
            "entire>>>entir\n",
            "spectrum>>>spectrum\n",
            ".>>>.\n",
            "All>>>all\n",
            "I>>>I\n",
            "need>>>need\n",
            "is>>>is\n",
            "an>>>an\n",
            "emotion>>>emot\n",
            ",>>>,\n",
            "and>>>and\n",
            "the>>>the\n",
            "rest>>>rest\n",
            "is>>>is\n",
            "taken>>>taken\n",
            "care>>>care\n",
            "of>>>of\n",
            ".>>>.\n",
            "Thank>>>thank\n",
            "you>>>you\n",
            "for>>>for\n",
            "being>>>be\n",
            "the>>>the\n",
            "listener>>>listen\n",
            "you>>>you\n",
            "are>>>are\n",
            ".>>>.\n",
            "This>>>thi\n",
            "has>>>ha\n",
            "been>>>been\n",
            "a>>>a\n",
            "selfish>>>selfish\n",
            "post>>>post\n",
            ".>>>.\n",
            "I>>>I\n",
            "used>>>use\n",
            "your>>>your\n",
            "letter>>>letter\n",
            "for>>>for\n",
            "my>>>my\n",
            "sense>>>sens\n",
            "of>>>of\n",
            "clarity>>>clariti\n",
            ".>>>.\n",
            "You>>>you\n",
            "always>>>alway\n",
            "make>>>make\n",
            "me>>>me\n",
            "reflect>>>reflect\n",
            "and>>>and\n",
            "dig>>>dig\n",
            "deep>>>deep\n",
            ".>>>.\n",
            "I>>>I\n",
            "can>>>can\n",
            "finally>>>final\n",
            "sleep>>>sleep\n",
            "now>>>now\n",
            ".>>>.\n",
            "Goodnight>>>goodnight\n",
            ".>>>.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aGmaNxR-9wAy",
        "colab_type": "text"
      },
      "source": [
        "**Lemmatization**: \n",
        "\n",
        "retrieving the source of the word Lemmatization reduces words to their base word, which is linguistically correct lemmas. It transforms root word with the use of vocabulary and morphological analysis. Lemmatization is usually more sophisticated than stemming. Stemmer works on an individual word without knowledge of the context. For example, The word \"better\" has \"good\" as its lemma. This thing will miss by stemming because it requires a dictionary look-up."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wFFaMiH1tDQ_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "3ef91928-6cb6-459f-b62a-b983351d1c1a"
      },
      "source": [
        "from nltk.stem import WordNetLemmatizer\n",
        "lemmatizer= WordNetLemmatizer()\n",
        "import nltk\n",
        "nltk.download('wordnet')\n",
        "for word in u:\n",
        "  print(f\"{word}>>>{lemmatizer.lemmatize(word)}\")"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/wordnet.zip.\n",
            "I>>>I\n",
            "must>>>must\n",
            "be>>>be\n",
            "honest>>>honest\n",
            "with>>>with\n",
            "you>>>you\n",
            ".>>>.\n",
            "I>>>I\n",
            "was>>>wa\n",
            "about>>>about\n",
            "to>>>to\n",
            "go>>>go\n",
            "to>>>to\n",
            "sleep>>>sleep\n",
            "before>>>before\n",
            "I>>>I\n",
            "opened>>>opened\n",
            "my>>>my\n",
            "mail>>>mail\n",
            "and>>>and\n",
            "read>>>read\n",
            "your>>>your\n",
            "letter>>>letter\n",
            ".>>>.\n",
            "Now>>>Now\n",
            "I>>>I\n",
            "am>>>am\n",
            "wide>>>wide\n",
            "awake>>>awake\n",
            ",>>>,\n",
            "sitting>>>sitting\n",
            "upright>>>upright\n",
            ",>>>,\n",
            "because>>>because\n",
            "you>>>you\n",
            "made>>>made\n",
            "me>>>me\n",
            "question>>>question\n",
            "my>>>my\n",
            "preferences>>>preference\n",
            "between>>>between\n",
            "shoes>>>shoe\n",
            "and>>>and\n",
            "slippers>>>slipper\n",
            ".>>>.\n",
            "Be>>>Be\n",
            "patient>>>patient\n",
            "with>>>with\n",
            "me>>>me\n",
            ".>>>.\n",
            "I>>>I\n",
            "need>>>need\n",
            "proper>>>proper\n",
            "peaceful>>>peaceful\n",
            "sleep>>>sleep\n",
            "and>>>and\n",
            "let>>>let\n",
            "’>>>’\n",
            "s>>>s\n",
            "sort>>>sort\n",
            "this>>>this\n",
            "out>>>out\n",
            ".>>>.\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "shoe>>>shoe\n",
            "person>>>person\n",
            ".>>>.\n",
            "Correction>>>Correction\n",
            ":>>>:\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "person>>>person\n",
            "obsessed>>>obsessed\n",
            "with>>>with\n",
            "dry>>>dry\n",
            ",>>>,\n",
            "dust>>>dust\n",
            "->>>-\n",
            "free>>>free\n",
            "feet>>>foot\n",
            ".>>>.\n",
            "Whatever>>>Whatever\n",
            "helps>>>help\n",
            "me>>>me\n",
            "keep>>>keep\n",
            "a>>>a\n",
            "grip>>>grip\n",
            "on>>>on\n",
            "my>>>my\n",
            "walk>>>walk\n",
            "and>>>and\n",
            "posture>>>posture\n",
            "in>>>in\n",
            "spite>>>spite\n",
            "of>>>of\n",
            "my>>>my\n",
            "profusely>>>profusely\n",
            "sweaty>>>sweaty\n",
            "feet>>>foot\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "that>>>that\n",
            "person>>>person\n",
            ".>>>.\n",
            "Whatever>>>Whatever\n",
            "keeps>>>keep\n",
            "the>>>the\n",
            "grainy>>>grainy\n",
            ",>>>,\n",
            "pokey>>>pokey\n",
            "sensation>>>sensation\n",
            "of>>>of\n",
            "soil>>>soil\n",
            "or>>>or\n",
            "sand>>>sand\n",
            "in>>>in\n",
            "parts>>>part\n",
            "of>>>of\n",
            "my>>>my\n",
            "skin>>>skin\n",
            "away>>>away\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "that>>>that\n",
            "person>>>person\n",
            ".>>>.\n",
            "I>>>I\n",
            "guess>>>guess\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "shoe>>>shoe\n",
            "person>>>person\n",
            "because>>>because\n",
            "it>>>it\n",
            "is>>>is\n",
            "loyal>>>loyal\n",
            "to>>>to\n",
            "my>>>my\n",
            "whole>>>whole\n",
            "feet>>>foot\n",
            ".>>>.\n",
            "Not>>>Not\n",
            "making>>>making\n",
            "parts>>>part\n",
            "of>>>of\n",
            "it>>>it\n",
            "lying>>>lying\n",
            "exposed>>>exposed\n",
            "and>>>and\n",
            "parts>>>part\n",
            "of>>>of\n",
            "it>>>it\n",
            "covered>>>covered\n",
            ",>>>,\n",
            "without>>>without\n",
            "any>>>any\n",
            "symmetry>>>symmetry\n",
            ".>>>.\n",
            "I>>>I\n",
            "am>>>am\n",
            "also>>>also\n",
            "cotton>>>cotton\n",
            "sock>>>sock\n",
            "person>>>person\n",
            ",>>>,\n",
            "preferably>>>preferably\n",
            "red>>>red\n",
            ".>>>.\n",
            "Sometimes>>>Sometimes\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "clean>>>clean\n",
            "matte>>>matte\n",
            "(>>>(\n",
            "anti>>>anti\n",
            "->>>-\n",
            "slippery>>>slippery\n",
            ")>>>)\n",
            "floor>>>floor\n",
            "person>>>person\n",
            ".>>>.\n",
            "Basically>>>Basically\n",
            ",>>>,\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "person>>>person\n",
            "with>>>with\n",
            "issues>>>issue\n",
            ".>>>.\n",
            "I>>>I\n",
            "really>>>really\n",
            "liked>>>liked\n",
            "your>>>your\n",
            "host>>>host\n",
            "->>>-\n",
            "guest>>>guest\n",
            "theory>>>theory\n",
            "of>>>of\n",
            "travelling>>>travelling\n",
            ",>>>,\n",
            "how>>>how\n",
            "we>>>we\n",
            "do>>>do\n",
            "not>>>not\n",
            "just>>>just\n",
            "get>>>get\n",
            "out>>>out\n",
            "of>>>of\n",
            "home>>>home\n",
            "when>>>when\n",
            "we>>>we\n",
            "travel>>>travel\n",
            ",>>>,\n",
            "the>>>the\n",
            "home>>>home\n",
            "expands>>>expands\n",
            "when>>>when\n",
            "we>>>we\n",
            "step>>>step\n",
            "foot>>>foot\n",
            "at>>>at\n",
            "a>>>a\n",
            "place>>>place\n",
            "engulfing>>>engulfing\n",
            "it>>>it\n",
            "into>>>into\n",
            "our>>>our\n",
            "comfort>>>comfort\n",
            "zone>>>zone\n",
            ".>>>.\n",
            "But>>>But\n",
            "the>>>the\n",
            "thing>>>thing\n",
            "is>>>is\n",
            "I>>>I\n",
            "wear>>>wear\n",
            "socks>>>sock\n",
            "all>>>all\n",
            "the>>>the\n",
            "time>>>time\n",
            "in>>>in\n",
            "my>>>my\n",
            "house>>>house\n",
            ",>>>,\n",
            "and>>>and\n",
            "open>>>open\n",
            "them>>>them\n",
            "when>>>when\n",
            "I>>>I\n",
            "go>>>go\n",
            "to>>>to\n",
            "bed>>>bed\n",
            ",>>>,\n",
            "replacing>>>replacing\n",
            "the>>>the\n",
            "sock>>>sock\n",
            "with>>>with\n",
            "a>>>a\n",
            "comforter>>>comforter\n",
            ".>>>.\n",
            "I>>>I\n",
            "guess>>>guess\n",
            "I>>>I\n",
            "need>>>need\n",
            "that>>>that\n",
            "comfort>>>comfort\n",
            "zone>>>zone\n",
            "close>>>close\n",
            "and>>>and\n",
            "maybe>>>maybe\n",
            ",>>>,\n",
            "mine>>>mine\n",
            "is>>>is\n",
            "restricted>>>restricted\n",
            "to>>>to\n",
            "my>>>my\n",
            "own>>>own\n",
            "skin>>>skin\n",
            ".>>>.\n",
            "So>>>So\n",
            "much>>>much\n",
            "so>>>so\n",
            ",>>>,\n",
            "that>>>that\n",
            "it>>>it\n",
            "doesn>>>doesn\n",
            "’>>>’\n",
            "t>>>t\n",
            "even>>>even\n",
            "include>>>include\n",
            "my>>>my\n",
            "own>>>own\n",
            "home>>>home\n",
            ".>>>.\n",
            "When>>>When\n",
            "I>>>I\n",
            "travel>>>travel\n",
            ",>>>,\n",
            "I>>>I\n",
            "like>>>like\n",
            "wearing>>>wearing\n",
            "shoes>>>shoe\n",
            ",>>>,\n",
            "with>>>with\n",
            "a>>>a\n",
            "firm>>>firm\n",
            "grip>>>grip\n",
            "on>>>on\n",
            "myself>>>myself\n",
            ",>>>,\n",
            "dust>>>dust\n",
            "free>>>free\n",
            ",>>>,\n",
            "protected>>>protected\n",
            ".>>>.\n",
            "When>>>When\n",
            "I>>>I\n",
            "face>>>face\n",
            "situations>>>situation\n",
            "where>>>where\n",
            "I>>>I\n",
            "have>>>have\n",
            "to>>>to\n",
            "open>>>open\n",
            "them>>>them\n",
            "and>>>and\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "want>>>want\n",
            "to>>>to\n",
            ",>>>,\n",
            "I>>>I\n",
            "usually>>>usually\n",
            "wrinkle>>>wrinkle\n",
            "my>>>my\n",
            "nose>>>nose\n",
            ".>>>.\n",
            "It>>>It\n",
            "doesn>>>doesn\n",
            "’>>>’\n",
            "t>>>t\n",
            "get>>>get\n",
            "better>>>better\n",
            "that>>>that\n",
            "I>>>I\n",
            "can>>>can\n",
            "’>>>’\n",
            "t>>>t\n",
            "wear>>>wear\n",
            "those>>>those\n",
            "shoes>>>shoe\n",
            "again>>>again\n",
            "with>>>with\n",
            "a>>>a\n",
            "dirty>>>dirty\n",
            "feet>>>foot\n",
            ".>>>.\n",
            "I>>>I\n",
            "take>>>take\n",
            "a>>>a\n",
            "handkerchief>>>handkerchief\n",
            ".>>>.\n",
            "Imagine>>>Imagine\n",
            "my>>>my\n",
            "situation>>>situation\n",
            "at>>>at\n",
            "the>>>the\n",
            "beach>>>beach\n",
            "or>>>or\n",
            "in>>>in\n",
            "a>>>a\n",
            "hill>>>hill\n",
            "stream>>>stream\n",
            ",>>>,\n",
            "when>>>when\n",
            "the>>>the\n",
            "water>>>water\n",
            "feels>>>feel\n",
            "soothing>>>soothing\n",
            "to>>>to\n",
            "my>>>my\n",
            "sweaty>>>sweaty\n",
            "feet>>>foot\n",
            "and>>>and\n",
            "removes>>>remove\n",
            "the>>>the\n",
            "dust>>>dust\n",
            ",>>>,\n",
            "only>>>only\n",
            "to>>>to\n",
            "attract>>>attract\n",
            "more>>>more\n",
            "of>>>of\n",
            "it>>>it\n",
            "when>>>when\n",
            "I>>>I\n",
            "walk>>>walk\n",
            "on>>>on\n",
            "the>>>the\n",
            "sand>>>sand\n",
            ".>>>.\n",
            "I>>>I\n",
            "let>>>let\n",
            "it>>>it\n",
            "irritate>>>irritate\n",
            "me>>>me\n",
            "those>>>those\n",
            "times>>>time\n",
            ",>>>,\n",
            "because>>>because\n",
            "I>>>I\n",
            "always>>>always\n",
            "have>>>have\n",
            "the>>>the\n",
            "ocean>>>ocean\n",
            "and>>>and\n",
            "the>>>the\n",
            "river>>>river\n",
            "on>>>on\n",
            "my>>>my\n",
            "side>>>side\n",
            ".>>>.\n",
            "I>>>I\n",
            "like>>>like\n",
            "my>>>my\n",
            "bare>>>bare\n",
            "feet>>>foot\n",
            "drowning>>>drowning\n",
            "with>>>with\n",
            "no>>>no\n",
            "air>>>air\n",
            "bubbles>>>bubble\n",
            "left>>>left\n",
            ".>>>.\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "like>>>like\n",
            "it>>>it\n",
            "when>>>when\n",
            "droplets>>>droplet\n",
            "of>>>of\n",
            "water>>>water\n",
            "make>>>make\n",
            "only>>>only\n",
            "a>>>a\n",
            "part>>>part\n",
            "of>>>of\n",
            "my>>>my\n",
            "feet>>>foot\n",
            "wet>>>wet\n",
            ",>>>,\n",
            "only>>>only\n",
            "to>>>to\n",
            "let>>>let\n",
            "me>>>me\n",
            "realise>>>realise\n",
            "and>>>and\n",
            "miss>>>miss\n",
            "the>>>the\n",
            "comfort>>>comfort\n",
            "of>>>of\n",
            "dry>>>dry\n",
            "skin>>>skin\n",
            "and>>>and\n",
            "scowl>>>scowl\n",
            "at>>>at\n",
            "the>>>the\n",
            "linger>>>linger\n",
            "of>>>of\n",
            "irritation>>>irritation\n",
            "due>>>due\n",
            "to>>>to\n",
            "a>>>a\n",
            "little>>>little\n",
            "dampness>>>dampness\n",
            ".>>>.\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "like>>>like\n",
            "it>>>it\n",
            "when>>>when\n",
            "I>>>I\n",
            "have>>>have\n",
            "to>>>to\n",
            "keep>>>keep\n",
            "on>>>on\n",
            "brushing>>>brushing\n",
            "my>>>my\n",
            "left>>>left\n",
            "leg>>>leg\n",
            "against>>>against\n",
            "my>>>my\n",
            "right>>>right\n",
            "to>>>to\n",
            "flatten>>>flatten\n",
            "the>>>the\n",
            "spheres>>>sphere\n",
            "of>>>of\n",
            "water>>>water\n",
            ".>>>.\n",
            "And>>>And\n",
            "repeat>>>repeat\n",
            "it>>>it\n",
            "with>>>with\n",
            "the>>>the\n",
            "right>>>right\n",
            ".>>>.\n",
            "I>>>I\n",
            "just>>>just\n",
            "realised>>>realised\n",
            "why>>>why\n",
            "I>>>I\n",
            "don>>>don\n",
            "’>>>’\n",
            "t>>>t\n",
            "like>>>like\n",
            "drizzling>>>drizzling\n",
            "rains>>>rain\n",
            ".>>>.\n",
            "Eventually>>>Eventually\n",
            ",>>>,\n",
            "when>>>when\n",
            "I>>>I\n",
            "have>>>have\n",
            "to>>>to\n",
            "get>>>get\n",
            "out>>>out\n",
            "of>>>of\n",
            "the>>>the\n",
            "water>>>water\n",
            ",>>>,\n",
            "I>>>I\n",
            "give>>>give\n",
            "up>>>up\n",
            "my>>>my\n",
            "fight>>>fight\n",
            "with>>>with\n",
            "the>>>the\n",
            "dirt>>>dirt\n",
            "and>>>and\n",
            "wear>>>wear\n",
            "those>>>those\n",
            "damn>>>damn\n",
            "shoes>>>shoe\n",
            "with>>>with\n",
            "muddy>>>muddy\n",
            "feet>>>foot\n",
            ".>>>.\n",
            "Those>>>Those\n",
            "are>>>are\n",
            "the>>>the\n",
            "times>>>time\n",
            "I>>>I\n",
            "miss>>>miss\n",
            "my>>>my\n",
            "slippery>>>slippery\n",
            "slippers>>>slipper\n",
            ".>>>.\n",
            "Don>>>Don\n",
            "’>>>’\n",
            "t>>>t\n",
            "let>>>let\n",
            "my>>>my\n",
            "socks>>>sock\n",
            "or>>>or\n",
            "shoes>>>shoe\n",
            "hear>>>hear\n",
            "me>>>me\n",
            "say>>>say\n",
            "that>>>that\n",
            ".>>>.\n",
            "They>>>They\n",
            "do>>>do\n",
            "not>>>not\n",
            "know>>>know\n",
            "that>>>that\n",
            "sometimes>>>sometimes\n",
            "I>>>I\n",
            "keep>>>keep\n",
            "a>>>a\n",
            "back>>>back\n",
            "->>>-\n",
            "up>>>up\n",
            "pair>>>pair\n",
            "of>>>of\n",
            "red>>>red\n",
            "flip>>>flip\n",
            "->>>-\n",
            "flops>>>flop\n",
            "for>>>for\n",
            "such>>>such\n",
            "scenarios>>>scenario\n",
            ".>>>.\n",
            "Does>>>Does\n",
            "this>>>this\n",
            "make>>>make\n",
            "me>>>me\n",
            "a>>>a\n",
            "slipper>>>slipper\n",
            "person>>>person\n",
            "?>>>?\n",
            "I>>>I\n",
            "guess>>>guess\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "slipper>>>slipper\n",
            "person>>>person\n",
            "when>>>when\n",
            "my>>>my\n",
            "feet>>>foot\n",
            "are>>>are\n",
            "ankle>>>ankle\n",
            "deep>>>deep\n",
            "in>>>in\n",
            "mud>>>mud\n",
            ",>>>,\n",
            "so>>>so\n",
            "that>>>that\n",
            "there>>>there\n",
            "is>>>is\n",
            "no>>>no\n",
            "place>>>place\n",
            "for>>>for\n",
            "me>>>me\n",
            "to>>>to\n",
            "complain>>>complain\n",
            ".>>>.\n",
            "No>>>No\n",
            "alternative>>>alternative\n",
            ".>>>.\n",
            "No>>>No\n",
            "uneasy>>>uneasy\n",
            "half>>>half\n",
            "->>>-\n",
            "done>>>done\n",
            "feeling>>>feeling\n",
            ".>>>.\n",
            "Otherwise>>>Otherwise\n",
            ",>>>,\n",
            "not>>>not\n",
            "much>>>much\n",
            "of>>>of\n",
            "a>>>a\n",
            "slipper>>>slipper\n",
            "person>>>person\n",
            ".>>>.\n",
            "I>>>I\n",
            "am>>>am\n",
            "a>>>a\n",
            "person>>>person\n",
            "who>>>who\n",
            "likes>>>like\n",
            "symmetry>>>symmetry\n",
            ".>>>.\n",
            "All>>>All\n",
            "in>>>in\n",
            "or>>>or\n",
            "none>>>none\n",
            "at>>>at\n",
            "all>>>all\n",
            ".>>>.\n",
            "It>>>It\n",
            "’>>>’\n",
            "s>>>s\n",
            "difficult>>>difficult\n",
            "being>>>being\n",
            "me>>>me\n",
            "in>>>in\n",
            "this>>>this\n",
            "world>>>world\n",
            "with>>>with\n",
            "its>>>it\n",
            "shades>>>shade\n",
            "of>>>of\n",
            "grey>>>grey\n",
            ".>>>.\n",
            "But>>>But\n",
            "then>>>then\n",
            ",>>>,\n",
            "my>>>my\n",
            "favourite>>>favourite\n",
            "colour>>>colour\n",
            "is>>>is\n",
            "red>>>red\n",
            "and>>>and\n",
            "thankfully>>>thankfully\n",
            "it>>>it\n",
            "covers>>>cover\n",
            "the>>>the\n",
            "entire>>>entire\n",
            "spectrum>>>spectrum\n",
            ".>>>.\n",
            "All>>>All\n",
            "I>>>I\n",
            "need>>>need\n",
            "is>>>is\n",
            "an>>>an\n",
            "emotion>>>emotion\n",
            ",>>>,\n",
            "and>>>and\n",
            "the>>>the\n",
            "rest>>>rest\n",
            "is>>>is\n",
            "taken>>>taken\n",
            "care>>>care\n",
            "of>>>of\n",
            ".>>>.\n",
            "Thank>>>Thank\n",
            "you>>>you\n",
            "for>>>for\n",
            "being>>>being\n",
            "the>>>the\n",
            "listener>>>listener\n",
            "you>>>you\n",
            "are>>>are\n",
            ".>>>.\n",
            "This>>>This\n",
            "has>>>ha\n",
            "been>>>been\n",
            "a>>>a\n",
            "selfish>>>selfish\n",
            "post>>>post\n",
            ".>>>.\n",
            "I>>>I\n",
            "used>>>used\n",
            "your>>>your\n",
            "letter>>>letter\n",
            "for>>>for\n",
            "my>>>my\n",
            "sense>>>sense\n",
            "of>>>of\n",
            "clarity>>>clarity\n",
            ".>>>.\n",
            "You>>>You\n",
            "always>>>always\n",
            "make>>>make\n",
            "me>>>me\n",
            "reflect>>>reflect\n",
            "and>>>and\n",
            "dig>>>dig\n",
            "deep>>>deep\n",
            ".>>>.\n",
            "I>>>I\n",
            "can>>>can\n",
            "finally>>>finally\n",
            "sleep>>>sleep\n",
            "now>>>now\n",
            ".>>>.\n",
            "Goodnight>>>Goodnight\n",
            ".>>>.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a2HsI8netDOT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 320
        },
        "outputId": "596a5ffe-84c7-4b6b-ac84-59a3d68751d7"
      },
      "source": [
        "from nltk.tokenize import word_tokenize\n",
        "tokenized_word=word_tokenize(text)\n",
        "from nltk.probability import FreqDist\n",
        "fdist = FreqDist(tokenized_word)\n",
        "print(fdist)\n",
        "\n",
        "fdist.most_common(2)\n",
        "[('is', 3), (',', 2)]\n",
        "# Frequency Distribution Plot\n",
        "import matplotlib.pyplot as plt\n",
        "fdist.plot(30,cumulative=False)\n",
        "plt.show()"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<FreqDist with 301 samples and 743 outcomes>\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEeCAYAAACNLn6mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xb9dXH8c/xyHAmWcTZC8IISYhNCIQHElZpSymljFI2lLSUUlpaGp4uoH06KC2UUjYpqy27FBJ2IYNAINjBmQSyySQJZDvDjs/zx71OFVmyZVuybOv7fr30snT107lHlnR09bu/+7vm7oiISObISncCIiLSsFT4RUQyjAq/iEiGUeEXEckwKvwiIhkmJ90JJKJLly7er1+/Oj12586dtG7dOqltFVMxFVMxG1vMWIqLize6e9cqd7h7o78UFBR4XRUVFSW9rWIqpmIqZmOLGQtQ5DFqqrp6REQyjAq/iEiGUeEXEckwKvwiIhlGhV9EJMOo8IuIZJhmXfi37SqjQrOPiojsp1kX/p//ex7XvfYZz5esZm+FvgBERKAZF/6de/ZStHwTK7eWc+0TJZz0pyk89f5K9pRXpDs1EZG0araFv3WLbCb/eAxXFbSnT6c8ln9Wyk+encOYWyfz6Izl7Crbm+4URUTSotkWfoAWOVmcPCCPN390An8+bziDurVlzZZd/PL5+Rx3y2Tun7aEHbvL052miEiDataFv1JOdhZnHtmT135wPPdcMILDe7Rn4/bd/PalhYy+5U3+8sYituwsS3eaIiINIiMKf6WsLOOLR+Qz6ZrjeOjSoxjRpyObS8u47fWPGf37NylZtzvdKYqIpFxGFf5KZsbYQ7rx7FXH8s8rj2Zk/05s313OvxZuT3dqIiIpl5GFv5KZcezALtz1zREALNlUrmGfItLsZXThr9S1XUt6dGjFrnJn6QZt9YtI86bCHxraqyMAs1dtSXMmIiKppcIfGtq7AwBzVm1OcyYiIqmlwh8api1+EckQKvyhIT2DLf4P12zVtA4i0qylvPCbWbaZfWBmk8Lb/c3sPTNbbGZPmlmLVOeQiA6tc+nRNps9eyv4aN22dKcjIpIyDbHFfy3wYcTtW4Db3X0QsAm4ogFySMigTrkAzFY/v4g0Yykt/GbWC/gy8GB424ATgWfCJo8AZ6Yyh9oYGBZ+7eAVkebMPIUnKjGzZ4DfAe2AHwOXAu+GW/uYWW/gZXcfEuOx44BxAPn5+QUTJ06sUw6lpaXk5eUl1LZk1VZ+PaOUPh1yuP3ULkmJmWhbxVRMxVTM+raNVlhYWOzuhVXucPeUXIDTgbvD62OASUAXYHFEm97AvJpiFRQUeF0VFRUl3Pbtd9/3Af/7ove/YZLv2F2WlJiJtlVMxVRMxaxv22hAkceoqans6hkNnGFmy4EnCLp47gA6mllO2KYXsDqFOdRKyxzj4APbUeEwf83WdKcjIpISKSv87v6/7t7L3fsB3wDedPcLgMnA2WGzS4DnU5VDXQzrFQzrnL1S/fwi0jylYxz/eOA6M1sMdAYmpCGHuCqnbpijA7lEpJnKqblJ/bn7FGBKeH0pMLIh1lsXQ3tp6gYRad505G6Uwd3b0TIni+WflbKlVGflEpHmR4U/Sm52Fof1aA/AnNXa6heR5keFP4Zh6ucXkWZMhT+GoRrZIyLNmAp/DBrZIyLNmQp/DAO6tKFdyxzWbd3F+q270p2OiEhSqfDHkJVl++bn14lZRKS5UeGPY1jvyu4e9fOLSPOiwh/HvqkbtMUvIs2MCn8cQyO2+D2FU1eLiDQ0Ff44enRoRZe2LdhcWsbKz3emOx0RkaRR4Y/DzPYN69SpGEWkOVHhr4YmbBOR5kiFvxrD9m3xaweviDQfKvzVqNzin7d6C3srtINXRJoHFf5qdG7bkp4dW1O6Zy9LNmxPdzoiIkmhwl+DYb01YZuINC8pK/xm1srMZprZbDObb2Y3h8sfNrNlZlYSXoanKodk0IRtItLcpPLUi7uBE919u5nlAtPN7OXwvuvd/ZkUrjtpNLJHRJqblG3xe6CyYzw3vDS5PaRH9OyAGXy4dht7yivSnY6ISL1ZKqcjMLNsoBgYBNzl7uPN7GHgGIJfBG8AN7j77hiPHQeMA8jPzy+YOHFinXIoLS0lLy+vXm2//8oGVm/byy0ndWZQp9ykxExFnoqpmIqZuTFjKSwsLHb3wip3uHvKL0BHYDIwBMgHDGgJPAL8sqbHFxQUeF0VFRXVu+0Pn/jA+46f5I/OWJ60mHVtp5iKqZiKmSigyGPU1AYZ1ePum8PCf5q7rw1z2g08BIxsiBzqY18/v0b2iEgzkMpRPV3NrGN4vTVwCrDQzPLDZQacCcxLVQ7J8t+ZOjWyR0SavlSO6skHHgn7+bOAp9x9kpm9aWZdCbp7SoDvpDCHpDgsvz05Wcai9dso3VOe7nREROolZYXf3ecAR8ZYfmKq1pkqrXKzGdy9HfPXbGXe6q1kpzshEZF60JG7CfrvgVzq5xeRpk2FP0E6FaOINBcq/AnSFr+INBcq/Ak6+MC2tMrNYsVnpWzboyN4RaTpUuFPUE52Fof3CLp7lnxeluZsRETqToW/FioP5Fq8SYVfRJouFf5aqDwV42Jt8YtIE6bCXwv7tvhV+EWkCVPhr4V+ndvQrlUOm3ZVsG7LrnSnIyJSJyr8tZCVZQwP5+2Z9cmmNGcjIlI3Kvy1VND3AACKlqvwi0jTpMJfS4V9OwFQvOLzNGciIlI3Kvy1NLxPR7KA+Wu2snPP3nSnIyJSayr8tdS2ZQ59O+ZQXuGU6MQsItIEqfDXwSFdWgDq7hGRpkmFvw4O6ZwLQNEK7eAVkaYnladebGVmM81stpnNN7Obw+X9zew9M1tsZk+aWYtU5ZAqlVv8s1ZsoqLC05yNiEjtpHKLfzdworsPA4YDp5nZKOAW4HZ3HwRsAq5IYQ4p0SUvmx4dWrF1VzmL1m9PdzoiIrWSssLvgcqqmBteHDgReCZc/gjBCdebnBHheP5idfeISBNj7qnrqghPtF4MDALuAm4F3g239jGz3sDL7j4kxmPHAeMA8vPzCyZOnFinHEpLS8nLy0tq29LSUqasdiaUbGNM31ZcM7JjUmKmIk/FVEzFzIyYsRQWFha7e2GVO9w95RegIzAZOA5YHLG8NzCvpscXFBR4XRUVFSW9bVFRkc9dtdn7jp/kx//hzaTFTJRiKqZiKmYigCKPUVMbZFSPu28OC/8xQEczywnv6gWsbogcku2Q7u3Ia5HNis9K2bBtd7rTERFJWCpH9XQ1s47h9dbAKcCHBF8AZ4fNLgGeT1UOqZSTncWRfYIuHo3nF5GmJJVb/PnAZDObA7wPvO7uk4DxwHVmthjoDExIYQ4pVRDO26MJ20SkKcmpuUnduPsc4MgYy5cCI1O13oZUWDlTp0b2iEgToiN36+HIPh3JMpi/Zgu7yjRhm4g0DSr89dCuVS6Du7enbK8zWxO2iUgTocJfTwV9gx286u4RkaZChb+eKk/MMkuFX0SaCBX+eqo8FWPxJ5qwTUSaBhX+eup1QGsObN+SzaVlLN2oCdtEpPFT4a8nM9vX3aPx/CLSFKjwJ0GBxvOLSBOiwp8Ehf00RbOINB0q/ElwaH57Wudms2zjDjZu14RtItK4qfAnQW52FsN7V07Ypq1+EWncVPiTpEBn5BKRJqLWhd/MDjCzoalIpikrCPv5i5ZrimYRadwSKvxmNsXM2ptZJ2AW8ICZ3Zba1JqWEX0OwAzmrd6qCdtEpFFLdIu/g7tvBc4CHnX3o4GTU5dW09OhdS4Hd2vHnr0VzFu9Jd3piIjElWjhzzGzfOBcYFIK82nS9nX3qJ9fRBqxRAv/zcCrBCdKf9/MBgCLqnuAmfU2s8lmtsDM5pvZteHym8xstZmVhJcv1e8pNB77TsyiI3hFpBFL9Axca9193w5dd1+aQB9/OfAjd59lZu2AYjN7Pbzvdnf/Yx3ybdT2zdT5ySbcHTNLc0YiIlUlusV/Z4LL9nH3te4+K7y+jeBE6z1rl17T0rtTa7q2a8nnO/awdOOOdKcjIhKTucefStjMjgGOBX4A3B5xV3vga+4+LKGVmPUDpgFDgOuAS4GtQBHBr4IqfSNmNg4YB5Cfn18wceLERFZVRWlpKXl5eUltW127W9/ZxLurd/Pdwvac1D8vKTFTkadiKqZiNq+YsRQWFha7e2GVO9w97gU4AbgRWBv+rbxcBxxU3WMjYrQFioGzwtsHAtkEvzZ+A/ytphgFBQVeV0VFRUlvW127B6Yt8b7jJ/n1T5ckLWZd2yqmYipm5sSMBSjyGDW12j5+d58KTDWzh919RW2/bcwsF3gW+Ie7/yuM+WnE/Q/QzEYJaaZOEWnsEt2529LM7gf6RT7G3U+M9wAL9mxOAD5099silue7+9rw5teAebVNujE7vEcHWuZksXTDDj7fsSfd6YiIVJFo4X8auBd4EEj0sNTRwEXAXDMrCZf9FDjfzIYDDiwHvp1wtk1Ai5wshvXuyMxlnzNrxSYOSHdCIiJREi385e5+T20Cu/t0INZ4xpdqE6cpKux7ADOXfU7Rik2c0i3d2YiI7C/R4ZwTzey7ZpZvZp0qLynNrAn774lZNGGbiDQ+iW7xXxL+vT5imQMDkptO8zCiT1D4Z6/aQtmIrmnORkRkfwkVfnfvn+pEmpOOeS04qFtbFq3fztLNZYxKd0IiIhESKvxmdnGs5e7+aHLTaT4K+x3AovXbWbixLN2piIjsJ9E+/qMiLv8D3ASckaKcmoWR/YNdIP9euJ35azRNs4g0HgkVfne/JuJyJTCC4IhciePLR/Rg7OCubN3jnH//u5Ss3JzulEREgLqfc3cHoH7/arTIyeK+iwo5umdLtu4q58IH3+N9nZZRRBqBRE+9ONHMXggvLwIfAc+lNrWmr0VOFteN6shXhvVg++5yLp4wk3cWb0x3WiKS4RIdzhk5d345sMLdV6Ugn2YnJ8v483nDaZmTxTPFq7js4fe576ICxgzWkV0ikh6J9vFPBRYC7YADAE1CUwvZWcYfvj6UC47uw+7yCq58tIjX5q9Ld1oikqES7eo5F5gJnENw3t33zOzsVCbW3GRlGf935hAuH92fsr3Od/8xi0lz1qQ7LRHJQIl29fwMOMrd1wOYWVfgP8AzqUqsOTIzfnH6obTMzeKeKUv4/uMfsLusgq8X9Ep3aiKSQRIt/FmVRT/0GXUfEZTRzIyffGEwrXKyuf0/H/PjZ2azu7yCwYm+EiIi9ZRo8X7FzF41s0vN7FLgRTJgls1UMTOuPfkgbvjiIbjDT5+bywsf76CiIv5pMEVEkqXawm9mg8xstLtfD9wHDA0vM4D7GyC/Zu07Jwzkxq8cBsAjs7dx2h3TeL5kNeV7K9KcmYg0ZzVt8f+Z4KTouPu/3P06d7+OYAz/n1OdXCa4bHR/7vjGcDq3zuLjT7dz7RMlnHzbVJ58/xP2lOsLQESSr6ae5QPdfW70Qnefa2b9UpJRBvrq8J4cWLaW5d6Ne6YuYflnpYx/di53/GcR3xkzkHMLe9MqNzvdaYpIM1HTFn/Hau5rXd0Dzay3mU02swVmNt/Mrg2XdzKz181sUfhXZycEcrOMb4zswxvXncCfzxvOoG5tWbNlF798fj7H3TKZ+6ctYcfu8nSnKSLNQE2Fv8jMroxeaGbfAopreGw58CN3PwwYBVxtZocBNwBvuPtBwBvhbQnlZGdx5pE9ee0Hx3PvhSM4vEd7Nm7fzW9fWsjoW97kL28sYmeZuoBEpO5q6ur5AfCcmV3Afwt9IdAC+Fp1D3T3tcDa8Po2M/sQ6Al8FRgTNnsEmAKMr0PuzVpWlnHakHy+cHh3pny0gTvfXMSsTzZz2+sf079jDs8esYdObVqkO00RaYLMveYhhGY2FhgS3pzv7m/WaiXB/oBpYYxP3L1juNyATZW3ox4zDhgHkJ+fXzBx4sTarHKf0tJS8vLykto2HTHdnXkb9nBv8VbWbd9L7/Y53HjCARzQKn7ff3N57oqpmIqZWNtohYWFxe5eWOUOd0/phWDe/mLgrPD25qj7N9UUo6CgwOuqqKgo6W3TGfPTLTt99G9e8b7jJ/nYWyf7ms2l9Y5Zm7aKqZiK2ThjxgIUeYyamtKjb80sF3gW+Ie7/ytc/KmZ5Yf35wPr4z1equrWvhW/GtOJQ/Pbs3TjDs69bwYrPy9Nd1oi0oSkrPCH3TgTgA/d/baIu14ALgmvXwI8n6ocmqsOLbN4/MqjGdarAys/38l5981g2cYd6U5LRJqIVG7xjwYuAk40s5Lw8iXg98ApZrYIODm8LbXUMa8Fj33raAr6HsCaLbs4774ZLPp0W7rTEpEmIGWF392nu7u5+1B3Hx5eXnL3z9z9JHc/yN1Pdnedj7CO2rfK5dHLR3LMgM6s37abb9z/LgvWbE13WiLSyGmGzSauTcscHrrsKI4/uCuf7djD+Q+8y5xVOrG7iMSnwt8MtMrN5oGLCzj50APZsrOMCx54j+IV+iElIrGp8DcTLXOyuefCEXz5iHy27S7nogkzmbd+d7rTEpFGSIW/GcnNzuKObwznrCN7UrpnL795axNTP96Q7rREpJFR4W9mcrKz+OM5wzh/ZG/2VMCVjxTx+oJP052WiDQiKvzNUFaW8duvHcGXBuWxZ28FV/29mBfnrE13WiLSSKjwN1NmxuXD2/HtEwZQXuFc8/gsnvtgVbrTEpFGQIW/GTMzbjjtEK496SAqHK57ajZPzPwk3WmJSJqp8DdzZsYPTzmYn5w2GHe44V9zeXTG8nSnJSJppMKfIb47ZhC/OD04sfsvn5/P/dOWpDkjEUkXFf4McsVx/fm/M4PTKvz2pYXc+caiNGckIumgwp9hLhzVl1vPHkqWwZ9e/5hbX11YeV4EEckQNZ16UZqhcwp70zI3mx8+WcJdk5ewq6yCL3ZX8RfJFNriz1BnDOvBXd8cQW62MWH6Mm6eton3ln6W7rREpAGo8Gew04Z05/6LCmnbMoe56/dw3v3vcs697zD14w3q/hFpxlT4M9zYQ7oxffxYzj2sDe1b5fD+8k1c8reZnPHXt3l1/joqKvQFINLcpPLUi38zs/VmNi9i2U1mtjrqjFySZh3zWnDe4e14+4YTGX/aIXRu04K5q7fw7ceK+eIdb/F8yWr26gtApNlI5Rb/w8BpMZbfHnlGrhSuX2qpXatcrhozkOnjT+TGrxxG9/at+OjTbVz7RAkn3zaVp4pWUq4vAJEmL2Wjetx9mpn1S1V8SZ3WLbK5bHR/vnl0H54tXs09UxezbOMOfvLMHHq0y+bFw/ZwQJsW6U5TROrIUrkTLyz8k9x9SHj7JuBSYCtQBPzI3TfFeew4YBxAfn5+wcSJE+uUQ2lpKXl5eUltm2kx91Y401fu4qn521m3Yy/H9GrFj0Z1wMwaVZ6KqZiZHDOWwsLCYncvrHKHu6fsAvQD5kXcPhDIJuhi+g3wt0TiFBQUeF0VFRUlvW2mxlyxcYcP/tmL3nf8JH9u1qqkrb8pPHfFVMzGHjMWoMhj1NQGHdXj7p+6+153rwAeAEY25Pqlfvp0zuOyYe0A+MXz81i7ZWeaMxKRumjQwm9m+RE3vwbMi9dWGqeT+rfm5EO7sW1XOdc/PUfDPUWaoFQO53wcmAEMNrNVZnYF8Aczm2tmc4CxwA9TtX5JDTPjd2cNpVObFkxfvFFTPIs0Qakc1XN+jMUTUrU+aThd27Xkt187gu/8vZjfvbyQ4w7qyqBubdOdlogkSEfuSp2cNqQ7Z43oye7yCn70VAlleyvSnZKIJEiFX+rspjMOp2fH1sxetYW7Ji9OdzoikiAVfqmz9q1yufWcoQDc+eZi5qzanOaMRCQRKvxSL8cO7MLlo/uzt8L54ZMl7Crbm+6URKQGKvxSbz85bTCDurVlyYYd/P7lhelOR0RqoMIv9dYqN5s/nzecnCzj4XeW8/bijelOSUSqocIvSTGkZweuPekgAH789Gy27CxLc0YiEo8KvyTNVWMGMrx3R9Zu2cVNL8xPdzoiEocKvyRNTnYWt507jFa5WTz3wWqeXrCd0j3l6U5LRKKo8EtSDejalp9/+TAAnpi/ndG/f5O/vrmIrbvU9SPSWKjwS9JdOKovD116FAd3ymVTaRl/fO1jRv/+Tf746kd8vmNPutMTyXgpm6tHMtvYQ7rRbnsn9nTsx51vLmbG0s/46+TFTJi+jAuO7sO44wfQrX2rdKcpkpFU+CVlzIxjB3Xh2EFdKF7xOX99czGTP9rAg9OX8ei7KzivsDffPmFAutMUyTgq/NIgCvp24qHLRjJvdTCvz8vz1vHYuyt4fOYnDOmaS9f5RQnF2bx5Mx0TaJuzZxud+u6gf5c29U1dpNlR4ZcGNaRnB+65sIBFn27j7ilLeL5kNSWf7oFPP008yJrE2r76pymcPrQHV48dxODu7eqYsUjzo8IvaXHQge24/bzhXHfKwUyc/gEDBw5M6HFLliypsa07PP32Aqau2MULs9fwwuw1nHrYgXzvxEEM7dUxGemLNGkpK/xm9jfgdGC9uw8Jl3UCniQ4Cfty4Fx335SqHKTx690pj6N7tqLg8O4JtS/etTqhtl13r+ZX5x3D/VOX8Pj7K3ltwae8tuBTjj+4K9ecOIij+nWqb+oiTVYqh3M+DJwWtewG4A13Pwh4I7wtkhI9O7bm5q8OYfpPxjLu+AHktchm2scbOOfeGZx33wzeWrQBd50zWDJPKk+9OM3M+kUt/iowJrz+CDAFGJ+qHEQAurVvxU+/dChXnTCQh95exkPvLOe9ZZ/z3oSZDO3Vgc45e+i8dHaNcWznNnofvItu7TQMVZo2S+UWT1j4J0V09Wx2947hdQM2Vd6O8dhxwDiA/Pz8gokTJ9Yph9LSUvLy8pLaVjGbdswdZRW8uriUiR/vYOue2r3/W2TBSQPy+OrgNnTNy05pnoqpmLVtG62wsLDY3Qur3OHuKbsQ9OXPi7i9Oer+TYnEKSgo8LoqKipKelvFbB4xd+wu84mzV/sfnp7mT77/SfWXmZ/4uX953fuOn+R9x0/yQT990X/y9GxftmF7yvNUTMWsK6DIY9TUhh7V86mZ5bv7WjPLB9Y38PpF9slrkcPpQ3uQX7aWgoLeNbYfmLWetj0P5u4pi5k4ew1PFq3k6eKVnDGsB98dO4iDD9SQUWkaGnqunheAS8LrlwDPN/D6ReplcPd23PGNI3njR2M4t7AXWWb8u2QNp94+je88Vsy81VvSnaJIjVI5nPNxgh25XcxsFXAj8HvgKTO7AlgBnJuq9YukUv8ubfjD2cP4/kkHcf+0pTzx/kpemb+OV+av4/iDu9KBHby8dkGNcfZu38HBh5fRrlVuA2QtEkjlqJ7z49x1UqrWKdLQeh2Qx6++OoTvjR3Eg9OX8fd3VzDt4w3BnR8vSyjGswvf5NLR/bl8dD865rVIYbYiAR25K5IEkUNGX5y7lkVLV9CrV69qH1Phzr/fX8KHG8v4yxuLmPDWUi48pi/fOm4AXdu1bKDMJROp8Isk0QFtWnDhqL4U526koKDmmUcL226m/IB+/HXyYt5atJH7pi7l4beXc/7IYOrqHh1bN0DWkmlU+EXS7OgBnTl6QGdmr9zMXycv5vUFn/LwO8v5x3sr+PqIXlw1ZiB9O2uWUUkeFX6RRmJY7448cHEhC9dt5a7JS5g0Zw1PvL+Sp4pWcvrQHrQp3847mxfVGKdsy06GDNtLy5z4B5hJZlPhF2lkDunenjvPP5IfnHwQ90xZwr8/WM0Ls9cEd87/OKEYT380hXHHD+AbR/WhdQt9Acj+VPhFGqmBXdvyx3OGce1JB/HC7DUs/WQV+d3zq31MhTuTZq3gky27uHniAu6avJgrjhvARcf0pW1LfdwloHeCSCPXu1MeV48dRHHxFgoKBtfYfmyX7Wxq3Yu/Tl7MnFVbuOWVhdw7dQmXje7HpcdqyKg0/JG7IpJiWWacenh3nr96NI9cPpKR/TqxZWcZf/7PIkb//k1+//JCNm7fne40JY20xS/STJkZJxzclRMO7sp7Sz/bN2T03qlLePidZZxX2JsWu0pZsGd5jbGytu6hIPUpSwNR4RfJALGGjD4yY0Vw5wfzE4qxp+0yLhvdP4VZSkNR4RfJIJVDRj9cu5V/zVrFJ2vW0bVr12ofs31XOf8uWcPNExewq6yCq8Ykdn5kabxU+EUy0KH57fnZlw+juHgnBQVH1Ni+e/Z27pu1lVteWcju8r1ce9JBBOdSkqZIO3dFpEanDMjjT+cMI8vgz/9ZxC2vfKTzFTdhKvwikpCzRvTiL+cfSU6Wce/UJdw8cYGKfxOlwi8iCTt9aA/uvmAELbKzePid5fzs3/OoqFDxb2pU+EWkVk49vDv3X1xAy5ws/vneJ1z/zBz2qvg3KSr8IlJrYwZ346FLj6J1bjbPzlrFtU98QNneinSnJQlKS+E3s+VmNtfMSsysKB05iEj9HDuoC49dMZK2LXOYNGctV/9jFrvL96Y7LUlAOodzjnX3jWlcv4jUU2G/Tvz9W0dz8YT3eG3Bp3z7sWKO61bO9srTT1Zj0brdCbXLzTLYq66kZNI4fhGpl+G9O/L4uFFcNGEmUz7awJSPgLdmJvbgBNt1bJXFd3ct4YKj+9JGs4zWm6VjOJaZLQM2AQ7c5+73x2gzDhgHkJ+fXzBx4sQ6rau0tJS8vLyktlVMxVTMqlZtLeepBdvZurOMrOyazwFQsXdvQu0+21nBqq3lALRtYXzloDZ8cVAebVpU7aluTv/PurSNVlhYWOzuhVXucPcGvwA9w7/dgNnA8dW1Lygo8LoqKipKelvFVEzFbLiYFRUVfv/Et/2su9/2vuMned/xk3zIL1/xP7zyoW/ctqvR5JnumLEARR6jpqZl5667rw7/rgeeA0amIw8RafzMjBH5LXnmO8fw+JWjGD2oM9t2l3PX5CUcd8tkfj1pAZ9u3ZXuNJuUBu8sM7M2QJa7bwuvnwr8qqHzEJGmxcw4ZmBnjhnYmVmfbOKuNxfzxsL1TJi+jMdmrOCcwl4clreHnJWbE4q3+POyhNom2i5VMTeWJn+kVDr2khwIPBdO8JQD/DyLVVAAABVPSURBVNPdX0lDHiLSRI3ocwATLj2Keau3cPeUxbw8bx3/eO+T4M7Jbyce6I0E2ybaLgUxzxzchi/8T+KrT0SDF353XwoMa+j1ikjzM6RnB+6+oIDF67dx39SlzFq6jjZt2iT02B07diTUNtF2qYrZOS/5B8ZpXJSINHmDurXj1nOGUVxcTkFBYucKKy4uTqhtou1SGTPZNGWDiEiGUeEXEckwKvwiIhlGhV9EJMOo8IuIZBgVfhGRDKPCLyKSYVT4RUQyTFqmZa4tM9sArKjjw7sAiZ7wJdG2iqmYiqmYjS1mLH3dvWuVpbGm7GxOF+JMS1qftoqpmIqpmI0tZm0u6uoREckwKvwiIhkmEwp/ldM6JqGtYiqmYipmY4uZsCaxc1dERJInE7b4RUQkggq/iEiGUeEXEckwKvx1ZGb5ZtYyzn2dU7C+A8xspJkdX3mJuv+x8O+1yV53UxHrucf7f5hZazMbXE2so1O17lrErPL+irPslkSWSWIs0DvdeaSSdu7GYGbd3X1dDW3+AwwEnnX3H0fdtwgoAR4CXvYY/2QzOxD4LdDD3b9oZocBx7j7hBhtvwVcC/QK444CZrj7iRFtFgAnAy8DYwCLjOHun9f0vKt5rgcCR4U3Z7r7+mraHgv0I+K0nu7+aIx2pwO/BvqGbS1o6u1jtO0Z0a4y5rQY7Wa5+4ioZR+4+5FRy74C/BFo4e79zWw48Ct3PyOizTTAga8l8r9LdN0R92UDB0Y9p08SiJnosjnuPrSafGt8ncxsNFDi7jvM7EJgBHCHu6+IajcQWOXuu81sDDAUeNTdN8dYb0vg6zHW/asYbfOAHwF93P1KMzsIGOzukyLaXBzr+cV6z4Xtb3H38Qksm+vuR8SKEdXuMXe/qKZl4fKDgXuAA919iJkNBc5w9/+LajcauImqn40BNeWTqGZV+M1sG8GHtcpdxCkqceK86O5fTqCdAYe5+/wYy08GLicomE8BD7v7xxFtXib4YviZuw8zsxzgg1hvNjObG8Z5192Hm9khwG/d/ayINt8HrgIGAKtjPPcBYbt4/yMIGu73PzKzc4FbgSlhrP8Brnf3Z2Lk+RjBl2EJsPe/If37MdouBs4C5sb6YoxodwtwHrAgKmZkkT4f+CZwHPBWxMPbARXuflJUzGLgRGBKZWGO9UEPXxPcvbya/Gq17vAx1wA3Ap8ClWfS9spCbWbdgZ7A38PYlV/i7YF73f2QsN1VwHcJXvMlUet+290vjJNzQq+Tmc0BhhEU8oeBB4Fz3f2EqHYlQCFBMX8JeB443N2/FGPdrwBbgOKIdePuf4rR9smw3cVhocwD3nH34RFt7ox4SCvgJGCWu58d57kn9CVpZo8Af3X392PFiRcvfM/McffDYrSdClwP3Bfxvpvn7kOi2i0EfkjV/9Fn1eVSK8k+FFiXKodbjyUoxJuBqQRb9QDvh38/iGhbEidGZdsSoGV4fX6ctvcQfFivCS/D4rT7NUHRaEdQUK4i2OqNbjcb6BZxuyswO07MDwk3JhL4v0wGshJo91Hlc66mTV+CXzkzgBMiLiOAnBjt343xv59Tx9e3VusOH7MY6FxNzEvC/8+28G/l5QXgrIh2HQiK7eNhHpWXTjXknNDrRFBAAX4JXBG5LE6764Frov+3UW3n1eJ/WxTjdYr53ou4vyPwSozlVwFzgR3AnIjLMuDvMdovJCi6S8J2cyPfI8D/hq9PObA14vIZ8Ls4uSX0mQfeq8t7sTaXfT+1JHnCPv4LgYuBdQQF+AVgOPA00B/YEbbz8DGjCLaEYlllZh2BfwOvm9km4k9at5BgS/FfBFuKj5nZA+5+Z1S7M9x9WMTte8xsNsGHPFKW79+18xnx9w3NA7oDa+PcH+knwEvhVtDuyoXufltUu6VAbmSbaB50PawAjklgvQDzzeybQHbYffB94J0EHxt33WbWFzjI3f9jZq2B1gTFIdpK4r/WuPsjwCNm9nV3f7aadlvCOOfXMu1EX6dtZva/BO/l480si+C1iFYW/vK5BPhKuCxWO4B3zOwId5+bQJ57wv9j5WdkINW8D0I7CD5f0f5J0A36O+CGiOXbPHZX3heAAwh+4QJMI9h4A8Ddfwf8zsx+B/wBOJjgFwfE/0W9MXwOlc/nbGK/BpPN7FaCz3DkZ2NWnLi1psKfGjOAxwiKa2S3S5GZ3Rtev47gy2Cgmb1NsCUd8+epu38tvHqTmU0m2NJ7Jc66rwBGufsO2NdVMgOILvw7zOwC4AmCN+L5BB+aaC+b2asEW5UQdLu8FNnAzCaGMdoBC8xsJvu/Yc+gqt8A2wk+LC3iPBeAUqDEzN6IirmvW8LMprv7cTG6seJ18V0D/CyM90/gVYJfQHVmZlcC44BOBN0ovYB7Cboeoi0FppjZi1Tzpefuz5rZl4HD+W9RwWP0h9dSFxJ7nc4j6Gq6wt3XmVkfgm6/aJcB3wF+4+7LzKw/wfs/luOAy8xsabjuytco1v6IGwne573N7B/AaODSyAYR7z2AbOBQgq7V/UR+SZrZcQRf0A+ZWRcz6+/uy6IecibwLSI2oIAHqPo5WkrwpbDf/jeCrsRoVxMchXuIma0m+LVxQYx2lQMLCiOfQpyYddKs+vgbCzM7CvgpVXdIRvcj5gCDCd5YH7l7WRLWPRc4yt13hbdbEfzEjO6/7gfcQfBhcuBt4Afuvjyq3S3AewQfWAj6sUd5xM4wM9uvzzeau0+NkWeVvs04z+eSODEfqemx1cQsJCj8/fjv6xOv+CQaswQYSfAzPe5+g3D5jbFiuPvNUe3uBfIIugsfJNgwmOnuV9Q1zzBuzNcr1utUi5itCXbCflRDu77E2JL2qB3GYdtOBJ+NUeHfd4F2kUU66rmUAyvcfVU167+RoKAOdveDzawH8LS7j45qN4egW7ZyA6oNwYCK6M9wjfvfItpmu/veMFaWu8f6NdgwUt2XlIkXgn7prxD85NzX9xqj3bEEW1QXV16SsO7rCPrlbwovJQQFva7xYvXpxuwPB25JZFm4/A/AqQnm0AIYEl5yG+r1qWXM98K/H4R/c+L9nyIek1fD/XOi/rYF3qrv80/guUwP/25j//7rbcDWGO2/Ev5Pl4W3hwMvxIl9LUF/+c3Arwj6z6+J0/ZtoH3E7UOJsY+AYHTU6eGlWw3PrYTgS6Ta/Tthjq0ibrciGIgQ3a42+98+IdjiP4lq9rEQ/KK/DSgKL38COiTzNVZXT2pscPeJ1TWIN7ICiDkMLVHufpuZTeG/W+iXufsHMdbfFbiSqsPqLg/v3zdiJNz6qdSO4AMZyynA+KhlX4yxDIKdbT82s91AGXG6ZSwYHvgIsDxs09vMLvEYwzlrocbXpw6mmtlPgdZmdgrB/y7mOszsGGACQSHvY2bDgG+7+3ejmu4M/5aGW6afAfl1TTDRLjF3Py782y7B0DcR/NqZEj6uxMziDT1MtCsSguHOE83sS8AhBJ+N/bpGYow6u9PMYo46C+1xdzezyn72NnHaPQS8Z2bPhbfPJHjNotVm/9shBF9OVwMTzGwS8IS7T49q9zeC/TDnhrcvCvOp8iuirtTVkwJmdhJBn3l0v/S/Itp8SDAUNC0vgJm9Q9BtEz1k7Nnw/g4EP8lr3BlWj2GFnYCD2L//empUm2Lgmx52IVgwFvpxdy+ozfONilnj61OHmFkERe1UggL0KvBgrNfXzN4j6LZ5wasf1vcLgoJ4EnAXQbF+0N1/Udc8U8HM3nX3URZx3EKsIZLh8oS6IiPan0kwEKAd8HWPGBId3j8bOMXDAQjhBs1/fP+BC5Htf0zwnjuF4L19OfBPrzr4ATMbQUQXZ6wNqKj2JxDuf3P3PTW0PYCgq/UCd8+Ouq/EI4asxltWH9riT43LCL7dc4kYp02wo6hSbUbApEKeRx20EslrN2KktiMmsNgHpb1D1Z2huR7Rb+zuH5tZvBEjiUrk9akVd68ws78D07yGfu6w/Uqz/Y6x2xujTeUO52fDrcNW4evS2NRmlFSNW9IWjM2P/MLsQLBB8T0zw/c/3qA2o85w9z+Gv8i2Euxf+6W7vx6n7Swg4ZE00RstsYRfDucBpxF045wbo9lOMzuu8peABQd07YzRrs5U+FPjKHePOR1AHUfApMIkM/uSu79Uc9MaubsvN7Oro+8ws05xiv+1/Hen2NjKnWIx2hWZ2YMEQ1QhGFpYVM98474+dWVmZxB0ObQA+luMo4EjrLTgyFkPv8SuJRhbHyvufkfYhoWvXt2BKRA5SupxqhkllWBXZPTrW1zNumscdRYjh9eBmMU+lcxsOfABwaij6yu7u2L4DvBo+KsbYBPBUNnk5aKunuQzs4eAW919QYz7TiDoCriF4CfsvrsIdoTWao6YeuS4DWhD8GGN28eeYKxJ7n66mS0j+FKL3JR1j3GouZm97+5HhaNhjvbgcP/57n54VLuWBH2ikaOK7nb3msZzV5dv3NenHjETOho4XN6F4Gf+yQT/q9eAaz3qyMx4+4E8xpHQjYGZtQVw9+0NuM7vExwXUTlK6C13fy5Gu6Qc1V8fZtbe3bdWc/91UXlV7n/YQZBj9DEudaYt/tQYRTD2fBlRY5Urfw6aWW6M/uzWDZWgu7eL1cdex1inh1ffJjg6+S13X1jDwxLaKRYW+NuA28J8e9Wn6Ifivj71iFnm7luium9iblW5+0Zij9+OVkga9wMlysyOINjx2im8vRG4xN3n1THeU+5+brg/oMpzj3qduhF0Lc0i2Cn6aqyYtdhRnUp7wl/F0cdlXB5ercxxMMGv4ecJ3psXAjOTmYi2+FPAgrHKVbj7irruCE22eH3sHmNumVrEHEuw5fU/BFuqswi+BO6o4XFxd4qF3QJnEGykFAPrwzx/WI88474+9Yg5gWBn8Q0Ek5B9n2D/xHditK12RFVEu6eB77t7uvYDJSQcKPAzd58c3h5DMJb92DrGy3f3tYm+ThZ8255KsO+mkKArZYK7L4nx8LQJX8+FBEO4f0Xw5f+hu18b1W4a8GUPx/mbWTvgRXc/niRR4W9gtRktk+I8Ej7wpJZxs8O4Ywn6Knd6OKlYHeN94O5Hhl9Uvd39xngjRtLBwpkYLRjK2Yb9R/X8unL0StRjahpRFbkfaDjB1l469gMlxMxmR4+iibUsxTkMIyj8pxHMazQKeN3df1LtAxtQxHt5jrsPDffvvOXuo6LafQQMrfxlG3Z3zknmfil19TSwWo6WSaVd7r7LzDCzlu6+0KqZnz4RFkyr0IZgXPZbBDtR407hnKAcM8snGP3ws3rGSoUCC8bYn0fwZRc5y2QeUKXwU8OIKoIpoyv3A50ZsbxyWWOzNBx6WjlNw4UEUxnUSW364y0458HFwEaCo5uvd/cyC4bXLmL//WjpVnlk/mYzG0Iwj1e3GO0eBWZGjXx6OJmJqPBnrtoceJKoOUABwRG2Wwje4DPcvT5D0W4m2Hqe7u7vW3Bg0KJ65plM9xJ08Qxg/9EoRlC8Yh3IVO2IqsayH6gWLid4nSqHw74VLquTWvbHdyKYsXS/9244vPb0OI9Jl/stGL//c4J5utoCVY7JcPffWDBte+UO65gHYdaHunqkVgeeJBivHcFkWj8Gurt7zDOVJRAnm6CP+/b65pRqZnaPu19VQ5vILdm2BN03lXP979uSbSz7gSS5bP+T0FQei+Je/0n3ap+LCr8ki5l9j2ArpYBgioW3CPow36xHzJnuPjI5GTYOFh7oRfC/qTJ+v7HsB0qUBUdT/5iqO6uTNptkc2C1OAlNynNR4ZdkseBw+LeAYq/mrFW1jHk7wdbRk0RMG+1JnJu8odV19FNjZcG0CfdStaBVd+BVxrEEZ6RtCCr80qhZcP6BaN7UtyaTPfopncys2Osxd1KmMLP7gTs9sZPQpDYXFX6RhhVj9NP0JIx+anDhAXUQHLOwHniO/YedNrpuqXSIOBAth+CAyUROQpPanFT4pTEzswMJ5vDp4e5fNLPDCE6QEWuK3CYh7L4qIPjwv03Q31/f0U8NLt4UHfuuxJiqIxPFOxCtUn0OHKwrFX5p1MJhbQ8RHBk6zIKzln3gcabxbUqSNfop3SyYE/8Vd98ajucfQXDwWpPdD9PcxZ2+VKSR6OLuTxFOnxzuNK4yhXFTYmbfM7MnCWZq/CrBHDNfTG9W9fLzsOgfRzBR3YPAPWnOSaqhA7iksdthZp0JuxDMbBTBkLimrBXBxHNJG/2UZpVfxF8GHnD3F83s/9KZkFRPXT3SqFlwFqQ7CWY0nA90Bc529znVPlAajAUniVlNcFarEQQnDZnZkHP1SO2o8EujZsGp+b4HfIHgZN8zCIbExZoDR9LAzPIIJkeb6+6LwrmVjnD319KcmsShwi+Nmpk9RXCavH+Ei74JdHT3c9KXlUjTpsIvjZqZLXD3w2paJiKJ06geaexmhTt0ATCzo6n/OXdFMpq2+KVRM7MPCU5F90m4qA/wEcGslmk56lGkqVPhl0atMR71KNLUqfCLiGQY9fGLiGQYFX4RkQyjwi8Zx8x+ZmbzzWyOmZWEI4VSta4pZlaYqvgidaG5eiSjmNkxwOnACHffbWZdgBZpTkukQWmLXzJNPrDR3XcDuPtGd19jZr80s/fNbJ6Z3W9mBvu22G83syIz+9DMjjKzf5nZosqJyMysn5ktNLN/hG2eCacx2I+ZnWpmM8xslpk9bWZtw+W/N7MF4S+QPzbg/0IylAq/ZJrXgN5m9rGZ3W1mJ4TL/+ruR4XnRG1N8Kug0h53LyQ4r+zzwNXAEODScOZQCI41uNvdDyWYYuK7kSsNf1n8HDjZ3UcQHIR2Xfj4rwGHh8ckaFZLSTkVfsko7r6d4OxX44ANwJNmdikw1szeC0+TdyLBbKCVXgj/zgXmu/va8BfDUqB3eN9Kd387vP534LioVY8CDgPeNrMS4BKgL8EU07uACWZ2FlCatCcrEof6+CXjuPteYAowJSz03waGAoXuvtLMbiKYM79S5XlkKyKuV96u/AxFHxATfduA1939/Oh8zGwkcBJwNsFMpE36RPLS+GmLXzKKmQ02s4MiFg0nmAICYGPY7352HUL3CXccQzCD6PSo+98FRpvZoDCPNmZ2cLi+Du7+EvBDQHPYS8ppi18yTVvgTjPrSDDfz2KCbp/NwDxgHfB+HeJ+BFxtZn8DFhB16kF33xB2KT1uZpXn1v05wTkGng/PO2DAdXVYt0itaMoGkXoys37ApHDHsEijp64eEZEMoy1+EZEMoy1+EZEMo8IvIpJhVPhFRDKMCr+ISIZR4RcRyTD/D69ats8G2WlUAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "soctm-Ne9QRN",
        "colab_type": "text"
      },
      "source": [
        "**Sentence segmentation using nltk**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r_Q2f8qfxyWJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 768
        },
        "outputId": "5ad1b32e-1477-4aec-e91c-8645e7cd325d"
      },
      "source": [
        "sent_tokenize(text)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['I must be honest with you.',\n",
              " 'I was about to go to sleep before I opened my mail and read your letter.',\n",
              " 'Now I am wide awake, sitting upright, because you made me question my preferences between shoes and slippers.',\n",
              " 'Be patient with me.',\n",
              " 'I need proper peaceful sleep and let’s sort this out.I am a shoe person.',\n",
              " 'Correction: I am a person obsessed with dry, dust-free feet.',\n",
              " 'Whatever helps me keep a grip on my walk and posture in spite of my profusely sweaty feet, I am that person.',\n",
              " 'Whatever keeps the grainy, pokey sensation of soil or sand in parts of my skin away, I am that person.',\n",
              " 'I guess, I am a shoe person because it is loyal to my whole feet.',\n",
              " 'Not making parts of it lying exposed and parts of it covered, without any symmetry.',\n",
              " 'I am also cotton sock person, preferably red.',\n",
              " 'Sometimes, I am a clean matte (anti-slippery) floor person.',\n",
              " 'Basically, I am a person with issues.I really liked your host-guest theory of travelling, how we do not just get out of home when we travel, the home expands when we step foot at a place engulfing it into our comfort zone.',\n",
              " 'But the thing is I wear socks all the time in my house, and open them when I go to bed, replacing the sock with a comforter.',\n",
              " 'I guess I need that comfort zone close and maybe, mine is restricted to my own skin.',\n",
              " 'So much so, that it doesn’t even include my own home.',\n",
              " 'When I travel, I like wearing shoes, with a firm grip on myself, dust free, protected.',\n",
              " 'When I face situations where I have to open them and I don’t want to, I usually wrinkle my nose.',\n",
              " 'It doesn’t get better that I can’t wear those shoes again with a dirty feet.',\n",
              " 'I take a handkerchief.Imagine my situation at the beach or in a hill stream, when the water feels soothing to my sweaty feet and removes the dust, only to attract more of it when I walk on the sand.',\n",
              " 'I let it irritate me those times, because I always have the ocean and the river on my side.',\n",
              " 'I like my bare feet drowning with no air bubbles left.',\n",
              " 'I don’t like it when droplets of water make only a part of my feet wet, only to let me realise and miss the comfort of dry skin and scowl at the linger of irritation due to a little dampness.',\n",
              " 'I don’t like it when I have to keep on brushing my left leg against my right to flatten the spheres of water.',\n",
              " 'And repeat it with the right.',\n",
              " 'I just realised why I don’t like drizzling rains.Eventually, when I have to get out of the water, I give up my fight with the dirt and wear those damn shoes with muddy feet.',\n",
              " 'Those are the times I miss my slippery slippers.',\n",
              " 'Don’t let my socks or shoes hear me say that.',\n",
              " 'They do not know that sometimes I keep a back-up pair of red flip-flops for such scenarios.',\n",
              " 'Does this make me a slipper person?',\n",
              " 'I guess I am a slipper person when my feet are ankle deep in mud, so that there is no place for me to complain.',\n",
              " 'No alternative.',\n",
              " 'No uneasy half-done feeling.',\n",
              " 'Otherwise, not much of a slipper person.I am a person who likes symmetry.',\n",
              " 'All in or none at all.',\n",
              " 'It’s difficult being me in this world with its shades of grey.',\n",
              " 'But then, my favourite colour is red and thankfully it covers the entire spectrum.',\n",
              " 'All I need is an emotion, and the rest is taken care of.Thank you for being the listener you are.',\n",
              " 'This has been a selfish post.',\n",
              " 'I used your letter for my sense of clarity.',\n",
              " 'You always make me reflect and dig deep.',\n",
              " 'I can finally sleep now.',\n",
              " 'Goodnight.']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aa73QyL19PAd",
        "colab_type": "text"
      },
      "source": [
        "**from scratch using for loop**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "du_AF81i-ymd",
        "colab_type": "text"
      },
      "source": [
        "**Stopwords**\n",
        "\n",
        "Stopwords considered as noise in the text. Text may contain stop words such as is, am, are, this, a, an, the, etc.\n",
        "In NLTK for removing stopwords, you need to create a list of stopwords and filter out your list of tokens from these words."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ypu1Di9z0CiE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "12a068fe-5b79-405a-beff-33aee60bd00f"
      },
      "source": [
        "x = word_tokenize(text)\n",
        "stopwords = [';', ':', ',', '.', '!', '?']\n",
        "new =[]\n",
        "storey = \"\"\n",
        "for i in x:\n",
        "    if i in stopwords:\n",
        "      storey = storey + i\n",
        "      new.append(storey)\n",
        "      storey = \" \"\n",
        "    else:\n",
        "      storey= storey +\" \"+ i\n",
        "print(new)"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[' I must be honest with you.', '  I was about to go to sleep before I opened my mail and read your letter.', '  Now I am wide awake,', '  sitting upright,', '  because you made me question my preferences between shoes and slippers.', '  Be patient with me.', '  I need proper peaceful sleep and let ’ s sort this out.I am a shoe person.', '  Correction:', '  I am a person obsessed with dry,', '  dust-free feet.', '  Whatever helps me keep a grip on my walk and posture in spite of my profusely sweaty feet,', '  I am that person.', '  Whatever keeps the grainy,', '  pokey sensation of soil or sand in parts of my skin away,', '  I am that person.', '  I guess,', '  I am a shoe person because it is loyal to my whole feet.', '  Not making parts of it lying exposed and parts of it covered,', '  without any symmetry.', '  I am also cotton sock person,', '  preferably red.', '  Sometimes,', '  I am a clean matte ( anti-slippery ) floor person.', '  Basically,', '  I am a person with issues.I really liked your host-guest theory of travelling,', '  how we do not just get out of home when we travel,', '  the home expands when we step foot at a place engulfing it into our comfort zone.', '  But the thing is I wear socks all the time in my house,', '  and open them when I go to bed,', '  replacing the sock with a comforter.', '  I guess I need that comfort zone close and maybe,', '  mine is restricted to my own skin.', '  So much so,', '  that it doesn ’ t even include my own home.', '  When I travel,', '  I like wearing shoes,', '  with a firm grip on myself,', '  dust free,', '  protected.', '  When I face situations where I have to open them and I don ’ t want to,', '  I usually wrinkle my nose.', '  It doesn ’ t get better that I can ’ t wear those shoes again with a dirty feet.', '  I take a handkerchief.Imagine my situation at the beach or in a hill stream,', '  when the water feels soothing to my sweaty feet and removes the dust,', '  only to attract more of it when I walk on the sand.', '  I let it irritate me those times,', '  because I always have the ocean and the river on my side.', '  I like my bare feet drowning with no air bubbles left.', '  I don ’ t like it when droplets of water make only a part of my feet wet,', '  only to let me realise and miss the comfort of dry skin and scowl at the linger of irritation due to a little dampness.', '  I don ’ t like it when I have to keep on brushing my left leg against my right to flatten the spheres of water.', '  And repeat it with the right.', '  I just realised why I don ’ t like drizzling rains.Eventually,', '  when I have to get out of the water,', '  I give up my fight with the dirt and wear those damn shoes with muddy feet.', '  Those are the times I miss my slippery slippers.', '  Don ’ t let my socks or shoes hear me say that.', '  They do not know that sometimes I keep a back-up pair of red flip-flops for such scenarios.', '  Does this make me a slipper person?', '  I guess I am a slipper person when my feet are ankle deep in mud,', '  so that there is no place for me to complain.', '  No alternative.', '  No uneasy half-done feeling.', '  Otherwise,', '  not much of a slipper person.I am a person who likes symmetry.', '  All in or none at all.', '  It ’ s difficult being me in this world with its shades of grey.', '  But then,', '  my favourite colour is red and thankfully it covers the entire spectrum.', '  All I need is an emotion,', '  and the rest is taken care of.Thank you for being the listener you are.', '  This has been a selfish post.', '  I used your letter for my sense of clarity.', '  You always make me reflect and dig deep.', '  I can finally sleep now.', '  Goodnight.']\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}