DW02_Regular_Expression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5adfa3f4",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "4afb93ea",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Why-regular-expressions?\" data-toc-modified-id=\"Why-regular-expressions?-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Why regular expressions?</a></span></li><li><span><a href=\"#Example-1:-Validate-emails\" data-toc-modified-id=\"Example-1:-Validate-emails-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>Example 1: Validate emails</a></span><ul class=\"toc-item\"><li><span><a href=\"#Raw-string\" data-toc-modified-id=\"Raw-string-2.1\"><span class=\"toc-item-num\">2.1&nbsp;&nbsp;</span>Raw string</a></span></li><li><span><a href=\"#Group\" data-toc-modified-id=\"Group-2.2\"><span class=\"toc-item-num\">2.2&nbsp;&nbsp;</span>Group</a></span></li><li><span><a href=\"#Begin\" data-toc-modified-id=\"Begin-2.3\"><span class=\"toc-item-num\">2.3&nbsp;&nbsp;</span>Begin</a></span></li><li><span><a href=\"#End\" data-toc-modified-id=\"End-2.4\"><span class=\"toc-item-num\">2.4&nbsp;&nbsp;</span>End</a></span></li><li><span><a href=\"#Match-characters\" data-toc-modified-id=\"Match-characters-2.5\"><span class=\"toc-item-num\">2.5&nbsp;&nbsp;</span>Match characters</a></span></li><li><span><a href=\"#Loop\" data-toc-modified-id=\"Loop-2.6\"><span class=\"toc-item-num\">2.6&nbsp;&nbsp;</span>Loop</a></span></li><li><span><a href=\"#Escape\" data-toc-modified-id=\"Escape-2.7\"><span class=\"toc-item-num\">2.7&nbsp;&nbsp;</span>Escape</a></span></li></ul></li><li><span><a href=\"#Example-2:-Character-sets-[]\" data-toc-modified-id=\"Example-2:-Character-sets-[]-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>Example 2: Character sets <code>[]</code></a></span><ul class=\"toc-item\"><li><span><a href=\"#Positive-character-sets\" data-toc-modified-id=\"Positive-character-sets-3.1\"><span class=\"toc-item-num\">3.1&nbsp;&nbsp;</span>Positive character sets</a></span></li><li><span><a href=\"#Negative-character-sets\" data-toc-modified-id=\"Negative-character-sets-3.2\"><span class=\"toc-item-num\">3.2&nbsp;&nbsp;</span>Negative character sets</a></span></li><li><span><a href=\"#Metacharacters-inside-character-sets\" data-toc-modified-id=\"Metacharacters-inside-character-sets-3.3\"><span class=\"toc-item-num\">3.3&nbsp;&nbsp;</span>Metacharacters inside character sets</a></span></li><li><span><a href=\"#Shorthand-character-sets\" data-toc-modified-id=\"Shorthand-character-sets-3.4\"><span class=\"toc-item-num\">3.4&nbsp;&nbsp;</span>Shorthand character sets</a></span></li><li><span><a href=\"#Is-[^\\d\\s]-the-same-as-[\\D\\S]?\" data-toc-modified-id=\"Is-[^\\d\\s]-the-same-as-[\\D\\S]?-3.5\"><span class=\"toc-item-num\">3.5&nbsp;&nbsp;</span>Is <code>[^\\d\\s]</code> the same as <code>[\\D\\S]</code>?</a></span></li></ul></li><li><span><a href=\"#Example-3:-Repetition-Meta-characters\" data-toc-modified-id=\"Example-3:-Repetition-Meta-characters-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span>Example 3: Repetition Meta-characters</a></span></li><li><span><a href=\"#Example-4:-Grouping\" data-toc-modified-id=\"Example-4:-Grouping-5\"><span class=\"toc-item-num\">5&nbsp;&nbsp;</span>Example 4: Grouping</a></span></li><li><span><a href=\"#Example-5:-Greedy-vs-Lazy-regex\" data-toc-modified-id=\"Example-5:-Greedy-vs-Lazy-regex-6\"><span class=\"toc-item-num\">6&nbsp;&nbsp;</span>Example 5: Greedy vs Lazy regex</a></span></li><li><span><a href=\"#Example-6:-Alternation-|\" data-toc-modified-id=\"Example-6:-Alternation-|-7\"><span class=\"toc-item-num\">7&nbsp;&nbsp;</span>Example 6: Alternation <code>|</code></a></span></li><li><span><a href=\"#Example-7:-backslash-plague-\\\" data-toc-modified-id=\"Example-7:-backslash-plague-\\-8\"><span class=\"toc-item-num\">8&nbsp;&nbsp;</span>Example 7: backslash plague <code>\\</code></a></span></li><li><span><a href=\"#Example-8:-Raw-String-r&quot;...&quot;\" data-toc-modified-id=\"Example-8:-Raw-String-r&quot;...&quot;-9\"><span class=\"toc-item-num\">9&nbsp;&nbsp;</span>Example 8: Raw String <code>r\"...\"</code></a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49c8f6d8",
   "metadata": {},
   "source": [
    "# Note - Regular Expression with Python"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1be87a1",
   "metadata": {},
   "source": [
    "## Why regular expressions?\n",
    "\n",
    "- Useful in **finding, replacing and extracting information from text**, such as log files, HTML/XML files, and other documents\n",
    "\n",
    "- Useful in **verifying whether input fits into the text pattern**, such as verifying"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94ed3128",
   "metadata": {},
   "source": [
    "## Example 1: Validate emails\n",
    "\n",
    "`r\"(^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\\.[a-zA-Z0-9-.]+$)\"`\n",
    "\n",
    "\n",
    "![](image/regex_valid_email.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72f42f02",
   "metadata": {},
   "source": [
    "### Raw string\n",
    "\n",
    "`r\"\"` stand for raw string in python."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0839850a",
   "metadata": {},
   "source": [
    "### Group\n",
    "\n",
    "`()` creates a capture group and indicates precedence."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ca0d63f",
   "metadata": {},
   "source": [
    "### Begin\n",
    "\n",
    "`^` matches beginning of string."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9936b0ff",
   "metadata": {},
   "source": [
    "### End\n",
    "\n",
    "`$` matches end of string."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8418ba4a",
   "metadata": {},
   "source": [
    "### Match characters\n",
    "\n",
    "`[a-zA-Z0-9_.+-]` match **one character** in any of \"a\" to \"z\", \"A\" to \"Z\", \"0\" to \"9\", \"_\", \".\", \"+\" and \"-\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e535649",
   "metadata": {},
   "source": [
    "### Loop\n",
    "\n",
    "`+` after `[]` indicates loop to find **one and more** matched character until the next character outside the pattern.\n",
    "\n",
    "`*` indicates loop to find **zero and more** matched character until the next character outside the pattern.\n",
    "\n",
    "`{m,n}` indicates loop to find the $m_{th}$ to at most $n_{th}$ matched character."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "155e7cbe",
   "metadata": {},
   "source": [
    "### Escape\n",
    "\n",
    "`\\` escape from special characters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "756b5476",
   "metadata": {},
   "source": [
    "## Example 2: Character sets `[]`\n",
    "\n",
    "### Positive character sets\n",
    "\n",
    "To match the following two words:\n",
    "\n",
    "```\n",
    "grey gray\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8d18b2f",
   "metadata": {},
   "source": [
    "Regular expression: `gr[ea]y`\n",
    "\n",
    "![](image/regex_match_two_words.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59a8332f",
   "metadata": {},
   "source": [
    "### Negative character sets\n",
    "\n",
    "To match the words not `bog` from the following words:\n",
    "\n",
    "```\n",
    "hog dog bog\n",
    "```\n",
    "\n",
    "Regular expression: `[^b]og`\n",
    "\n",
    "\n",
    "`[^]`: if the `^` is the **first character** insider the character set `[]`, all the characters that are **not in the set** will be matched."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c9fad4a",
   "metadata": {},
   "source": [
    "### Metacharacters inside character sets\n",
    "\n",
    "We are going to match the following strings:\n",
    "\n",
    "```\n",
    "var(9), var[0]\n",
    "```\n",
    "\n",
    "Regular expression: `var[([][0-9][)\\]]`\n",
    "\n",
    "![](image/regex_metacharacter.png)\n",
    "\n",
    "For metaharacters **except** `]`, `-`, `^`, `\\` **inside** the character set `[]`, all other metacharacters **already escaped**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48415a34",
   "metadata": {},
   "source": [
    "### Shorthand character sets\n",
    "\n",
    "![](image/regex_shorthand_character.png)\n",
    "\n",
    "![](image/regex_shorthand_example.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27ee934f",
   "metadata": {},
   "source": [
    "### Is `[^\\d\\s]` the same as `[\\D\\S]`?\n",
    "\n",
    "![](image/regex_character_caution.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b34461c",
   "metadata": {},
   "source": [
    "## Example 3: Repetition Meta-characters\n",
    "\n",
    "![](image/regex_repetition.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cd549dd",
   "metadata": {},
   "source": [
    "To match the following words:\n",
    "\n",
    "```\n",
    "oops ooops ooooops oooooops\n",
    "```\n",
    "\n",
    "But not\n",
    "\n",
    "```\n",
    "ops\n",
    "```\n",
    "\n",
    "The regular expression should be: `ooo*ps` or `oo+ps`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70e69a61",
   "metadata": {},
   "source": [
    "![](image/regex_repetition_mn.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c65551b",
   "metadata": {},
   "source": [
    "![](image/regex_repetition_exe.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7dba1a44",
   "metadata": {},
   "source": [
    "## Example 4: Grouping\n",
    "\n",
    "`()`: matches **whatever** regular expression is **inside** the parentheses, and indicates the start and end of a group.\n",
    "\n",
    "- **Apply repetition** operators to a group of regular expressions\n",
    "\n",
    "- Makes regular expressions **easier to read**\n",
    "\n",
    "- **Capture groups** for use in matching, replacing and extraction, i.e., the contents of a group can be retrieved.\n",
    "\n",
    "- **Cannot be used insides a character set**.\n",
    "\n",
    "![](image/regex_group_example.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64fbd40b",
   "metadata": {},
   "source": [
    "## Example 5: Greedy vs Lazy regex\n",
    "\n",
    "![](image/regex_group_greedy.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67cb9acb",
   "metadata": {},
   "source": [
    "![](image/regex_lazy.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66ac74f1",
   "metadata": {},
   "source": [
    "## Example 6: Alternation `|`\n",
    "\n",
    "![](image/regex_alternation.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34258f14",
   "metadata": {},
   "source": [
    "## Example 7: backslash plague `\\`\n",
    "\n",
    "![](image/regex_backslash.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71ad6ce2",
   "metadata": {},
   "source": [
    "## Example 8: Raw String `r\"...\"`\n",
    "\n",
    "![](image/regex_raw_string.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}