UniProt_MBDBMetrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZWtNO2K7QjS_"
   },
   "source": [
    "# Molecular Biology Database Metrics Jupyter Notebook\n",
    "At [UniProt](https://uniprot.org) we have developed some useful metrics over the years to measure and report our progress and impact to funders and policy makers. As part of an administrative supplement awarded by NIH [ODSS](https://datascience.nih.gov/about/odss) we have created [a Google Colab page](https://colab.research.google.com/drive/1aEmSQR9DGQIZmHAIuQV9mLv7Mw9Ppkin) and this Jupyter notebook to enable other data resource providers to easily run these metrics.\n",
    "\n",
    "Both allow you to investigate how often a resource is mentioned in the full text literature and how often it is mentioned in papers that cite specific funding sources. You need not limit yourself to only looking at database resources. You can use any keywords to investigate the impact of experimental techniques or anything else you can think of.\n",
    "\n",
    "If you have specific feedback on this notebook then please contact Alex Bateman (<agb@ebi.ac.uk>).\n",
    "We would like to fund the fantastic work of [Europe PubMed Central](https://europepmc.org/) and [ChEMBL](https://www.ebi.ac.uk/chembl/) for making the APIs that power much of this page.\n",
    "\n",
    "\n",
    "## Contents\n",
    "\n",
    "This notebook will perform queries in publication ([EuropePMC](https://europepmc.org/)) and patents ([SureCHEMBL](https://www.surechembl.org/)) databases to compute and display graphically the impact of a resource.\n",
    "In particular plots will be generated that\n",
    "- show how the number of mentions changes over time\n",
    "- group publications by the grant agency they acknowledge\n",
    "- find in which section of a publication a resource is mentioned.\n",
    "\n",
    "# Instructions\n",
    "\n",
    "Start by executing BOTH the (⏵︎) **Code** and the **Parameters** cells (wherein you can specify what to query for).\n",
    "\n",
    "After doing that, to (re)generate the plots, execute (⏵︎) the corresponding cells.\n",
    "Alternatively, after choosing what to query in the **Parameters** section, you can choose **Run all** from the **Runtime** menu at the top of the page.\n",
    "\n",
    "If this is your first time using an interactive notebook or colab, you may want to check the [intro to jupyter notebooks](https://jupyter.org/try-jupyter/retro/notebooks/?path=notebooks/Intro.ipynb) or the [colab introductory page](https://colab.research.google.com/)\n",
    "\n",
    "# Caveat\n",
    "\n",
    "To find all mentions of the data resource it may be important to provide synonyms. Each search term must match exactly, thus “UniProt” and “UniProtKB” will match different sets of papers.\n",
    "\n",
    "If the name of the resource matches a common English word such as STRING or PRINTS then you will get many false positive matches.\n",
    "\n",
    "Also keep in mind that this tool can only search the full text of the open access portion of EuropePMC, thus the count of papers will be an underestimate of the total number of mentions in the whole research literature.\n",
    "\n",
    "### Changelog:\n",
    "\n",
    "    2023.03.12 first colab version with basic query capabilities\n",
    "    2023.03.13 added plot generation via plotly and parameters selection via form\n",
    "    2023.03.14 interface and plot improvements; added capability to download data as csv, introduction text and first tests of patent info retrieval\n",
    "    2023.03.15 improved plot interface, also with possibility to draw on plots; plots can be saved with meaningful filenames; added retrieval of patent information from SureChEMBL\n",
    "    2023.03.16 added retrieval of total patents in timeframe; added progress bars and error messages if Code has not been initialised\n",
    "    2023.06.27 changed to ascii progress bar to avoid incompatibility with plotly display\n",
    "    2023.08.25 converted from colab version to run independently as jupyter notebook\n",
    "    2023.09.14 added downloadable list of API url links for totals of publication mentions\n",
    "    2023.11.03 added possibility to specify custom comma separated lists of granters\n",
    "    2023.11.13 expanded list of granters in dropdown, added link to a list of the top 1000 ones\n",
    "    2024.01.22 updated to new SureCHEMBL API for retrieving patent data\n",
    "\n",
    "### Credits:\n",
    "\n",
    "Code by Alex Bateman, Alex Ignatchenko and [Giuseppe Insana](https://insana.net)\n",
    "\n",
    "© 2023- [UniProt consortium](https://www.uniprot.org/help/about)\n",
    "\n",
    "### Tips\n",
    "\n",
    "- Add your own query search terms: simply modify the `options` array in the `query_widget = widgets.Dropdown` definition.\n",
    "- Add more funding agencies: simply modify the `options` array in the `granters_widget = widgets.Dropdown` definition.\n",
    "- Export all generated plots as svg (Scalable Vector Graphics format): look for `plotly_config` at the beginning of the Code cell and three lines below that change from `'format': 'png'` to `'format': 'svg'`. Once that is done and the cell re-run, the *Download plots as* button in the plot interface (the camera icon) will allow to export the plots in svg format.\n",
    "\n",
    "### Citation\n",
    "\n",
    "If you find this software useful, please consider citing the [journal article](https://doi.org/10.1093/bioadv/vbad180) ([pubmed 38130879](https://pubmed.ncbi.nlm.nih.gov/38130879)):\n",
    "\n",
    "Bibtex:\n",
    "```\n",
    "@article{10.1093/bioadv/vbad180,\n",
    "    author = {Insana, Giuseppe and Ignatchenko, Alex and Martin, Maria and Bateman, Alex and UniProt Consortium },\n",
    "    title = \"{MBDBMetrics: an online metrics tool to measure the impact of biological data resources}\",\n",
    "    journal = {Bioinformatics Advances},\n",
    "    volume = {3},\n",
    "    number = {1},\n",
    "    pages = {vbad180},\n",
    "    year = {2023},\n",
    "    month = {12},\n",
    "    issn = {2635-0041},\n",
    "    doi = {10.1093/bioadv/vbad180},\n",
    "    url = {https://doi.org/10.1093/bioadv/vbad180},\n",
    "    eprint = {https://academic.oup.com/bioinformaticsadvances/article-pdf/3/1/vbad180/54717401/vbad180.pdf},\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "71bks8wI2fj_",
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# imports\n",
    "import os\n",
    "import sys\n",
    "import re\n",
    "import pandas as pd\n",
    "import plotly.graph_objects as go\n",
    "import requests\n",
    "from requests.adapters import HTTPAdapter, Retry\n",
    "import datetime\n",
    "from time import sleep\n",
    "from tqdm import tqdm\n",
    "import plotly.io as pio\n",
    "\n",
    "pio.renderers.default = \"notebook\"\n",
    "import ipywidgets as widgets  # for jupyter input dialogs\n",
    "from IPython.display import clear_output\n",
    "\n",
    "RED = \"\\033[91m\"\n",
    "BOLD = \"\\033[1m\"\n",
    "END = \"\\033[0m\"\n",
    "\n",
    "# plotly config\n",
    "plotly_config = {\n",
    "    \"displayModeBar\": True,\n",
    "    \"toImageButtonOptions\": {\n",
    "        \"format\": \"png\",  # one of png, svg, jpeg, webp\n",
    "        \"filename\": \"impact_plot\",\n",
    "        #'width': 1280,\n",
    "        #'height': 1024,\n",
    "    },\n",
    "    \"modeBarButtonsToRemove\": [\"lasso2d\"],\n",
    "    \"modeBarButtonsToAdd\": [\"drawline\", \"drawopenpath\"],\n",
    "}\n",
    "\n",
    "# paper sections:\n",
    "paper_sections = [\n",
    "    \"ABBR\",\n",
    "    \"ACK_FUND\",\n",
    "    \"APPENDIX\",\n",
    "    \"AUTH_CON\",\n",
    "    \"CASE\",\n",
    "    \"COMP_INT\",\n",
    "    \"FIG\",\n",
    "    \"OTHER\",\n",
    "    \"REF\",\n",
    "    \"TABLE\",\n",
    "    \"ABSTRACT\",\n",
    "    \"INTRO\",\n",
    "    \"KEYWORD\",\n",
    "    \"METHODS\",\n",
    "    \"RESULTS\",\n",
    "    \"DISCUSS\",\n",
    "    \"CONCL\",\n",
    "    \"SUPPL\",\n",
    "]\n",
    "# grant agencies' lists:\n",
    "granters_nih = [\n",
    "    \"NIH\",\n",
    "    \"NHGRI NIH HHS\",\n",
    "    \"NIGMS NIH HHS\",\n",
    "    \"NCI NIH HHS\",\n",
    "    \"NHLBI NIH HHS\",\n",
    "    \"NIAID NIH HHS\",\n",
    "    \"NIDDK NIH HHS\",\n",
    "    \"NINDS NIH HHS\",\n",
    "    \"NIMH NIH HHS\",\n",
    "    \"NCRR NIH HHS\",\n",
    "    \"NICHD NIH HHS\",\n",
    "    \"NIA NIH HHS\",\n",
    "    \"NIDA NIH HHS\",\n",
    "    \"NEI NIH HHS\",\n",
    "    \"NIAMS NIH HHS\",\n",
    "    \"NIEHS NIH HHS\",\n",
    "    \"NIAAA NIH HHS\",\n",
    "    \"NIDCR NIH HHS\",\n",
    "    \"NIDCD NIH HHS\",\n",
    "    \"NIBIB NIH HHS\",\n",
    "]\n",
    "granters_us = [\n",
    "    \"Department of Energy\",\n",
    "    \"National Science Foundation\",\n",
    "    \"Bill and Melinda Gates\",\n",
    "    \"Department of Defense\",\n",
    "    \"Department of Agriculture\",\n",
    "    \"Food and Drug Administration\",\n",
    "    \"Environmental Protection Agency\",\n",
    "    \"Howard Hughes Medical Institute\",\n",
    "]\n",
    "granters_uk = [\n",
    "    \"Wellcome Trust\",\n",
    "    \"Medical Research Council\",\n",
    "    \"Biotechnology and Biological Sciences Research Council\",\n",
    "    \"Engineering and Physical Sciences Research Council\",\n",
    "    \"Natural Environment Research Council\",\n",
    "    \"Economic and Social Research Council\",\n",
    "    \"Arts and Humanities Research Council\",\n",
    "    \"Science and Technology Facilities Council\",\n",
    "]\n",
    "css_colors = {\n",
    "    \"aliceblue\",\n",
    "    \"antiquewhite\",\n",
    "    \"aqua\",\n",
    "    \"aquamarine\",\n",
    "    \"azure\",\n",
    "    \"beige\",\n",
    "    \"bisque\",\n",
    "    \"black\",\n",
    "    \"blanchedalmond\",\n",
    "    \"blue\",\n",
    "    \"blueviolet\",\n",
    "    \"brown\",\n",
    "    \"burlywood\",\n",
    "    \"cadetblue\",\n",
    "    \"chartreuse\",\n",
    "    \"chocolate\",\n",
    "    \"coral\",\n",
    "    \"cornflowerblue\",\n",
    "    \"cornsilk\",\n",
    "    \"crimson\",\n",
    "    \"cyan\",\n",
    "    \"darkblue\",\n",
    "    \"darkcyan\",\n",
    "    \"darkgoldenrod\",\n",
    "    \"darkgray\",\n",
    "    \"darkgrey\",\n",
    "    \"darkgreen\",\n",
    "    \"darkkhaki\",\n",
    "    \"darkmagenta\",\n",
    "    \"darkolivegreen\",\n",
    "    \"darkorange\",\n",
    "    \"darkorchid\",\n",
    "    \"darkred\",\n",
    "    \"darksalmon\",\n",
    "    \"darkseagreen\",\n",
    "    \"darkslateblue\",\n",
    "    \"darkslategray\",\n",
    "    \"darkslategrey\",\n",
    "    \"darkturquoise\",\n",
    "    \"darkviolet\",\n",
    "    \"deeppink\",\n",
    "    \"deepskyblue\",\n",
    "    \"dimgray\",\n",
    "    \"dimgrey\",\n",
    "    \"dodgerblue\",\n",
    "    \"firebrick\",\n",
    "    \"floralwhite\",\n",
    "    \"forestgreen\",\n",
    "    \"fuchsia\",\n",
    "    \"gainsboro\",\n",
    "    \"ghostwhite\",\n",
    "    \"gold\",\n",
    "    \"goldenrod\",\n",
    "    \"gray\",\n",
    "    \"grey\",\n",
    "    \"green\",\n",
    "    \"greenyellow\",\n",
    "    \"honeydew\",\n",
    "    \"hotpink\",\n",
    "    \"indianred\",\n",
    "    \"indigo\",\n",
    "    \"ivory\",\n",
    "    \"khaki\",\n",
    "    \"lavender\",\n",
    "    \"lavenderblush\",\n",
    "    \"lawngreen\",\n",
    "    \"lemonchiffon\",\n",
    "    \"lightblue\",\n",
    "    \"lightcoral\",\n",
    "    \"lightcyan\",\n",
    "    \"lightgoldenrodyellow\",\n",
    "    \"lightgray\",\n",
    "    \"lightgrey\",\n",
    "    \"lightgreen\",\n",
    "    \"lightpink\",\n",
    "    \"lightsalmon\",\n",
    "    \"lightseagreen\",\n",
    "    \"lightskyblue\",\n",
    "    \"lightslategray\",\n",
    "    \"lightslategrey\",\n",
    "    \"lightsteelblue\",\n",
    "    \"lightyellow\",\n",
    "    \"lime\",\n",
    "    \"limegreen\",\n",
    "    \"linen\",\n",
    "    \"magenta\",\n",
    "    \"maroon\",\n",
    "    \"mediumaquamarine\",\n",
    "    \"mediumblue\",\n",
    "    \"mediumorchid\",\n",
    "    \"mediumpurple\",\n",
    "    \"mediumseagreen\",\n",
    "    \"mediumslateblue\",\n",
    "    \"mediumspringgreen\",\n",
    "    \"mediumturquoise\",\n",
    "    \"mediumvioletred\",\n",
    "    \"midnightblue\",\n",
    "    \"mintcream\",\n",
    "    \"mistyrose\",\n",
    "    \"moccasin\",\n",
    "    \"navajowhite\",\n",
    "    \"navy\",\n",
    "    \"oldlace\",\n",
    "    \"olive\",\n",
    "    \"olivedrab\",\n",
    "    \"orange\",\n",
    "    \"orangered\",\n",
    "    \"orchid\",\n",
    "    \"palegoldenrod\",\n",
    "    \"palegreen\",\n",
    "    \"paleturquoise\",\n",
    "    \"palevioletred\",\n",
    "    \"papayawhip\",\n",
    "    \"peachpuff\",\n",
    "    \"peru\",\n",
    "    \"pink\",\n",
    "    \"plum\",\n",
    "    \"powderblue\",\n",
    "    \"purple\",\n",
    "    \"red\",\n",
    "    \"rosybrown\",\n",
    "    \"royalblue\",\n",
    "    \"rebeccapurple\",\n",
    "    \"saddlebrown\",\n",
    "    \"salmon\",\n",
    "    \"sandybrown\",\n",
    "    \"seagreen\",\n",
    "    \"seashell\",\n",
    "    \"sienna\",\n",
    "    \"silver\",\n",
    "    \"skyblue\",\n",
    "    \"slateblue\",\n",
    "    \"slategray\",\n",
    "    \"slategrey\",\n",
    "    \"snow\",\n",
    "    \"springgreen\",\n",
    "    \"steelblue\",\n",
    "    \"tan\",\n",
    "    \"teal\",\n",
    "    \"thistle\",\n",
    "    \"tomato\",\n",
    "    \"turquoise\",\n",
    "    \"violet\",\n",
    "    \"wheat\",\n",
    "    \"white\",\n",
    "    \"whitesmoke\",\n",
    "    \"yellow\",\n",
    "    \"yellowgreen\",\n",
    "}\n",
    "HEX_COLOR_REGEX = re.compile(r\"^#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})$\")\n",
    "\n",
    "\n",
    "# helper functions\n",
    "def eprint(*myargs, **kwargs):\n",
    "    \"\"\"\n",
    "    print to stderr, useful for error messages\n",
    "    \"\"\"\n",
    "    print(*myargs, file=sys.stderr, **kwargs)\n",
    "\n",
    "\n",
    "def query2publications(query, description=\"\", debug=False, apilinks=None):\n",
    "    \"\"\"\n",
    "    return total number of hits of a query submitted to Europe PMC database\n",
    "    \"\"\"\n",
    "    headers = {\"user-agent\": \"resourceimpact/0.5\", \"Accept\": \"application/json\"}\n",
    "    params = {\"resultType\": \"idlist\", \"format\": \"json\", \"pageSize\": 1, \"query\": query}\n",
    "    api_url = \"https://www.ebi.ac.uk/europepmc/webservices/rest/search\"\n",
    "\n",
    "    s = requests.Session()  # for retries\n",
    "    retries = Retry(total=5, backoff_factor=1, status_forcelist=[502, 503, 504])\n",
    "    s.mount(\"http://\", HTTPAdapter(max_retries=retries))\n",
    "\n",
    "    r = s.get(api_url, params=params, headers=headers)\n",
    "    if r.status_code in (200, 206):\n",
    "        parsed = r.json()\n",
    "        hits = parsed[\"hitCount\"]\n",
    "        if debug:\n",
    "            eprint(r.url)\n",
    "            eprint(\"{}: {}\".format(description, hits))\n",
    "        if apilinks is not None and os.path.isfile(apilinks):\n",
    "            with open(apilinks, \"a\") as apifh:\n",
    "                apifh.write('<a href=\"{}\">{}</a><br />\\n'.format(r.url, description))\n",
    "        return hits\n",
    "    else:\n",
    "        eprint(\n",
    "            \"ERROR: problems querying EuropePMC, http status code is '{}'({})\".format(\n",
    "                r.status_code, r.reason\n",
    "            )\n",
    "        )\n",
    "        return -1\n",
    "\n",
    "\n",
    "def query2patents(query, description=\"\", debug=False):\n",
    "    \"\"\"\n",
    "    return total number of hits of a query submitted to patent website\n",
    "    query format example: query=(uniprot) AND ((pnctry:(US OR EP OR WO OR JP))) AND (pd:20220608)\n",
    "    \"\"\"\n",
    "    query = \"({}) AND ((pnctry:(US OR EP OR WO OR JP)))\".format(query)\n",
    "    headers = {\"user-agent\": \"resourceimpact/0.5\", \"Accept\": \"application/json\"}\n",
    "    params = {\"itemsPerPage\": 0, \"page\": 1, \"query\": query}\n",
    "    api_url = \"https://www.surechembl.org/api/search/content\"\n",
    "\n",
    "    s = requests.Session()  # for retries\n",
    "    retries = Retry(total=5, backoff_factor=1, status_forcelist=[502, 503, 504])\n",
    "    s.mount(\"http://\", HTTPAdapter(max_retries=retries))\n",
    "\n",
    "    r = s.post(api_url, params=params, headers=headers)\n",
    "    if r.status_code in (200, 206):\n",
    "        parsed = r.json()\n",
    "        hits = parsed[\"data\"][\"results\"][\"total_hits\"]\n",
    "        if debug:\n",
    "            eprint(r.url)\n",
    "            eprint(\"{}: {}\".format(description, hits))\n",
    "        return hits\n",
    "    else:\n",
    "        eprint(\n",
    "            \"ERROR: problems querying SureCHEMBL, http status code is '{}'({})\".format(\n",
    "                r.status_code, r.reason\n",
    "            )\n",
    "        )\n",
    "        return -1\n",
    "\n",
    "\n",
    "def barplot(\n",
    "    x, y, title=\"\", xaxis=\"\", resulttype=\"papers\", filename=\"resource_impact\", sort=True\n",
    "):\n",
    "    \"\"\"\n",
    "    plot a bar plot\n",
    "    return a dataframe with the data used\n",
    "    \"\"\"\n",
    "    layout = go.Layout(\n",
    "        title=\"Number of {} mentioning {}<br />{}\".format(resulttype, query, title),\n",
    "        # barmode=\"stack\",\n",
    "        xaxis_title=xaxis,\n",
    "        # xaxis_showticklabels = False,\n",
    "        yaxis_title=\"Number of {}\".format(resulttype),\n",
    "    )\n",
    "    trace = go.Bar(x=x, y=y, marker_color=bar_colors, text=y)\n",
    "    plot = go.Figure([trace], layout)\n",
    "    plot.update_xaxes(type=\"category\")\n",
    "\n",
    "    if sort:\n",
    "        plot.update_xaxes(categoryorder=\"total ascending\")\n",
    "\n",
    "    if filename:\n",
    "        # update filename for downloadable plot image\n",
    "        plotly_config[\"toImageButtonOptions\"][\"filename\"] = filename\n",
    "\n",
    "    data = {}\n",
    "    data[xaxis] = x\n",
    "    data[resulttype] = y\n",
    "    df = pd.DataFrame(data)\n",
    "    plot.show(config=plotly_config, renderer=\"notebook\")\n",
    "    return df\n",
    "\n",
    "\n",
    "def show_form():\n",
    "    display(query_widget)\n",
    "    display(years_slider)\n",
    "    display(granters_widget)\n",
    "    display(bars_color_scheme_widget)\n",
    "\n",
    "\n",
    "def print_parameters():\n",
    "    clear_output(wait=True)\n",
    "    show_form()\n",
    "    print(\"Your chosen parameters:\\n\\tquery string is: {}\".format(query))\n",
    "    print(\"\\ttime frame: from {} to {}\".format(year_from, year_to))\n",
    "    print(\"\\tgrant agencies: {}\".format(\", \".join(grants)))\n",
    "\n",
    "\n",
    "def query_handler(change):\n",
    "    global query, query_short, query_keywords, resource_searchstring\n",
    "    query = change.new\n",
    "    query_keywords = query.split(\",\")\n",
    "    if len(query_keywords) == 2:\n",
    "        query_short = \", \".join(query_keywords[0:2])\n",
    "    elif len(query_keywords) > 2:\n",
    "        query_short = \", \".join(query_keywords[0:2]) + \", ...\"\n",
    "    else:\n",
    "        query_short = query_keywords[0]\n",
    "\n",
    "    resource_searchstring = '(\"{}\")'.format('\" OR \"'.join(query_keywords))\n",
    "    print_parameters()\n",
    "\n",
    "\n",
    "def years_handler(change):\n",
    "    global year_from, year_to, query_time_constraint\n",
    "    year_from, year_to = [date.strftime(\"%Y\") for date in change.new]\n",
    "    # year_from, year_to = [date.strftime('%Y') for date in years_slider.value]\n",
    "    if int(year_from) > int(year_to):\n",
    "        eprint(\n",
    "            \"\\t\\tATTENTION: 'year_to' cannot be smaller than 'year_from'! Defaulting year_to same as year_from: '{}'\".format(\n",
    "                year_from\n",
    "            )\n",
    "        )\n",
    "        year_to = year_from\n",
    "\n",
    "    if int(year_to) > int(current_year):\n",
    "        eprint(\n",
    "            \"\\t\\tNOTICE: 'year_to' cannot be later than current year. Defaulting year_to to '{}'\".format(\n",
    "                current_year\n",
    "            )\n",
    "        )\n",
    "        year_to = str(current_year)\n",
    "\n",
    "    query_time_constraint = \" and (FIRST_PDATE:[{}-01-01 TO {}-12-31])\".format(\n",
    "        year_from, year_to\n",
    "    )\n",
    "    print_parameters()\n",
    "\n",
    "\n",
    "def granters_handler(change):\n",
    "    global granters, grants, all_grant_agencies_constraint\n",
    "    granters = change.new\n",
    "    if granters == \"US funders\":\n",
    "        grants = granters_nih + granters_us\n",
    "    # elif granters == 'US not including NIH':\n",
    "    #  grants = granters_us\n",
    "    elif granters == \"UK funders\":\n",
    "        grants = granters_uk\n",
    "    elif granters == \"NIH Institutes\":\n",
    "        grants = granters_nih\n",
    "    elif granters == \"UK and US incl. NIH\":\n",
    "        grants = granters_nih + granters_us + granters_uk\n",
    "    elif \",\" in granters:\n",
    "        grants = granters.replace(\", \", \",\").split(\",\")\n",
    "    else:\n",
    "        grants = [granters]\n",
    "\n",
    "    all_grant_agencies_constraint = ' and (GRANT_AGENCY:\"{}\")'.format(\n",
    "        '\" or GRANT_AGENCY:\"'.join(grants)\n",
    "    )\n",
    "    print_parameters()\n",
    "\n",
    "\n",
    "def color_scheme_handler(change):\n",
    "    global bars_color_scheme, bar_colors\n",
    "    bars_color_scheme = change.new\n",
    "    # colors validator\n",
    "    if bars_color_scheme == \"multicolor\":\n",
    "        # color list from plotly.express.colors.qualitative.Alphabet\n",
    "        bar_colors = [\n",
    "            \"#AA0DFE\",\n",
    "            \"#3283FE\",\n",
    "            \"#85660D\",\n",
    "            \"#782AB6\",\n",
    "            \"#565656\",\n",
    "            \"#1C8356\",\n",
    "            \"#16FF32\",\n",
    "            \"#F7E1A0\",\n",
    "            \"#E2E2E2\",\n",
    "            \"#1CBE4F\",\n",
    "            \"#C4451C\",\n",
    "            \"#DEA0FD\",\n",
    "            \"#FE00FA\",\n",
    "            \"#325A9B\",\n",
    "            \"#FEAF16\",\n",
    "            \"#F8A19F\",\n",
    "            \"#90AD1C\",\n",
    "            \"#F6222E\",\n",
    "            \"#1CFFCE\",\n",
    "            \"#2ED9FF\",\n",
    "            \"#B10DA1\",\n",
    "            \"#C075A6\",\n",
    "            \"#FC1CBF\",\n",
    "            \"#B00068\",\n",
    "            \"#FBE426\",\n",
    "            \"#FA0087\",\n",
    "        ]\n",
    "    elif bars_color_scheme == \"default(muted blue)\":\n",
    "        bar_colors = \"#1f77b4\"\n",
    "    elif HEX_COLOR_REGEX.search(bars_color_scheme) or bars_color_scheme in css_colors:\n",
    "        bar_colors = bars_color_scheme\n",
    "    else:\n",
    "        eprint(\"WARNING: specified color is not valid. Defaulting to muted blue\")\n",
    "        bar_colors = \"#1f77b4\"\n",
    "    print_parameters()\n",
    "\n",
    "\n",
    "# QUERY string\n",
    "query = \"UniprotKB,Uniprot,Trembl,SwissProt,Swiss-Prot,Uniparc,Uniref,UniRef100,UniRef90,UniRef50\"\n",
    "query_keywords = [\n",
    "    \"UniprotKB\",\n",
    "    \"Uniprot\",\n",
    "    \"Trembl\",\n",
    "    \"SwissProt\",\n",
    "    \"Swiss-Prot\",\n",
    "    \"Uniparc\",\n",
    "    \"Uniref\",\n",
    "    \"UniRef100\",\n",
    "    \"UniRef90\",\n",
    "    \"UniRef50\",\n",
    "]\n",
    "query_short = \"UniprotKB, Uniprot, ...\"\n",
    "resource_searchstring = '(\"UniprotKB\" OR \"Uniprot\" OR \"Trembl\" OR \"SwissProt\" OR \"Swiss-Prot\" OR \"Uniparc\" OR \"Uniref\" OR \"UniRef100\" OR \"UniRef90\" OR \"UniRef50\")'\n",
    "query_widget = widgets.Dropdown(\n",
    "    options=[\n",
    "        \"UniprotKB,Uniprot,Trembl,SwissProt,Swiss-Prot,Uniparc,Uniref,UniRef100,UniRef90,UniRef50\",\n",
    "        \"Ensembl,EnsemblGenomes,EnsemblPlants,EnsemblBacteria,EnsemblFungi,EnsemblProtists,EnsemblVertebrates\",\n",
    "        \"RefSeq\",\n",
    "        \"OMIM,Online Mendelian Inheritance in Man\",\n",
    "        \"SGD,Saccharomyces Genome Database,Yeastgenome.org,yeast genome database\",\n",
    "        \"Zfin,zfin.org,Zebrafish Information Network\",\n",
    "        \"MGI,MGD,Mouse Genome Informatics,informatics.jax.org\",\n",
    "        \"Rat genome database,rgd.mcw.edu\",\n",
    "        \"FlyBase,flybase.org\",\n",
    "        \"Gene Ontology,geneontology\",\n",
    "        \"Wormbase,wormbase.org\",\n",
    "        \"TAIR,arabidopsis.org,Arabidopsis information resource\",\n",
    "    ],\n",
    "    value=query,\n",
    "    description=\"Query:\",\n",
    "    layout={\"width\": \"30em\"},\n",
    "    disabled=False,\n",
    ")\n",
    "query_widget.observe(query_handler, names=\"value\")\n",
    "\n",
    "# YEAR range\n",
    "current_year = datetime.datetime.now().date().year\n",
    "year_from, year_to = str(current_year - 5), str(current_year)\n",
    "query_time_constraint = \" and (FIRST_PDATE:[{}-01-01 TO {}-12-31])\".format(\n",
    "    current_year - 5, current_year\n",
    ")\n",
    "start_date = datetime.date(1980, 1, 1)\n",
    "end_date = datetime.date(current_year, 12, 31)\n",
    "\n",
    "dates = pd.date_range(start_date, end_date, freq=\"Y\")\n",
    "options = [(date.strftime(\" %Y \"), date) for date in dates]\n",
    "index = (0, len(options) - 1)\n",
    "\n",
    "years_slider = widgets.SelectionRangeSlider(\n",
    "    options=options,\n",
    "    index=index,\n",
    "    description=\"Years range\",\n",
    "    orientation=\"horizontal\",\n",
    "    layout={\"width\": \"30em\"},\n",
    ")\n",
    "years_slider.value = (dates[-6], dates[-1])  # default: last 6 years\n",
    "years_slider.observe(years_handler, names=\"value\")\n",
    "\n",
    "# GRANTERS dropdown\n",
    "granters = \"US funders\"\n",
    "grants = granters_nih + granters_us\n",
    "all_grant_agencies_constraint = ' and (GRANT_AGENCY:\"{}\")'.format(\n",
    "    '\" or GRANT_AGENCY:\"'.join(grants)\n",
    ")\n",
    "resource_searchstring = '(\"UniprotKB\" OR \"Uniprot\" OR \"Trembl\" OR \"SwissProt\" OR \"Swiss-Prot\" OR \"Uniparc\" OR \"Uniref\")'\n",
    "granters_widget = widgets.Dropdown(\n",
    "    options=[\n",
    "        \"US funders\",\n",
    "        \"UK funders\",\n",
    "        \"NIH Institutes\",\n",
    "        \"UK and US incl. NIH\",\n",
    "        \"National Natural Science Foundation of China\",\n",
    "        \"NCI NIH HHS\",\n",
    "        \"NIGMS NIH HHS\",\n",
    "        \"NHLBI NIH HHS\",\n",
    "        \"NIAID NIH HHS\",\n",
    "        \"NIDDK NIH HHS\",\n",
    "        \"NINDS NIH HHS\",\n",
    "        \"National Institutes of Health\",\n",
    "        \"NICHD NIH HHS\",\n",
    "        \"NIMH NIH HHS\",\n",
    "        \"NCRR NIH HHS\",\n",
    "        \"NIA NIH HHS\",\n",
    "        \"Medical Research Council\",\n",
    "        \"Wellcome Trust\",\n",
    "        \"NIDA NIH HHS\",\n",
    "        \"National Science Foundation\",\n",
    "        \"NCATS NIH HHS\",\n",
    "        \"National Institute for Health Research (NIHR)\",\n",
    "        \"NEI NIH HHS\",\n",
    "        \"European Research Council\",\n",
    "        \"Deutsche Forschungsgemeinschaft\",\n",
    "        \"NIEHS NIH HHS\",\n",
    "        \"NIAMS NIH HHS\",\n",
    "        \"PHS HHS\",\n",
    "        \"Japan Society for the Promotion of Science\",\n",
    "        \"Biotechnology and Biological Sciences Research Council\",\n",
    "        \"National Research Foundation of Korea\",\n",
    "        \"Canadian Institutes of Health Research\",\n",
    "        \"Intramural NIH HHS\",\n",
    "        \"NIH HHS\",\n",
    "        \"Engineering and Physical Sciences Research Council\",\n",
    "        \"National Key Research and Development Program of China\",\n",
    "        \"NIAAA NIH HHS\",\n",
    "        \"NIBIB NIH HHS\",\n",
    "        \"Ministry of Education, Culture, Sports, Science and Technology\",\n",
    "        \"NIH\",\n",
    "        \"NIDCR NIH HHS\",\n",
    "        \"NIADDK NIH HHS\",\n",
    "        \"NIDCD NIH HHS\",\n",
    "        \"Swiss National Science Foundation\",\n",
    "        \"Fundamental Research Funds for the Central Universities\",\n",
    "        \"CIHR\",\n",
    "        \"National Cancer Institute\",\n",
    "        \"Natural Sciences and Engineering Research Council of Canada\",\n",
    "        \"Conselho Nacional de Desenvolvimento Científico e Tecnológico\",\n",
    "        \"China Postdoctoral Science Foundation\",\n",
    "        \"Cancer Research UK\",\n",
    "        \"National Institute of General Medical Sciences\",\n",
    "        \"Howard Hughes Medical Institute\",\n",
    "        \"Ministry of Science and Technology of the People&apos;s Republic of China\",\n",
    "        \"Coordenação de Aperfeiçoamento de Pessoal de Nível Superior\",\n",
    "        \"NHGRI NIH HHS\",\n",
    "        \"Australian Research Council\",\n",
    "        \"British Heart Foundation\",\n",
    "        \"National Health and Medical Research Council\",\n",
    "        \"Agence Nationale de la Recherche\",\n",
    "        \"National Institute of Mental Health\",\n",
    "        \"Austrian Science Fund FWF\",\n",
    "        \"U.S. Department of Energy\",\n",
    "        \"Dutch Research Council (NWO)\",\n",
    "        \"AHRQ HHS\",\n",
    "        \"FIC NIH HHS\",\n",
    "        \"National Heart, Lung, and Blood Institute\",\n",
    "        \"European Regional Development Fund\",\n",
    "        \"NIMHD NIH HHS\",\n",
    "        \"Ministry of Science and Technology, Taiwan\",\n",
    "        \"European Commission\",\n",
    "        \"National Institute on Aging\",\n",
    "        \"China Scholarship Council\",\n",
    "        \"Chinese Academy of Sciences\",\n",
    "        \"Natural Environment Research Council\",\n",
    "        \"NINR NIH HHS\",\n",
    "        \"Fundação de Amparo à Pesquisa do Estado de São Paulo\",\n",
    "        \"National Institute of Allergy and Infectious Diseases\",\n",
    "        \"Natural Science Foundation of Jiangsu Province\",\n",
    "        \"Novo Nordisk Fonden\",\n",
    "        \"Economic and Social Research Council\",\n",
    "    ],\n",
    "    value=granters,\n",
    "    description=\"Granters:\",\n",
    "    layout={\"width\": \"30em\"},\n",
    "    disabled=False,\n",
    ")\n",
    "granters_widget.observe(granters_handler, names=\"value\")\n",
    "\n",
    "# COLOR scheme\n",
    "bars_color_scheme = \"default(muted blue)\"\n",
    "bar_colors = \"#1f77b4\"\n",
    "bars_color_scheme_widget = widgets.Dropdown(\n",
    "    options=[\n",
    "        \"default(muted blue)\",\n",
    "        \"black\",\n",
    "        \"multicolor\",\n",
    "        \"darkgreen\",\n",
    "        \"darkgray\",\n",
    "        \"goldenrod\",\n",
    "        \"teal\",\n",
    "        \"slateblue\",\n",
    "        \"blueviolet\",\n",
    "        \"#ff7f0e\",\n",
    "    ],\n",
    "    value=bars_color_scheme,\n",
    "    description=\"Color scheme:\",\n",
    "    layout={\"width\": \"30em\"},\n",
    "    disabled=False,\n",
    ")\n",
    "bars_color_scheme_widget.observe(color_scheme_handler, names=\"value\")\n",
    "\n",
    "initialized_code = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    initialized_code\n",
    "except:\n",
    "    print(\n",
    "        \"ERROR: You need first to execute the CODE cell (the one above this one) in order to use this notebook. After doing that, re-execute this cell.\"\n",
    "    )\n",
    "    # raise SystemExit\n",
    "else:\n",
    "    # DISPLAY\n",
    "    show_form()\n",
    "    print_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 594
    },
    "id": "5nKdS-OCwzHl",
    "outputId": "29163faa-9df6-4b4f-8ff5-2c075ab0a78d",
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# @title **Generate plot of total results** { display-mode: \"form\" }\n",
    "try:\n",
    "    initialized_code\n",
    "except:\n",
    "    print(\n",
    "        \"ERROR: You need first to execute the CODE and PARAMETERS cells (top of the page) in order to generate plots. After doing that, re-execute this cell.\"\n",
    "    )\n",
    "    # raise SystemExit\n",
    "else:\n",
    "    # total result(s)\n",
    "    hits = []\n",
    "    names = []\n",
    "    apilinksfile = \"totals_api.html\"\n",
    "    open(apilinksfile, \"w\").close()  # init file\n",
    "    hits.append(\n",
    "        query2publications(resource_searchstring, \"Total papers\", apilinks=apilinksfile)\n",
    "    )\n",
    "    names.append(\"Total papers\")\n",
    "    if int(year_from) > int(year_to):\n",
    "        eprint(\"ATTENTION: 'year_to' cannot be smaller than 'year_from'!\")\n",
    "    else:\n",
    "        eprint(\"Querying publication and patent databases\")\n",
    "        t = tqdm(total=4, unit=\"query\", ascii=\" ▖▘▝▗▚▞█\")\n",
    "        desc = \"Papers from {} to {}\".format(year_from, year_to)\n",
    "        hits.append(\n",
    "            query2publications(\n",
    "                resource_searchstring + query_time_constraint,\n",
    "                desc,\n",
    "                apilinks=apilinksfile,\n",
    "            )\n",
    "        )\n",
    "        names.append(desc)\n",
    "        t.update(1)\n",
    "\n",
    "        desc = 'Papers acknowledging \"{}\"'.format(granters)\n",
    "        hits.append(\n",
    "            query2publications(\n",
    "                resource_searchstring + all_grant_agencies_constraint,\n",
    "                desc,\n",
    "                apilinks=apilinksfile,\n",
    "            )\n",
    "        )\n",
    "        names.append(desc)\n",
    "        t.update(1)\n",
    "\n",
    "        desc = 'Acknowledging \"{}\", {} to {}'.format(granters, year_from, year_to)\n",
    "        hits.append(\n",
    "            query2publications(\n",
    "                resource_searchstring\n",
    "                + query_time_constraint\n",
    "                + all_grant_agencies_constraint,\n",
    "                desc,\n",
    "                apilinks=apilinksfile,\n",
    "            )\n",
    "        )\n",
    "        names.append(desc)\n",
    "        t.update(1)\n",
    "\n",
    "        desc = \"Patents from {} to {}\".format(year_from, year_to)\n",
    "        patent_hits = query2patents(\n",
    "            resource_searchstring\n",
    "            + \" AND (pd:[{}0101 TO {}1231])\".format(year_from, year_to),\n",
    "            desc,\n",
    "        )\n",
    "        hits.append(patent_hits)\n",
    "        names.append(desc)\n",
    "        t.update(1)\n",
    "\n",
    "        t.close()\n",
    "\n",
    "        if patent_hits == -1:\n",
    "            print(\n",
    "                '\"{}{}WARNING! Timeout when retrieving patent data.\"{}'.format(\n",
    "                    BOLD, RED, END\n",
    "                )\n",
    "            )\n",
    "\n",
    "        df = barplot(\n",
    "            names,\n",
    "            hits,\n",
    "            title=\"{}\".format(\"; \".join(names)),\n",
    "            xaxis=\"Set\",\n",
    "            filename=\"totals\",\n",
    "            resulttype=\"papers/patents\",\n",
    "            sort=False,\n",
    "        )\n",
    "        df.to_csv(\"totals.csv\", encoding=\"utf-8-sig\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oXxg5a6xxTnf"
   },
   "source": [
    "If you wish to download the data from the previous plot (once it is completely generated): [totals.csv](./totals.csv)\n",
    "\n",
    "If you wish to explore further the list of mentions, check the API URL used for fetching publication data: [totals_api.html](./totals_api.html).  \n",
    "Please refer to the [EuropePMC API documentation](https://europepmc.org/RestfulWebService) for the available options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "v1s3nWg5wCmi",
    "outputId": "e5703fe5-0b40-44d7-bf20-a239d3dd3e05",
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# @title **Generate plots by year** (both in total and acknowledging any of the grant agencies specified) { display-mode: \"form\" }\n",
    "try:\n",
    "    initialized_code\n",
    "except:\n",
    "    print(\n",
    "        \"ERROR: You need first to execute the CODE and PARAMETERS cells (top of the page) in order to generate plots. After doing that, re-execute this cell.\"\n",
    "    )\n",
    "    # raise SystemExit\n",
    "else:\n",
    "    # time plot\n",
    "    if resource_searchstring:\n",
    "        if int(year_from) > int(year_to):\n",
    "            eprint(\"ATTENTION: 'year_to' cannot be smaller than 'year_from'!\")\n",
    "        else:\n",
    "            years = list(range(int(year_from), int(year_to) + 1))\n",
    "            hits_by_year = []\n",
    "            hits_by_year_selected_grants = []\n",
    "            eprint(\"Querying publication database for {} years\".format(len(years)))\n",
    "        for year in tqdm(years, unit=\"query\", ascii=\" ▖▘▝▗▚▞█\"):\n",
    "            query_year_constraint = (\n",
    "                \" and (FIRST_PDATE:[{0}-01-01 TO {0}-12-31])\".format(year)\n",
    "            )\n",
    "            hits_by_year.append(\n",
    "                query2publications(\n",
    "                    resource_searchstring + query_year_constraint, \"{}\".format(year)\n",
    "                )\n",
    "            )\n",
    "            hits_by_year_selected_grants.append(\n",
    "                query2publications(\n",
    "                    resource_searchstring\n",
    "                    + query_year_constraint\n",
    "                    + all_grant_agencies_constraint,\n",
    "                    \"{}\".format(year),\n",
    "                )\n",
    "            )\n",
    "        df = barplot(\n",
    "            years,\n",
    "            hits_by_year,\n",
    "            title=\"from {} to {}\".format(year_from, year_to),\n",
    "            xaxis=\"Year\",\n",
    "            filename=\"by_year\",\n",
    "            sort=False,\n",
    "        )\n",
    "        df.to_csv(\"by_year.csv\", encoding=\"utf-8-sig\")\n",
    "        df = barplot(\n",
    "            years,\n",
    "            hits_by_year_selected_grants,\n",
    "            title='acknowledging \"{}\", from {} to {}'.format(\n",
    "                granters, year_from, year_to\n",
    "            ),\n",
    "            xaxis=\"Year\",\n",
    "            filename=\"by_year_grant\",\n",
    "            sort=False,\n",
    "        )\n",
    "        df.to_csv(\"by_year_grant.csv\", encoding=\"utf-8-sig\")\n",
    "    else:\n",
    "        eprint(\"ERROR: query string not specified\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nDiPcaZty_oc"
   },
   "source": [
    "If you wish to download the data from the previous plots (once they are completely generated):\n",
    "[by_year.csv](./by_year.csv)\n",
    "and\n",
    "[by_year_grant.csv](./by_year_grant.csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 577
    },
    "id": "LdLdroqUy8XF",
    "outputId": "48abbf47-e9e1-463e-f9ab-2e777581dd95",
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# @title **Generate plots by grant agency** (in the timeframe specified) { display-mode: \"form\" }\n",
    "try:\n",
    "    initialized_code\n",
    "except:\n",
    "    print(\n",
    "        \"ERROR: You need first to execute the CODE and PARAMETERS cells (top of the page) in order to generate plots. After doing that, re-execute this cell.\"\n",
    "    )\n",
    "    # raise SystemExit\n",
    "else:\n",
    "    # grants plot\n",
    "    if resource_searchstring and len(grants) and int(year_from) <= int(year_to):\n",
    "        # hits_by_grant = []\n",
    "        hits_by_grant_timeframe = []\n",
    "        eprint(\"Querying publication database for {} funders\".format(len(grants)))\n",
    "        for grant in tqdm(grants, unit=\"query\", ascii=\" ▖▘▝▗▚▞█\"):\n",
    "            grant_agency_constraint = ' and (GRANT_AGENCY:\"{}\")'.format(grant)\n",
    "            # hits_by_grant.append(query2publications(resource_searchstring + grant_agency_constraint))\n",
    "            hits_by_grant_timeframe.append(\n",
    "                query2publications(\n",
    "                    resource_searchstring\n",
    "                    + grant_agency_constraint\n",
    "                    + query_time_constraint\n",
    "                )\n",
    "            )\n",
    "        # barplot(grants, hits_by_grant, title='by grant agency', xaxis='Grant agency', sort=True)\n",
    "        df = barplot(\n",
    "            grants,\n",
    "            hits_by_grant_timeframe,\n",
    "            title=\"by grant agency acknowledged, from {} to {}\".format(\n",
    "                year_from, year_to\n",
    "            ),\n",
    "            xaxis=\"Grant agency\",\n",
    "            filename=\"by_grant_agency\",\n",
    "            sort=True,\n",
    "        )\n",
    "        df.to_csv(\"by_grant_agency.csv\", encoding=\"utf-8-sig\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r_hZXKCrzp1W"
   },
   "source": [
    "If you wish to download the data from the previous plot (once it is completely generated): \n",
    "[by_grant_agency.csv](./by_grant_agency.csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 577
    },
    "id": "doOJCpLtU8Td",
    "outputId": "d21553d4-c075-4f81-b8a8-8b8110218ea6",
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# @title **Generate plot by paper section** { display-mode: \"form\" }\n",
    "try:\n",
    "    initialized_code\n",
    "except:\n",
    "    print(\n",
    "        \"ERROR: You need first to execute the CODE and PARAMETERS cells (top of the page) in order to generate plots. After doing that, re-execute this cell.\"\n",
    "    )\n",
    "    # raise SystemExit\n",
    "else:\n",
    "    # paper section plot\n",
    "    # hits_by_section = []\n",
    "    hits_by_section_timeframe = []\n",
    "    eprint(\n",
    "        \"Querying publication database for {} paper sections\".format(\n",
    "            len(paper_sections)\n",
    "        )\n",
    "    )\n",
    "    for section in tqdm(paper_sections, unit=\"query\", ascii=\" ▖▘▝▗▚▞█\"):\n",
    "        section_keywords = []\n",
    "        for keyword in query_keywords:\n",
    "            section_keywords.append('({}:\"{}\")'.format(section, keyword))\n",
    "            section_resource_searchstring = \"({})\".format(\" OR \".join(section_keywords))\n",
    "        # hits_by_section.append(query2publications(section_resource_searchstring, 'section {}'.format(section)))\n",
    "        hits_by_section_timeframe.append(\n",
    "            query2publications(\n",
    "                section_resource_searchstring + query_time_constraint,\n",
    "                \"section {}\".format(section),\n",
    "            )\n",
    "        )\n",
    "    # barplot(paper_sections, hits_by_section, title='by paper section', xaxis='Paper section', sort=True)\n",
    "    df = barplot(\n",
    "        paper_sections,\n",
    "        hits_by_section_timeframe,\n",
    "        title=\"by paper section, from {} to {}\".format(year_from, year_to),\n",
    "        xaxis=\"Paper section\",\n",
    "        filename=\"by_paper_section\",\n",
    "        sort=True,\n",
    "    )\n",
    "    df.to_csv(\"by_paper_section.csv\", encoding=\"utf-8-sig\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z_Qtrr0y0CeA"
   },
   "source": [
    "If you wish to download the data from the previous plot (once it is completely generated): [by_paper_section.csv](./by_paper_section.csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cellView": "form",
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 577
    },
    "id": "k7opYjpckeGn",
    "outputId": "0254669c-3263-40c5-b47b-ced70e32bd53",
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "# @title **Generate plots by year of patent mentions**\n",
    "try:\n",
    "    initialized_code\n",
    "except:\n",
    "    print(\n",
    "        \"ERROR: You need first to execute the CODE and PARAMETERS cells (top of the page) in order to generate plots. After doing that, re-execute this cell.\"\n",
    "    )\n",
    "    # raise SystemExit\n",
    "else:\n",
    "    if resource_searchstring:\n",
    "        if int(year_from) > int(year_to):\n",
    "            eprint(\"ATTENTION: 'year_to' cannot be smaller than 'year_from'!\")\n",
    "        else:\n",
    "            years = list(range(int(year_from), int(year_to) + 1))\n",
    "            patents_by_year = []\n",
    "            eprint(\"Querying patent database for {} years\".format(len(years)))\n",
    "            for year in tqdm(years, unit=\"query\", ascii=\" ▖▘▝▗▚▞█\"):\n",
    "                patent_hits = query2patents(\n",
    "                    resource_searchstring\n",
    "                    + \" AND (pd:[{0}0101 TO {0}1231])\".format(year),\n",
    "                    year,\n",
    "                )\n",
    "                patents_by_year.append(patent_hits)\n",
    "                if patent_hits == -1:\n",
    "                    print(\n",
    "                        '\"{}{}WARNING! Timeout when retrieving patent data for {}.\"{}'.format(\n",
    "                            BOLD, RED, year, END\n",
    "                        )\n",
    "                    )\n",
    "            df = barplot(\n",
    "                years,\n",
    "                patents_by_year,\n",
    "                title=\"from {} to {}\".format(year_from, year_to),\n",
    "                xaxis=\"Year\",\n",
    "                filename=\"patents_by_year\",\n",
    "                resulttype=\"patents\",\n",
    "                sort=False,\n",
    "            )\n",
    "            df.to_csv(\"patents_by_year.csv\", encoding=\"utf-8-sig\")\n",
    "    else:\n",
    "        eprint(\"ERROR: query string not specified\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wGMkyidwlpiW"
   },
   "source": [
    "If you wish to download the data from the previous plot (once it is completely generated): [patents_by_year.csv](./patents_by_year.csv)"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}