data_extraction/notebooks/retrieval_from_pdf_tables.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrieval from pdf tables using OCR table extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Table of contents**<a id='toc0_'></a>    \n",
    "- 1. [Load documents](#toc1_)    \n",
    "  - 1.1. [Unstructured Pytesseract loader](#toc1_1_)    \n",
    "  - 1.2. [Paddle OCR loader](#toc1_2_)    \n",
    "- 2. [Simple Retrieval](#toc2_)    \n",
    "  - 2.1. [Text splitting](#toc2_1_)    \n",
    "  - 2.2. [Embedding and storing](#toc2_2_)    \n",
    "  - 2.3. [Model and retrieval](#toc2_3_)    \n",
    "- 3. [Multivector retrieval](#toc3_)    \n",
    "  - 3.1. [Text splitting](#toc3_1_)    \n",
    "  - 3.2. [Summarization](#toc3_2_)    \n",
    "  - 3.3. [Embedding and storing](#toc3_3_)    \n",
    "  - 3.4. [Model and retrieval](#toc3_4_)    \n",
    "\n",
    "<!-- vscode-jupyter-toc-config\n",
    "\tnumbering=true\n",
    "\tanchor=true\n",
    "\tflat=false\n",
    "\tminLevel=2\n",
    "\tmaxLevel=6\n",
    "\t/vscode-jupyter-toc-config -->\n",
    "<!-- THIS CELL WILL BE REPLACED ON TOC UPDATE. DO NOT WRITE YOUR TEXT IN THIS CELL -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook includes an example of a simple retierver, and an example of a  multivector retriever using extracted data from pdf tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tqdm as notebook_tqdm\n",
    "import os\n",
    "import sys\n",
    "\n",
    "current_dir = os.getcwd()\n",
    "kit_dir = os.path.abspath(os.path.join(current_dir, '..'))\n",
    "repo_dir = os.path.abspath(os.path.join(kit_dir, '..'))\n",
    "\n",
    "sys.path.append(kit_dir)\n",
    "sys.path.append(repo_dir)\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from utils.model_wrappers.langchain_llms import SambaNovaCloud, SambaStudio\n",
    "from langchain.embeddings import HuggingFaceInstructEmbeddings\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.vectorstores import FAISS\n",
    "from langchain.chains import RetrievalQA\n",
    "from langchain.prompts import PromptTemplate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "\n",
    "# Provide location of pdf files\n",
    "folder_loc = os.path.join(kit_dir, 'data/sample_data/sample_pdfs')\n",
    "pdf_files = [f for f in glob.glob(f'{folder_loc}/*.pdf')]\n",
    "sample_pdf = pdf_files[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. <a id='toc1_'></a>[Load documents](#toc0_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1. <a id='toc1_1_'></a>[Unstructured Pytesseract loader](#toc0_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For runing this loader you should install the pyteseract and poppler-utils packages in your machine, or run this notebook over the data_extarction docker container\n",
    "\n",
    "This loader uses behind the scenes Unstructured and pytesseract module to perform a layout detection, then transcribe text, and tables as Html tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/luiss/.pyenv/versions/3.11.3/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "Some weights of the model checkpoint at microsoft/table-transformer-structure-recognition were not used when initializing TableTransformerForObjectDetection: ['model.backbone.conv_encoder.model.layer2.0.downsample.1.num_batches_tracked', 'model.backbone.conv_encoder.model.layer3.0.downsample.1.num_batches_tracked', 'model.backbone.conv_encoder.model.layer4.0.downsample.1.num_batches_tracked']\n",
      "- This IS expected if you are initializing TableTransformerForObjectDetection from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing TableTransformerForObjectDetection from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "commercial and public sector industries and is as disruptive decades ago. And like the internet—Al promises decisive to organizations that can leverage it for innovation sooner\n",
      "\n",
      "\n",
      "\n",
      "Accelerate Data-Driven Decision-Making with State-of-the-Art AI\n",
      "\n",
      "AI is increasingly being adopted across commercial and public sector industries and is as disruptive today as the advent of the internet a few decades ago. And like the internet—AI promises decisive competitive and operational advantages to organizations that can leverage it for innovation sooner rather than later.\n",
      "\n",
      "ROADBLOCKS TO INNOVATION\n",
      "\n",
      "Today, all but the big tech giants face seemingly insurmountable obstacles to bring AI applications into production. These challenges include:\n",
      "\n",
      "Critical skill gaps due to machine learning talent scarcity • Lack of expertise in computing architectures • Difficulty in keeping on top of latest models and techniques • Investment justiﬁcation without proof of prior impact\n",
      "\n",
      "UNLOCKING THE FUTURE\n",
      "\n",
      "The AI talent shortage continues to burden companies across all industries. And the reality is, unless you’re a company like Google or Facebook, you’ll be hard-pressed to attract top talent, resulting in a subpar AI solution. It’s time you demand a solution to this challenge and it’s time that companies meet this need.\n",
      "\n",
      "HARNESS THE POWER OF NEXT-GENERATION AI SambaNova enables organizations to build and deploy AI solutions for natural language processing, high-resolution computer vision, and recommendation. With state-of-the-art accuracy, unmatched scalability, and ease of use, SambaNova delivers AI capabilities at a fraction of the time and expense it takes to develop complex in-house infrastructure and machine learning expertise.\n",
      "\n",
      "Affordable and attainable, SambaNova’s AI platform can be delivered as a CapEx-modeled rack Infrastructure solution or an OpEx modeled subscription service that is fully managed by SambaNova and comes completely conﬁgured with pretrained models that can save signiﬁcant time for customers.\n",
      "\n",
      "4? po fon oO e E\n",
      "\n",
      "OUR PRODUCTS Dataﬂow-as-a-ServiceTM Dataﬂow-as-a-ServiceTM is the industry’s ﬁrst subscription-based AI acceleration services platform that enables enterprises to jump-start AI initiatives by leveraging SambaNova’s expertise to augment their existing ML capabilities and staffing. Dataﬂow-as-a-service provides comprehensive API services and models for businesses to deploy fully managed AI solutions within weeks as opposed to the typical 12-18 months.\n",
      "\n",
      "Computer Vision\n",
      "\n",
      "I\n",
      "\n",
      "Natural Language Processing\n",
      "\n",
      "Recommendation Systems\n",
      "\n",
      "DataScale®\n",
      "\n",
      "ONE PLATFORM\n",
      "\n",
      "DataScale® Built on SambaNova Systems’ Reconﬁgurable Dataﬂow Architecture, DataScale® is an integrated AI software and hardware accelerator that delivers unrivaled performance, accuracy, scale, and ease of use. Optimized from algorithms to silicon, DataScale delivers efficiency with a software-deﬁned-hardware approach and a highly ﬂexible modular architecture. DataScale can scale seamlessly from one to hundreds of systems to meet the demands of modern AI computing.\n",
      "\n",
      "YOUR CHOICE OF DEPLOYMENT\n",
      "\n",
      "<table><tbody><tr><td>DATAFLOW-AS-A-SERVICE™</td><td>DATASCALE®</td></tr><tr><td>SambaNova managed</td><td>Customer managed</td></tr><tr><td>Optimized pretrained models</td><td>Build your own models</td></tr><tr><td>Ability to fine-tune with your custom data</td><td>Your data</td></tr><tr><td>Opex</td><td>Opex + Capex</td></tr></tbody></table>\n",
      "\n",
      "Copyright © 2021 SambaNova Systems, Inc. All rights reserved.\n",
      "\n",
      "01\n",
      "\n",
      "eau 10 (Oro)\n",
      "\n",
      "AX\n",
      "\n",
      "DataScales\n",
      "\n",
      "Al Oil\n",
      "\n",
      "AI Use Cases Across Industries\n",
      "\n",
      "\n",
      "\n",
      "ue ee\n",
      "\n",
      "ete (S)\n",
      "\n",
      "Manufacturing & Automotive Digital transformation, predictive maintenance, visual inspection\n",
      "\n",
      "Healthcare & Life Sciences Genomics, drug discovery, telehealth, disease diagnosis\n",
      "\n",
      "Financial Services Compliance, risk mitigation, algorithmic trading\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "ANN “TDL:\n",
      "\n",
      "Oil & Gas, Energy Predictive asset maintenance, seismic processing, exploration\n",
      "\n",
      "Public Sector Intelligent decision-making, ISR, predictive analytics\n",
      "\n",
      "Retail & E-Commerce Digital acceleration, personalized services delivery\n",
      "\n",
      "DON’T JUST TAKE OUR WORD FOR IT\n",
      "\n",
      "”We’re interested in technology that can train models faster than individual GPUs, that has more scalability and has more performance and so the SambaNova DataScale System met that criteria.”\n",
      "\n",
      "RICK STEVENS ASSOCIATE LABORATORY DIRECTOR ARGONNE NATIONAL LABORATORY\n",
      "\n",
      "”In this partnership with SambaNova, we’re providing businesses with never-before-seen access to artiﬁcial intelligence that will certainly lead to greater adoption of AI in Asia. Together, we are enabling ScaleWorX end-users access to disruptive and innovative solutions that can quickly generate new revenue opportunities with AI.”\n",
      "\n",
      "JUN MATSUURA MANAGING DIRECTOR\n",
      "\n",
      "”The support for running multiple models built into the hardware was a signiﬁcant aspect of our selection of SambaNova Systems. We’re seeing roughly a 5x improvement compared to a comparable GPU running the same models.”\n",
      "\n",
      "BRONIS DE SUPINSKI, CHIEF TECHNOLOGY OFFICER LAWRENCE LIVERMORE NATIONAL LABORATORY\n",
      "\n",
      "To learn how your organization can innovate with next-generation AI, please visit us at SambaNova.AI.\n",
      "\n",
      "in ¥ f &\n",
      "\n",
      "G)SambaNo\n",
      "\n",
      "SambaNova Systems is an AI innovation company that empowers organizations to deploy best-in-class solutions for computer vision, natural language processing, recommendation systems, and AI for science with conﬁdence. SambaNova’s ﬂagship offering, Dataﬂow-as-a-Service, helps organizations rapidly deploy AI in days, unlocking new revenue and boosting operational efficiency. SambaNova’s DataScale® is an integrated software and hardware system using Reconﬁgurable Dataﬂow Architecture (RDA), along with open standards and user interfaces. Headquartered in Palo Alto, California, SambaNova Systems was founded in 2017 by industry luminaries, hardware, and software design experts from Sun/Oracle and Stanford University. Investors include SoftBank Vision Fund 2, funds and accounts managed by BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and several others.\n",
      "\n",
      "Copyright © 2021 SambaNova Systems, Inc. All rights reserved. | Palo Alto, CA and Austin, TX | (650) 263-1153\n",
      "\n",
      "02\n",
      "---\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from data_extraction.src.pdf_table_text_extraction import UnstructuredPdfPytesseractLoader\n",
    "\n",
    "loader = UnstructuredPdfPytesseractLoader(sample_pdf)\n",
    "docs = loader.load()\n",
    "\n",
    "for doc in docs:\n",
    "    print(f'{doc.page_content}\\n---\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. <a id='toc1_2_'></a>[Paddle OCR loader](#toc0_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For runing this loader you should run this notebook over the paddle-ocr environment the data_extarction_paddel docker container\n",
    "\n",
    "This loader uses behind the scenes Paddle OCR and Paddle Structure modules to perform a layout detection, then mask images and equations, transcribe text, and tables as Html tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2024/02/06 15:47:02] ppocr DEBUG: Namespace(help='==SUPPRESS==', use_gpu=False, use_xpu=False, ir_optim=True, use_tensorrt=False, min_subgraph_size=15, shape_info_filename=None, precision='fp32', gpu_mem=500, image_dir=None, det_algorithm='DB', det_model_dir='/Users/jorgep/.paddleocr/whl/det/en/en_PP-OCRv3_det_infer', det_limit_side_len=960, det_limit_type='max', det_db_thresh=0.3, det_db_box_thresh=0.6, det_db_unclip_ratio=1.5, max_batch_size=10, use_dilation=False, det_db_score_mode='fast', det_east_score_thresh=0.8, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_sast_score_thresh=0.5, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_pse_thresh=0, det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_box_type='quad', det_pse_scale=1, scales=[8, 16, 32], alpha=1.0, beta=1.0, fourier_degree=5, det_fce_box_type='poly', rec_algorithm='SVTR_LCNet', rec_model_dir='/Users/jorgep/.paddleocr/whl/rec/en/en_PP-OCRv3_rec_infer', rec_image_shape='3, 48, 320', rec_batch_num=6, max_text_length=25, rec_char_dict_path='/Users/jorgep/Documents/ask_public_own/paddle_env/lib/python3.10/site-packages/paddleocr/ppocr/utils/en_dict.txt', use_space_char=True, vis_font_path='./doc/fonts/simfang.ttf', drop_score=0.5, e2e_algorithm='PGNet', e2e_model_dir=None, e2e_limit_side_len=768, e2e_limit_type='max', e2e_pgnet_score_thresh=0.5, e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_pgnet_valid_set='totaltext', e2e_pgnet_mode='fast', use_angle_cls=False, cls_model_dir='/Users/jorgep/.paddleocr/whl/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_image_shape='3, 48, 192', label_list=['0', '180'], cls_batch_num=6, cls_thresh=0.9, enable_mkldnn=False, cpu_threads=10, use_pdserving=False, warmup=False, sr_model_dir=None, sr_image_shape='3, 32, 128', sr_batch_num=1, draw_img_save_dir='./inference_results', save_crop_res=False, crop_res_save_dir='./output', use_mp=False, total_process_num=1, process_id=0, benchmark=False, save_log_path='./log_output/', show_log=True, use_onnx=False, output='./output', table_max_len=488, table_algorithm='TableAttn', table_model_dir=None, merge_no_span_structure=True, table_char_dict_path=None, layout_model_dir=None, layout_dict_path=None, layout_score_threshold=0.5, layout_nms_threshold=0.5, kie_algorithm='LayoutXLM', ser_model_dir=None, ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ocr_order_method=None, mode='structure', image_orientation=False, layout=True, table=True, ocr=True, recovery=False, save_pdf=False, lang='en', det=True, rec=True, type='ocr', ocr_version='PP-OCRv3', structure_version='PP-Structurev2')\n",
      "True\n",
      "True\n",
      "True\n",
      "**figure l**\n",
      "Al is increasingly being adopted across commercial and public sector industries and is as disruptive\n",
      "today as the advent of the internet a few decades ago.And like the internet-Al promises decisive\n",
      "competitive and operational advantages to organizations that can leverage it for innovation sooner\n",
      "rather than later.\n",
      "Today,all but the big tech giants face seemingly insurmountable obstacles to bring Al applications into production.These\n",
      "challenges include:\n",
      "Critical skill gaps due to machine learning talent scarcity\n",
      "Lack of expertise in computing architectures\n",
      "Difficulty in keeping on top of latest models and technigues\n",
      "Investment justification without proof of prior impact\n",
      "UNLOCKING THE FUTURE\n",
      "The Al talent shortage continues to burden companies across all industries. And the reality is, unless you're a company lik\n",
      "Google or Facebook,you'll be hard-pressed to attract top talent,resulting in a subpar Al solution.It's time you demand\n",
      "solution to this challenge and it's time that companies meet this need.\n",
      "HARNESS THE POWER OF NEXT-GENERATION AI\n",
      "SambaNova enables organizations to build and deploy Al solutions for natural language processing,high-resolution\n",
      "computer vision,and recommendation.With state-of-the-art accuracy,unmatched scalability,and ease of use\n",
      "SambaNova delivers Al capabilities at a fraction of the time and expense it takes to develop complex in-house\n",
      "infrastructure and machine learning expertise\n",
      "Affordable and attainable, SambaNova's Al platform can be delivered as a CapEx-modeled rack Infrastructure solution\n",
      "or an OpEx modeled subscription service that is fully managed by SambaNova and comes completely configured with\n",
      "pretrained models that can save significant time for customers.\n",
      "OUR PRODUCTS\n",
      "Dataflow-as-a-ServiceTM\n",
      "Dataflow-as-a-ServiceTM is the industry's first\n",
      "subscripfion-basedAlacceleration services\n",
      "platform that enables enterprises to jump-start Al\n",
      "initiatives by leveraging SambaNova's expertise to\n",
      "augment their existing ML capabilities and staffing\n",
      "Dataflow-as-a-service provides comprehensive\n",
      "API services and models for businesses to deploy\n",
      "fully managed Al solutions within weeks as\n",
      "opposed to the typical 12-18 months.\n",
      "DataScale\n",
      "../data/extraction/SambaNova_Accelerate-Data-Driven-Decision-Making/SambaNova_Accelerate-Data-Driven-Decision-Making_page_0_figure_0.jpg\n",
      "ght 2021 SambaNova Systems, Inc.All rights reserved.\n",
      "ONE PLATFORM\n",
      "01\n",
      "---\n",
      "\n",
      "Al Use Cases\n",
      "Across Industries\n",
      "../data/extraction/SambaNova_Accelerate-Data-Driven-Decision-Making/SambaNova_Accelerate-Data-Driven-Decision-Making_page_1_figure_0.jpg\n",
      "../data/extraction/SambaNova_Accelerate-Data-Driven-Decision-Making/SambaNova_Accelerate-Data-Driven-Decision-Making_page_1_figure_2.jpg\n",
      "DON'T JUSTTAKEOURWORD FORIT\n",
      "'re interested in technology that can train models faster than individual Gp\n",
      "re scalability and has more performance and so the SambaNova DataSco\n",
      "t that criteria.\"\n",
      "In this partnership with sambaNova,we're providing businesses with never-before-seen\n",
      "access to artificial intelligence thatwill certainlylead to greaferadoption ofAl in Asia\n",
      "Together, we are enabling Scaleworx end-users access to disruptive and innovative\n",
      "solutions that can quickly generate new revenue opportunities with Al.'\n",
      "JUN MATSUURA\n",
      "MANAGING DIRECTOR\n",
      "The support for running multiple models built into the hardware was a significant aspect of\n",
      "our selection of SambaNova Systems.We're seeing roughly a 5x improvement compared\n",
      "to a comparable GPUrunning the same models.\n",
      "BRONIS DE SUPINSKI,\n",
      "CHIEF TECHNOLOGY OFFICER\n",
      "LAWRENCE LIVERMORE NATIONAL LABORATORY\n",
      "earn how your organization can innovate with next-generation Al, please vi\n",
      "nbaNova.Al\n",
      "iny\n",
      "SSambaNova\n",
      "../data/extraction/SambaNova_Accelerate-Data-Driven-Decision-Making/SambaNova_Accelerate-Data-Driven-Decision-Making_page_1_figure_1.jpg\n",
      "RICK STEVENS\n",
      "ASSOCIATE LABORATORY DIRECTOR\n",
      "ARGONNE NATIONALLABORATORY\n",
      "---\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from data_extraction.src.multi_column_ocr import PaddleOCRLoader\n",
    "\n",
    "loader = PaddleOCRLoader(sample_pdf, output_folder=os.path.join(kit_dir, 'data/extraction'))\n",
    "docs = loader.load()\n",
    "\n",
    "for doc in docs:\n",
    "    print(f'{doc.page_content}\\n---\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. <a id='toc2_'></a>[Simple Retrieval](#toc0_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an example of a simple RAG pipeline for QA over the document data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Documents are split into smaller chunks and their embeddings are stored in a vector database\n",
    "\n",
    "then, for each user question the **k** most related chunks are retrieved and used as a context for QA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1. <a id='toc2_1_'></a>[Text splitting](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "commercial and public sector industries and is as disruptive decades ago. And like the internet—Al promises decisive to organizations that can leverage it for innovation sooner\n",
      "\n",
      "\n",
      "\n",
      "Accelerate Data-Driven Decision-Making with State-of-the-Art AI\n",
      "\n",
      "AI is increasingly being adopted across commercial and public sector industries and is as disruptive today as the advent of the internet a few decades ago. And like the internet—AI promises decisive competitive and operational advantages to organizations that can leverage it for innovation sooner rather than later.\n",
      "\n",
      "ROADBLOCKS TO INNOVATION\n",
      "\n",
      "Today, all but the big tech giants face seemingly insurmountable obstacles to bring AI applications into production. These challenges include:\n",
      "\n",
      "Critical skill gaps due to machine learning talent scarcity • Lack of expertise in computing architectures • Difficulty in keeping on top of latest models and techniques • Investment justiﬁcation without proof of prior impact\n",
      "\n",
      "UNLOCKING THE FUTURE\n",
      "---\n",
      "\n",
      "UNLOCKING THE FUTURE\n",
      "\n",
      "The AI talent shortage continues to burden companies across all industries. And the reality is, unless you’re a company like Google or Facebook, you’ll be hard-pressed to attract top talent, resulting in a subpar AI solution. It’s time you demand a solution to this challenge and it’s time that companies meet this need.\n",
      "\n",
      "HARNESS THE POWER OF NEXT-GENERATION AI SambaNova enables organizations to build and deploy AI solutions for natural language processing, high-resolution computer vision, and recommendation. With state-of-the-art accuracy, unmatched scalability, and ease of use, SambaNova delivers AI capabilities at a fraction of the time and expense it takes to develop complex in-house infrastructure and machine learning expertise.\n",
      "---\n",
      "\n",
      "Affordable and attainable, SambaNova’s AI platform can be delivered as a CapEx-modeled rack Infrastructure solution or an OpEx modeled subscription service that is fully managed by SambaNova and comes completely conﬁgured with pretrained models that can save signiﬁcant time for customers.\n",
      "\n",
      "4? po fon oO e E\n",
      "\n",
      "OUR PRODUCTS Dataﬂow-as-a-ServiceTM Dataﬂow-as-a-ServiceTM is the industry’s ﬁrst subscription-based AI acceleration services platform that enables enterprises to jump-start AI initiatives by leveraging SambaNova’s expertise to augment their existing ML capabilities and staffing. Dataﬂow-as-a-service provides comprehensive API services and models for businesses to deploy fully managed AI solutions within weeks as opposed to the typical 12-18 months.\n",
      "\n",
      "Computer Vision\n",
      "\n",
      "I\n",
      "\n",
      "Natural Language Processing\n",
      "\n",
      "Recommendation Systems\n",
      "\n",
      "DataScale®\n",
      "\n",
      "ONE PLATFORM\n",
      "---\n",
      "\n",
      "Computer Vision\n",
      "\n",
      "I\n",
      "\n",
      "Natural Language Processing\n",
      "\n",
      "Recommendation Systems\n",
      "\n",
      "DataScale®\n",
      "\n",
      "ONE PLATFORM\n",
      "\n",
      "DataScale® Built on SambaNova Systems’ Reconﬁgurable Dataﬂow Architecture, DataScale® is an integrated AI software and hardware accelerator that delivers unrivaled performance, accuracy, scale, and ease of use. Optimized from algorithms to silicon, DataScale delivers efficiency with a software-deﬁned-hardware approach and a highly ﬂexible modular architecture. DataScale can scale seamlessly from one to hundreds of systems to meet the demands of modern AI computing.\n",
      "\n",
      "YOUR CHOICE OF DEPLOYMENT\n",
      "\n",
      "<table><tbody><tr><td>DATAFLOW-AS-A-SERVICE™</td><td>DATASCALE®</td></tr><tr><td>SambaNova managed</td><td>Customer managed</td></tr><tr><td>Optimized pretrained models</td><td>Build your own models</td></tr><tr><td>Ability to fine-tune with your custom data</td><td>Your data</td></tr><tr><td>Opex</td><td>Opex + Capex</td></tr></tbody></table>\n",
      "---\n",
      "\n",
      "Copyright © 2021 SambaNova Systems, Inc. All rights reserved.\n",
      "\n",
      "01\n",
      "\n",
      "eau 10 (Oro)\n",
      "\n",
      "AX\n",
      "\n",
      "DataScales\n",
      "\n",
      "Al Oil\n",
      "\n",
      "AI Use Cases Across Industries\n",
      "\n",
      "\n",
      "\n",
      "ue ee\n",
      "\n",
      "ete (S)\n",
      "\n",
      "Manufacturing & Automotive Digital transformation, predictive maintenance, visual inspection\n",
      "\n",
      "Healthcare & Life Sciences Genomics, drug discovery, telehealth, disease diagnosis\n",
      "\n",
      "Financial Services Compliance, risk mitigation, algorithmic trading\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "ANN “TDL:\n",
      "\n",
      "Oil & Gas, Energy Predictive asset maintenance, seismic processing, exploration\n",
      "\n",
      "Public Sector Intelligent decision-making, ISR, predictive analytics\n",
      "\n",
      "Retail & E-Commerce Digital acceleration, personalized services delivery\n",
      "\n",
      "DON’T JUST TAKE OUR WORD FOR IT\n",
      "\n",
      "”We’re interested in technology that can train models faster than individual GPUs, that has more scalability and has more performance and so the SambaNova DataScale System met that criteria.”\n",
      "\n",
      "RICK STEVENS ASSOCIATE LABORATORY DIRECTOR ARGONNE NATIONAL LABORATORY\n",
      "---\n",
      "\n",
      "RICK STEVENS ASSOCIATE LABORATORY DIRECTOR ARGONNE NATIONAL LABORATORY\n",
      "\n",
      "”In this partnership with SambaNova, we’re providing businesses with never-before-seen access to artiﬁcial intelligence that will certainly lead to greater adoption of AI in Asia. Together, we are enabling ScaleWorX end-users access to disruptive and innovative solutions that can quickly generate new revenue opportunities with AI.”\n",
      "\n",
      "JUN MATSUURA MANAGING DIRECTOR\n",
      "\n",
      "”The support for running multiple models built into the hardware was a signiﬁcant aspect of our selection of SambaNova Systems. We’re seeing roughly a 5x improvement compared to a comparable GPU running the same models.”\n",
      "\n",
      "BRONIS DE SUPINSKI, CHIEF TECHNOLOGY OFFICER LAWRENCE LIVERMORE NATIONAL LABORATORY\n",
      "\n",
      "To learn how your organization can innovate with next-generation AI, please visit us at SambaNova.AI.\n",
      "\n",
      "in ¥ f &\n",
      "\n",
      "G)SambaNo\n",
      "---\n",
      "\n",
      "in ¥ f &\n",
      "\n",
      "G)SambaNo\n",
      "\n",
      "SambaNova Systems is an AI innovation company that empowers organizations to deploy best-in-class solutions for computer vision, natural language processing, recommendation systems, and AI for science with conﬁdence. SambaNova’s ﬂagship offering, Dataﬂow-as-a-Service, helps organizations rapidly deploy AI in days, unlocking new revenue and boosting operational efficiency. SambaNova’s DataScale® is an integrated software and hardware system using Reconﬁgurable Dataﬂow Architecture (RDA), along with open standards and user interfaces. Headquartered in Palo Alto, California, SambaNova Systems was founded in 2017 by industry luminaries, hardware, and software design experts from Sun/Oracle and Stanford University. Investors include SoftBank Vision Fund 2, funds and accounts managed by BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and several others.\n",
      "---\n",
      "\n",
      "Copyright © 2021 SambaNova Systems, Inc. All rights reserved. | Palo Alto, CA and Austin, TX | (650) 263-1153\n",
      "\n",
      "02\n",
      "---\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Define a recursive character text splitter and use en of table </table> for splitting\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=1000, chunk_overlap=200, length_function=len, separators=['\\n\\n', '</table>', '\\n']\n",
    ")\n",
    "chunks = text_splitter.split_documents(docs)\n",
    "\n",
    "for chunk in chunks:\n",
    "    print(f'{chunk.page_content}\\n---\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2. <a id='toc2_2_'></a>[Embedding and storing](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load INSTRUCTOR_Transformer\n",
      "max_seq_length  512\n"
     ]
    }
   ],
   "source": [
    "# Define HuggingFace embeding model\n",
    "encode_kwargs = {'normalize_embeddings': True}\n",
    "embeddings = HuggingFaceInstructEmbeddings(\n",
    "    model_name='BAAI/bge-large-en',\n",
    "    query_instruction='Represent this sentence for searching relevant passages: ',\n",
    "    encode_kwargs=encode_kwargs,\n",
    ")\n",
    "\n",
    "# Store the embeddings on an in-memory simple vector database\n",
    "vectorstore = FAISS.from_documents(documents=chunks, embedding=embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3. <a id='toc2_3_'></a>[Model and retrieval](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define SambaNova llm model\n",
    "load_dotenv(os.path.join(repo_dir, '.env'))\n",
    "\n",
    "llm = SambaNovaCloud(model='llama3-405b', max_tokens=1024)\n",
    "\n",
    "# Select expert should be removed if not using Bundle\n",
    "\n",
    "# llm = SambaStudio(\n",
    "#         model_kwargs={\n",
    "#             'do_sample': False,\n",
    "#             'max_tokens': 500,\n",
    "#             'temperature': 0.0,\n",
    "#             'model': 'llama-2-70b-chat-hf',\n",
    "#     },\n",
    "# )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define retriever\n",
    "retriever = vectorstore.as_retriever(\n",
    "    search_type='similarity_score_threshold',\n",
    "    search_kwargs={'score_threshold': 0.5, 'k': 4},\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define retrieval chain\n",
    "simple_retrieval_chain = RetrievalQA.from_llm(\n",
    "    llm=llm,\n",
    "    retriever=retriever,\n",
    "    return_source_documents=True,\n",
    "    input_key='question',\n",
    "    output_key='answer',\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "custom_prompt_template = \"\"\"<s>[INST] <<SYS>>\\nUse the following pieces of context to answer the question at the end. \n",
    "Be careful when reading tables in html format do not mix information between rows or columns,\n",
    "If the answer is not in context for answering, say that you don't know, don't try to make up an answer or provide an answer not extracted from provided context. \n",
    "Cross check if the answer is contained in provided context. If not than say \"I do not have information regarding this.\"\n",
    "\n",
    "Context\n",
    "{context}\n",
    "End of context\n",
    "<</SYS>>\n",
    "\n",
    "Question: {question}\n",
    "Helpful Answer: [/INST]\"\"\"\n",
    "\n",
    "CUSTOMPROMPT = PromptTemplate(template=custom_prompt_template, input_variables=['context', 'question'])\n",
    "\n",
    "## Inject custom prompt\n",
    "simple_retrieval_chain.combine_documents_chain.llm_chain.prompt = CUSTOMPROMPT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/luiss/.pyenv/versions/3.11.3/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The method `Chain.__call__` was deprecated in langchain 0.1.0 and will be removed in 0.3.0. Use invoke instead.\n",
      "  warn_deprecated(\n"
     ]
    }
   ],
   "source": [
    "user_question = (\n",
    "    \"What is the main advantage of using SambaNova's DataScale platform compared to traditional AI computing solutions?\"\n",
    ")\n",
    "response = simple_retrieval_chain({'question': user_question})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Response =According to the provided context, the main advantage of using SambaNova's DataScale platform is that it delivers \"unrivaled performance, accuracy, scale, and ease of use\" compared to traditional AI computing solutions. Additionally, it is mentioned that DataScale can \"scale seamlessly from one to hundreds of systems to meet the demands of modern AI computing\", which implies that it offers greater scalability than traditional solutions. \n",
      "\n",
      "It is also mentioned that SambaNova's DataScale platform can train models \"faster than individual GPUs, that has more scalability and has more performance\" according to Rick Stevens, Associate Laboratory Director at Argonne National Laboratory. \n",
      "\n",
      "Therefore, the main advantage of using SambaNova's DataScale platform is its superior performance, accuracy, scalability, and ease of use compared to traditional AI computing solutions.\n"
     ]
    }
   ],
   "source": [
    "print(f'Response ={response[\"answer\"]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. <a id='toc3_'></a>[Multivector retrieval](#toc0_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an exmaple of a multivector RAG pipeline for QA over the document data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Documents are split into smaller chunks. chunks are summarized by an llm,\n",
    "\n",
    "then, the summaries embeddings and the original chunks are stored in a vector database. \n",
    "\n",
    "later on, for each user question the most related summaries idicate the original chunks to be retrieved and used as a context for QA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.documents import Document\n",
    "from langchain.retrievers.multi_vector import MultiVectorRetriever\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_community.vectorstores import Chroma\n",
    "from langchain.storage import InMemoryByteStore\n",
    "import uuid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1. <a id='toc3_1_'></a>[Text splitting](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "commercial and public sector industries and is as disruptive decades ago. And like the internet—Al promises decisive to organizations that can leverage it for innovation sooner\n",
      "\n",
      "\n",
      "\n",
      "Accelerate Data-Driven Decision-Making with State-of-the-Art AI\n",
      "\n",
      "AI is increasingly being adopted across commercial and public sector industries and is as disruptive today as the advent of the internet a few decades ago. And like the internet—AI promises decisive competitive and operational advantages to organizations that can leverage it for innovation sooner rather than later.\n",
      "\n",
      "ROADBLOCKS TO INNOVATION\n",
      "\n",
      "Today, all but the big tech giants face seemingly insurmountable obstacles to bring AI applications into production. These challenges include:\n",
      "\n",
      "Critical skill gaps due to machine learning talent scarcity • Lack of expertise in computing architectures • Difficulty in keeping on top of latest models and techniques • Investment justiﬁcation without proof of prior impact\n",
      "\n",
      "UNLOCKING THE FUTURE\n",
      "---\n",
      "\n",
      "UNLOCKING THE FUTURE\n",
      "\n",
      "The AI talent shortage continues to burden companies across all industries. And the reality is, unless you’re a company like Google or Facebook, you’ll be hard-pressed to attract top talent, resulting in a subpar AI solution. It’s time you demand a solution to this challenge and it’s time that companies meet this need.\n",
      "\n",
      "HARNESS THE POWER OF NEXT-GENERATION AI SambaNova enables organizations to build and deploy AI solutions for natural language processing, high-resolution computer vision, and recommendation. With state-of-the-art accuracy, unmatched scalability, and ease of use, SambaNova delivers AI capabilities at a fraction of the time and expense it takes to develop complex in-house infrastructure and machine learning expertise.\n",
      "---\n",
      "\n",
      "Affordable and attainable, SambaNova’s AI platform can be delivered as a CapEx-modeled rack Infrastructure solution or an OpEx modeled subscription service that is fully managed by SambaNova and comes completely conﬁgured with pretrained models that can save signiﬁcant time for customers.\n",
      "\n",
      "4? po fon oO e E\n",
      "\n",
      "OUR PRODUCTS Dataﬂow-as-a-ServiceTM Dataﬂow-as-a-ServiceTM is the industry’s ﬁrst subscription-based AI acceleration services platform that enables enterprises to jump-start AI initiatives by leveraging SambaNova’s expertise to augment their existing ML capabilities and staffing. Dataﬂow-as-a-service provides comprehensive API services and models for businesses to deploy fully managed AI solutions within weeks as opposed to the typical 12-18 months.\n",
      "\n",
      "Computer Vision\n",
      "\n",
      "I\n",
      "\n",
      "Natural Language Processing\n",
      "\n",
      "Recommendation Systems\n",
      "\n",
      "DataScale®\n",
      "\n",
      "ONE PLATFORM\n",
      "---\n",
      "\n",
      "Computer Vision\n",
      "\n",
      "I\n",
      "\n",
      "Natural Language Processing\n",
      "\n",
      "Recommendation Systems\n",
      "\n",
      "DataScale®\n",
      "\n",
      "ONE PLATFORM\n",
      "\n",
      "DataScale® Built on SambaNova Systems’ Reconﬁgurable Dataﬂow Architecture, DataScale® is an integrated AI software and hardware accelerator that delivers unrivaled performance, accuracy, scale, and ease of use. Optimized from algorithms to silicon, DataScale delivers efficiency with a software-deﬁned-hardware approach and a highly ﬂexible modular architecture. DataScale can scale seamlessly from one to hundreds of systems to meet the demands of modern AI computing.\n",
      "\n",
      "YOUR CHOICE OF DEPLOYMENT\n",
      "\n",
      "<table><tbody><tr><td>DATAFLOW-AS-A-SERVICE™</td><td>DATASCALE®</td></tr><tr><td>SambaNova managed</td><td>Customer managed</td></tr><tr><td>Optimized pretrained models</td><td>Build your own models</td></tr><tr><td>Ability to fine-tune with your custom data</td><td>Your data</td></tr><tr><td>Opex</td><td>Opex + Capex</td></tr></tbody></table>\n",
      "---\n",
      "\n",
      "Copyright © 2021 SambaNova Systems, Inc. All rights reserved.\n",
      "\n",
      "01\n",
      "\n",
      "eau 10 (Oro)\n",
      "\n",
      "AX\n",
      "\n",
      "DataScales\n",
      "\n",
      "Al Oil\n",
      "\n",
      "AI Use Cases Across Industries\n",
      "\n",
      "\n",
      "\n",
      "ue ee\n",
      "\n",
      "ete (S)\n",
      "\n",
      "Manufacturing & Automotive Digital transformation, predictive maintenance, visual inspection\n",
      "\n",
      "Healthcare & Life Sciences Genomics, drug discovery, telehealth, disease diagnosis\n",
      "\n",
      "Financial Services Compliance, risk mitigation, algorithmic trading\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "ANN “TDL:\n",
      "\n",
      "Oil & Gas, Energy Predictive asset maintenance, seismic processing, exploration\n",
      "\n",
      "Public Sector Intelligent decision-making, ISR, predictive analytics\n",
      "\n",
      "Retail & E-Commerce Digital acceleration, personalized services delivery\n",
      "\n",
      "DON’T JUST TAKE OUR WORD FOR IT\n",
      "\n",
      "”We’re interested in technology that can train models faster than individual GPUs, that has more scalability and has more performance and so the SambaNova DataScale System met that criteria.”\n",
      "\n",
      "RICK STEVENS ASSOCIATE LABORATORY DIRECTOR ARGONNE NATIONAL LABORATORY\n",
      "---\n",
      "\n",
      "RICK STEVENS ASSOCIATE LABORATORY DIRECTOR ARGONNE NATIONAL LABORATORY\n",
      "\n",
      "”In this partnership with SambaNova, we’re providing businesses with never-before-seen access to artiﬁcial intelligence that will certainly lead to greater adoption of AI in Asia. Together, we are enabling ScaleWorX end-users access to disruptive and innovative solutions that can quickly generate new revenue opportunities with AI.”\n",
      "\n",
      "JUN MATSUURA MANAGING DIRECTOR\n",
      "\n",
      "”The support for running multiple models built into the hardware was a signiﬁcant aspect of our selection of SambaNova Systems. We’re seeing roughly a 5x improvement compared to a comparable GPU running the same models.”\n",
      "\n",
      "BRONIS DE SUPINSKI, CHIEF TECHNOLOGY OFFICER LAWRENCE LIVERMORE NATIONAL LABORATORY\n",
      "\n",
      "To learn how your organization can innovate with next-generation AI, please visit us at SambaNova.AI.\n",
      "\n",
      "in ¥ f &\n",
      "\n",
      "G)SambaNo\n",
      "---\n",
      "\n",
      "in ¥ f &\n",
      "\n",
      "G)SambaNo\n",
      "\n",
      "SambaNova Systems is an AI innovation company that empowers organizations to deploy best-in-class solutions for computer vision, natural language processing, recommendation systems, and AI for science with conﬁdence. SambaNova’s ﬂagship offering, Dataﬂow-as-a-Service, helps organizations rapidly deploy AI in days, unlocking new revenue and boosting operational efficiency. SambaNova’s DataScale® is an integrated software and hardware system using Reconﬁgurable Dataﬂow Architecture (RDA), along with open standards and user interfaces. Headquartered in Palo Alto, California, SambaNova Systems was founded in 2017 by industry luminaries, hardware, and software design experts from Sun/Oracle and Stanford University. Investors include SoftBank Vision Fund 2, funds and accounts managed by BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and several others.\n",
      "---\n",
      "\n",
      "Copyright © 2021 SambaNova Systems, Inc. All rights reserved. | Palo Alto, CA and Austin, TX | (650) 263-1153\n",
      "\n",
      "02\n",
      "---\n",
      "\n"
     ]
    }
   ],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=1000, chunk_overlap=200, length_function=len, separators=['\\n\\n', '</table>', '\\n']\n",
    ")\n",
    "\n",
    "chunks = text_splitter.split_documents(docs)\n",
    "for chunk in chunks:\n",
    "    print(f'{chunk.page_content}\\n---\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2. <a id='toc3_2_'></a>[Summarization](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define summarization chain\n",
    "chain = (\n",
    "    {'doc': lambda x: x.page_content}\n",
    "    | ChatPromptTemplate.from_template('<s>[INST] <<SYS>>Summarize the following document :<<SYS>>\\n\\n{doc}[/INST]')\n",
    "    | llm\n",
    "    | StrOutputParser()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here is a summary of the document:\n",
      "\n",
      "Artificial Intelligence (AI) is transforming industries, just like the internet did a few decades ago. Organizations that adopt AI early will gain a competitive edge. However, many companies face challenges in implementing AI, including a shortage of skilled talent, lack of expertise in computing architectures, and difficulty in keeping up with the latest models and techniques. These roadblocks hinder innovation and make it hard for companies to justify investing in AI without prior proof of its impact.\n",
      "---\n",
      "\n",
      "The document discusses the challenge of attracting top AI talent, which is a problem for most companies except for a few large ones like Google and Facebook. To address this issue, SambaNova offers a solution that enables organizations to build and deploy AI solutions for various applications, including natural language processing and computer vision, with high accuracy, scalability, and ease of use, at a lower cost and time than developing in-house infrastructure and expertise.\n",
      "---\n",
      "\n",
      "Here is a summary of the document:\n",
      "\n",
      "SambaNova offers an affordable and attainable AI platform that can be delivered as a capital expenditure (CapEx) rack infrastructure solution or as an operating expense (OpEx) subscription service. The platform comes pre-configured with pre-trained models, saving customers significant time. SambaNova's Dataflow-as-a-Service is a subscription-based AI acceleration platform that enables enterprises to quickly deploy AI solutions, leveraging SambaNova's expertise to augment their existing machine learning capabilities. The platform provides comprehensive API services and models for businesses to deploy fully managed AI solutions within weeks, rather than the typical 12-18 months. The platform supports various applications, including computer vision, natural language processing, and recommendation systems.\n",
      "---\n",
      "\n",
      "The document describes DataScale, an integrated AI software and hardware accelerator built on SambaNova Systems' Reconfigurable Dataflow Architecture. It offers high performance, accuracy, and scalability for AI computing. DataScale can be deployed in two ways: Dataflow-as-a-Service (managed by SambaNova) or on-premises (customer-managed). The former provides optimized pre-trained models and fine-tuning capabilities, while the latter allows customers to build and train their own models. The deployment options also differ in terms of operational and capital expenditures (Opex and Capex).\n",
      "---\n",
      "\n",
      "The document appears to be a promotional material from SambaNova Systems, Inc. It highlights the company's DataScale System and its applications across various industries, including:\n",
      "\n",
      "1. Manufacturing & Automotive: digital transformation, predictive maintenance, and visual inspection\n",
      "2. Healthcare & Life Sciences: genomics, drug discovery, telehealth, and disease diagnosis\n",
      "3. Financial Services: compliance, risk mitigation, and algorithmic trading\n",
      "4. Oil & Gas, Energy: predictive asset maintenance, seismic processing, and exploration\n",
      "5. Public Sector: intelligent decision-making, ISR, and predictive analytics\n",
      "6. Retail & E-Commerce: digital acceleration and personalized services delivery\n",
      "\n",
      "The document also includes a testimonial from Rick Stevens, Associate Laboratory Director at Argonne National Laboratory, who expresses interest in technology that can train models faster and more scalable than individual GPUs, and mentions that SambaNova's DataScale System meets those criteria.\n",
      "---\n",
      "\n",
      "The document appears to be a promotional piece for SambaNova Systems, a company that provides artificial intelligence (AI) solutions. The text quotes three individuals:\n",
      "\n",
      "1. Rick Stevens, Associate Laboratory Director at Argonne National Laboratory, who mentions a partnership with SambaNova that will lead to greater adoption of AI in Asia.\n",
      "2. Jun Matsuura, Managing Director, who highlights the benefits of SambaNova's hardware, specifically its ability to run multiple models, resulting in a 5x improvement compared to a comparable GPU.\n",
      "3. Bronis de Supinski, Chief Technology Officer at Lawrence Livermore National Laboratory, who is also quoted, but his statement is not provided in the text.\n",
      "\n",
      "The document invites readers to visit SambaNova's website to learn more about innovating with next-generation AI.\n",
      "---\n",
      "\n",
      "Here is a summary of the document:\n",
      "\n",
      "SambaNova Systems is an AI innovation company that provides solutions for computer vision, natural language processing, and other AI applications. Their flagship product, Dataflow-as-a-Service, enables organizations to deploy AI quickly, resulting in increased revenue and operational efficiency. SambaNova was founded in 2017 by industry experts from Sun/Oracle and Stanford University, and is headquartered in Palo Alto, California. The company has received investments from prominent firms such as SoftBank, BlackRock, and Intel Capital.\n",
      "---\n",
      "\n",
      "The document appears to be a copyright notice from SambaNova Systems, Inc. It states that the company holds all rights reserved and provides contact information, including locations in Palo Alto, CA, and Austin, TX, as well as a phone number (650) 263-1153. There is no actual content to summarize beyond this notice.\n",
      "---\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# call summarization chain in batch\n",
    "summaries = chain.batch(chunks, {'max_concurrency': 1})\n",
    "\n",
    "# create an id for each original chunk\n",
    "doc_ids = [str(uuid.uuid4()) for _ in chunks]\n",
    "\n",
    "# create a document over each summary and store in the metadata de doc_id of the original chunks\n",
    "id_key = 'doc_id'\n",
    "summarized_docs = [Document(page_content=s, metadata={id_key: doc_ids[i]}) for i, s in enumerate(summaries)]\n",
    "\n",
    "for summary in summarized_docs:\n",
    "    print(f'{summary.page_content}\\n---\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3. <a id='toc3_3_'></a>[Embedding and storing](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load INSTRUCTOR_Transformer\n",
      "max_seq_length  512\n"
     ]
    }
   ],
   "source": [
    "# Define HuggingFace embeding model\n",
    "encode_kwargs = {'normalize_embeddings': True}\n",
    "embedding = HuggingFaceInstructEmbeddings(\n",
    "    model_name='BAAI/bge-large-en',\n",
    "    query_instruction='Represent this sentence for searching relevant passages: ',\n",
    "    encode_kwargs=encode_kwargs,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/luiss/.pyenv/versions/3.11.3/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: The class `Chroma` was deprecated in LangChain 0.2.9 and will be removed in 0.4. An updated version of the class exists in the langchain-chroma package and should be used instead. To use it run `pip install -U langchain-chroma` and import as `from langchain_chroma import Chroma`.\n",
      "  warn_deprecated(\n"
     ]
    }
   ],
   "source": [
    "# Define The vectorstore to use to index the summarized chunks\n",
    "vectorstore = Chroma(collection_name='summaries', embedding_function=embedding)\n",
    "\n",
    "# Define the storage layer for the parent documents (original chunks)\n",
    "store = InMemoryByteStore()\n",
    "\n",
    "# define the retriever (empty to start), passing the vectorstore and the store of original chunks\n",
    "multivector_retriever = MultiVectorRetriever(\n",
    "    vectorstore=vectorstore,\n",
    "    byte_store=store,\n",
    "    id_key=id_key,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# emmbeding and adding the summarized_chunks to the vectorstore\n",
    "multivector_retriever.vectorstore.add_documents(summarized_docs)\n",
    "\n",
    "# storing the original chunks and their respective doc_id to the docstore\n",
    "multivector_retriever.docstore.mset(list(zip(doc_ids, chunks)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4. <a id='toc3_4_'></a>[Model and retrieval](#toc0_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define retrieval chain\n",
    "multivector_retrieval_chain = RetrievalQA.from_llm(\n",
    "    llm=llm,\n",
    "    retriever=multivector_retriever,\n",
    "    return_source_documents=True,\n",
    "    input_key='question',\n",
    "    output_key='answer',\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "custom_prompt_template = \"\"\"<s>[INST] <<SYS>>\\nUse the following pieces of context to answer the question at the end. \n",
    "Be careful when reading tables in html format do not mix information between rows or columns,\n",
    "If the answer is not in context for answering, say that you don't know, don't try to make up an answer or provide an answer not extracted from provided context. \n",
    "Cross check if the answer is contained in provided context. If not than say \"I do not have information regarding this.\"\n",
    "\n",
    "Context\n",
    "{context}\n",
    "End of context\n",
    "<</SYS>>\n",
    "\n",
    "Question: {question}\n",
    "Helpful Answer: [/INST]\"\"\"\n",
    "CUSTOMPROMPT = PromptTemplate(template=custom_prompt_template, input_variables=['context', 'question'])\n",
    "## Inject custom prompt\n",
    "multivector_retrieval_chain.combine_documents_chain.llm_chain.prompt = CUSTOMPROMPT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_question = (\n",
    "    \"What is the main advantage of using SambaNova's DataScale platform compared to traditional AI computing solutions?\"\n",
    ")\n",
    "response = multivector_retrieval_chain({'question': user_question})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Response =According to the provided context, the main advantage of using SambaNova's DataScale platform is that it delivers \"unrivaled performance, accuracy, scale, and ease of use\" compared to traditional AI computing solutions. Additionally, it is optimized from algorithms to silicon, providing efficiency with a software-defined-hardware approach and a highly flexible modular architecture. This allows DataScale to scale seamlessly from one to hundreds of systems to meet the demands of modern AI computing.\n"
     ]
    }
   ],
   "source": [
    "print(f'Response ={response[\"answer\"]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}