12_5_Summarize private documents using RAG LangChain and LLMs__M.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"text-align:center\">\n",
    "    <a href=\"https://skills.network\" target=\"_blank\">\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n",
    "    </a>\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Summarize Private Documents Using RAG, LangChain, and LLMs**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Estimated time needed: **45** minutes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Imagine it's your first day at an exciting new job at a fast-growing tech company, Innovatech. You're filled with a mix of anticipation and nerves, eager to make a great first impression and contribute to your team. As you find your way to your desk, decorated with a welcoming note and some company swag, you can't help but feel a surge of pride. This is the moment you've been working towards, and it's finally here.\n",
    "\n",
    "Your manager, Alex, greets you with a warm smile. \"Welcome aboard! We're thrilled to have you with us. I have sent you a folder. Inside this folder, you'll find everything you need to get up to speed on our company policies, culture, and the projects your team is working on. Please keep them private.\"\n",
    "\n",
    "You thank Alex and open the folder, only to be greeted by a mountain of documents. manuals, guidelines, technical documents, project summaries, and more await you. It's overwhelming. You think to yourself, \"How am I supposed to absorb all of this information in a short time? And they are private and I cannot just upload it to GPT to summarize them.\" \"Why not create an agent to read and summarize them for you, and then you can just ask it?\" your colleague, Jordan, suggests with an encouraging grin. You're intrigued, but uncertain; the world of large language models (LLMs) is one that you've only scratched the surface of. Sensing your hesitation, Jordan elaborates, \"Imagine having a personal assistant who's not only exceptionally fast at reading but can also understand and condense the information into easy-to-digest summaries. That's what an LLM can do for you, especially when enhanced with LangChain and Retrieval-Augmented Generation (RAG) technology.\" \n",
    "\n",
    "\"But, how do I get started? And how long will it take to set up something like that?\" you ask. Jordan says, \"Let's dive into a project that will not only help you tackle this immediate challenge but also equip you with a skill set that's becoming indispensable in this field.\"\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/C-rBNv5ZbCn1Qe9a-c_RwQ.png\" style=\"width:50%;margin:auto;display:flex\" alt=\"indexing\"/>\n",
    "\n",
    "---------\n",
    "\n",
    "So, this project steps you through the fascinating world of LLMs and RAG, starting from the basics of what these technologies are, to building a practical application that can read and summarize documents for you. By the end of this tutorial, you have a working tool capable of processing the pile of documents on your desk, allowing you to focus on making meaningful contributions to your projects sooner.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## __Table of Contents__\n",
    "\n",
    "<ol>\n",
    "    <li><a href=\"#Background\">Background</a>\n",
    "        <ol>\n",
    "            <li><a href=\"#What-is-RAG?\">What is RAG?</a></li>\n",
    "            <li><a href=\"#RAG-architecture\">RAG architecture</a></li>\n",
    "        </ol>\n",
    "    </li>\n",
    "    <li>\n",
    "        <a href=\"#Objectives\">Objectives</a>\n",
    "    </li>\n",
    "    <li>\n",
    "        <a href=\"#Setup\">Setup</a>\n",
    "        <ol>\n",
    "            <li><a href=\"#Installing-required-libraries\">Installing required libraries</a></li>\n",
    "            <li><a href=\"#Importing-required-libraries\">Importing required libraries</a></li>\n",
    "        </ol>\n",
    "    </li>\n",
    "    <li>\n",
    "        <a href=\"#Preprocessing\">Preprocessing</a>\n",
    "        <ol>\n",
    "            <li><a href=\"#Load-the-document\">Load the document</a></li>\n",
    "            <li><a href=\"#Splitting-the-document-into-chunks\">Splitting the document into chunks</a></li>\n",
    "            <li><a href=\"#Embedding-and-storing\">Embedding and storing</a></li>\n",
    "        </ol>\n",
    "    </li>\n",
    "    <li>\n",
    "        <a href=\"#LLM-model-construction\">LLM model construction</a>\n",
    "    </li>\n",
    "    <li>\n",
    "        <a href=\"#Integrating-LangChain\">Integrating LangChain</a>\n",
    "    </li>\n",
    "    <li>\n",
    "        <a href=\"#Dive-deeper\">Dive deeper</a>\n",
    "        <ol>\n",
    "            <li><a href=\"#Using-prompt-template\">Using prompt template</a></li>\n",
    "            <li><a href=\"#Make-the-conversation-have-memory\">Make the conversation have memory</a></li>\n",
    "            <li><a href=\"#Wrap-up-and-make-it-an-agent\">Wrap up and make it an agent</a></li>\n",
    "        </ol>\n",
    "    </li>\n",
    "</ol>\n",
    "\n",
    "<a href=\"#Exercises\">Exercises</a>\n",
    "<ol>\n",
    "    <li><a href=\"#Exercise-1:-Work-on-your-own-document\">Exercise 1: Work on your own document</a></li>\n",
    "    <li><a href=\"#Exercise-2:-Return-the-source-from-the-document\">Exercise 2: Return the source from the document</a></li>\n",
    "    <li><a href=\"#Exercise-3:-Use-another-LLM-model\">Exercise 3: Use another LLM model</a></li>\n",
    "</ol>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background\n",
    "\n",
    "### What is RAG?\n",
    "One of the most powerful applications enabled by LLMs is sophisticated question-answering (Q&A) chatbots. These are applications that can answer questions about specific source information. These applications use a technique known as retrieval-augmented generation (RAG). RAG is a technique for augmenting LLM knowledge with additional data, which can be your own data.\n",
    "\n",
    "LLMs can reason about wide-ranging topics, but their knowledge is limited to public data up to the specific point in time that they were trained. If you want to build AI applications that can reason about private data or data introduced after a model’s cut-off date, you must augment the knowledge of the model with the specific information that it needs. The process of bringing and inserting the appropriate information into the model prompt is known as RAG.\n",
    "\n",
    "LangChain has several components that are designed to help build Q&A applications and RAG applications, more generally.\n",
    "\n",
    "### RAG architecture\n",
    "A typical RAG application has two main components:\n",
    "\n",
    "* **Indexing**: A pipeline for ingesting and indexing data from a source. This usually happens offline.\n",
    "\n",
    "* **Retrieval and generation**: The actual RAG chain takes the user query at run time and retrieves the relevant data from the index, then passes that to the model.\n",
    "\n",
    "The most common full sequence from raw data to answer looks like the following examples.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **Indexing**\n",
    "1. Load: First, you must load your data. This is done with [DocumentLoaders](https://python.langchain.com/docs/modules/data_connection/document_loaders/).\n",
    "\n",
    "2. Split: [Text splitters](https://python.langchain.com/docs/modules/data_connection/document_transformers/) break large `Documents` into smaller chunks. This is useful both for indexing data and for passing it into a model because large chunks are harder to search and won’t fit in a model’s finite context window.\n",
    "\n",
    "3. Store: You need somewhere to store and index your splits so that they can later be searched. This is often done using a [VectorStore](https://python.langchain.com/docs/modules/data_connection/vectorstores/) and [Embeddings](https://python.langchain.com/docs/modules/data_connection/text_embedding/) model.\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/WEE3pjeJvSZP0R7UL7CYTA.png\" width=\"50%\" alt=\"indexing\"/> <br>\n",
    "<span style=\"font-size: 10px;\">[source](https://python.langchain.com/docs/use_cases/question_answering/)</span>\n",
    "\n",
    "\n",
    "- **Retrieval and generation**\n",
    "1. Retrieve: Given a user input, relevant splits are retrieved from storage using a retriever.\n",
    "2. Generate: A ChatModel / LLM produces an answer using a prompt that includes the question and the retrieved data.\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/SwPO26VeaC8VTZwtmWh5TQ.png\" width=\"50%\" alt=\"retrieval\"/> <br>\n",
    "<span style=\"font-size: 10px;\">[source](https://python.langchain.com/docs/use_cases/question_answering/)</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives\n",
    "\n",
    "After completing this lab, you will be able to:\n",
    "\n",
    " - Master the technique of splitting and embedding documents into formats that LLMs can efficiently read and interpret.\n",
    " - Know how to access and utilize different LLMs from IBM watsonx.ai, selecting the optimal model for their specific document processing needs.\n",
    " - Implement different retrieval chains from LangChain, tailoring the document retrieval process to support various purposes.\n",
    " - Develop an agent that uses integrated LLM, LangChain, and RAG technologies for interactive and efficient document retrieval and summarization, making conversation have memory.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this lab, you are going to use the following libraries:\n",
    "\n",
    "*   [`ibm-watsonx-ai`](https://ibm.github.io/watson-machine-learning-sdk/index.html) for using LLMs from IBM's watsonx.ai\n",
    "*   [`LangChain`](https://www.langchain.com/) for using its different chain and prompt functions\n",
    "*   [`Hugging Face`](https://huggingface.co/models?other=embeddings) and [`Hugging Face Hub`](https://huggingface.co/models?other=embeddings) for their embedding methods for processing text data\n",
    "*   [`SentenceTransformers`](https://www.sbert.net/) for transforming sentences into high-dimensional vectors\n",
    "*   [`Chroma DB`](https://www.trychroma.com/) for efficient storage and retrieval of high-dimensional text vector data\n",
    "*   [`wget`](https://pypi.org/project/wget/) for downloading files from remote systems\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Installing required libraries\n",
    "\n",
    "The following required libraries are __not__ preinstalled in the Skills Network Labs environment. __You must run the following cell__ to install them:\n",
    "\n",
    "**Note:** The version has been pinned here to specify the version. It's recommended that you do this as well. Even though the library will be updated in the future, the library could still support this lab work.\n",
    "\n",
    "This might take approximately 3-5 minutes. \n",
    "\n",
    "As `%%capture` is used to capture the installation, you won't see the output process. But once the installation is done, you will see a number beside the cell.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "!pip install \"ibm-watsonx-ai==0.2.6\"\n",
    "!pip install \"langchain==0.1.16\" \n",
    "!pip install \"langchain-ibm==0.1.4\"\n",
    "!pip install \"huggingface == 0.0.1\"\n",
    "!pip install \"huggingface-hub == 0.23.4\"\n",
    "!pip install \"sentence-transformers == 2.5.1\"\n",
    "!pip install \"chromadb == 0.4.24\"\n",
    "!pip install \"wget == 3.2\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the installation of libraries is completed, restart your kernel. You can do that by clicking the **Restart the kernel** icon.\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/rfWX6bPefx_DHiwFMktGBw/restart-kernel.jpg\" style=\"width:80%;margin:auto;display:flex\" alt=\"Restart kernel\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing required libraries\n",
    "_It is recommended that you import all required libraries in one place (here):_\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can use this section to suppress warnings generated by your code:\n",
    "def warn(*args, **kwargs):\n",
    "    pass\n",
    "import warnings\n",
    "warnings.warn = warn\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "from langchain.document_loaders import TextLoader\n",
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "from langchain.vectorstores import Chroma\n",
    "from langchain.embeddings import HuggingFaceEmbeddings\n",
    "from langchain.chains import RetrievalQA\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.chains import ConversationalRetrievalChain\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "\n",
    "from ibm_watsonx_ai.foundation_models import Model\n",
    "from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams\n",
    "from ibm_watsonx_ai.foundation_models.utils.enums import ModelTypes, DecodingMethods\n",
    "from ibm_watson_machine_learning.foundation_models.extensions.langchain import WatsonxLLM\n",
    "import wget"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing\n",
    "### Load the document\n",
    "\n",
    "The document, which is provided in a TXT format, outlines some company policies and serves as an example data set for the project.\n",
    "\n",
    "This is the `load` step in `Indexing`.<br>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/MPdUH7bXpHR5muZztZfOQg.png\" width=\"50%\" alt=\"split\"/>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file downloaded\n"
     ]
    }
   ],
   "source": [
    "filename = 'companyPolicies.txt'\n",
    "url = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/6JDbUb_L3egv_eOkouY71A.txt'\n",
    "\n",
    "# Use wget to download the file\n",
    "wget.download(url, out=filename)\n",
    "print('file downloaded')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the file is downloaded and imported into this lab environment, you can use the following code to look at the document.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.\tCode of Conduct\n",
      "\n",
      "Our Code of Conduct outlines the fundamental principles and ethical standards that guide every member of our organization. We are committed to maintaining a workplace that is built on integrity, respect, and accountability.\n",
      "Integrity: We hold ourselves to the highest ethical standards. This means acting honestly and transparently in all our interactions, whether with colleagues, clients, or the broader community. We respect and protect sensitive information, and we avoid conflicts of interest.\n",
      "Respect: We embrace diversity and value each individual's contributions. Discrimination, harassment, or any form of disrespectful behavior is unacceptable. We create an inclusive environment where differences are celebrated and everyone is treated with dignity and courtesy.\n",
      "Accountability: We take responsibility for our actions and decisions. We follow all relevant laws and regulations, and we strive to continuously improve our practices. We report any potential violations of this code and support the investigation of such matters.\n",
      "Safety: We prioritize the safety of our employees, clients, and the communities we serve. We maintain a culture of safety, including reporting any unsafe conditions or practices.\n",
      "Environmental Responsibility: We are committed to minimizing our environmental footprint and promoting sustainable practices.\n",
      "Our Code of Conduct is not just a set of rules; it is the foundation of our organization's culture. We expect all employees to uphold these principles and serve as role models for others, ensuring we maintain our reputation for ethical conduct, integrity, and social responsibility.\n",
      "\n",
      "2.\tRecruitment Policy\n",
      "\n",
      "Our Recruitment Policy reflects our commitment to attracting, selecting, and onboarding the most qualified and diverse candidates to join our organization. We believe that the success of our company relies on the talents, skills, and dedication of our employees.\n",
      "Equal Opportunity: We are an equal opportunity employer and do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, or any other protected status. We actively promote diversity and inclusion.\n",
      "Transparency: We maintain transparency in our recruitment processes. All job vacancies are advertised internally and externally when appropriate. Job descriptions and requirements are clear and accurately represent the role.\n",
      "Selection Criteria: Our selection process is based on the qualifications, experience, and skills necessary for the position. Interviews and assessments are conducted objectively, and decisions are made without bias.\n",
      "Data Privacy: We are committed to protecting the privacy of candidates' personal information and adhere to all relevant data protection laws and regulations.\n",
      "Feedback: Candidates will receive timely and constructive feedback on their application and interview performance.\n",
      "Onboarding: New employees receive comprehensive onboarding to help them integrate into the organization effectively. This includes information on our culture, policies, and expectations.\n",
      "Employee Referrals: We encourage and appreciate employee referrals as they contribute to building a strong and engaged team.\n",
      "Our Recruitment Policy is a foundation for creating a diverse, inclusive, and talented workforce. It ensures that we attract and hire the best candidates who align with our company values and contribute to our continued success. We continuously review and update this policy to reflect evolving best practices in recruitment.\n",
      "\n",
      "3.\tInternet and Email Policy\n",
      "\n",
      "Our Internet and Email Policy is established to guide the responsible and secure use of these essential tools within our organization. We recognize their significance in daily business operations and the importance of adhering to principles that maintain security, productivity, and legal compliance.\n",
      "Acceptable Use: Company-provided internet and email services are primarily meant for job-related tasks. Limited personal use is allowed during non-work hours, provided it doesn't interfere with work responsibilities.\n",
      "Security: Safeguard your login credentials, avoiding the sharing of passwords. Exercise caution with email attachments and links from unknown sources. Promptly report any unusual online activity or potential security breaches.\n",
      "Confidentiality: Reserve email for the transmission of confidential information, trade secrets, and sensitive customer data only when encryption is applied. Exercise discretion when discussing company matters on public forums or social media.\n",
      "Harassment and Inappropriate Content: Internet and email usage must not involve harassment, discrimination, or the distribution of offensive or inappropriate content. Show respect and sensitivity to others in all online communications.\n",
      "Compliance: Ensure compliance with all relevant laws and regulations regarding internet and email usage, including those related to copyright and data protection.\n",
      "Monitoring: The company retains the right to monitor internet and email usage for security and compliance purposes.\n",
      "Consequences: Policy violations may lead to disciplinary measures, including potential termination.\n",
      "Our Internet and Email Policy aims to promote safe, responsible usage of digital communication tools that align with our values and legal obligations. Each employee is expected to understand and follow this policy. Regular reviews ensure its alignment with evolving technology and security standards.\n",
      "\n",
      "4.\tMobile Phone Policy\n",
      "\n",
      "The Mobile Phone Policy sets forth the standards and expectations governing the appropriate and responsible usage of mobile devices in the organization. The purpose of this policy is to ensure that employees utilize mobile phones in a manner consistent with company values and legal compliance.\n",
      "Acceptable Use: Mobile devices are primarily intended for work-related tasks. Limited personal usage is allowed, provided it does not disrupt work obligations.\n",
      "Security: Safeguard your mobile device and access credentials. Exercise caution when downloading apps or clicking links from unfamiliar sources. Promptly report security concerns or suspicious activities related to your mobile device.\n",
      "Confidentiality: Avoid transmitting sensitive company information via unsecured messaging apps or emails. Be discreet when discussing company matters in public spaces.\n",
      "Cost Management: Keep personal phone usage separate from company accounts and reimburse the company for any personal charges on company-issued phones.\n",
      "Compliance: Adhere to all pertinent laws and regulations concerning mobile phone usage, including those related to data protection and privacy.\n",
      "Lost or Stolen Devices: Immediately report any lost or stolen mobile devices to the IT department or your supervisor.\n",
      "Consequences: Non-compliance with this policy may lead to disciplinary actions, including the potential loss of mobile phone privileges.\n",
      "The Mobile Phone Policy is aimed at promoting the responsible and secure use of mobile devices in line with legal and ethical standards. Every employee is expected to comprehend and abide by these guidelines. Regular reviews of the policy ensure its ongoing alignment with evolving technology and security best practices.\n",
      "\n",
      "5.\tSmoking Policy\n",
      "\n",
      "Policy Purpose: The Smoking Policy has been established to provide clear guidance and expectations concerning smoking on company premises. This policy is in place to ensure a safe and healthy environment for all employees, visitors, and the general public.\n",
      "Designated Smoking Areas: Smoking is only permitted in designated smoking areas, as marked by appropriate signage. These areas have been chosen to minimize exposure to secondhand smoke and to maintain the overall cleanliness of the premises.\n",
      "Smoking Restrictions: Smoking inside company buildings, offices, meeting rooms, and other enclosed spaces is strictly prohibited. This includes electronic cigarettes and vaping devices.\n",
      "Compliance with Applicable Laws: All employees and visitors must adhere to relevant federal, state, and local smoking laws and regulations.\n",
      "Disposal of Smoking Materials: Properly dispose of cigarette butts and related materials in designated receptacles. Littering on company premises is prohibited.\n",
      "No Smoking in Company Vehicles: Smoking is not permitted in company vehicles, whether they are owned or leased, to maintain the condition and cleanliness of these vehicles.\n",
      "Enforcement and Consequences: All employees and visitors are expected to adhere to this policy. Non-compliance may lead to appropriate disciplinary action, which could include fines, or, in the case of employees, possible termination of employment.\n",
      "Review of Policy: This policy will be reviewed periodically to ensure its alignment with evolving legal requirements and best practices for maintaining a healthy and safe workplace.\n",
      "We appreciate your cooperation in maintaining a smoke-free and safe environment for all.\n",
      "\n",
      "6.\tDrug and Alcohol Policy\n",
      "\n",
      "Policy Objective: The Drug and Alcohol Policy is established to establish clear expectations and guidelines for the responsible use of drugs and alcohol within the organization. This policy aims to maintain a safe, healthy, and productive workplace.\n",
      "Prohibited Substances: The use, possession, distribution, or sale of illegal drugs or unauthorized controlled substances is strictly prohibited on company premises or during work-related activities. This includes the misuse of prescription drugs.\n",
      "Alcohol Consumption: The consumption of alcoholic beverages is not allowed during work hours, on company property, or while performing company-related duties. Exception may be made for company-sanctioned events.\n",
      "Impairment: Employees are expected to perform their job duties without impairment from drugs or alcohol. The use of substances that could impair job performance or pose a safety risk is prohibited.\n",
      "Testing and Searches: The organization reserves the right to conduct drug and alcohol testing as per applicable laws and regulations. Employees may be subject to testing in cases of reasonable suspicion, post-accident, or as part of routine workplace safety measures.\n",
      "Reporting: Employees should report any concerns related to drug or alcohol misuse by themselves or their colleagues, as well as safety concerns arising from such misuse.\n",
      "Treatment and Assistance: Employees with substance abuse issues are encouraged to seek help. The organization is committed to providing support, resources, and information to assist those seeking treatment.\n",
      "Consequences: Violation of this policy may result in disciplinary actions, up to and including termination of employment. Legal action may also be pursued when necessary.\n",
      "Policy Review: This policy will undergo periodic review to ensure its continued relevance and compliance with evolving legal requirements and best practices for a safe and productive work environment.\n",
      "Your adherence to this policy is appreciated as it helps to maintain a safe and drug-free workplace for all.\n",
      "\n",
      "7.\tHealth and Safety Policy\n",
      "\n",
      "Our commitment to health and safety is paramount. We prioritize the well-being of our employees, customers, and the public. We diligently comply with all relevant health and safety laws and regulations. Our objective is to maintain a workplace free from hazards, preventing accidents, injuries, and illnesses. Every individual within our organization is responsible for upholding these standards. We regularly assess and improve our safety measures, provide adequate training, and encourage open communication regarding safety concerns. Through collective dedication, we aim to ensure a safe, healthy, and secure environment for all. Your cooperation is essential in achieving this common goal.\n",
      "\n",
      "8.\tAnti-discrimination and Harassment Policy\n",
      "\n",
      "The Anti-Discrimination and Harassment Policy is a testament to the commitment of this organization in fostering a workplace that is free from discrimination, harassment, and any form of unlawful bias. This policy applies to every individual within the organization, including employees, contractors, visitors, and clients.\n",
      "Non-Discrimination: This organization strictly prohibits discrimination based on race, color, religion, gender, national origin, age, disability, sexual orientation, or any other legally protected characteristic in all aspects of employment, including recruitment, hiring, compensation, benefits, promotions, and terminations.\n",
      "Harassment: Harassment in any form, whether based on the aforementioned characteristics or any other protected status, is unacceptable. This encompasses unwelcome advances, offensive jokes, slurs, and other verbal or physical conduct that creates a hostile or intimidating work environment.\n",
      "Reporting: Individuals who experience or witness any form of discrimination or harassment are encouraged to promptly report the incident to their supervisor, manager, or the designated HR representative. The organization is committed to a timely and confidential investigation of such complaints.\n",
      "Consequences: Violation of this policy may result in disciplinary action, including termination of employment. The organization is committed to taking appropriate action against any individual found to be in violation of this policy.\n",
      "Review and Update: This policy is subject to regular review and update to remain aligned with evolving legal requirements and best practices in preventing discrimination and harassment. This organization considers it a collective responsibility to ensure a workplace free from discrimination and harassment, and it is essential that every individual within the organization plays their part in upholding these principles.\n",
      "\n",
      "9.\tDiscipline and Termination Policy\n",
      "\n",
      "The Discipline and Termination Policy underscores the organization's commitment to maintaining a productive, ethical, and respectful work environment. This policy applies to all personnel, including employees, contractors, and temporary staff.\n",
      "Performance and Conduct Expectations: Employees are expected to meet performance standards and adhere to conduct guidelines. The organization will provide clear expectations, feedback, and opportunities for improvement when performance or conduct issues arise.\n",
      "Disciplinary Actions: When necessary, disciplinary actions will be taken, which may include verbal warnings, written warnings, suspension, or other appropriate measures. Disciplinary actions are designed to address issues constructively and maintain performance standards.\n",
      "Termination: In situations where an employee's performance or conduct issues persist, the organization may resort to termination. Termination may also occur for reasons such as redundancy, violation of policies, or restructuring.\n",
      "Termination Procedure: The organization will follow appropriate procedures, ensuring fairness and adherence to legal requirements during the termination process. Employees may be eligible for notice periods, severance pay, or other benefits as per employment agreements and applicable laws.\n",
      "Exit Process: The organization will conduct an exit process to ensure a smooth transition for departing employees, including the return of company property, final pay, and cancellation of access and benefits.\n",
      "This policy serves as a framework for handling discipline and termination. The organization recognizes the importance of fairness and consistency in these processes, and decisions will be made after careful consideration. Every employee is expected to understand and adhere to this policy, contributing to a respectful and productive workplace. Regular reviews will ensure its alignment with evolving legal requirements and best practices.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open(filename, 'r') as file:\n",
    "    # Read the contents of the file\n",
    "    contents = file.read()\n",
    "    print(contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the content, you see that the document discusses nine fundamental policies within a company.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Splitting the document into chunks\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step, you are splitting the document into chunks, which is basically the `split` process in `Indexing`.\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/0JFmAV5e_mejAXvCilgHWg.png\" width=\"50%\" alt=\"split\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`LangChain` is used to split the document and create chunks. It helps you divide a long story (document) into smaller parts, which are called `chunks`, so that it's easier to handle. \n",
    "\n",
    "For the splitting process, the goal is to ensure that each segment is as extensive as if you were to count to a certain number of characters and meet the split separator. This certain number is called `chunk size`. Let's set 1000 as the chunk size in this project. Though the chunk size is 1000, the splitting is happening randomly. This is an issue with LangChain. `CharacterTextSplitter` uses `\\n\\n` as the default split separator. You can change it by adding the `separator` parameter in the `CharacterTextSplitter` function; for example, `separator=\"\\n\"`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Created a chunk of size 1624, which is longer than the specified 1000\n",
      "Created a chunk of size 1885, which is longer than the specified 1000\n",
      "Created a chunk of size 1903, which is longer than the specified 1000\n",
      "Created a chunk of size 1729, which is longer than the specified 1000\n",
      "Created a chunk of size 1678, which is longer than the specified 1000\n",
      "Created a chunk of size 2032, which is longer than the specified 1000\n",
      "Created a chunk of size 1894, which is longer than the specified 1000\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16\n"
     ]
    }
   ],
   "source": [
    "loader = TextLoader(filename)\n",
    "documents = loader.load()\n",
    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "texts = text_splitter.split_documents(documents)\n",
    "print(len(texts))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the ouput of print, you see that the document has been split into 16 chunks\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Embedding and storing\n",
    "This step is the `embed` and `store` processes in `Indexing`. <br>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/u_oJz3v2cSR_lr0YvU6PaA.png\" width=\"50%\" alt=\"split\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step, you're taking the pieces of the story, your \"chunks,\" converting the text into numbers, and making them easier for your computer to understand and remember by using a process called \"embedding.\" Think of embedding like giving each chunk its own special code. This code helps the computer quickly find and recognize each chunk later on. \n",
    "\n",
    "You do this embedding process during a phase called \"Indexing.\" The reason why is to make sure that when you need to find specific information or details within your larger document, the computer can do so swiftly and accurately.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code creates a default embedding model from Hugging Face and ingests them to Chromadb.\n",
    "\n",
    "When it's completed, print \"document ingested\".\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b3dad67d761d46dc8255027808c23e42",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "modules.json:   0%|          | 0.00/349 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f897ab4dc7f546688c2b42a379958573",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config_sentence_transformers.json:   0%|          | 0.00/116 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e7eac4f357564ca380bf74e1e341e442",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md:   0%|          | 0.00/10.6k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "be74875722c84ffe95dc7e9779dc7628",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "sentence_bert_config.json:   0%|          | 0.00/53.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e5e93942846640fba5893f28f5a7be74",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/571 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1dc635da7991498cb80f9af16eac0af8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/438M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b547e468c08f450bafe542f1cdcc4493",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/363 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d4dd677b462a4dcabe1d8e5668ae7f77",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4913d370d1f24cef8b25eafbf10f7e8d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "510aeb5c965e4980aecf41350ab4fb94",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/239 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9f0c73c2fbc6437f9125282bfe51e984",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "1_Pooling/config.json:   0%|          | 0.00/190 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "document ingested\n"
     ]
    }
   ],
   "source": [
    "embeddings = HuggingFaceEmbeddings()\n",
    "docsearch = Chroma.from_documents(texts, embeddings)  # store the embedding in docsearch using Chromadb\n",
    "print('document ingested')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Up to this point, you've been performing the `Indexing` task. The next step is the `Retrieval` task.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LLM model construction\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, you'll build an LLM model from IBM watsonx.ai. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, define a model ID and choose which model that you want to use. There are many other model options. Refer to [Foundation Models](https://ibm.github.io/watsonx-ai-python-sdk/foundation_models.html) for other model options. This tutorial uses the `FLAN_UL2` model as an example.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_id = 'google/flan-ul2'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define parameters for the model.\n",
    "\n",
    "The decoding method is set to `greedy` to get a deterministic output.\n",
    "\n",
    "For other commonly used parameters, you can refer to [Foundation model parameters: decoding and stopping criteria](https://www.ibm.com/docs/en/watsonx-as-a-service?utm_source=skills_network&utm_content=in_lab_content_link&utm_id=Lab-RAG_v1_1711546843&topic=lab-model-parameters-prompting).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    GenParams.DECODING_METHOD: DecodingMethods.GREEDY,  \n",
    "    GenParams.MIN_NEW_TOKENS: 130, # this controls the minimum number of tokens in the generated output\n",
    "    GenParams.MAX_NEW_TOKENS: 256,  # this controls the maximum number of tokens in the generated output\n",
    "    GenParams.TEMPERATURE: 0.5 # this randomness or creativity of the model's responses\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define `credentials` and `project_id`,  which are necessary parameters to successfully run LLMs from watsonx.ai.\n",
    "\n",
    "(Keep `credentials` and `project_id` as they are now so that you do not need to create your own keys to run models. This supports you in running the model inside this lab environment. However, if you want to run the model locally, refer to this [tutorial](https://medium.com/the-power-of-ai/ibm-watsonx-ai-the-interface-and-api-e8e1c7227358) for creating your own keys.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "credentials = {\n",
    "    \"url\": \"https://us-south.ml.cloud.ibm.com\"\n",
    "}\n",
    "\n",
    "project_id = \"skills-network\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wrap the parameters to the model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Model(\n",
    "    model_id=model_id,\n",
    "    params=parameters,\n",
    "    credentials=credentials,\n",
    "    project_id=project_id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build a model called `flan_ul2_llm` from watsonx.ai.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flan_ul2_llm = WatsonxLLM(model=model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This completes the `LLM` part of the `Retrieval` task. <br>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/UZXQ44Tgv4EQ2-mTcu5e-A.png\" width=\"50%\" alt=\"split\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Integrating LangChain\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LangChain has a number of components that are designed to help retrieve information from the document and build question-answering applications, which helps you complete the `retrieve` part of the `Retrieval` task. <br>\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/M4WpkkMMbfK0Wkz0W60Jiw.png\" width=\"50%\" alt=\"split\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following steps, you create a simple Q&A application over the document source using LangChain's `RetrievalQA`.\n",
    "\n",
    "Then, you ask the query \"what is mobile policy?\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'what is mobile policy?',\n",
       " 'result': 'The Mobile Phone Policy sets forth the standards and expectations governing the appropriate and responsible usage of mobile devices in the organization. The purpose of this policy is to ensure that employees utilize mobile phones in a manner consistent with company values and legal compliance. Acceptable Use: Mobile devices are primarily intended for work-related tasks. Limited personal usage is allowed, provided it does not disrupt work obligations. Security: Safeguard your mobile device and access credentials. Exercise caution when downloading apps or clicking links from unfamiliar sources. Promptly report security concerns or suspicious activities related to your mobile device. Confidentiality: Avoid transmitting sensitive company information via unsecured messaging apps or emails. Be discreet when discussing company matters in public spaces. Cost Management: Keep personal phone usage separate from company accounts and reimburse the company for any personal charges on company-issued phones. Compliance: Adhere to all pertinent laws and regulations concerning mobile phone usage, including those related to data protection and privacy. Lost or Stolen Devices: Immediately report any lost or stolen mobile devices to the IT department or your supervisor. Consequences: Non-compliance with this policy may lead to disciplinary actions, including the potential loss of mobile phone privileges. The Mobile'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(llm=flan_ul2_llm, \n",
    "                                 chain_type=\"stuff\", \n",
    "                                 retriever=docsearch.as_retriever(), \n",
    "                                 return_source_documents=False)\n",
    "query = \"what is mobile policy?\"\n",
    "qa.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the response, it seems fine. The model's response is the relevant information about the mobile policy from the document.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, try to ask a more high-level question.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'Can you summarize the document for me?',\n",
       " 'result': \"Code of Conduct, Health and Safety Policy, Anti-discrimination and Harassment Policy, Recruitment Policy.. I think that's it.. I'm not sure.. I'm not sure if it's all in there.. I'm not sure if it's all in there.. I'm not sure if it's all in there.. I'm not sure if it's all in there.. I'm not sure if it's all in there.. I'm not sure if it's all in there..\"}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(llm=flan_ul2_llm, \n",
    "                                 chain_type=\"stuff\", \n",
    "                                 retriever=docsearch.as_retriever(), \n",
    "                                 return_source_documents=False)\n",
    "query = \"Can you summarize the document for me?\"\n",
    "qa.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this time, the model seems to not have the ability to summarize the document. This is because of the limitation of the `FLAN_UL2` model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, try to use another model, `LLAMA_3_70B_INSTRUCT`. You should do the model construction again.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_id = 'meta-llama/llama-3-70b-instruct'\n",
    "\n",
    "parameters = {\n",
    "    GenParams.DECODING_METHOD: DecodingMethods.GREEDY,  \n",
    "    GenParams.MAX_NEW_TOKENS: 256,  # this controls the maximum number of tokens in the generated output\n",
    "    GenParams.TEMPERATURE: 0.5 # this randomness or creativity of the model's responses\n",
    "}\n",
    "\n",
    "credentials = {\n",
    "    \"url\": \"https://us-south.ml.cloud.ibm.com\"\n",
    "}\n",
    "\n",
    "project_id = \"skills-network\"\n",
    "\n",
    "model = Model(\n",
    "    model_id=model_id,\n",
    "    params=parameters,\n",
    "    credentials=credentials,\n",
    "    project_id=project_id\n",
    ")\n",
    "\n",
    "llama_3_llm = WatsonxLLM(model=model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try the same query again on this model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'Can you summarize the document for me?',\n",
       " 'result': \" This document appears to be a company's policies and code of conduct. It outlines the organization's commitment to integrity, respect, accountability, safety, and environmental responsibility. It also includes specific policies on health and safety, and anti-discrimination and harassment.\"}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(llm=llama_3_llm, \n",
    "                                 chain_type=\"stuff\", \n",
    "                                 retriever=docsearch.as_retriever(), \n",
    "                                 return_source_documents=False)\n",
    "query = \"Can you summarize the document for me?\"\n",
    "qa.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, you've created a simple Q&A application for your own document. Congratulations!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dive deeper\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section dives deeper into how you can improve this application. You might want to ask \"How to add the prompt in retrieval using LangChain?\" <br>\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/bvw3pPRCYRUsv-Z2m33hmQ.png\" width=\"50%\" alt=\"split\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You use prompts to guide the responses from an LLM the way that you want. For instance, if the LLM is uncertain about an answer, you instruct it to simply state, \"I do not know,\" instead of attempting to generate a speculative response.\n",
    "\n",
    "Let's see an example.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'Can I eat in company vehicles?',\n",
       " 'result': 'No Smoking in Company Vehicles: Smoking is not permitted in company vehicles, whether they are owned or leased, to maintain the condition and cleanliness of these vehicles. So no eating..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..?..'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(llm=flan_ul2_llm, \n",
    "                                 chain_type=\"stuff\", \n",
    "                                 retriever=docsearch.as_retriever(), \n",
    "                                 return_source_documents=False)\n",
    "query = \"Can I eat in company vehicles?\"\n",
    "qa.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the query is asking something that does not exist in the document. The LLM responds with information that actually is not true. You don't want this to happen, so you must add a prompt to the LLM.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using prompt template\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following code, you create a prompt template using `PromptTemplate`.\n",
    "\n",
    "`context` and `question` are keywords in the RetrievalQA, so LangChain can automatically recognize them as document content and query.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_template = \"\"\"Use the information from the document to answer the question at the end. If you don't know the answer, just say that you don't know, definately do not try to make up an answer.\n",
    "\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "\n",
    "PROMPT = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
    ")\n",
    "\n",
    "chain_type_kwargs = {\"prompt\": PROMPT}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can ask the same question that does not have an answer in the document again.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'Can I eat in company vehicles?',\n",
       " 'result': \"Answer: The document does not mention eating in company vehicles, it only mentions that smoking is not permitted in company vehicles. Therefore, I don't know if eating is allowed or not.\"}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(llm=llama_3_llm, \n",
    "                                 chain_type=\"stuff\", \n",
    "                                 retriever=docsearch.as_retriever(), \n",
    "                                 chain_type_kwargs=chain_type_kwargs, \n",
    "                                 return_source_documents=False)\n",
    "\n",
    "query = \"Can I eat in company vehicles?\"\n",
    "qa.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the answer, you can see that the model responds with \"don't know\".\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make the conversation have memory\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do you want your conversations with an LLM to be more like a dialogue with a friend who remembers what you talked about last time? An LLM that retains the memory of your previous exchanges builds a more coherent and contextually rich conversation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look at a situation in which an LLM does not have memory.\n",
    "\n",
    "You start a new query, \"What I cannot do in it?\". You do not specify what \"it\" is. In this case, \"it\" means \"company vehicles\" if you refer to the last query.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'query': 'What I cannot do in it?', 'result': \"Answer: I don't know.\"}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = \"What I cannot do in it?\"\n",
    "qa.invoke(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the response, you see that the model does not have the memory because it does not provide the correct answer, which is something related to \"smoking is not permitted in company vehicles.\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make the LLM have memory, you introduce the `ConversationBufferMemory` function from LangChain.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "memory = ConversationBufferMemory(memory_key = \"chat_history\", return_message = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a `ConversationalRetrievalChain` to retrieve information and talk with the LLM.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "qa = ConversationalRetrievalChain.from_llm(llm=llama_3_llm, \n",
    "                                           chain_type=\"stuff\", \n",
    "                                           retriever=docsearch.as_retriever(), \n",
    "                                           memory = memory, \n",
    "                                           get_chat_history=lambda h : h, \n",
    "                                           return_source_documents=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a `history` list to store the chat history.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "history = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The Mobile Phone Policy sets forth the standards and expectations governing the appropriate and responsible usage of mobile devices in the organization.\n"
     ]
    }
   ],
   "source": [
    "query = \"What is mobile policy?\"\n",
    "result = qa.invoke({\"question\":query}, {\"chat_history\": history})\n",
    "print(result[\"answer\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Append the previous query and answer to the history.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "history.append((query, result[\"answer\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The key points of a mobile phone policy include acceptable use, security, confidentiality, cost management, compliance, lost or stolen devices, and consequences.\n"
     ]
    }
   ],
   "source": [
    "query = \"List points in it?\"\n",
    "result = qa({\"question\": query}, {\"chat_history\": history})\n",
    "print(result[\"answer\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Append the previous query and answer to the chat history again.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "history.append((query, result[\"answer\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The purpose of a mobile phone policy is to ensure that employees utilize mobile phones in a manner consistent with company values and legal compliance.\n"
     ]
    }
   ],
   "source": [
    "query = \"What is the aim of it?\"\n",
    "result = qa({\"question\": query}, {\"chat_history\": history})\n",
    "print(result[\"answer\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wrap up and make it an agent\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code defines a function to make an agent, which can retrieve information from the document and has the conversation memory.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def qa():\n",
    "    memory = ConversationBufferMemory(memory_key = \"chat_history\", return_message = True)\n",
    "    qa = ConversationalRetrievalChain.from_llm(llm=llama_3_llm, \n",
    "                                               chain_type=\"stuff\", \n",
    "                                               retriever=docsearch.as_retriever(), \n",
    "                                               memory = memory, \n",
    "                                               get_chat_history=lambda h : h, \n",
    "                                               return_source_documents=False)\n",
    "    history = []\n",
    "    while True:\n",
    "        query = input(\"Question: \")\n",
    "        \n",
    "        if query.lower() in [\"quit\",\"exit\",\"bye\"]:\n",
    "            print(\"Answer: Goodbye!\")\n",
    "            break\n",
    "            \n",
    "        result = qa({\"question\": query}, {\"chat_history\": history})\n",
    "        \n",
    "        history.append((query, result[\"answer\"]))\n",
    "        \n",
    "        print(\"Answer: \", result[\"answer\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the function.\n",
    "\n",
    "Feel free to answer questions for your chatbot. For example: \n",
    "\n",
    "_What is the smoking policy? Can you list all points of it? Can you summarize it?_\n",
    "\n",
    "To **stop** the agent, you can type in 'quit', 'exit', 'bye'. Otherwise you cannot run other cells. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Question:  What is the trickiest part in the smoking policy?\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer:   I don't know.\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Question:  Try again!\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer:   I don't know.\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Question:  Define tricky\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer:   I don't know. The provided context appears to be a list of company policies, and it doesn't seem to be related to the definition of the word \"tricky\".\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Question:  Got it! Thank you! Please list the most important points in the smocking policy\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer:   The key points in the smoking policy are:\n",
      "\n",
      "* Smoking is only permitted in designated smoking areas.\n",
      "* Smoking is strictly prohibited inside company buildings, offices, meeting rooms, and other enclosed spaces.\n",
      "* Electronic cigarettes and vaping devices are also prohibited in these areas.\n",
      "* All employees and visitors must comply with relevant federal, state, and local smoking laws and regulations.\n",
      "* Proper disposal of cigarette butts and related materials is required.\n",
      "* Smoking is not permitted in company vehicles.\n",
      "* Non-compliance with the policy may lead to disciplinary action, including fines or termination of employment.\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Question:  Bye\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: Goodbye!\n"
     ]
    }
   ],
   "source": [
    "qa()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations! You have finished the project. Following are three exercises to help you to extend your knowledge.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1: Work on your own document\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You are welcome to use your own document to practice. Another document has also been prepared that you can use for practice. Can you load this document and make the LLM read it for you? <br>\n",
    "Here is the URL to the document: https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/XVnuuEg94sAE4S_xAsGxBA.txt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file downloaded\n"
     ]
    }
   ],
   "source": [
    "# Add your code here\n",
    "\n",
    "filename = 'stateOfUnion.txt'\n",
    "url = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/XVnuuEg94sAE4S_xAsGxBA.txt'\n",
    "\n",
    "wget.download(url, out=filename)\n",
    "print('file downloaded')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here for solution</summary>\n",
    "<br>\n",
    "    \n",
    "```python\n",
    "filename = 'stateOfUnion.txt'\n",
    "url = 'https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/XVnuuEg94sAE4S_xAsGxBA.txt'\n",
    "\n",
    "wget.download(url, out=filename)\n",
    "print('file downloaded')\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2: Return the source from the document\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, you not only want the LLM to summarize for you, but you also want the model to return the exact content source from the document to you for reference. Can you adjust the code to make it happen?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "page_content='Policy Purpose: The Smoking Policy has been established to provide clear guidance and expectations concerning smoking on company premises. This policy is in place to ensure a safe and healthy environment for all employees, visitors, and the general public.\\nDesignated Smoking Areas: Smoking is only permitted in designated smoking areas, as marked by appropriate signage. These areas have been chosen to minimize exposure to secondhand smoke and to maintain the overall cleanliness of the premises.\\nSmoking Restrictions: Smoking inside company buildings, offices, meeting rooms, and other enclosed spaces is strictly prohibited. This includes electronic cigarettes and vaping devices.\\nCompliance with Applicable Laws: All employees and visitors must adhere to relevant federal, state, and local smoking laws and regulations.\\nDisposal of Smoking Materials: Properly dispose of cigarette butts and related materials in designated receptacles. Littering on company premises is prohibited.\\nNo Smoking in Company Vehicles: Smoking is not permitted in company vehicles, whether they are owned or leased, to maintain the condition and cleanliness of these vehicles.\\nEnforcement and Consequences: All employees and visitors are expected to adhere to this policy. Non-compliance may lead to appropriate disciplinary action, which could include fines, or, in the case of employees, possible termination of employment.\\nReview of Policy: This policy will be reviewed periodically to ensure its alignment with evolving legal requirements and best practices for maintaining a healthy and safe workplace.\\nWe appreciate your cooperation in maintaining a smoke-free and safe environment for all.' metadata={'source': 'companyPolicies.txt'}\n"
     ]
    }
   ],
   "source": [
    "# Add your code \n",
    "\n",
    "qa = RetrievalQA.from_chain_type(llm=llama_3_llm, chain_type=\"stuff\", retriever=docsearch.as_retriever(), return_source_documents=True)\n",
    "query = \"Can I smoke in company vehicles?\"\n",
    "results = qa.invoke(query)\n",
    "print(results['source_documents'][0]) ## this will return you the source content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here for a hint</summary>\n",
    "All you must do is change the return_source_documents to True when you create the chain. And when you print, print the ['source_documents'][0] \n",
    "<br><br>\n",
    "\n",
    "    \n",
    "```python\n",
    "qa = RetrievalQA.from_chain_type(llm=llama_3_llm, chain_type=\"stuff\", retriever=docsearch.as_retriever(), return_source_documents=True)\n",
    "query = \"Can I smoke in company vehicles?\"\n",
    "results = qa.invoke(query)\n",
    "print(results['source_documents'][0]) ## this will return you the source content\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3: Use another LLM model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "IBM watsonx.ai also has many other LLM models that you can use; for example, `GRANITE_13B_CHAT_V2`. Can you change the model to see the difference of the response?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add your code here\n",
    "\n",
    "model_id = ModelTypes.GRANITE_13B_CHAT_V2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Click here for a hint</summary>\n",
    "\n",
    "To use the Granite model in your notebook, go to the cell where the `model_id` is specified to `LLAMA_3_70B_INSTRUCT` and replace the current `model_id` with the following code. Expect different results and performance when using other models. \n",
    "\n",
    "```python\n",
    "model_id = ModelTypes.GRANITE_13B_CHAT_V2\n",
    "```\n",
    "</br>\n",
    "\n",
    "After updating, run the remaining cells in the notebook to ensure the Granite model is used for subsequent operations.\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Kang Wang](https://author.skills.network/instructors/kang_wang) <br>\n",
    "Kang Wang is a Data Scientist Intern in IBM. He is also a PhD Candidate in the University of Waterloo.\n",
    "\n",
    "[Faranak Heidari](https://www.linkedin.com/in/faranakhdr/) <br>\n",
    "Faranak Heidari is a Data Scientist Intern in IBM with a strong background in applied machine learning. Experienced in managing complex data to establish business insights and foster data-driven decision-making in complex settings such as healthcare. She is also a PhD candidate at the University of Toronto.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other Contributors\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Sina Nazeri](https://author.skills.network/instructors/sina_nazeri) <br>\n",
    "I am grateful to have had the opportunity to work as a Research Associate, Ph.D., and IBM Data Scientist. Through my work, I have gained experience in unraveling complex data structures to extract insights and provide valuable guidance.\n",
    "\n",
    "[Wojciech Fulmyk](https://author.skills.network/instructors/wojciech_fulmyk) <br>\n",
    "As a data scientist at the Ecosystems Skills Network at IBM and a Ph.D. candidate in Economics at the University of Calgary, I bring a wealth of experience in unraveling complex problems through the lens of data. What sets me apart is my ability to seamlessly merge technical expertise with effective communication, translating intricate data findings into actionable insights for stakeholders at all levels. From modeling to storytelling, I bring a holistic approach to data science. Leveraging machine learning algorithms, I construct predictive models tailored to both real-world challenges as well as old, well-understood problems. My knack for data-driven storytelling ensures that the insights uncovered resonate with both technical and non-technical audiences. Open to collaboration, I'm eager to take on new challenges and contribute to transformative data-driven endeavors. Whether you seek to extract insights, enhance predictive models, or explore untapped potential within your datasets, I'm here to help. Feel free to connect to me via my LinkedIn profile. Let's learn from each other!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{## Change Log}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{|Date (YYYY-MM-DD)|Version|Changed By|Change Description||-|-|-|-||2024-03-22|0.1|Kang Wang|Create the Project|}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "© Copyright IBM Corporation. All rights reserved.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "prev_pub_hash": "f5d5d9ce8fcc51797e3f39bb0ee855e389584dc0de35ac67358310c105e91083"
 },
 "nbformat": 4,
 "nbformat_minor": 4
}