python-recipes/llm-session-manager/00_llm_session_manager.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n",
    "\n",
    "# LLM Session Memory - Multiple Sessions\n",
    "\n",
    "Large Language Models are inherently stateless and have no knowledge of previous interactions with a user, or even of previous parts of the current conversation. While this may not be noticeable when asking simple questions, it becomes a hinderance when engaging in long running conversations that rely on conversational context.\n",
    "\n",
    "The solution to this problem is to append the previous conversation history to each subsequent call to the LLM.\n",
    "\n",
    "This notebook will show how to use Redis to structure and store and retrieve this conversational session memory.\n",
    "\n",
    "## Let's Begin!\n",
    "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/session-manager/00_session_manager.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Environment setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install cohere \"redisvl>=0.4.1\" sentence-transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set Cohere API Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import getpass\n",
    "\n",
    "\n",
    "if \"COHERE_API_KEY\" not in os.environ:\n",
    "    os.environ[\"COHERE_API_KEY\"] = getpass.getpass(\"COHERE_API_KEY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Run local redis (for colab)\n",
    "Use the shell script below to download, extract, and install [Redis Stack](https://redis.io/docs/getting-started/install-stack/) directly from the Redis package archive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "%%sh\n",
    "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
    "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
    "sudo apt-get update  > /dev/null 2>&1\n",
    "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
    "redis-stack-server --daemonize yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### For Alternative Environments\n",
    "There are many ways to get the necessary redis-stack instance running\n",
    "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n",
    "own version of Redis Enterprise running, that works too!\n",
    "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n",
    "3. With docker: `docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the Redis Connection URL and Client\n",
    "\n",
    "By default this notebook connects to the local instance of Redis Stack. **If you have your own Redis Enterprise instance** - replace REDIS_PASSWORD, REDIS_HOST and REDIS_PORT values with your own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from redis import Redis\n",
    "\n",
    "# Replace values below with your own if using Redis Cloud instance\n",
    "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") # ex: \"redis-18374.c253.us-central1-1.gce.cloud.redislabs.com\"\n",
    "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\")      # ex: 18374\n",
    "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\")  # ex: \"1TNxTEdYRDgIDKM2gDfasupCADXXXX\"\n",
    "\n",
    "# If SSL is enabled on the endpoint, use rediss:// as the URL prefix\n",
    "REDIS_URL = f\"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}\"\n",
    "\n",
    "redis_client = Redis.from_url(REDIS_URL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Initialize CohereClient as LLM layer\n",
    "To demonstrate how a real LLM conversation may flow we'll use a real LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Dict, List\n",
    "import cohere\n",
    "import os\n",
    "\n",
    "class CohereClient():\n",
    "    def __init__(self, api_key: str = None, model: str = 'command-r-plus'):\n",
    "        api_key = api_key or os.getenv(\"COHERE_API_KEY\")\n",
    "        self.client = cohere.Client(api_key)\n",
    "        self._model = model\n",
    "\n",
    "    def converse(self, prompt: str, context: List[Dict]) -> str:\n",
    "        context = self.remap(context)\n",
    "        response = self.client.chat(\n",
    "                model=self._model,\n",
    "                chat_history = context,\n",
    "                message=prompt,\n",
    "                )\n",
    "        return response.text\n",
    "\n",
    "    def remap(self, context) -> List[Dict]:\n",
    "        ''' re-index the chat history to match the Cohere API requirements '''\n",
    "        new_context = []\n",
    "        for statement in context:\n",
    "            if statement[\"role\"] == \"user\":\n",
    "                new_statement = {\"role\": \"USER\", \"message\": statement[\"content\"]}\n",
    "            elif statement[\"role\"] == \"llm\":\n",
    "                new_statement = {\"role\": \"CHATBOT\", \"message\": statement[\"content\"]}\n",
    "            elif statement[\"role\"] == \"system\":\n",
    "                new_statement = {\"role\": \"SYSTEM\", \"message\": statement[\"content\"]}\n",
    "            else:\n",
    "                raise ValueError(f'Unknown chat role {statement[\"role\"]}')\n",
    "            new_context.append(new_statement)\n",
    "        return new_context\n",
    "\n",
    "client = CohereClient()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import SemanticSessionManager\n",
    "\n",
    "redisvl provides the SemanticSessionManager for easy management of session state."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from redisvl.extensions.session_manager import SemanticSessionManager\n",
    "\n",
    "user_session = SemanticSessionManager(name=\"llm chef\")\n",
    "user_session.add_message({\"role\":\"system\", \"content\":\"You are a helpful chef, assisting people in making delicious meals\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### On each call to the LLM we can first retrieve the recent history of the conversation. On the first call it will just be the preamble we set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "USER:  can you give me some ideas for breakfast?\n",
      "\n",
      "LLM:  Sure! Here are a few breakfast ideas:\n",
      "\n",
      "- Avocado toast: This is a quick and easy option that's both delicious and nutritious. Simply toast some bread, smash an avocado on top, and add a squeeze of lemon or lime juice and a pinch of salt and pepper to taste. You can also add additional toppings like tomato, feta cheese, or a fried egg.\n",
      "- Oatmeal: Oatmeal is a hearty and healthy breakfast option that can be prepared in a variety of ways. You can cook it on the stovetop or in the microwave, and top it with fruits, nuts, seeds, or a drizzle of honey.\n",
      "- Smoothie bowl: Blend together your favorite fruits, vegetables, and liquid base (such as juice, milk, or yogurt) to make a smoothie, and then pour it into a bowl. Top it with granola, nuts, seeds, or fresh fruit for a beautiful and tasty breakfast.\n",
      "- Eggs: There are endless ways to prepare eggs for breakfast. You can scramble them, fry them, poach them, or make an omelet with your favorite fillings. Serve them with toast, hash browns, or roasted vegetables for a filling and satisfying meal.\n",
      "- Pancakes or waffles: Treat yourself to a stack of fluffy pancakes or waffles for a classic breakfast option. You can top them with butter and syrup, or get creative with fruit, whipped cream, or chocolate chips.\n",
      "- Yogurt parfait: Layer yogurt, granola, and fresh fruit in a bowl or jar to make a delicious and beautiful yogurt parfait. This is a quick and easy option that can be customized to your taste.\n",
      "\n",
      "Feel free to experiment and get creative with your breakfast options! You can also try preparing larger batches of certain items (like oatmeal or pancakes) on the weekends, so you have leftovers to enjoy during the busy workweek.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"can you give me some ideas for breakfast?\"\n",
    "context = user_session.get_recent()\n",
    "response = client.converse(prompt=prompt, context=context)\n",
    "user_session.store(prompt, response)\n",
    "print('USER: ', prompt)\n",
    "print('\\nLLM: ', response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's continue this conversation and inspect the context we're passing to the LLM as we go."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "USER:  can you give me the recipe for those pancakes?\n",
      "\n",
      "LLM:  Sure, here's a basic recipe for fluffy pancakes:\n",
      "\n",
      "Ingredients:\n",
      "\n",
      "- 2 cups all-purpose flour\n",
      "- 2 teaspoons baking powder\n",
      "- 1/2 teaspoon salt\n",
      "- 2 tablespoons sugar\n",
      "- 2 cups milk\n",
      "- 2 eggs\n",
      "- 2 tablespoons melted butter, plus more for the pan\n",
      "- 1 teaspoon vanilla extract (optional)\n",
      "\n",
      "Instructions:\n",
      "\n",
      "1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\n",
      "\n",
      "2. In a separate bowl, whisk together the milk, eggs, melted butter, and vanilla extract (if using).\n",
      "\n",
      "3. Pour the wet ingredients into the dry ingredients and stir until just combined. Be careful not to overmix the batter; a few lumps are okay.\n",
      "\n",
      "4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\n",
      "\n",
      "5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\n",
      "\n",
      "6. Flip the pancakes carefully and cook for an additional 1-2 minutes on the other side until golden brown.\n",
      "\n",
      "7. Repeat with the remaining batter, greasing the pan as needed.\n",
      "\n",
      "8. Serve the pancakes warm with your favorite toppings such as butter, syrup, fruit, whipped cream, or chocolate chips.\n",
      "\n",
      "This recipe should make about 10-12 pancakes, depending on their size. You can keep them warm in a low oven (about 200°F/90°C) while you cook the rest of the batch, or simply enjoy them as they come off the griddle!\n"
     ]
    }
   ],
   "source": [
    "prompt = \"can you give me the recipe for those pancakes?\"\n",
    "context = user_session.get_recent()\n",
    "response = client.converse(prompt=prompt, context=context)\n",
    "user_session.store(prompt, response)\n",
    "print('USER: ', prompt)\n",
    "print('\\nLLM: ', response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'role': 'system', 'content': 'You are a helpful chef, assisting people in making delicious meals'}\n",
      "{'role': 'user', 'content': 'can you give me some ideas for breakfast?'}\n",
      "{'role': 'llm', 'content': \"Sure! Here are a few breakfast ideas:\\n\\n- Avocado toast: This is a quick and easy option that's both delicious and nutritious. Simply toast some bread, smash an avocado on top, and add a squeeze of lemon or lime juice and a pinch of salt and pepper to taste. You can also add additional toppings like tomato, feta cheese, or a fried egg.\\n- Oatmeal: Oatmeal is a hearty and healthy breakfast option that can be prepared in a variety of ways. You can cook it on the stovetop or in the microwave, and top it with fruits, nuts, seeds, or a drizzle of honey.\\n- Smoothie bowl: Blend together your favorite fruits, vegetables, and liquid base (such as juice, milk, or yogurt) to make a smoothie, and then pour it into a bowl. Top it with granola, nuts, seeds, or fresh fruit for a beautiful and tasty breakfast.\\n- Eggs: There are endless ways to prepare eggs for breakfast. You can scramble them, fry them, poach them, or make an omelet with your favorite fillings. Serve them with toast, hash browns, or roasted vegetables for a filling and satisfying meal.\\n- Pancakes or waffles: Treat yourself to a stack of fluffy pancakes or waffles for a classic breakfast option. You can top them with butter and syrup, or get creative with fruit, whipped cream, or chocolate chips.\\n- Yogurt parfait: Layer yogurt, granola, and fresh fruit in a bowl or jar to make a delicious and beautiful yogurt parfait. This is a quick and easy option that can be customized to your taste.\\n\\nFeel free to experiment and get creative with your breakfast options! You can also try preparing larger batches of certain items (like oatmeal or pancakes) on the weekends, so you have leftovers to enjoy during the busy workweek.\"}\n"
     ]
    }
   ],
   "source": [
    "for ctx in context:\n",
    "    print(ctx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "USER:  I am vegetarian. Can you remove the eggs?\n",
      "\n",
      "LLM:  Absolutely! Here's an egg-free version of the fluffy pancake recipe:\n",
      "\n",
      "Ingredients:\n",
      "\n",
      "- 2 cups all-purpose flour\n",
      "- 2 teaspoons baking powder\n",
      "- 1/2 teaspoon salt\n",
      "- 2 tablespoons sugar\n",
      "- 2 cups milk (dairy or non-dairy alternative)\n",
      "- 2 tablespoons melted butter, plus more for the pan\n",
      "- 1 teaspoon vanilla extract (optional)\n",
      "\n",
      "Instructions:\n",
      "\n",
      "1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\n",
      "\n",
      "2. In a separate bowl, whisk together the milk, melted butter, and vanilla extract (if using).\n",
      "\n",
      "3. Pour the wet ingredients into the dry ingredients and stir until just combined. Again, be careful not to overmix the batter; a few lumps are okay.\n",
      "\n",
      "4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\n",
      "\n",
      "5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\n",
      "\n",
      "6. Carefully flip the pancakes and cook for an additional 1-2 minutes on the other side until golden brown.\n",
      "\n",
      "7. Repeat with the remaining batter, adding more butter to the pan as needed.\n",
      "\n",
      "8. Serve the pancakes warm with your favorite toppings.\n",
      "\n",
      "By eliminating the eggs, you may notice a slight difference in texture, but the pancakes should still turn out delicious and fluffy. Enjoy experimenting with different toppings and serving suggestions!\n"
     ]
    }
   ],
   "source": [
    "prompt =\"I am vegetarian. Can you remove the eggs?\"\n",
    "context = user_session.get_recent()\n",
    "response = client.converse(prompt=prompt, context=context)\n",
    "user_session.store(prompt, response)\n",
    "print('USER: ', prompt)\n",
    "print('\\nLLM: ', response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'role': 'system', 'content': 'You are a helpful chef, assisting people in making delicious meals'}\n",
      "{'role': 'user', 'content': 'can you give me some ideas for breakfast?'}\n",
      "{'role': 'llm', 'content': \"Sure! Here are a few breakfast ideas:\\n\\n- Avocado toast: This is a quick and easy option that's both delicious and nutritious. Simply toast some bread, smash an avocado on top, and add a squeeze of lemon or lime juice and a pinch of salt and pepper to taste. You can also add additional toppings like tomato, feta cheese, or a fried egg.\\n- Oatmeal: Oatmeal is a hearty and healthy breakfast option that can be prepared in a variety of ways. You can cook it on the stovetop or in the microwave, and top it with fruits, nuts, seeds, or a drizzle of honey.\\n- Smoothie bowl: Blend together your favorite fruits, vegetables, and liquid base (such as juice, milk, or yogurt) to make a smoothie, and then pour it into a bowl. Top it with granola, nuts, seeds, or fresh fruit for a beautiful and tasty breakfast.\\n- Eggs: There are endless ways to prepare eggs for breakfast. You can scramble them, fry them, poach them, or make an omelet with your favorite fillings. Serve them with toast, hash browns, or roasted vegetables for a filling and satisfying meal.\\n- Pancakes or waffles: Treat yourself to a stack of fluffy pancakes or waffles for a classic breakfast option. You can top them with butter and syrup, or get creative with fruit, whipped cream, or chocolate chips.\\n- Yogurt parfait: Layer yogurt, granola, and fresh fruit in a bowl or jar to make a delicious and beautiful yogurt parfait. This is a quick and easy option that can be customized to your taste.\\n\\nFeel free to experiment and get creative with your breakfast options! You can also try preparing larger batches of certain items (like oatmeal or pancakes) on the weekends, so you have leftovers to enjoy during the busy workweek.\"}\n",
      "{'role': 'user', 'content': 'can you give me the recipe for those pancakes?'}\n",
      "{'role': 'llm', 'content': \"Sure, here's a basic recipe for fluffy pancakes:\\n\\nIngredients:\\n\\n- 2 cups all-purpose flour\\n- 2 teaspoons baking powder\\n- 1/2 teaspoon salt\\n- 2 tablespoons sugar\\n- 2 cups milk\\n- 2 eggs\\n- 2 tablespoons melted butter, plus more for the pan\\n- 1 teaspoon vanilla extract (optional)\\n\\nInstructions:\\n\\n1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\\n\\n2. In a separate bowl, whisk together the milk, eggs, melted butter, and vanilla extract (if using).\\n\\n3. Pour the wet ingredients into the dry ingredients and stir until just combined. Be careful not to overmix the batter; a few lumps are okay.\\n\\n4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\\n\\n5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\\n\\n6. Flip the pancakes carefully and cook for an additional 1-2 minutes on the other side until golden brown.\\n\\n7. Repeat with the remaining batter, greasing the pan as needed.\\n\\n8. Serve the pancakes warm with your favorite toppings such as butter, syrup, fruit, whipped cream, or chocolate chips.\\n\\nThis recipe should make about 10-12 pancakes, depending on their size. You can keep them warm in a low oven (about 200°F/90°C) while you cook the rest of the batch, or simply enjoy them as they come off the griddle!\"}\n"
     ]
    }
   ],
   "source": [
    "for ctx in context:\n",
    "    print(ctx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "USER:  I am also vegan. Can you replace the butter too?\n",
      "\n",
      "LLM:  Sure, to make the recipe completely vegan, you can replace the butter with a vegan alternative. Here's the updated recipe:\n",
      "\n",
      "Ingredients:\n",
      "\n",
      "- 2 cups all-purpose flour\n",
      "- 2 teaspoons baking powder\n",
      "- 1/2 teaspoon salt\n",
      "- 2 tablespoons sugar\n",
      "- 2 cups plant-based milk (such as soy, almond, or oat milk)\n",
      "- 2 tablespoons neutral-flavored oil (such as canola or vegetable oil), plus more for the pan\n",
      "- 1 teaspoon vanilla extract (optional)\n",
      "\n",
      "Instructions:\n",
      "\n",
      "1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\n",
      "\n",
      "2. In a separate bowl, whisk together the plant-based milk, oil, and vanilla extract (if using).\n",
      "\n",
      "3. Pour the wet ingredients into the dry ingredients and stir until just combined. It's important not to overmix the batter; a few lumps are totally fine.\n",
      "\n",
      "4. Heat a nonstick pan or griddle over medium heat and lightly grease it with a small amount of oil.\n",
      "\n",
      "5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook the pancakes until you see small bubbles appearing on the surface and the edges look dry, which should take around 2-3 minutes.\n",
      "\n",
      "6. Carefully flip the pancakes and cook for an additional 1-2 minutes on the other side until they're golden brown.\n",
      "\n",
      "7. Repeat the process with the remaining batter, adding more oil to the pan as needed.\n",
      "\n",
      "8. Serve the pancakes warm with your choice of vegan toppings, such as maple syrup, fresh fruit, or vegan butter.\n",
      "\n",
      "By substituting plant-based milk and oil for dairy milk and butter, you've made these pancakes suitable for a vegan diet while still maintaining their delicious flavor and fluffy texture. Enjoy your vegan-friendly breakfast!\n"
     ]
    }
   ],
   "source": [
    "prompt = \"I am also vegan. Can you replace the butter too?\"\n",
    "context = user_session.get_recent()\n",
    "response = client.converse(prompt=prompt, context=context)\n",
    "user_session.store(prompt, response)\n",
    "print('USER: ', prompt)\n",
    "print('\\nLLM: ', response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'role': 'llm', 'content': \"Sure! Here are a few breakfast ideas:\\n\\n- Avocado toast: This is a quick and easy option that's both delicious and nutritious. Simply toast some bread, smash an avocado on top, and add a squeeze of lemon or lime juice and a pinch of salt and pepper to taste. You can also add additional toppings like tomato, feta cheese, or a fried egg.\\n- Oatmeal: Oatmeal is a hearty and healthy breakfast option that can be prepared in a variety of ways. You can cook it on the stovetop or in the microwave, and top it with fruits, nuts, seeds, or a drizzle of honey.\\n- Smoothie bowl: Blend together your favorite fruits, vegetables, and liquid base (such as juice, milk, or yogurt) to make a smoothie, and then pour it into a bowl. Top it with granola, nuts, seeds, or fresh fruit for a beautiful and tasty breakfast.\\n- Eggs: There are endless ways to prepare eggs for breakfast. You can scramble them, fry them, poach them, or make an omelet with your favorite fillings. Serve them with toast, hash browns, or roasted vegetables for a filling and satisfying meal.\\n- Pancakes or waffles: Treat yourself to a stack of fluffy pancakes or waffles for a classic breakfast option. You can top them with butter and syrup, or get creative with fruit, whipped cream, or chocolate chips.\\n- Yogurt parfait: Layer yogurt, granola, and fresh fruit in a bowl or jar to make a delicious and beautiful yogurt parfait. This is a quick and easy option that can be customized to your taste.\\n\\nFeel free to experiment and get creative with your breakfast options! You can also try preparing larger batches of certain items (like oatmeal or pancakes) on the weekends, so you have leftovers to enjoy during the busy workweek.\"}\n",
      "{'role': 'user', 'content': 'can you give me the recipe for those pancakes?'}\n",
      "{'role': 'llm', 'content': \"Sure, here's a basic recipe for fluffy pancakes:\\n\\nIngredients:\\n\\n- 2 cups all-purpose flour\\n- 2 teaspoons baking powder\\n- 1/2 teaspoon salt\\n- 2 tablespoons sugar\\n- 2 cups milk\\n- 2 eggs\\n- 2 tablespoons melted butter, plus more for the pan\\n- 1 teaspoon vanilla extract (optional)\\n\\nInstructions:\\n\\n1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\\n\\n2. In a separate bowl, whisk together the milk, eggs, melted butter, and vanilla extract (if using).\\n\\n3. Pour the wet ingredients into the dry ingredients and stir until just combined. Be careful not to overmix the batter; a few lumps are okay.\\n\\n4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\\n\\n5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\\n\\n6. Flip the pancakes carefully and cook for an additional 1-2 minutes on the other side until golden brown.\\n\\n7. Repeat with the remaining batter, greasing the pan as needed.\\n\\n8. Serve the pancakes warm with your favorite toppings such as butter, syrup, fruit, whipped cream, or chocolate chips.\\n\\nThis recipe should make about 10-12 pancakes, depending on their size. You can keep them warm in a low oven (about 200°F/90°C) while you cook the rest of the batch, or simply enjoy them as they come off the griddle!\"}\n",
      "{'role': 'user', 'content': 'I am vegetarian. Can you remove the eggs?'}\n",
      "{'role': 'llm', 'content': \"Absolutely! Here's an egg-free version of the fluffy pancake recipe:\\n\\nIngredients:\\n\\n- 2 cups all-purpose flour\\n- 2 teaspoons baking powder\\n- 1/2 teaspoon salt\\n- 2 tablespoons sugar\\n- 2 cups milk (dairy or non-dairy alternative)\\n- 2 tablespoons melted butter, plus more for the pan\\n- 1 teaspoon vanilla extract (optional)\\n\\nInstructions:\\n\\n1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\\n\\n2. In a separate bowl, whisk together the milk, melted butter, and vanilla extract (if using).\\n\\n3. Pour the wet ingredients into the dry ingredients and stir until just combined. Again, be careful not to overmix the batter; a few lumps are okay.\\n\\n4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\\n\\n5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\\n\\n6. Carefully flip the pancakes and cook for an additional 1-2 minutes on the other side until golden brown.\\n\\n7. Repeat with the remaining batter, adding more butter to the pan as needed.\\n\\n8. Serve the pancakes warm with your favorite toppings.\\n\\nBy eliminating the eggs, you may notice a slight difference in texture, but the pancakes should still turn out delicious and fluffy. Enjoy experimenting with different toppings and serving suggestions!\"}\n"
     ]
    }
   ],
   "source": [
    "for ctx in context:\n",
    "    print(ctx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### This works, but our context keeps growing, increasing our LLM token count, which increases latency and cost.\n",
    "#### Conversation histories can be truncated, but that can break quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "USER:  I changed my mind. Can you give me the first recipe from your list?\n",
      "\n",
      "LLM:  Sure, here's the first recipe for fluffy pancakes that I provided:\n",
      "\n",
      "Ingredients:\n",
      "\n",
      "- 2 cups all-purpose flour\n",
      "- 2 teaspoons baking powder\n",
      "- 1/2 teaspoon salt\n",
      "- 2 tablespoons sugar\n",
      "- 2 cups milk\n",
      "- 2 eggs\n",
      "- 2 tablespoons melted butter, plus more for the pan\n",
      "- 1 teaspoon vanilla extract (optional)\n",
      "\n",
      "Instructions:\n",
      "\n",
      "1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\n",
      "\n",
      "2. In a separate bowl, whisk together the milk, eggs, melted butter, and vanilla extract (if using).\n",
      "\n",
      "3. Pour the wet ingredients into the dry ingredients and stir until just combined. Be careful not to overmix the batter; a few lumps are okay and will ensure your pancakes are nice and fluffy.\n",
      "\n",
      "4. Heat a nonstick pan or griddle over medium heat. Grease it with a small amount of butter to prevent the pancakes from sticking.\n",
      "\n",
      "5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook the pancakes until the edges look dry and small bubbles form on the surface, which should take about 2-3 minutes.\n",
      "\n",
      "6. Flip the pancakes carefully and cook for an additional 1-2 minutes on the other side until they're golden brown.\n",
      "\n",
      "7. Repeat this process with the remaining batter, adding more butter to the pan as needed.\n",
      "\n",
      "8. Serve the pancakes warm with your favorite toppings such as butter, maple syrup, fresh fruit, or whipped cream.\n",
      "\n",
      "This classic recipe is a great starting point for fluffy pancakes, and you can always add your own twist with different mix-ins or toppings. Enjoy your breakfast!\n"
     ]
    }
   ],
   "source": [
    "prompt = \"I changed my mind. Can you give me the first recipe from your list?\"\n",
    "context = user_session.get_recent(top_k=5)\n",
    "response = client.converse(prompt=prompt, context=context)\n",
    "user_session.store(prompt, response)\n",
    "print('USER: ', prompt)\n",
    "print('\\nLLM: ', response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'role': 'llm', 'content': \"Sure, here's a basic recipe for fluffy pancakes:\\n\\nIngredients:\\n\\n- 2 cups all-purpose flour\\n- 2 teaspoons baking powder\\n- 1/2 teaspoon salt\\n- 2 tablespoons sugar\\n- 2 cups milk\\n- 2 eggs\\n- 2 tablespoons melted butter, plus more for the pan\\n- 1 teaspoon vanilla extract (optional)\\n\\nInstructions:\\n\\n1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\\n\\n2. In a separate bowl, whisk together the milk, eggs, melted butter, and vanilla extract (if using).\\n\\n3. Pour the wet ingredients into the dry ingredients and stir until just combined. Be careful not to overmix the batter; a few lumps are okay.\\n\\n4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\\n\\n5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\\n\\n6. Flip the pancakes carefully and cook for an additional 1-2 minutes on the other side until golden brown.\\n\\n7. Repeat with the remaining batter, greasing the pan as needed.\\n\\n8. Serve the pancakes warm with your favorite toppings such as butter, syrup, fruit, whipped cream, or chocolate chips.\\n\\nThis recipe should make about 10-12 pancakes, depending on their size. You can keep them warm in a low oven (about 200°F/90°C) while you cook the rest of the batch, or simply enjoy them as they come off the griddle!\"}\n",
      "{'role': 'user', 'content': 'I am vegetarian. Can you remove the eggs?'}\n",
      "{'role': 'llm', 'content': \"Absolutely! Here's an egg-free version of the fluffy pancake recipe:\\n\\nIngredients:\\n\\n- 2 cups all-purpose flour\\n- 2 teaspoons baking powder\\n- 1/2 teaspoon salt\\n- 2 tablespoons sugar\\n- 2 cups milk (dairy or non-dairy alternative)\\n- 2 tablespoons melted butter, plus more for the pan\\n- 1 teaspoon vanilla extract (optional)\\n\\nInstructions:\\n\\n1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\\n\\n2. In a separate bowl, whisk together the milk, melted butter, and vanilla extract (if using).\\n\\n3. Pour the wet ingredients into the dry ingredients and stir until just combined. Again, be careful not to overmix the batter; a few lumps are okay.\\n\\n4. Heat a nonstick pan or griddle over medium heat and grease it with a small amount of butter.\\n\\n5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook until the edges look dry and there are small bubbles on the surface, about 2-3 minutes.\\n\\n6. Carefully flip the pancakes and cook for an additional 1-2 minutes on the other side until golden brown.\\n\\n7. Repeat with the remaining batter, adding more butter to the pan as needed.\\n\\n8. Serve the pancakes warm with your favorite toppings.\\n\\nBy eliminating the eggs, you may notice a slight difference in texture, but the pancakes should still turn out delicious and fluffy. Enjoy experimenting with different toppings and serving suggestions!\"}\n",
      "{'role': 'user', 'content': 'I am also vegan. Can you replace the butter too?'}\n",
      "{'role': 'llm', 'content': \"Sure, to make the recipe completely vegan, you can replace the butter with a vegan alternative. Here's the updated recipe:\\n\\nIngredients:\\n\\n- 2 cups all-purpose flour\\n- 2 teaspoons baking powder\\n- 1/2 teaspoon salt\\n- 2 tablespoons sugar\\n- 2 cups plant-based milk (such as soy, almond, or oat milk)\\n- 2 tablespoons neutral-flavored oil (such as canola or vegetable oil), plus more for the pan\\n- 1 teaspoon vanilla extract (optional)\\n\\nInstructions:\\n\\n1. In a large bowl, whisk together the flour, baking powder, salt, and sugar until well combined.\\n\\n2. In a separate bowl, whisk together the plant-based milk, oil, and vanilla extract (if using).\\n\\n3. Pour the wet ingredients into the dry ingredients and stir until just combined. It's important not to overmix the batter; a few lumps are totally fine.\\n\\n4. Heat a nonstick pan or griddle over medium heat and lightly grease it with a small amount of oil.\\n\\n5. Pour about 1/4 cup of batter onto the pan for each pancake. Cook the pancakes until you see small bubbles appearing on the surface and the edges look dry, which should take around 2-3 minutes.\\n\\n6. Carefully flip the pancakes and cook for an additional 1-2 minutes on the other side until they're golden brown.\\n\\n7. Repeat the process with the remaining batter, adding more oil to the pan as needed.\\n\\n8. Serve the pancakes warm with your choice of vegan toppings, such as maple syrup, fresh fruit, or vegan butter.\\n\\nBy substituting plant-based milk and oil for dairy milk and butter, you've made these pancakes suitable for a vegan diet while still maintaining their delicious flavor and fluffy texture. Enjoy your vegan-friendly breakfast!\"}\n"
     ]
    }
   ],
   "source": [
    "for ctx in context:\n",
    "    print(ctx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The LLM has forgotten about the avocado toast it mentioned because that was too far back in it's conversation history!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Semantic session memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Rather than naively pass the most recent conversation we can use RedisVL to find only the semantically relevant sections of our conversation based on our new prompt. We can do this with the `fetch_relevant()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "USER:  Can you give me the avocado one?\n",
      "\n",
      "LLM:  Certainly! Here's a simple recipe for avocado toast:\n",
      "\n",
      "Ingredients:\n",
      "- 1 ripe avocado\n",
      "- 2 slices of your favorite bread\n",
      "- 1/2 lemon or lime\n",
      "- Salt and pepper to taste\n",
      "- Optional toppings: crushed red pepper flakes, chopped tomato, feta cheese, or a fried egg\n",
      "\n",
      "Directions:\n",
      "\n",
      "1. Toast the bread to your desired level.\n",
      "2. While the bread is toasting, cut the avocado in half, remove the pit, and scoop the flesh into a small bowl.\n",
      "3. Using a fork, mash the avocado until it reaches your desired consistency. You can leave it chunkier or make it smoother, depending on your preference.\n",
      "4. Add a squeeze of lemon or lime juice to the mashed avocado. This will help to brighten the flavor and prevent the avocado from browning too quickly.\n",
      "5. Add a pinch of salt and pepper to taste.\n",
      "6. Spread the avocado mixture onto the toasted bread.\n",
      "7. If desired, add additional toppings such as crushed red pepper flakes for a spicy kick, chopped tomato for freshness, feta cheese for a tangy flavor, or a fried egg for extra protein.\n",
      "8. Enjoy your delicious and nutritious avocado toast!\n",
      "\n",
      "Feel free to experiment with different types of bread and additional toppings to create your own unique version of avocado toast. It's a quick, easy, and healthy breakfast option that can be customized to your taste.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"Can you give me the avocado one?\"\n",
    "user_session.set_distance_threshold(0.75)\n",
    "context = user_session.get_relevant(prompt=prompt)\n",
    "response = client.converse(prompt=prompt, context=context)\n",
    "user_session.store(prompt, response)\n",
    "print('USER: ', prompt)\n",
    "print('\\nLLM: ', response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'role': 'llm', 'content': \"Sure! Here are a few breakfast ideas:\\n\\n- Avocado toast: This is a quick and easy option that's both delicious and nutritious. Simply toast some bread, smash an avocado on top, and add a squeeze of lemon or lime juice and a pinch of salt and pepper to taste. You can also add additional toppings like tomato, feta cheese, or a fried egg.\\n- Oatmeal: Oatmeal is a hearty and healthy breakfast option that can be prepared in a variety of ways. You can cook it on the stovetop or in the microwave, and top it with fruits, nuts, seeds, or a drizzle of honey.\\n- Smoothie bowl: Blend together your favorite fruits, vegetables, and liquid base (such as juice, milk, or yogurt) to make a smoothie, and then pour it into a bowl. Top it with granola, nuts, seeds, or fresh fruit for a beautiful and tasty breakfast.\\n- Eggs: There are endless ways to prepare eggs for breakfast. You can scramble them, fry them, poach them, or make an omelet with your favorite fillings. Serve them with toast, hash browns, or roasted vegetables for a filling and satisfying meal.\\n- Pancakes or waffles: Treat yourself to a stack of fluffy pancakes or waffles for a classic breakfast option. You can top them with butter and syrup, or get creative with fruit, whipped cream, or chocolate chips.\\n- Yogurt parfait: Layer yogurt, granola, and fresh fruit in a bowl or jar to make a delicious and beautiful yogurt parfait. This is a quick and easy option that can be customized to your taste.\\n\\nFeel free to experiment and get creative with your breakfast options! You can also try preparing larger batches of certain items (like oatmeal or pancakes) on the weekends, so you have leftovers to enjoy during the busy workweek.\"}\n",
      "{'role': 'user', 'content': 'can you give me some ideas for breakfast?'}\n"
     ]
    }
   ],
   "source": [
    "for ctx in context:\n",
    "    print(ctx)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Looking at the context we see that only the conversation section that had to do with avocados was retrieved. We don't need to know about pancakes to make avocado toast."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_session.clear()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}