examples/agents/inline_agent/inline-agent-api-usage.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Building Dynamic AI Assistants with Amazon Bedrock Inline Agents\n",
    "\n",
    "In this notebook, we'll walk through the process of setting up and invoking an inline agent, showcasing its flexibility and power in creating dynamic AI assistants. By following our progressive approach, you will gain a comprehensive understanding of how to use inline agents for various use cases and complexity levels. Throughout a single interactive conversation, we will demonstrate how the agent can be enhanced `on the fly` with new tools and instructions while maintaining context of our ongoing discussion.\n",
    "\n",
    "We'll follow a progressive approach to building our assistant:\n",
    "\n",
    "1. Simple Inline Agent: We'll start with a basic inline agent with a code interpreter.\n",
    "2. Adding Knowledge Bases: We'll enhance our agent by incorporating a knowledge base with role-based access.\n",
    "3. Integrating Action Groups: Finally, we'll add custom tools to extend the agent's functionality.\n",
    "\n",
    "## What are Inline Agents?\n",
    "\n",
    "[Inline agents](https://docs.aws.amazon.com/bedrock/latest/userguide/agents-create-inline.html) are a powerful feature of Amazon Bedrock that allow developers to create flexible and adaptable AI assistants. \n",
    "\n",
    "Unlike traditional static agents, inline agents can be dynamically configured at runtime, enabling real time adjustments to their behavior, capabilities, and knowledge base.\n",
    "\n",
    "Key features of inline agents include:\n",
    "\n",
    "1. **Dynamic configuration**: Modify the agent's instructions, action groups, and other parameters on the fly.\n",
    "2. **Flexible integration**: Easily incorporate external APIs and services as needed for each interaction.\n",
    "3. **Contextual adaptation**: Adjust the agent's responses based on user roles, preferences, or specific scenarios.\n",
    "\n",
    "## Why Use Inline Agents?\n",
    "\n",
    "Inline agents offer several advantages for building AI applications:\n",
    "\n",
    "1. **Rapid prototyping**: Quickly experiment with different configurations without redeploying your application.\n",
    "2. **Personalization**: Tailor the agent's capabilities to individual users or use cases in real time.\n",
    "3. **Scalability**: Efficiently manage a single agent that can adapt to multiple roles or functions.\n",
    "4. **Cost effectiveness**: Optimize resource usage by dynamically selecting only the necessary tools and knowledge for each interaction.\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "Before you begin, make sure that you have:\n",
    "\n",
    "1. An active AWS account with access to Amazon Bedrock.\n",
    "2. Necessary permissions to create and invoke inline agents.\n",
    "3. Be sure to complete additonal prerequisites, visit [Amazon Bedrock Inline Agent prerequisites documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/inline-agent-prereq.html) to learn more.\n",
    "\n",
    "### Installing prerequisites\n",
    "Let's begin with installing the required packages. This step is important as you need `boto3` version `1.35.68` or later to use inline agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# uncomment to install the required python packages\n",
    "!pip install --upgrade -r requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# # restart kernel\n",
    "from IPython.core.display import HTML\n",
    "HTML(\"<script>Jupyter.notebook.kernel.restart()</script>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Setup and Imports\n",
    "\n",
    "First, let's import the necessary libraries and set up our Bedrock client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "from pprint import pprint\n",
    "import boto3\n",
    "from datetime import datetime\n",
    "import random\n",
    "import pprint\n",
    "from termcolor import colored\n",
    "from rich.console import Console\n",
    "from rich.markdown import Markdown\n",
    "\n",
    "session = boto3.session.Session()\n",
    "region = session.region_name\n",
    "\n",
    "# Runtime Endpoints\n",
    "bedrock_rt_client = boto3.client(\n",
    "    \"bedrock-agent-runtime\",\n",
    "    region_name=region\n",
    ")\n",
    "\n",
    "sts_client = boto3.client(\"sts\")\n",
    "account_id = sts_client.get_caller_identity()[\"Account\"]\n",
    "\n",
    "# To manage session id:\n",
    "random_int = random.randint(1,100000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Configuring the Inline Agent\n",
    "\n",
    "Next, we'll set up the basic configuration for our Amazon Bedrock Inline Agent. This includes specifying the foundation model, session management, and basic instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# change model id as needed:\n",
    "model_id = \"anthropic.claude-3-sonnet-20240229-v1:0\"\n",
    "\n",
    "sessionId = f'custom-session-id-{random_int}'\n",
    "endSession = False\n",
    "enableTrace = True\n",
    "\n",
    "# customize instructions of inline agent:\n",
    "agent_instruction = \"\"\"You are a helpful AI assistant helping Octank Inc employees with their questions and processes. \n",
    "You write short and direct responses while being cheerful. You have access to python coding environment that helps you extend your capabilities.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Basic Inline Agent Invocation\n",
    "\n",
    "Let's start by invoking a simple inline agent with just the foundation model and basic instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# prepare request parameters before invoking inline agent\n",
    "request_params = {\n",
    "    \"instruction\": agent_instruction,\n",
    "    \"foundationModel\": model_id,\n",
    "    \"sessionId\": sessionId,\n",
    "    \"endSession\": endSession,\n",
    "    \"enableTrace\": enableTrace,\n",
    "}\n",
    "\n",
    "# define code interpreter tool\n",
    "code_interpreter_tool = {\n",
    "    \"actionGroupName\": \"UserInputAction\",\n",
    "    \"parentActionGroupSignature\": \"AMAZON.CodeInterpreter\"\n",
    "}\n",
    "\n",
    "# add the tool to request parameter of inline agent\n",
    "request_params[\"actionGroups\"] = [code_interpreter_tool]\n",
    "\n",
    "# enable traces\n",
    "request_params[\"enableTrace\"] = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# enter the question you want the inline agent to answer\n",
    "request_params['inputText'] = 'what is the time right now in pacific timezone?'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Invoking a simple Inline Agent\n",
    "\n",
    "We'll send a request to the agent asking it to perform a simple calculation or code execution task. This will showcase how the agent can interpret and run code on the fly.\n",
    "\n",
    "To do so, we will use the [InvokeInlineAgent](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeInlineAgent.html) API via boto3 `bedrock-agent-runtime` client.\n",
    "\n",
    "Our function `invoke_inline_agent_helper` also helps us processing the agent trace request and format it for easier readibility. You do not have to use this function in your system, but it will make it easier to observe the code used by code interpreter, the function invocations and the knowledge base content.\n",
    "\n",
    "We also provide the metrics for the agent invocation time and the input and output tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def invoke_inline_agent_helper(client, request_params, trace_level=\"core\"):\n",
    "    _time_before_call = datetime.now()\n",
    "\n",
    "    _agent_resp = client.invoke_inline_agent(\n",
    "        **request_params\n",
    "    )\n",
    "\n",
    "    if request_params[\"enableTrace\"]:\n",
    "        if trace_level == \"all\":\n",
    "            print(f\"invokeAgent API response object: {_agent_resp}\")\n",
    "        else:\n",
    "            print(\n",
    "                f\"invokeAgent API request ID: {_agent_resp['ResponseMetadata']['RequestId']}\"\n",
    "            )\n",
    "            session_id = request_params[\"sessionId\"]\n",
    "            print(f\"invokeAgent API session ID: {session_id}\")\n",
    "\n",
    "    # Return error message if invoke was unsuccessful\n",
    "    if _agent_resp[\"ResponseMetadata\"][\"HTTPStatusCode\"] != 200:\n",
    "        _error_message = f\"API Response was not 200: {_agent_resp}\"\n",
    "        if request_params[\"enableTrace\"] and trace_level == \"all\":\n",
    "            print(_error_message)\n",
    "        return _error_message\n",
    "\n",
    "    _total_in_tokens = 0\n",
    "    _total_out_tokens = 0\n",
    "    _total_llm_calls = 0\n",
    "    _orch_step = 0\n",
    "    _sub_step = 0\n",
    "    _trace_truncation_lenght = 300\n",
    "    _time_before_orchestration = datetime.now()\n",
    "\n",
    "    _agent_answer = \"\"\n",
    "    _event_stream = _agent_resp[\"completion\"]\n",
    "\n",
    "    try:\n",
    "        for _event in _event_stream:\n",
    "            _sub_agent_alias_id = None\n",
    "\n",
    "            if \"chunk\" in _event:\n",
    "                _data = _event[\"chunk\"][\"bytes\"]\n",
    "                _agent_answer = _data.decode(\"utf8\")\n",
    "\n",
    "            if \"trace\" in _event and request_params[\"enableTrace\"]:\n",
    "                if \"failureTrace\" in _event[\"trace\"][\"trace\"]:\n",
    "                    print(\n",
    "                        colored(\n",
    "                            f\"Agent error: {_event['trace']['trace']['failureTrace']['failureReason']}\",\n",
    "                            \"red\",\n",
    "                        )\n",
    "                    )\n",
    "\n",
    "                if \"orchestrationTrace\" in _event[\"trace\"][\"trace\"]:\n",
    "                    _orch = _event[\"trace\"][\"trace\"][\"orchestrationTrace\"]\n",
    "\n",
    "                    if trace_level in [\"core\", \"outline\"]:\n",
    "                        if \"rationale\" in _orch:\n",
    "                            _rationale = _orch[\"rationale\"]\n",
    "                            print(colored(f\"{_rationale['text']}\", \"blue\"))\n",
    "\n",
    "                        if \"invocationInput\" in _orch:\n",
    "                            # NOTE: when agent determines invocations should happen in parallel\n",
    "                            # the trace objects for invocation input still come back one at a time.\n",
    "                            _input = _orch[\"invocationInput\"]\n",
    "                            print(_input)\n",
    "\n",
    "                            if \"actionGroupInvocationInput\" in _input:\n",
    "                                if 'function' in _input['actionGroupInvocationInput']:\n",
    "                                    tool = _input['actionGroupInvocationInput']['function']\n",
    "                                elif 'apiPath' in _input['actionGroupInvocationInput']:\n",
    "                                    tool = _input['actionGroupInvocationInput']['apiPath']\n",
    "                                else:\n",
    "                                    tool = 'undefined'\n",
    "                                if trace_level == \"outline\":\n",
    "                                    print(\n",
    "                                        colored(\n",
    "                                            f\"Using tool: {tool}\",\n",
    "                                            \"magenta\",\n",
    "                                        )\n",
    "                                    )\n",
    "                                else:\n",
    "                                    print(\n",
    "                                        colored(\n",
    "                                            f\"Using tool: {tool} with these inputs:\",\n",
    "                                            \"magenta\",\n",
    "                                        )\n",
    "                                    )\n",
    "                                    if (\n",
    "                                        len(\n",
    "                                            _input[\"actionGroupInvocationInput\"][\n",
    "                                                \"parameters\"\n",
    "                                            ]\n",
    "                                        )\n",
    "                                        == 1\n",
    "                                    ) and (\n",
    "                                        _input[\"actionGroupInvocationInput\"][\n",
    "                                            \"parameters\"\n",
    "                                        ][0][\"name\"]\n",
    "                                        == \"input_text\"\n",
    "                                    ):\n",
    "                                        print(\n",
    "                                            colored(\n",
    "                                                f\"{_input['actionGroupInvocationInput']['parameters'][0]['value']}\",\n",
    "                                                \"magenta\",\n",
    "                                            )\n",
    "                                        )\n",
    "                                    else:\n",
    "                                        print(\n",
    "                                            colored(\n",
    "                                                f\"{_input['actionGroupInvocationInput']['parameters']}\\n\",\n",
    "                                                \"magenta\",\n",
    "                                            )\n",
    "                                        )\n",
    "\n",
    "                            elif \"codeInterpreterInvocationInput\" in _input:\n",
    "                                if trace_level == \"outline\":\n",
    "                                    print(\n",
    "                                        colored(\n",
    "                                            f\"Using code interpreter\", \"magenta\"\n",
    "                                        )\n",
    "                                    )\n",
    "                                else:\n",
    "                                    console = Console()\n",
    "                                    _gen_code = _input[\n",
    "                                        \"codeInterpreterInvocationInput\"\n",
    "                                    ][\"code\"]\n",
    "                                    _code = f\"```python\\n{_gen_code}\\n```\"\n",
    "\n",
    "                                    console.print(\n",
    "                                        Markdown(f\"**Generated code**\\n{_code}\")\n",
    "                                    )\n",
    "\n",
    "                        if \"observation\" in _orch:\n",
    "                            if trace_level == \"core\":\n",
    "                                _output = _orch[\"observation\"]\n",
    "                                if \"actionGroupInvocationOutput\" in _output:\n",
    "                                    print(\n",
    "                                        colored(\n",
    "                                            f\"--tool outputs:\\n{_output['actionGroupInvocationOutput']['text'][0:_trace_truncation_lenght]}...\\n\",\n",
    "                                            \"magenta\",\n",
    "                                        )\n",
    "                                    )\n",
    "\n",
    "                                if \"agentCollaboratorInvocationOutput\" in _output:\n",
    "                                    _collab_name = _output[\n",
    "                                        \"agentCollaboratorInvocationOutput\"\n",
    "                                    ][\"agentCollaboratorName\"]\n",
    "                                    _collab_output_text = _output[\n",
    "                                        \"agentCollaboratorInvocationOutput\"\n",
    "                                    ][\"output\"][\"text\"][0:_trace_truncation_lenght]\n",
    "                                    print(\n",
    "                                        colored(\n",
    "                                            f\"\\n----sub-agent {_collab_name} output text:\\n{_collab_output_text}...\\n\",\n",
    "                                            \"magenta\",\n",
    "                                        )\n",
    "                                    )\n",
    "\n",
    "                                if \"finalResponse\" in _output:\n",
    "                                    print(\n",
    "                                        colored(\n",
    "                                            f\"Final response:\\n{_output['finalResponse']['text'][0:_trace_truncation_lenght]}...\",\n",
    "                                            \"cyan\",\n",
    "                                        )\n",
    "                                    )\n",
    "\n",
    "\n",
    "                    if \"modelInvocationOutput\" in _orch:\n",
    "                        _orch_step += 1\n",
    "                        _sub_step = 0\n",
    "                        print(colored(f\"---- Step {_orch_step} ----\", \"green\"))\n",
    "\n",
    "                        _llm_usage = _orch[\"modelInvocationOutput\"][\"metadata\"][\n",
    "                            \"usage\"\n",
    "                        ]\n",
    "                        _in_tokens = _llm_usage[\"inputTokens\"]\n",
    "                        _total_in_tokens += _in_tokens\n",
    "\n",
    "                        _out_tokens = _llm_usage[\"outputTokens\"]\n",
    "                        _total_out_tokens += _out_tokens\n",
    "\n",
    "                        _total_llm_calls += 1\n",
    "                        _orch_duration = (\n",
    "                            datetime.now() - _time_before_orchestration\n",
    "                        )\n",
    "\n",
    "                        print(\n",
    "                            colored(\n",
    "                                f\"Took {_orch_duration.total_seconds():,.1f}s, using {_in_tokens+_out_tokens} tokens (in: {_in_tokens}, out: {_out_tokens}) to complete prior action, observe, orchestrate.\",\n",
    "                                \"yellow\",\n",
    "                            )\n",
    "                        )\n",
    "\n",
    "                        # restart the clock for next step/sub-step\n",
    "                        _time_before_orchestration = datetime.now()\n",
    "\n",
    "                elif \"preProcessingTrace\" in _event[\"trace\"][\"trace\"]:\n",
    "                    _pre = _event[\"trace\"][\"trace\"][\"preProcessingTrace\"]\n",
    "                    if \"modelInvocationOutput\" in _pre:\n",
    "                        _llm_usage = _pre[\"modelInvocationOutput\"][\"metadata\"][\n",
    "                            \"usage\"\n",
    "                        ]\n",
    "                        _in_tokens = _llm_usage[\"inputTokens\"]\n",
    "                        _total_in_tokens += _in_tokens\n",
    "\n",
    "                        _out_tokens = _llm_usage[\"outputTokens\"]\n",
    "                        _total_out_tokens += _out_tokens\n",
    "\n",
    "                        _total_llm_calls += 1\n",
    "\n",
    "                        print(\n",
    "                            colored(\n",
    "                                \"Pre-processing trace, agent came up with an initial plan.\",\n",
    "                                \"yellow\",\n",
    "                            )\n",
    "                        )\n",
    "                        print(\n",
    "                            colored(\n",
    "                                f\"Used LLM tokens, in: {_in_tokens}, out: {_out_tokens}\",\n",
    "                                \"yellow\",\n",
    "                            )\n",
    "                        )\n",
    "\n",
    "                elif \"postProcessingTrace\" in _event[\"trace\"][\"trace\"]:\n",
    "                    _post = _event[\"trace\"][\"trace\"][\"postProcessingTrace\"]\n",
    "                    if \"modelInvocationOutput\" in _post:\n",
    "                        _llm_usage = _post[\"modelInvocationOutput\"][\"metadata\"][\n",
    "                            \"usage\"\n",
    "                        ]\n",
    "                        _in_tokens = _llm_usage[\"inputTokens\"]\n",
    "                        _total_in_tokens += _in_tokens\n",
    "\n",
    "                        _out_tokens = _llm_usage[\"outputTokens\"]\n",
    "                        _total_out_tokens += _out_tokens\n",
    "\n",
    "                        _total_llm_calls += 1\n",
    "                        print(colored(\"Agent post-processing complete.\", \"yellow\"))\n",
    "                        print(\n",
    "                            colored(\n",
    "                                f\"Used LLM tokens, in: {_in_tokens}, out: {_out_tokens}\",\n",
    "                                \"yellow\",\n",
    "                            )\n",
    "                        )\n",
    "\n",
    "                if trace_level == \"all\":\n",
    "                    print(json.dumps(_event[\"trace\"], indent=2))\n",
    "\n",
    "            if \"files\" in _event.keys() and request_params[\"enableTrace\"]:\n",
    "                console = Console()\n",
    "                files_event = _event[\"files\"]\n",
    "                console.print(Markdown(\"**Files**\"))\n",
    "\n",
    "                files_list = files_event[\"files\"]\n",
    "                for this_file in files_list:\n",
    "                    print(f\"{this_file['name']} ({this_file['type']})\")\n",
    "                    file_bytes = this_file[\"bytes\"]\n",
    "\n",
    "                    # save bytes to file, given the name of file and the bytes\n",
    "                    file_name = os.path.join(\"output\", this_file[\"name\"])\n",
    "                    with open(file_name, \"wb\") as f:\n",
    "                        f.write(file_bytes)\n",
    "\n",
    "        if request_params[\"enableTrace\"]:\n",
    "            duration = datetime.now() - _time_before_call\n",
    "\n",
    "            if trace_level in [\"core\", \"outline\"]:\n",
    "                print(\n",
    "                    colored(\n",
    "                        f\"Agent made a total of {_total_llm_calls} LLM calls, \"\n",
    "                        + f\"using {_total_in_tokens+_total_out_tokens} tokens \"\n",
    "                        + f\"(in: {_total_in_tokens}, out: {_total_out_tokens})\"\n",
    "                        + f\", and took {duration.total_seconds():,.1f} total seconds\",\n",
    "                        \"yellow\",\n",
    "                    )\n",
    "                )\n",
    "\n",
    "            if trace_level == \"all\":\n",
    "                print(f\"Returning agent answer as: {_agent_answer}\")\n",
    "\n",
    "        return _agent_answer\n",
    "\n",
    "    except Exception as e:\n",
    "        print(f\"Caught exception while processing input to invokeAgent:\\n\")\n",
    "        input_text = request_params[\"inputText\"]\n",
    "        print(f\"  for input text:\\n{input_text}\\n\")\n",
    "        print(\n",
    "            f\"  request ID: {_agent_resp['ResponseMetadata']['RequestId']}, retries: {_agent_resp['ResponseMetadata']['RetryAttempts']}\\n\"\n",
    "        )\n",
    "        print(f\"Error: {e}\")\n",
    "        raise Exception(\"Unexpected exception: \", e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "invoke_inline_agent_helper(bedrock_rt_client, request_params, trace_level=\"core\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Adding a Knowledge Base\n",
    "\n",
    "Now, we'll demonstrate how to incorporate a knowledge base into our inline agent invocation. Let's first create a knowledge base using fictional HR policy documents that we will later use in with inline agent.\n",
    "\n",
    "We will use [Amazon Bedrock Knowledge Base](https://aws.amazon.com/bedrock/knowledge-bases/) to create our knowledge base. To do so, we use the support function `create_knowledge_base` available in the `create_knowledge_base.py` file. It will abstract away the work to create the underline vector database, the vector indexes with the appropriated chunking strategy as well as the indexation of the documents to the knowledge base. Take a look at the `create_knowledge_base.py` file for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from create_knowledge_base import create_knowledge_base\n",
    "\n",
    "# Configuration\n",
    "bucket_name = f\"inline-agent-bucket-{random_int}\"\n",
    "kb_name = f\"policy-kb-{random_int}\"\n",
    "data_path = \"policy_documents\"\n",
    "\n",
    "# Create knowledge base and upload documents\n",
    "kb_id, bucket_name, kb_metadata = create_knowledge_base(region, bucket_name, kb_name, data_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Setting up Knowledge Base configuration to invoke inline agent\n",
    "\n",
    "Let's now set up the knowledge base configuration to invoke our inline agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# define number of chunks to retrieve\n",
    "num_results = 3\n",
    "search_strategy = \"HYBRID\"\n",
    "\n",
    "# provide instructions about knowledge base that inline agent can use\n",
    "kb_description = 'This knowledge base contains information about company HR policies, code or conduct, performance reviews and much more'\n",
    "\n",
    "# lets define access level for metadata filtering\n",
    "user_profile = 'basic'\n",
    "access_filter = {\n",
    "    \"equals\": {\n",
    "        \"key\": \"access_level\",\n",
    "        \"value\": user_profile\n",
    "    }\n",
    "}\n",
    "\n",
    "# lets revise our Knowledge bases configuration\n",
    "kb_config = {\n",
    "    \"knowledgeBaseId\": kb_id,\n",
    "    \"description\": kb_description,\n",
    "    \"retrievalConfiguration\": {\n",
    "        \"vectorSearchConfiguration\": {\n",
    "            \"filter\": access_filter,\n",
    "            \"numberOfResults\": num_results,\n",
    "            \"overrideSearchType\": \"HYBRID\"\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "# lets add knowledge bases to our request parameters\n",
    "request_params[\"knowledgeBases\"] = [kb_config]\n",
    "    \n",
    "# update the agent instructions to inform inline agent that it has access to a knowlegde base\n",
    "new_capabilities = \"\"\"You have access to Octank Inc company policies knowledge base. \n",
    "Use this database to search for information about company policies, company HR policies, code or conduct, performance reviews and much more. And use them to briefly answer the use question.\"\"\"\n",
    "request_params[\"instruction\"] += f\"\\n\\n{new_capabilities}\"\n",
    "\n",
    "# check updated request parameters including instructions for the inline agent\n",
    "print(request_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Querying the Enhanced Agent\n",
    "\n",
    "We'll send a query that requires the agent to retrieve information from the knowledge base and provide an informed response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# enter the question that will use knowledge bases\n",
    "request_params['inputText'] = 'How much is the employee compensation bonus?'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# invoke the inline agent\n",
    "invoke_inline_agent_helper(bedrock_rt_client, request_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Analyzing the Knowledge Base Integration\n",
    "\n",
    "We see that there are two types of access levels defined in the knowledge base, basic and manager. Compensation related access is `Manager` only. Let's try the same query with proper filter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# lets define access level for metadata filtering\n",
    "user_profile = 'Manager'\n",
    "# user_profile = 'basic'\n",
    "access_filter = {\n",
    "    \"equals\": {\n",
    "        \"key\": \"access_level\",\n",
    "        \"value\": user_profile\n",
    "    }\n",
    "}\n",
    "\n",
    "# lets revise our Knowledge bases configuration\n",
    "kb_config = {\n",
    "    \"knowledgeBaseId\": kb_id,\n",
    "    \"description\": kb_description,\n",
    "    \"retrievalConfiguration\": {\n",
    "        \"vectorSearchConfiguration\": {\n",
    "            \"filter\": access_filter,\n",
    "            \"numberOfResults\": num_results,\n",
    "            \"overrideSearchType\": \"HYBRID\"\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "# lets add knowledge bases to our request parameters\n",
    "request_params[\"knowledgeBases\"] = [kb_config]\n",
    "\n",
    "# invoke the inline agent\n",
    "invoke_inline_agent_helper(bedrock_rt_client, request_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Integrating Action Groups\n",
    "\n",
    "In this section, we'll show how to add a custom tool (action group) to our agent invocation. This illustrates how to extend the agent's functionality with external services via the API.\n",
    "\n",
    "Let's first create a lambda function that we will later use in with inline agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# run lambda function creation\n",
    "from lambda_creator import create_lambda_function_and_its_resources\n",
    "import os\n",
    "\n",
    "present_directory = os.getcwd()\n",
    "lambda_function_code_path = str(present_directory) + \"/pto_lambda/lambda_function.py\"\n",
    "\n",
    "# Create all resources\n",
    "resources = create_lambda_function_and_its_resources(\n",
    "    region=region,\n",
    "    account_id=account_id,\n",
    "    custom_name=f\"hr-inlineagent-lambda-{random_int}\",\n",
    "    lambda_code_path=lambda_function_code_path\n",
    ")\n",
    "\n",
    "# Access the created resources\n",
    "lambda_function = resources['lambda_function']\n",
    "lambda_function_arn = lambda_function['FunctionArn']\n",
    "print(lambda_function_arn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Configuring the Agent with the Action Group\n",
    "\n",
    "We'll update our agent configuration to include the new action group, allowing it to interact with the external service.\n",
    "For this example we are providing an OpenAPI Schema to define our action group tools. You can also use function definition to do the same, but your lambda function even will change a bit. For more information see the documentation [here](https://docs.aws.amazon.com/bedrock/latest/userguide/action-define.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "apply_vacation_tool = {\n",
    "            'actionGroupName': 'FetchDetails',\n",
    "            \"actionGroupExecutor\": {\n",
    "                \"lambda\": lambda_function_arn\n",
    "            }, \"apiSchema\": {\n",
    "                \"payload\": \"\"\"\n",
    "    {\n",
    "    \"openapi\": \"3.0.0\",\n",
    "    \"info\": {\n",
    "        \"title\": \"Vacation Management API\",\n",
    "        \"version\": \"1.0.0\",\n",
    "        \"description\": \"API for managing vacation requests\"\n",
    "    },\n",
    "    \"paths\": {\n",
    "        \"/vacation\": {\n",
    "            \"post\": {\n",
    "                \"summary\": \"Process vacation request\",\n",
    "                \"description\": \"Process a vacation request or check balance\",\n",
    "                \"operationId\": \"processVacation\",\n",
    "                \"parameters\": [\n",
    "                    {\n",
    "                        \"name\": \"action\",\n",
    "                        \"in\": \"query\",\n",
    "                        \"description\": \"The type of vacation action to perform\",\n",
    "                        \"required\": true,\n",
    "                        \"schema\": {\n",
    "                            \"type\": \"string\",\n",
    "                            \"enum\": [\"check_balance\", \"check balance\", \"apply\", \"request\"],\n",
    "                            \"description\": \"Action type for vacation management\"\n",
    "                        }\n",
    "                    },\n",
    "                    {\n",
    "                        \"name\": \"days\",\n",
    "                        \"in\": \"query\",\n",
    "                        \"description\": \"Number of vacation days requested\",\n",
    "                        \"required\": false,\n",
    "                        \"schema\": {\n",
    "                            \"type\": \"integer\",\n",
    "                            \"minimum\": 1\n",
    "                        }\n",
    "                    }\n",
    "                ],\n",
    "                \"responses\": {\n",
    "                    \"200\": {\n",
    "                        \"description\": \"Request processed successfully\",\n",
    "                        \"content\": {\n",
    "                            \"application/json\": {\n",
    "                                \"schema\": {\n",
    "                                    \"type\": \"object\",\n",
    "                                    \"properties\": {\n",
    "                                        \"status\": {\n",
    "                                            \"type\": \"string\",\n",
    "                                            \"enum\": [\"approved\", \"pending\", \"rejected\", \"info\"],\n",
    "                                            \"description\": \"Status of the vacation request\"\n",
    "                                        },\n",
    "                                        \"message\": {\n",
    "                                            \"type\": \"string\",\n",
    "                                            \"description\": \"Detailed response message\"\n",
    "                                        },\n",
    "                                        \"ticket_url\": {\n",
    "                                            \"type\": \"string\",\n",
    "                                            \"description\": \"Ticket URL for long vacation requests\"\n",
    "                                        }\n",
    "                                    },\n",
    "                                    \"required\": [\"status\", \"message\"]\n",
    "                                }\n",
    "                            }\n",
    "                        }\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}\n",
    "    \"\"\"\n",
    "            },\n",
    "            \"description\": \"Process vacation and check leave balance\"\n",
    "}\n",
    "            \n",
    "# update the tools that inline agent has access to\n",
    "request_params[\"actionGroups\"] = [code_interpreter_tool, apply_vacation_tool]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Testing the Full Featured Agent\n",
    "\n",
    "We'll send a complex query that requires the agent to use its language understanding, access the knowledge base, and interact with the external service via the action group.\n",
    "\n",
    "### Analyzing the Complete Agent Behavior\n",
    "\n",
    "We'll examine the agent's response, focusing on how it orchestrates different capabilities (language model, knowledge base, and external actions) to handle complex queries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# ask question:\n",
    "request_params['inputText'] = 'I will be out of office from 2024/11/28 for the next 3 days'\n",
    "\n",
    "# invoke the inline agent\n",
    "invoke_inline_agent_helper(bedrock_rt_client, request_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Clean up\n",
    "\n",
    "Let's delete the resources that were created in this notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "lambda_client = boto3.client('lambda')\n",
    "iam_client = boto3.client('iam')\n",
    "\n",
    "def delete_iam_roles_and_policies(role_name, iam_client):\n",
    "    try:\n",
    "        iam_client.get_role(RoleName=role_name)\n",
    "    except iam_client.exceptions.NoSuchEntityException:\n",
    "        print(f\"Role {role_name} does not exist\") \n",
    "    attached_policies = iam_client.list_attached_role_policies(RoleName=role_name)[\"AttachedPolicies\"]\n",
    "    print(f\"======Attached policies with role {role_name}========\\n\", attached_policies)\n",
    "    for attached_policy in attached_policies:\n",
    "        policy_arn = attached_policy[\"PolicyArn\"]\n",
    "        policy_name = attached_policy[\"PolicyName\"]\n",
    "        iam_client.detach_role_policy(RoleName=role_name, PolicyArn=policy_arn)\n",
    "        print(f\"Detached policy {policy_name} from role {role_name}\")\n",
    "        if str(policy_arn.split(\"/\")[1]) == \"service-role\":\n",
    "            print(f\"Skipping deletion of service-linked role policy {policy_name}\")\n",
    "        else: \n",
    "            iam_client.delete_policy(PolicyArn=policy_arn)\n",
    "            print(f\"Deleted policy {policy_name} from role {role_name}\")\n",
    "\n",
    "    iam_client.delete_role(RoleName=role_name)\n",
    "    print(f\"Deleted role {role_name}\")\n",
    "    print(\"======== All IAM roles and policies deleted =========\")\n",
    "    \n",
    "# delete lambda function\n",
    "response = lambda_client.delete_function(\n",
    "    FunctionName=resources['lambda_function']['FunctionName']\n",
    ")\n",
    "# delete lamnda role and policy\n",
    "delete_iam_roles_and_policies(resources['lambda_role']['Role']['RoleName'], iam_client)\n",
    "# delete knowledge base\n",
    "kb_metadata.delete_kb(delete_s3_bucket=True, delete_iam_roles_and_policies=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "This notebook has demonstrated the key aspects of using the Amazon Bedrock Inline Agents API:\n",
    "\n",
    "1. Basic agent invocation\n",
    "2. Incorporating knowledge bases\n",
    "3. Adding custom action groups\n",
    "\n",
    "By leveraging these API capabilities, developers can create dynamic, adaptable AI assistants that can be easily customized for various use cases without redeploying applications.\n",
    "\n",
    "Key takeaways:\n",
    "1. Inline agents offer great flexibility through their API\n",
    "2. Knowledge bases and action groups can be easily integrated\n"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "conda_pytorch_p310",
   "language": "python",
   "name": "conda_pytorch_p310"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}