00-playground/47-reasoning-with-o1/L6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FbWoGOis4KoG"
   },
   "source": [
    "# Lesson 6: Metaprompting with o1\n",
    "\n",
    "Optimizing to production is one of the biggest painpoints we've seen developers experience with working with LLMs - with so much guidance for prompt engineering, RAG and fine-tuning out there, figuring out which optimization you need to hill-climb on your evals can be a difficult problem to frame and solve.\n",
    "\n",
    "Luckily, it appears to be one of the use cases that `o1` is capable at. In this session we'll focus on how to use `o1-mini` to work with a set of evals to optimize our prompt for the task and improve score on our evals.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"background-color:#fff6ff; padding:13px; border-width:3px; border-color:#efe6ef; border-style:solid; border-radius:6px\">\n",
    "<p> 💻 &nbsp; <b>Access <code>requirements.txt</code> and <code>helper.py</code> files:</b> 1) click on the <em>\"File\"</em> option on the top menu of the notebook and then 2) click on <em>\"Open\"</em>.\n",
    "\n",
    "<p> ⬇ &nbsp; <b>Download Notebooks:</b> 1) click on the <em>\"File\"</em> option on the top menu of the notebook and then 2) click on <em>\"Download as\"</em> and select <em>\"Notebook (.ipynb)\"</em>.</p>\n",
    "\n",
    "<p> 📒 &nbsp; For more help, please see the <em>\"Appendix – Tips, Help, and Download\"</em> Lesson.</p>\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"background-color:#f7fff8; padding:15px; border-width:3px; border-color:#e0f0e0; border-style:solid; border-radius:6px\"> 🚨\n",
    "&nbsp; <b>Different Run Results:</b> The output generated by AI models can vary with each execution due to their dynamic, probabilistic nature. Don't be surprised if your results differ from those shown in the video.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "height": 132
   },
   "outputs": [],
   "source": [
    "# Warning control\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "# Import OpenAI key\n",
    "from helper import get_openai_api_key\n",
    "openai_api_key = get_openai_api_key()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "height": 336
   },
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "from IPython.display import display, Markdown\n",
    "import pandas as pd\n",
    "from concurrent.futures import ThreadPoolExecutor\n",
    "from functionDefinitions import TOOLS\n",
    "import csv\n",
    "import json\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "GPT_MODEL = 'gpt-4o-mini'\n",
    "O1_MODEL = 'o1'\n",
    "\n",
    "client = OpenAI(api_key=openai_api_key)\n",
    "\n",
    "pd.set_option('display.max_colwidth', None)\n",
    "pd.set_option('display.max_rows', None)\n",
    "pd.set_option('display.max_columns', None)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "height": 268
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "This is the <span style='color:red;font-weight:700;text-decoration:line-through;'>original </span><span style='color:green;font-weight:700;'>modified </span>text <span style='color:green;font-weight:700;'>to compare </span>for <span style='color:red;font-weight:700;text-decoration:line-through;'>comparison purposes.</span><span style='color:green;font-weight:700;'>differences.</span>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from redlines import Redlines\n",
    "from IPython.display import display, Markdown\n",
    "\n",
    "def compare_strings(string1, string2):\n",
    "    # Create a Redlines object to compare the strings\n",
    "    diff = Redlines(string1, string2)\n",
    "    \n",
    "    # Display the differences using Markdown\n",
    "    display(Markdown(diff.output_markdown))\n",
    "\n",
    "# Example usage\n",
    "string_a = \"This is the original text for comparison purposes.\"\n",
    "string_b = \"This is the modified text to compare for differences.\"\n",
    "\n",
    "compare_strings(string_a, string_b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Generate 4o routine (prompt)\n",
    "\n",
    "We'll take the Flight Cancellation Policy that we have created and convert it to an LLM-based routine with the following prompt.\n",
    "\n",
    "This process is also a good use of O1, using a purely text document to create instructions and tools for an LLM to make use of. To dive deeper on this topic check out [this cookbook](https://cookbook.openai.com/examples/o1/using_reasoning_for_routine_generation)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "height": 81
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# **Internal Flight Cancellations and Changes Policy**\n",
      "\n",
      "**Purpose**: This document serves as a detailed guide for internal support agents to handle flight cancellations and changes. The focus is on providing clear instructions, ensuring efficiency, consistency, and customer satisfaction during calls.\n",
      "\n",
      "**Note**: Always maintain a calm, empathetic tone while assisting customers, especially during stressful situations involving cancellations or major changes.\n",
      "\n",
      "## **Table of Contents**\n",
      "\n",
      "1. General Guidelines for Handling Customer Requests  \n",
      "2. Cancellations: Types and Policies  \n",
      "3. Changes: Types and Policies  \n",
      "4. Rebooking Guidelines  \n",
      "5. Compensation and Refund Rules  \n",
      "6. Special Cases  \n",
      "7. FAQs for Common Scenarios\n",
      "\n",
      "### **1\\. General Guidelines for Handling Customer Requests**\n",
      "\n",
      "* **Confirm Identity**: Verify the customer's identity by asking for their booking reference and any additional required details (e.g., name and flight number).  \n",
      "* **Listen and Understand**: Clarify if the customer is looking to cancel, change, or inquire about compensation.  \n",
      "* **Check the Ticket Type**: Determine if the ticket is non-refundable, refundable, or flexible. This will affect the available options.  \n",
      "* **Be Empathetic**: Express empathy, especially when the customer is dealing with a disruption caused by the airline.\n",
      "* **Rely on Internal Data**: Only collect personal data from users (e.g. name, medical data, age, booking reference). All data regarding flights (such as fare type or refund amount) should be confirmed internally. \n",
      "\n",
      "### **2\\. Cancellations: Types and Policies**\n",
      "\n",
      "#### **2.1 Customer-Initiated Cancellations**\n",
      "\n",
      "* **Refundable Tickets**: Eligible for a full or partial refund based on cancellation timing.  \n",
      "  * **Within 24 hours of booking**: Full refund guaranteed.  \n",
      "  * **Beyond 24 hours**: Check the specific fare rules for applicable refund amounts.  \n",
      "* **Non-Refundable Tickets**:  \n",
      "  * **Flight Credit**: Offer flight credit for future use, subject to an administration fee.  \n",
      "  * **Penalty Fee**: Advise customers of any penalty fee applicable.\n",
      "\n",
      "#### **2.2 Airline-Initiated Cancellations**\n",
      "\n",
      "* **Weather-Related Cancellations**:  \n",
      "  * **Rebook Without Fees**: Allow the customer to rebook on the next available flight without any additional charges.  \n",
      "  * **Refund Options**: If the rebooking option is not acceptable, offer a full refund.  \n",
      "* **Mechanical Issues/Operational Changes**:  \n",
      "  * **Rebooking Priority**: Give affected customers priority for rebooking.  \n",
      "  * **Accommodation**: Offer hotel accommodation if the delay requires an overnight stay (for cancellations within 6 hours of departure).\n",
      "\n",
      "#### **2.3 No-Show Policy**\n",
      "\n",
      "* **No Refund for Non-Refundable Tickets**: No refunds are issued for no-shows; however, travel credits may be applied in certain cases.  \n",
      "* **Partial Refund for Refundable Tickets**: Refund any taxes or airport fees where applicable.\n",
      "\n",
      "### **3\\. Changes: Types and Policies**\n",
      "\n",
      "#### **3.1 Customer-Initiated Changes**\n",
      "\n",
      "* **Same-Day Changes**:  \n",
      "  * **Flexible Tickets**: Allow changes without any fees, subject to availability.  \n",
      "  * **Non-Flexible Tickets**: Charge a change fee. The customer must also pay any fare difference.  \n",
      "* **Changes in Advance**:  \n",
      "  * **Within 7 Days of Departure**: Standard change fee applies.  \n",
      "  * **Beyond 7 Days**: Lesser fee or free changes based on ticket type.\n",
      "\n",
      "#### **3.2 Airline-Initiated Changes**\n",
      "\n",
      "* **Schedule Changes**:  \n",
      "  * **Minor Changes (Less than 2 Hours)**: Inform the customer of the new schedule and offer an alternative only upon request.  \n",
      "  * **Major Changes (2+ Hours)**: Provide options to rebook, receive a full refund, or travel credit.\n",
      "\n",
      "### **4\\. Rebooking Guidelines**\n",
      "\n",
      "* **Priority Handling**: Rebook the customer on the next available flight operated by our airline first.  \n",
      "* **Interline Agreements**: If no suitable options are available, check our interline partners to facilitate alternative connections.  \n",
      "* **Upgrade Availability**: Offer a complimentary upgrade when an available flight is in a higher class but no economy seat is free.\n",
      "\n",
      "### **5\\. Compensation and Refund Rules**\n",
      "\n",
      "#### **5.1 Refund Processing**\n",
      "\n",
      "* **Refund Timeline**: Inform the customer that refunds will be processed within 7-10 business days for credit card payments and 14-20 business days for debit card payments.  \n",
      "* **Refund Options**:  \n",
      "  * **Original Payment Method**: Refunds will generally be credited back to the original method of payment.  \n",
      "  * **Travel Credits**: Offer travel credits as an alternative, with a bonus percentage (e.g., 10%) as an incentive.\n",
      "\n",
      "#### **5.2 Compensation**\n",
      "\n",
      "* **EU/US Compensation Rules**: Depending on the region, inform customers of their right to compensation based on the length of delay and cause.  \n",
      "  * **Meals and Accommodation**: If the delay exceeds four hours, provide meal vouchers or hotel accommodation.\n",
      "\n",
      "### **6\\. Special Cases**\n",
      "\n",
      "#### **6.1 Medical Emergencies**\n",
      "\n",
      "* **Flexible Cancellations**: Allow for full cancellation or flight credit when a medical certificate ID is provided.  \n",
      "* **Flight Changes**: Waive change fees if the customer or immediate family member is ill.\n",
      "\n",
      "#### **6.2 Bereavement Cases**\n",
      "\n",
      "* **Special Flexibility**: Offer flexibility on cancellations or changes for bereavement. Request proper documentation if needed.\n",
      "\n",
      "#### **6.3 Group Bookings**\n",
      "\n",
      "* **Partial Cancellations**: Allow individual cancellations within a group booking, but advise that this may affect the group rate.  \n",
      "* **Name Changes**: Permit one name change per passenger at no additional cost if made 7+ days before departure.\n",
      "\n",
      "#### **6.4 Unaccompanied Minors**\n",
      "\n",
      "* **Rebooking with Supervision**: Ensure proper supervision is arranged if changes are needed to flights involving unaccompanied minors.  \n",
      "* **Priority for Minors**: Always prioritize getting unaccompanied minors on the next available flight.\n",
      "\n",
      "### **7\\. FAQs for Common Scenarios**\n",
      "\n",
      "**Q1: What if a customer’s connecting flight is affected?**\n",
      "\n",
      "* **Policy**: Always prioritize rebooking for missed connections caused by airline delays. Ensure the customer is protected for onward flights without additional fees.\n",
      "\n",
      "**Q2: How do I handle customers seeking upgrades after a cancellation?**\n",
      "\n",
      "* **Policy**: Offer upgrades based on availability if the original fare class is sold out. Otherwise, advise on the fare difference.\n",
      "\n",
      "**Q3: What if a customer needs to change a destination?**\n",
      "\n",
      "* **Policy**: Allow changes to a different destination, but advise the customer of any applicable fare differences and change fees.\n",
      "\n",
      "**Q4: What are the rules around rebooking for third-party bookings?**\n",
      "\n",
      "* **Policy**: Request that the customer contacts the original booking agent (e.g., travel agency, online booking platform) to make changes. Provide assistance if the booking is eligible for direct airline handling.\n",
      "\n",
      "**Q5: What should I do if the customer requests compensation for inconvenience?**\n",
      "\n",
      "* **Policy**: Assess eligibility based on the airline's compensation policy. Inform the customer about meal vouchers, accommodation, or monetary compensation if applicable.\n",
      "\n",
      "**Q6: How do I handle a situation where a customer’s preferred flight is fully booked?**\n",
      "\n",
      "* **Policy**: Offer the customer the next available option and place them on a waitlist for their preferred flight if possible. Consider offering an upgrade to a higher class if available.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('originalPolicy/flightCancellationsPolicy.md', 'r') as file:\n",
    "    flight_cancellation_policy = file.read()\n",
    "    print(flight_cancellation_policy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "height": 540
   },
   "outputs": [],
   "source": [
    "CONVERSION_PROMPT = f\"\"\"\n",
    "You are a helpful assistant tasked with taking an external facing help center article and converting it into a internal-facing programmatically executable routine optimized for an LLM. \n",
    "The LLM using this routine will be tasked with reading the policy, answering incoming questions from customers, and helping drive the case toward resolution.\n",
    "\n",
    "Please follow these instructions:\n",
    "1. **Review the customer service policy carefully** to ensure every step is accounted for. It is crucial not to skip any steps or policies.\n",
    "2. **Organize the instructions into a logical, step-by-step order**, using the specified format. \n",
    "3. **Use the following format**:\n",
    "   - **Main actions are numbered** (e.g., 1, 2, 3).\n",
    "   - **Sub-actions are lettered** under their relevant main actions (e.g., 1a, 1b).\n",
    "      **Sub-actions should start on new lines**\n",
    "   - **Specify conditions using clear 'if...then...else' statements** (e.g., 'If the product was purchased within 30 days, then...').\n",
    "   - **For instructions that require more information from the customer**, provide polite and professional prompts to ask for additional information.\n",
    "   - **For actions that require data from external systems**, write a step to call a function using backticks for the function name (e.g., call the `check_delivery_date` function).\n",
    "      - **If a step requires the customer service agent to take an action** (e.g., process a refund), generate a function call for this action (e.g., call the `process_refund` function).\n",
    "      - **Only use the available set of functions that are defined below.\n",
    "   - **If there is an action an assistant can perform on behalf of the user**, include a function call for this action (e.g., call the `change_email_address` function), and ensure the function is defined with its purpose and required parameters.\n",
    "      - **Only use the available set of functions that are defined below.\n",
    "   - **The step prior to case resolution should always be to ask if there is anything more you can assist with**.\n",
    "   - **End with a final action for case resolution**: calling the `case_resolution` function should always be the final step.\n",
    "4. **Ensure compliance** by making sure all steps adhere to company policies, privacy regulations, and legal requirements.\n",
    "5. **Handle exceptions or escalations** by specifying steps for scenarios that fall outside the standard policy.\n",
    "6. **Ensure coverage** by checking that all of the conditions covered in the policy are also covered in the routines\n",
    "\n",
    "**Important**: Always wrap the functions you return in backticks i.e. `check_ticket_type`. Do not include the arguments to the functions.\n",
    "\n",
    "Here are the currently available set of functions in JSON format: \n",
    "TOOLS: {TOOLS}\n",
    "\n",
    "Please convert the following customer service policy into the formatted routine, ensuring it is easy to follow and execute programmatically. Ensure that you **only** use the functions provided and do **not** create net new functions.\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "height": 404
   },
   "outputs": [],
   "source": [
    "def generate_routine(policy):\n",
    "    try:\n",
    "        messages = [\n",
    "            {\n",
    "                \"role\": \"user\",\n",
    "                \"content\": f\"\"\"\n",
    "                    {CONVERSION_PROMPT}\n",
    "\n",
    "                    POLICY:\n",
    "                    {policy}\n",
    "                \"\"\"\n",
    "            }\n",
    "        ]\n",
    "\n",
    "        response = client.chat.completions.create(\n",
    "            model='o1-mini',\n",
    "            messages=messages\n",
    "        )\n",
    "        \n",
    "\n",
    "        return response.choices[0].message.content \n",
    "    except Exception as e:\n",
    "        print(f\"An error occurred: {e}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "flight_cancellation_routine = generate_routine(flight_cancellation_policy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "```markdown\n",
       "1. Verify Customer Identity\n",
       "   a. Call `verify_identity`\n",
       "      - Prompt the customer to provide their booking reference, full name, and flight number.\n",
       "\n",
       "2. Listen and Understand Customer Request\n",
       "   a. Call `ask_clarification` with prompt: \"How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?\"\n",
       "\n",
       "3. Determine Customer Request Type\n",
       "   a. If the customer wants to **cancel**:\n",
       "      i. Proceed to step 4.\n",
       "   b. Else if the customer wants to **change**:\n",
       "      i. Proceed to step 6.\n",
       "   c. Else if the customer wants to **inquire about compensation**:\n",
       "      i. Proceed to step 12.\n",
       "   d. Else:\n",
       "      i. Call `ask_clarification` with prompt: \"I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?\"\n",
       "\n",
       "4. Handle Cancellations\n",
       "   a. Call `check_ticket_type` to determine if the ticket is refundable, non-refundable, or flexible.\n",
       "   b. If ticket is **refundable**:\n",
       "      i. If cancellation is **within 24 hours of booking**:\n",
       "         - Call `process_full_refund`.\n",
       "      ii. Else:\n",
       "         - Call `check_fare_rules`.\n",
       "         - If partial refund is applicable:\n",
       "            - Call `process_partial_refund`.\n",
       "         - Else:\n",
       "            - Call `process_full_refund`.\n",
       "   c. Else if ticket is **non-refundable**:\n",
       "      i. Call `offer_flight_credit`.\n",
       "      ii. Inform the customer about the administration fee.\n",
       "      iii. Call `determine_refund_method`.\n",
       "      iv. If customer opts for travel credit:\n",
       "         - Call `apply_travel_credit`.\n",
       "      v. Else:\n",
       "         - Inform about penalty fees.\n",
       "   d. Else if ticket is **flexible**:\n",
       "      i. Proceed as refundable ticket.\n",
       "   e. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   f. Call `case_resolution`.\n",
       "\n",
       "5. Handle Airline-Initiated Cancellations\n",
       "   a. If cancellation is **weather-related**:\n",
       "      i. Call `rebook_without_fee`.\n",
       "      ii. If rebooking is not acceptable:\n",
       "         - Call `process_full_refund`.\n",
       "   b. Else if cancellation is due to **mechanical issues/operational changes**:\n",
       "      i. Call `prioritize_missed_connections`.\n",
       "      ii. If overnight stay is required:\n",
       "         - Call `offer_accommodation`.\n",
       "   c. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   d. Call `case_resolution`.\n",
       "\n",
       "6. Handle Changes\n",
       "   a. Call `check_ticket_type` to determine if the ticket is flexible or non-flexible.\n",
       "   b. If the change is **customer-initiated**:\n",
       "      i. If **same-day change**:\n",
       "         - If ticket is **flexible**:\n",
       "            * Call `rebook_without_fee`.\n",
       "         - Else:\n",
       "            * Call `apply_change_fee`.\n",
       "            * Call `check_fare_rules`.\n",
       "            * If fare difference exists:\n",
       "               - Inform the customer about the additional fare.\n",
       "      ii. Else if **changes in advance**:\n",
       "         - If **within 7 days of departure**:\n",
       "            * Call `apply_change_fee`.\n",
       "         - Else:\n",
       "            * Call `apply_lesser_change_fee`.\n",
       "   c. Else if the change is **airline-initiated**:\n",
       "      i. If schedule change is **minor (less than 2 hours)**:\n",
       "         - Inform the customer of the new schedule.\n",
       "         - If customer requests alternative:\n",
       "            * Call `provide_alternative_options`.\n",
       "      ii. Else if schedule change is **major (2+ hours)**:\n",
       "         - Offer options to `rebook_without_fee`, `process_full_refund`, or `offer_flight_credit`.\n",
       "   d. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   e. Call `case_resolution`.\n",
       "\n",
       "7. Rebooking Guidelines\n",
       "   a. Call `check_next_available_flight`.\n",
       "   b. If no suitable flight available:\n",
       "      i. Call `check_interline_partners`.\n",
       "      ii. If interline partners have available flights:\n",
       "         - Call `book_interline_flight`.\n",
       "      iii. Else:\n",
       "         - Inform the customer and offer alternative solutions.\n",
       "   c. If a higher class is available:\n",
       "      i. Call `offer_available_upgrades`.\n",
       "   d. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   e. Call `case_resolution`.\n",
       "\n",
       "8. Compensation and Refund Processing\n",
       "   a. Inform the customer about the refund timeline based on payment method.\n",
       "   b. Call `determine_refund_method`.\n",
       "      i. If refund to **original payment method**:\n",
       "         - Call `process_refund`.\n",
       "      ii. Else if offering **travel credits**:\n",
       "         - Call `apply_travel_credit`.\n",
       "   c. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   d. Call `case_resolution`.\n",
       "\n",
       "9. Handle Compensation Requests\n",
       "   a. Call `assess_compensation_request`.\n",
       "   b. If eligible for compensation:\n",
       "      i. Offer meal vouchers or hotel accommodation if delay exceeds four hours.\n",
       "      ii. Inform about monetary compensation if applicable.\n",
       "   c. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   d. Call `case_resolution`.\n",
       "\n",
       "10. Handle Special Cases\n",
       "    a. If **medical emergency**:\n",
       "       i. Call `process_flexible_cancellation` with medical certificate.\n",
       "       ii. Call `waive_change_fees` if applicable.\n",
       "    b. If **bereavement case**:\n",
       "       i. Call `apply_bereavement_flexibility` with documentation.\n",
       "    c. If **group booking**:\n",
       "       i. Call `process_partial_group_cancellation` for individual cancellations.\n",
       "       ii. If name change is requested:\n",
       "          - If made 7+ days before departure:\n",
       "             * Call `permit_name_change`.\n",
       "    d. If **unaccompanied minor**:\n",
       "       i. Call `arrange_supervision` for flight changes.\n",
       "       ii. Call `prioritize_minors_rebooking`.\n",
       "    e. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "    f. Call `case_resolution`.\n",
       "\n",
       "11. Handle FAQs\n",
       "    a. For **connecting flight affected**:\n",
       "       i. Call `prioritize_missed_connections`.\n",
       "    b. For **upgrade requests after cancellation**:\n",
       "       i. Call `offer_available_upgrades`.\n",
       "    c. For **destination changes**:\n",
       "       i. Call `process_destination_change`.\n",
       "    d. For **third-party bookings**:\n",
       "       i. Call `assist_third_party_booking`.\n",
       "    e. For **compensation requests**:\n",
       "       i. Call `assess_compensation_request`.\n",
       "    f. If **preferred flight is fully booked**:\n",
       "       i. Offer next available option.\n",
       "       ii. If possible, place on waitlist or offer upgrade.\n",
       "    g. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "    h. Call `case_resolution`.\n",
       "\n",
       "12. Finalize Case\n",
       "    a. Ensure all customer requests have been addressed.\n",
       "    b. Call `case_resolution`.\n",
       "```"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(flight_cancellation_routine))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data quality check\n",
    "\n",
    "We'll check that `o1-mini` has followed our instructions and used only the functions we provided.\n",
    "\n",
    "If it added additional ones, we'll need to write functions for them before proceeding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "height": 676
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Items in A that are not in B (or more of in A):\n",
      ": 1\n",
      "\n",
      "Items in B that are not in A (or more of in B):\n",
      "process_change_no_fee: 1\n",
      "book_flight: 1\n",
      "check_compensation_eligibility: 1\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "from collections import Counter\n",
    "\n",
    "def compare_lists(a, b):\n",
    "    # Create Counters for both lists\n",
    "    counter_a = Counter(a)\n",
    "    counter_b = Counter(b)\n",
    "    \n",
    "    # Find elements that are only in A or have a higher count in A\n",
    "    diff_a = counter_a - counter_b\n",
    "    # Find elements that are only in B or have a higher count in B\n",
    "    diff_b = counter_b - counter_a\n",
    "    \n",
    "    # Report differences\n",
    "    print(\"Items in A that are not in B (or more of in A):\")\n",
    "    for item, count in diff_a.items():\n",
    "        print(f\"{item}: {count}\")\n",
    "\n",
    "    print(\"\\nItems in B that are not in A (or more of in B):\")\n",
    "    for item, count in diff_b.items():\n",
    "        print(f\"{item}: {count}\")\n",
    "\n",
    "def extract_function_names(text):\n",
    "    # Use regex to match text between backticks and extract function names\n",
    "    pattern = r'`(.*?)`'\n",
    "    matches = re.findall(pattern, text)\n",
    "    return matches\n",
    "\n",
    "# Extract a unique list of the functions in the generated routine\n",
    "function_names_from_o1 = set(extract_function_names(flight_cancellation_routine))\n",
    "\n",
    "# Extract the list of TOOLs we provided the model\n",
    "function_names_defined = [tool[\"function\"][\"name\"] for tool in TOOLS if tool[\"type\"] == \"function\"]\n",
    "\n",
    "# Print the differences\n",
    "## Items that are in A and not in B need functions written for them\n",
    "## Items that are in B and not in A are fine, they are just unused in the routine the model has written\n",
    "compare_lists(function_names_from_o1, function_names_defined)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Evaluation\n",
    "\n",
    "Now that we have a routine generated with o1, we can run it against our evaluation suite and measure its accuracy.\n",
    "\n",
    "We'll start by creating an agent that is equipped with the policy and a list of tools. It will be given messages from an existing conversation and will be tasked with determining the next best action to take"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "height": 472
   },
   "outputs": [],
   "source": [
    "def agent_response(transcript, policy, model):\n",
    "    try:\n",
    "        messages = [\n",
    "            {\n",
    "                \"role\": \"system\",\n",
    "                \"content\": f\"\"\"\n",
    "You are a customer service agent that is responsible for handling airline related issues. Below is the exact policy that you must follow to address the customer's issue.\n",
    "\n",
    "POLICY:\n",
    "{policy}\n",
    "                \"\"\"\n",
    "            }\n",
    "        ]\n",
    "\n",
    "        messages.extend(transcript)\n",
    "        response = client.chat.completions.create(\n",
    "            model=model,\n",
    "            messages=messages,\n",
    "            tools=TOOLS,\n",
    "            parallel_tool_calls=False,\n",
    "        temperature=0)\n",
    "        \n",
    "        return response.choices[0].message \n",
    "    except Exception as e:\n",
    "        print(f\"An error occurred: {e}\")\n",
    "        print(messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will process each row in parallel to reduce runtime and compare the function call + inputs that the model selects against our expected function + parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "height": 3991
   },
   "outputs": [],
   "source": [
    "import math \n",
    "\n",
    "def safe_str(value):\n",
    "    return \"\" if (isinstance(value, float) and math.isnan(value)) else str(value)\n",
    "\n",
    "def process_row(row_number,test_row,policy,model,iteration=0, verbose=False):\n",
    "    # Extract variables from test_row\n",
    "    context = test_row['context']\n",
    "    request = test_row['request']\n",
    "    full_name = test_row['full_name']\n",
    "    booking_ref = test_row['booking_ref']\n",
    "    flight_no = test_row['flight_no']\n",
    "    ticket_type = test_row['ticket_type']\n",
    "    fare_rules = safe_str(test_row['fare_rules'])\n",
    "    medical_certificate = safe_str(test_row['medical_certificate'])\n",
    "    refund_amount = safe_str(test_row['refund_amount'])\n",
    "    expected_function = test_row['expected_function']\n",
    "    expected_inputs_str = test_row['expected_inputs']\n",
    "    expected_inputs = json.loads(expected_inputs_str)\n",
    "\n",
    "    CUSTOMER_PROMPT = f\"\"\"<objective>You are a customer named {full_name} trying to resolve an issue with a customer service agent.\n",
    "    Continue providing the information the agent requests so they can solve your issue.\n",
    "    If you have the information in your *details* then use that. Otherwise, generate an appropriate answer.</objective>\n",
    "    <details>\n",
    "    Booking Reference: {booking_ref}\n",
    "    Flight Number: {flight_no}\n",
    "    {f\"Medical Reference: {medical_certificate}\" if medical_certificate else \"\"}\n",
    "    Context: {context}\n",
    "    Your initial request: {request}\n",
    "    </details>\n",
    "    <guidance>\n",
    "    Remember that YOU are the customer.\n",
    "    If the agent needs some information, that must come from you. \n",
    "    Do not ask the agent for information.\n",
    "    </guidance>\n",
    "    \"\"\"\n",
    "\n",
    "    # Initialize transcript\n",
    "    transcript = [{\"role\": \"user\", \"content\": request}]\n",
    "    if verbose:\n",
    "        print(f'Starting request: {request}')\n",
    "    \n",
    "    # Initialize variables\n",
    "    is_correct = False\n",
    "\n",
    "    loop_count = 0\n",
    "    while True:\n",
    "        loop_count += 1\n",
    "        if loop_count > 10:\n",
    "            actual_function = None\n",
    "            actual_inputs = None\n",
    "            break\n",
    "        # Call agent_response with the current transcript, policy, model\n",
    "        response = agent_response(transcript, policy, GPT_MODEL)\n",
    "        \n",
    "        # Extract assistant message content and tool_calls\n",
    "        if response:\n",
    "            assistant_message_content = response.content\n",
    "            tool_calls = response.tool_calls\n",
    "        else:\n",
    "            assistant_message_content = None\n",
    "            tool_calls = None\n",
    "        \n",
    "        if not tool_calls:\n",
    "            # Append assistant's message to transcript\n",
    "            transcript.append({\"role\": \"assistant\", \"content\": assistant_message_content})\n",
    "            \n",
    "            # Get customer response\n",
    "            customer_messages = [{\"role\": \"system\", \"content\": CUSTOMER_PROMPT}]\n",
    "            customer_messages.extend(transcript)\n",
    "            \n",
    "            customer_response = client.chat.completions.create(model=GPT_MODEL, messages=customer_messages)\n",
    "            \n",
    "            # Append customer response to transcript\n",
    "            transcript.append({\"role\": \"user\", \"content\": customer_response.choices[0].message.content})\n",
    "            \n",
    "            # Continue the loop\n",
    "            continue\n",
    "        else:\n",
    "            #print(tool_calls)\n",
    "            # There is a tool call\n",
    "            tool_call = tool_calls[0]  # Assuming we're only interested in the first tool call\n",
    "            function_name = tool_call.function.name\n",
    "            arguments = json.loads(tool_call.function.arguments)\n",
    "            \n",
    "            if function_name == 'verify_identity':\n",
    "                # Append assistant's message to transcript\n",
    "                transcript.append(response)\n",
    "                \n",
    "                # Simulate the tool response\n",
    "                tool_response = \"True\"\n",
    "                \n",
    "                # Append the tool response to transcript\n",
    "                transcript.append({\"role\": \"tool\", \"content\": tool_response, \"tool_call_id\": tool_call.id})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            elif function_name == 'ask_clarification':\n",
    "                # Take the 'prompt' from arguments and use it as assistant message\n",
    "                clarification_prompt = arguments.get('prompt')\n",
    "                \n",
    "                # Append assistant's clarification message\n",
    "                transcript.append({\"role\": \"assistant\", \"content\": clarification_prompt})\n",
    "                \n",
    "                # Get customer response\n",
    "                customer_messages = [{\"role\": \"system\", \"content\": CUSTOMER_PROMPT}]\n",
    "                customer_messages.extend(transcript)\n",
    "                \n",
    "                customer_response = client.chat.completions.create(model=GPT_MODEL, messages=customer_messages)\n",
    "                \n",
    "                # Append customer response to transcript\n",
    "                transcript.append({\"role\": \"user\", \"content\": customer_response.choices[0].message.content})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            elif function_name == 'check_ticket_type':\n",
    "                # Append assistant's message to transcript\n",
    "                transcript.append(response)\n",
    "                \n",
    "                tool_response = f'Ticket type: {ticket_type}\\nFare rules: {fare_rules}'\n",
    "                \n",
    "                # Append the tool response to transcript\n",
    "                transcript.append({\"role\": \"tool\", \"content\": tool_response, \"tool_call_id\": tool_call.id})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            elif function_name == 'check_fare_rules':\n",
    "                # Append assistant's message to transcript\n",
    "                transcript.append(response)\n",
    "                \n",
    "                tool_response = f'Ticket type: {ticket_type}\\nFare rules: {fare_rules}'\n",
    "\n",
    "                if fare_rules == 'partial_refund':\n",
    "                    tool_response += f'\\nRefund amount: {refund_amount}'\n",
    "                \n",
    "                # Append the tool response to transcript\n",
    "                transcript.append({\"role\": \"tool\", \"content\": tool_response, \"tool_call_id\": tool_call.id})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            elif function_name == 'get_refund_amount':\n",
    "                # Append assistant's message to transcript\n",
    "                transcript.append(response)\n",
    "                \n",
    "                tool_response = f'Refund amount: {refund_amount}'\n",
    "                \n",
    "                # Append the tool response to transcript\n",
    "                transcript.append({\"role\": \"tool\", \"content\": tool_response, \"tool_call_id\": tool_call.id})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            elif function_name == 'check_next_available_flight':\n",
    "                # Append assistant's message to transcript\n",
    "                transcript.append(response)\n",
    "                \n",
    "                # simulating the tool response\n",
    "                tool_response = f'Next available flight: LMG091 at 10:00 AM Tomorrow'\n",
    "                \n",
    "                # Append the tool response to transcript\n",
    "                transcript.append({\"role\": \"tool\", \"content\": tool_response, \"tool_call_id\": tool_call.id})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            elif function_name == 'provide_alternative_options':\n",
    "                # Append assistant's message to transcript\n",
    "                transcript.append(response)\n",
    "                \n",
    "                # simulating the tool response\n",
    "                tool_response = f'Next available flight: FR9876 at 5:00 PM Tomorrow'\n",
    "                \n",
    "                # Append the tool response to transcript\n",
    "                transcript.append({\"role\": \"tool\", \"content\": tool_response, \"tool_call_id\": tool_call.id})\n",
    "                \n",
    "                # Continue the loop\n",
    "                continue\n",
    "            else:\n",
    "                if verbose:\n",
    "                    print('Got a successful function call')\n",
    "                # Stop and compare function call and arguments with expected_function and expected_inputs\n",
    "                transcript.append(response)\n",
    "                actual_function = function_name\n",
    "                actual_inputs = arguments\n",
    "                is_correct = (actual_function == expected_function) and (str(actual_inputs) == str(expected_inputs))\n",
    "                \n",
    "                # Break the loop\n",
    "                break\n",
    "    \n",
    "    # Return the results\n",
    "    return {\n",
    "        'request': request,\n",
    "        'context': context,\n",
    "        'expected_function': expected_function,\n",
    "        'expected_inputs': expected_inputs,\n",
    "        'actual_function': actual_function,\n",
    "        'actual_inputs': actual_inputs,\n",
    "        'is_correct': is_correct,\n",
    "        'transcript': transcript\n",
    "    }\n",
    "\n",
    "# added post-filming to address transcript size\n",
    "def filter_messages(message_list):\n",
    "    messages = []\n",
    "    for item in message_list:\n",
    "        # Convert the item to a string and check if 'ChatCompletionMessage' is in it.\n",
    "        if 'ChatCompletionMessage' in str(item):\n",
    "            #print(item)\n",
    "            message = {\"role\": \"assistant\",\n",
    "                       \"tool_call\": {\n",
    "                           \"name\": item.tool_calls[0].function.name,\n",
    "                           \"arguments\": item.tool_calls[0].function.arguments\n",
    "                       }\n",
    "                      }\n",
    "            messages.append(message)\n",
    "        else:\n",
    "            messages.append(item)\n",
    "    return messages\n",
    "\n",
    "\n",
    "def evaluate_function_calls(df, policy, model, i=0, verbose=False):\n",
    "    records = []\n",
    "\n",
    "    # Use ThreadPoolExecutor to process rows in parallel\n",
    "    with ThreadPoolExecutor() as executor:\n",
    "        futures = {executor.submit(process_row, row_number, row, policy, model, i, verbose): row_number for row_number, row in df.iterrows()}\n",
    "        for future in futures:\n",
    "            record = future.result()\n",
    "            records.append(record)\n",
    "\n",
    "\n",
    "    df = pd.DataFrame(records)\n",
    "    df['cleaned_transcript'] = df['transcript'].apply(filter_messages)\n",
    "    total_accuracy = df['is_correct'].mean()\n",
    "    return df, total_accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "height": 47
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>context</th>\n",
       "      <th>request</th>\n",
       "      <th>full_name</th>\n",
       "      <th>booking_ref</th>\n",
       "      <th>flight_no</th>\n",
       "      <th>expected_function</th>\n",
       "      <th>expected_inputs</th>\n",
       "      <th>ticket_type</th>\n",
       "      <th>fare_rules</th>\n",
       "      <th>medical_certificate</th>\n",
       "      <th>refund_amount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>I want to cancel my flight as I am initiating the cancellation myself.</td>\n",
       "      <td>I would like to cancel my flight, please.</td>\n",
       "      <td>John Doe</td>\n",
       "      <td>ABC123</td>\n",
       "      <td>XYZ789</td>\n",
       "      <td>process_partial_refund</td>\n",
       "      <td>{\"booking_reference\":\"ABC123\",\"refund_amount\":198}</td>\n",
       "      <td>refundable</td>\n",
       "      <td>partial_refund</td>\n",
       "      <td>NaN</td>\n",
       "      <td>198.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>I want to change my flight to a different date.</td>\n",
       "      <td>I'd like to initiate a change to my flight for a later date.</td>\n",
       "      <td>Jane Smith</td>\n",
       "      <td>DEF456</td>\n",
       "      <td>XYZ123</td>\n",
       "      <td>apply_change_fee</td>\n",
       "      <td>{\"booking_reference\":\"DEF456\"}</td>\n",
       "      <td>nonrefundable</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>I want to request a refund because my flight was canceled by the airline due to weather.</td>\n",
       "      <td>My flight was canceled by the airline, and I want a refund.</td>\n",
       "      <td>Emma Johnson</td>\n",
       "      <td>GHI789</td>\n",
       "      <td>ABC123</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{\"booking_reference\":\"GHI789\"}</td>\n",
       "      <td>refundable</td>\n",
       "      <td>full_refund</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>I want to cancel my flight but I think it is non-refundable.</td>\n",
       "      <td>I want to cancel my non-refundable ticket.</td>\n",
       "      <td>Michael Brown</td>\n",
       "      <td>JKL012</td>\n",
       "      <td>DEF456</td>\n",
       "      <td>offer_flight_credit</td>\n",
       "      <td>{\"booking_reference\":\"JKL012\"}</td>\n",
       "      <td>nonrefundable</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>I want to cancel my flight and I believe it is refundable.</td>\n",
       "      <td>I want to cancel my refundable ticket.</td>\n",
       "      <td>Sarah Brown</td>\n",
       "      <td>MNO345</td>\n",
       "      <td>STU678</td>\n",
       "      <td>process_partial_refund</td>\n",
       "      <td>{\"booking_reference\":\"MNO345\",\"refund_amount\":65}</td>\n",
       "      <td>refundable</td>\n",
       "      <td>partial_refund</td>\n",
       "      <td>NaN</td>\n",
       "      <td>65.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ID  \\\n",
       "0   0   \n",
       "1   1   \n",
       "2   2   \n",
       "3   3   \n",
       "4   4   \n",
       "\n",
       "                                                                                    context  \\\n",
       "0                    I want to cancel my flight as I am initiating the cancellation myself.   \n",
       "1                                           I want to change my flight to a different date.   \n",
       "2  I want to request a refund because my flight was canceled by the airline due to weather.   \n",
       "3                              I want to cancel my flight but I think it is non-refundable.   \n",
       "4                                I want to cancel my flight and I believe it is refundable.   \n",
       "\n",
       "                                                        request  \\\n",
       "0                     I would like to cancel my flight, please.   \n",
       "1  I'd like to initiate a change to my flight for a later date.   \n",
       "2   My flight was canceled by the airline, and I want a refund.   \n",
       "3                    I want to cancel my non-refundable ticket.   \n",
       "4                        I want to cancel my refundable ticket.   \n",
       "\n",
       "       full_name booking_ref flight_no       expected_function  \\\n",
       "0       John Doe      ABC123    XYZ789  process_partial_refund   \n",
       "1     Jane Smith      DEF456    XYZ123        apply_change_fee   \n",
       "2   Emma Johnson      GHI789    ABC123     process_full_refund   \n",
       "3  Michael Brown      JKL012    DEF456     offer_flight_credit   \n",
       "4    Sarah Brown      MNO345    STU678  process_partial_refund   \n",
       "\n",
       "                                      expected_inputs    ticket_type  \\\n",
       "0  {\"booking_reference\":\"ABC123\",\"refund_amount\":198}     refundable   \n",
       "1                      {\"booking_reference\":\"DEF456\"}  nonrefundable   \n",
       "2                      {\"booking_reference\":\"GHI789\"}     refundable   \n",
       "3                      {\"booking_reference\":\"JKL012\"}  nonrefundable   \n",
       "4   {\"booking_reference\":\"MNO345\",\"refund_amount\":65}     refundable   \n",
       "\n",
       "       fare_rules medical_certificate  refund_amount  \n",
       "0  partial_refund                 NaN          198.0  \n",
       "1             NaN                 NaN            NaN  \n",
       "2     full_refund                 NaN            NaN  \n",
       "3             NaN                 NaN            NaN  \n",
       "4  partial_refund                 NaN           65.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eval_df = pd.read_csv('evals/policyEvals.csv')\n",
    "eval_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "height": 353
   },
   "outputs": [],
   "source": [
    "from termcolor import colored\n",
    "\n",
    "def display_row(index):\n",
    "    eval_row = eval_df.iloc[index]\n",
    "    df_row = df.iloc[index][[\"actual_function\", \"actual_inputs\", \"is_correct\", \"cleaned_transcript\"]]\n",
    "    transcript_list = df_row[\"cleaned_transcript\"]\n",
    "\n",
    "    print(\"Eval DataFrame Row:\")\n",
    "    print(eval_row)\n",
    "    print(\"\\nDataFrame Columns:\")\n",
    "    print(\"actual_function:\", df_row[\"actual_function\"])\n",
    "    print(\"actual_inputs:\", df_row[\"actual_inputs\"])\n",
    "    print(\"is_correct:\", df_row[\"is_correct\"])\n",
    "    print(\"transcript:\")\n",
    "    for entry in transcript_list:\n",
    "        role = entry[\"role\"] if isinstance(entry, dict) else getattr(entry, \"role\", None)\n",
    "        color = \"blue\" if role == \"user\" else \"red\"\n",
    "        print(colored(str(entry), color))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "height": 132
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "### Accuracy: 58.82%"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>request</th>\n",
       "      <th>context</th>\n",
       "      <th>expected_function</th>\n",
       "      <th>expected_inputs</th>\n",
       "      <th>actual_function</th>\n",
       "      <th>actual_inputs</th>\n",
       "      <th>is_correct</th>\n",
       "      <th>transcript</th>\n",
       "      <th>cleaned_transcript</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I would like to cancel my flight, please.</td>\n",
       "      <td>I want to cancel my flight as I am initiating the cancellation myself.</td>\n",
       "      <td>process_partial_refund</td>\n",
       "      <td>{'booking_reference': 'ABC123', 'refund_amount': 198}</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'ABC123'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to cancel my flight, please.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight, please. My booking reference is ABC123, and the flight number is XYZ789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ansPc0MDF53FmKJBcELp4EZD', function=Function(arguments='{\"booking_reference\":\"ABC123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ansPc0MDF53FmKJBcELp4EZD'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_NQvw6o5ufa6LyNHMutEd8D7H', function=Function(arguments='{\"booking_reference\":\"ABC123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_NQvw6o5ufa6LyNHMutEd8D7H'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_0wVEPVZqo0Uy8tLD6e3XTdk4', function=Function(arguments='{\"booking_reference\":\"ABC123\"}', name='process_full_refund'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to cancel my flight, please.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight, please. My booking reference is ABC123, and the flight number is XYZ789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"ABC123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ansPc0MDF53FmKJBcELp4EZD'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_NQvw6o5ufa6LyNHMutEd8D7H'}, {'role': 'assistant', 'tool_call': {'name': 'process_full_refund', 'arguments': '{\"booking_reference\":\"ABC123\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>I'd like to initiate a change to my flight for a later date.</td>\n",
       "      <td>I want to change my flight to a different date.</td>\n",
       "      <td>apply_change_fee</td>\n",
       "      <td>{'booking_reference': 'DEF456'}</td>\n",
       "      <td>apply_change_fee</td>\n",
       "      <td>{'booking_reference': 'DEF456'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I'd like to initiate a change to my flight for a later date.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is DEF456, my name is Jane Smith, and my flight number is XYZ123.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ABMvPrO3nO7J5oPVVnp3HgrW', function=Function(arguments='{\"booking_reference\":\"DEF456\",\"full_name\":\"Jane Smith\",\"flight_number\":\"XYZ123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ABMvPrO3nO7J5oPVVnp3HgrW'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight to a later date.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_kwIlEJ5TwFobKnGeosQlFteA', function=Function(arguments='{\"booking_reference\":\"DEF456\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kwIlEJ5TwFobKnGeosQlFteA'}, {'role': 'assistant', 'content': 'Since your ticket is non-refundable, I will need to apply a change fee for your flight change. \n",
       "\n",
       "Could you please provide me with the new date you would like to change your flight to?'}, {'role': 'user', 'content': 'I would like to change my flight to May 15, 2024.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_qRjsvdTsTJgvjcoSSNF2wY4Q', function=Function(arguments='{\"booking_reference\":\"DEF456\"}', name='apply_change_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I'd like to initiate a change to my flight for a later date.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is DEF456, my name is Jane Smith, and my flight number is XYZ123.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"DEF456\",\"full_name\":\"Jane Smith\",\"flight_number\":\"XYZ123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ABMvPrO3nO7J5oPVVnp3HgrW'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight to a later date.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"DEF456\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kwIlEJ5TwFobKnGeosQlFteA'}, {'role': 'assistant', 'content': 'Since your ticket is non-refundable, I will need to apply a change fee for your flight change. \n",
       "\n",
       "Could you please provide me with the new date you would like to change your flight to?'}, {'role': 'user', 'content': 'I would like to change my flight to May 15, 2024.'}, {'role': 'assistant', 'tool_call': {'name': 'apply_change_fee', 'arguments': '{\"booking_reference\":\"DEF456\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>My flight was canceled by the airline, and I want a refund.</td>\n",
       "      <td>I want to request a refund because my flight was canceled by the airline due to weather.</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'GHI789'}</td>\n",
       "      <td>rebook_without_fee</td>\n",
       "      <td>{'booking_reference': 'GHI789'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ZtpsjhHYT4UHiwwP63vNyPHl', function=Function(arguments='{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ZtpsjhHYT4UHiwwP63vNyPHl'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_64iGmmS4z3YCUQZrOpvoLN5R', function=Function(arguments='{\"booking_reference\":\"GHI789\"}', name='rebook_without_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ZtpsjhHYT4UHiwwP63vNyPHl'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"GHI789\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>I want to cancel my non-refundable ticket.</td>\n",
       "      <td>I want to cancel my flight but I think it is non-refundable.</td>\n",
       "      <td>offer_flight_credit</td>\n",
       "      <td>{'booking_reference': 'JKL012'}</td>\n",
       "      <td>offer_flight_credit</td>\n",
       "      <td>{'booking_reference': 'JKL012'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my non-refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL012, my full name is Michael Brown, and my flight number is DEF456.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_sBbFdoQpFxfX0E17h9WVB7YP', function=Function(arguments='{\"booking_reference\":\"JKL012\",\"full_name\":\"Michael Brown\",\"flight_number\":\"DEF456\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_sBbFdoQpFxfX0E17h9WVB7YP'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_UuyTDOLHnEldsZxQNGttudvv', function=Function(arguments='{\"booking_reference\":\"JKL012\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_UuyTDOLHnEldsZxQNGttudvv'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_0sNtdz6t4wmS6fZwwUpvX0tT', function=Function(arguments='{\"booking_reference\":\"JKL012\"}', name='offer_flight_credit'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my non-refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL012, my full name is Michael Brown, and my flight number is DEF456.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"JKL012\",\"full_name\":\"Michael Brown\",\"flight_number\":\"DEF456\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_sBbFdoQpFxfX0E17h9WVB7YP'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"JKL012\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_UuyTDOLHnEldsZxQNGttudvv'}, {'role': 'assistant', 'tool_call': {'name': 'offer_flight_credit', 'arguments': '{\"booking_reference\":\"JKL012\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>I want to cancel my refundable ticket.</td>\n",
       "      <td>I want to cancel my flight and I believe it is refundable.</td>\n",
       "      <td>process_partial_refund</td>\n",
       "      <td>{'booking_reference': 'MNO345', 'refund_amount': 65}</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'MNO345'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is MNO345, my name is Sarah Brown, and my flight number is STU678.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_b2cK3XE6VqPh0b5X47mTyrLH', function=Function(arguments='{\"booking_reference\":\"MNO345\",\"full_name\":\"Sarah Brown\",\"flight_number\":\"STU678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_b2cK3XE6VqPh0b5X47mTyrLH'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking. Since my ticket is refundable, I would like to know the steps and whether I will receive a refund.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_pABr2YlVt3gyTj3JBG3qmUud', function=Function(arguments='{\"booking_reference\":\"MNO345\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_pABr2YlVt3gyTj3JBG3qmUud'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_a61u81PgoIi6uMIod9wrjC4h', function=Function(arguments='{\"booking_reference\":\"MNO345\"}', name='process_full_refund'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is MNO345, my name is Sarah Brown, and my flight number is STU678.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"MNO345\",\"full_name\":\"Sarah Brown\",\"flight_number\":\"STU678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_b2cK3XE6VqPh0b5X47mTyrLH'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking. Since my ticket is refundable, I would like to know the steps and whether I will receive a refund.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"MNO345\"}'}}, {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_pABr2YlVt3gyTj3JBG3qmUud'}, {'role': 'assistant', 'tool_call': {'name': 'process_full_refund', 'arguments': '{\"booking_reference\":\"MNO345\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>I missed my connecting flight because of the airline delay.</td>\n",
       "      <td>I want to address the issue of missing my connecting flight due to a delay caused by the airline.</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>check_compensation_eligibility</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YvAyvvsrTy7eWUioyGJuArdH', function=Function(arguments='{\"booking_reference\":\"PQR678\"}', name='check_compensation_eligibility'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"PQR678\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>I want to know if I am eligible for compensation because of a long delay.</td>\n",
       "      <td>I want to know if I can receive compensation due to a long delay on my flight.</td>\n",
       "      <td>check_compensation_eligibility</td>\n",
       "      <td>{'booking_reference': 'STU901'}</td>\n",
       "      <td>check_compensation_eligibility</td>\n",
       "      <td>{'booking_reference': 'STU901'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is STU901, my name is Jessica Green, and my flight number is ABC789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_DVnnLtN4WVeiMWg7Zg7oYtkj', function=Function(arguments='{\"booking_reference\":\"STU901\",\"full_name\":\"Jessica Green\",\"flight_number\":\"ABC789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_DVnnLtN4WVeiMWg7Zg7oYtkj'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MT6vdLGV0IIQRsFJ6ObxgMqd', function=Function(arguments='{\"booking_reference\":\"STU901\"}', name='check_compensation_eligibility'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is STU901, my name is Jessica Green, and my flight number is ABC789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"STU901\",\"full_name\":\"Jessica Green\",\"flight_number\":\"ABC789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_DVnnLtN4WVeiMWg7Zg7oYtkj'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"STU901\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>I would like to rebook my flight.</td>\n",
       "      <td>I want to rebook my flight, and I have provided my details for verification.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'VWX234', 'new_flight': 'LMG091'}</td>\n",
       "      <td>rebook_without_fee</td>\n",
       "      <td>{'booking_reference': 'VWX234'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_vnkfocr8WZxG7PztJiHEPpmI', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_vnkfocr8WZxG7PztJiHEPpmI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight, please. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_B79fEFFzAxQna5tYlH2L7bnT', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_B79fEFFzAxQna5tYlH2L7bnT'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_WpW2EWdTx0WjIqmnyjbowXjV', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='rebook_without_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_vnkfocr8WZxG7PztJiHEPpmI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight, please. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_B79fEFFzAxQna5tYlH2L7bnT'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>I want to cancel my flight because of a medical emergency.</td>\n",
       "      <td>I want to cancel my flight due to a medical emergency.</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'Medical emergency'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Please provide your booking reference, full name, and flight number so I can assist you with the cancellation.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_agAAAJPqFKkUEnT9tqiz5RRb', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_agAAAJPqFKkUEnT9tqiz5RRb'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_H0dvq31uvvNG1frGKccT9sxL', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}', name='process_flexible_cancellation'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Please provide your booking reference, full name, and flight number so I can assist you with the cancellation.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_agAAAJPqFKkUEnT9tqiz5RRb'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>I would like to change the passenger name on my group booking.</td>\n",
       "      <td>I want to change the name of one of the passengers in my group booking.</td>\n",
       "      <td>permit_name_change</td>\n",
       "      <td>{'booking_reference': 'BCD890'}</td>\n",
       "      <td>permit_name_change</td>\n",
       "      <td>{'booking_reference': 'BCD890'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to change the passenger name on my group booking.'}, {'role': 'assistant', 'content': 'I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change the name of one of the passengers in my group booking. The booking reference is BCD890, and the flight number is QRS123. What further information do you need from me to process this change?'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_7tpa7khnA1DEFjSGJPodNsLL', function=Function(arguments='{\"booking_reference\":\"BCD890\",\"full_name\":\"\",\"flight_number\":\"QRS123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_7tpa7khnA1DEFjSGJPodNsLL'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_xYTyqq9bcfoIMEUOKr58uQDs', function=Function(arguments='{\"booking_reference\":\"BCD890\"}', name='permit_name_change'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to change the passenger name on my group booking.'}, {'role': 'assistant', 'content': 'I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change the name of one of the passengers in my group booking. The booking reference is BCD890, and the flight number is QRS123. What further information do you need from me to process this change?'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"BCD890\",\"full_name\":\"\",\"flight_number\":\"QRS123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_7tpa7khnA1DEFjSGJPodNsLL'}, {'role': 'assistant', 'tool_call': {'name': 'permit_name_change', 'arguments': '{\"booking_reference\":\"BCD890\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>The airline canceled my flight, and I need a hotel for the night.</td>\n",
       "      <td>I want to find a hotel because my flight was canceled by the airline.</td>\n",
       "      <td>offer_accommodation</td>\n",
       "      <td>{'booking_reference': 'EFG123'}</td>\n",
       "      <td>offer_accommodation</td>\n",
       "      <td>{'booking_reference': 'EFG123'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_bjK14w3IYl3RMOaaWKLhZjmc', function=Function(arguments='{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_bjK14w3IYl3RMOaaWKLhZjmc'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_sJ3k9HY3ZJD9GC8ry4IlXtf6', function=Function(arguments='{\"booking_reference\":\"EFG123\"}', name='offer_accommodation'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_bjK14w3IYl3RMOaaWKLhZjmc'}, {'role': 'assistant', 'tool_call': {'name': 'offer_accommodation', 'arguments': '{\"booking_reference\":\"EFG123\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>I need to change my flight within the next 7 days.</td>\n",
       "      <td>I want to change my flight because it's within the next week and I hope I can do it without any extra fees.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'LMG091'}</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_e6L21PF8AfUJVQzeEJsSBbYI', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_e6L21PF8AfUJVQzeEJsSBbYI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. Since it's within the next week, I hope to do this without incurring any extra fees.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_TjqWp3bbENSTK5S4s0AmFMzq', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_TjqWp3bbENSTK5S4s0AmFMzq'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_zapiO6a2QuRdi0GuZRoLD2du', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='process_change_no_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_e6L21PF8AfUJVQzeEJsSBbYI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. Since it's within the next week, I hope to do this without incurring any extra fees.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_TjqWp3bbENSTK5S4s0AmFMzq'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>I want to change my non-flexible flight to a different date.</td>\n",
       "      <td>I want to change my flight date, but I know my ticket is non-flexible which might incur a fee.</td>\n",
       "      <td>apply_change_fee</td>\n",
       "      <td>{'booking_reference': 'JKL234'}</td>\n",
       "      <td>apply_change_fee</td>\n",
       "      <td>{'booking_reference': 'JKL234'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to change my non-flexible flight to a different date.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL234, my flight number is STU567, and my name is Robert Williams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_HougMQ5lxJAeKxi1Zxzxn7N7', function=Function(arguments='{\"booking_reference\":\"JKL234\",\"full_name\":\"Robert Williams\",\"flight_number\":\"STU567\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_HougMQ5lxJAeKxi1Zxzxn7N7'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight date. I understand that my ticket is non-flexible, so I am aware that this might incur a fee. Can you provide the options for new dates and any associated fees?'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_kDkNNGC3ldu1WJOEbUiLLyqv', function=Function(arguments='{\"booking_reference\":\"JKL234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kDkNNGC3ldu1WJOEbUiLLyqv'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Y1p2T3dNe1oQnFB2xY3SnHKQ', function=Function(arguments='{\"booking_reference\":\"JKL234\"}', name='apply_change_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to change my non-flexible flight to a different date.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL234, my flight number is STU567, and my name is Robert Williams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"JKL234\",\"full_name\":\"Robert Williams\",\"flight_number\":\"STU567\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_HougMQ5lxJAeKxi1Zxzxn7N7'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight date. I understand that my ticket is non-flexible, so I am aware that this might incur a fee. Can you provide the options for new dates and any associated fees?'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"JKL234\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kDkNNGC3ldu1WJOEbUiLLyqv'}, {'role': 'assistant', 'tool_call': {'name': 'apply_change_fee', 'arguments': '{\"booking_reference\":\"JKL234\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>The airline canceled my flight due to weather. Can I get rebooked?</td>\n",
       "      <td>I want to get rebooked on a flight since my original flight was canceled due to weather.</td>\n",
       "      <td>rebook_without_fee</td>\n",
       "      <td>{'booking_reference': 'MNO345'}</td>\n",
       "      <td>rebook_without_fee</td>\n",
       "      <td>{'booking_reference': 'MNO345'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'The airline canceled my flight due to weather. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to get rebooked on a flight, please. My original booking reference is MNO345, and the flight number was XYZ123.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Cv32WzCx8MnJ9vndtxPRVYOE', function=Function(arguments='{\"booking_reference\":\"MNO345\",\"full_name\":\"\",\"flight_number\":\"XYZ123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Cv32WzCx8MnJ9vndtxPRVYOE'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_h3xRdAF2T6AuN6RU43sh8ayI', function=Function(arguments='{\"booking_reference\":\"MNO345\"}', name='rebook_without_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'The airline canceled my flight due to weather. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to get rebooked on a flight, please. My original booking reference is MNO345, and the flight number was XYZ123.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"MNO345\",\"full_name\":\"\",\"flight_number\":\"XYZ123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Cv32WzCx8MnJ9vndtxPRVYOE'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"MNO345\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>I need a refund for my non-refundable flight.</td>\n",
       "      <td>I want to cancel my flight but I think it is non-refundable.</td>\n",
       "      <td>offer_flight_credit</td>\n",
       "      <td>{'booking_reference': 'PQR567'}</td>\n",
       "      <td>offer_flight_credit</td>\n",
       "      <td>{'booking_reference': 'PQR567'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need a refund for my non-refundable flight.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'Sure, my booking reference is PQR567, my full name is David White, and the flight number is XYZ456.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_pOwgDoDQm0UfxAzVa95he6y8', function=Function(arguments='{\"booking_reference\":\"PQR567\",\"full_name\":\"David White\",\"flight_number\":\"XYZ456\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_pOwgDoDQm0UfxAzVa95he6y8'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking. However, I understand that it is a non-refundable flight.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_j5OG05P8Q80wB2SbdYUmq18B', function=Function(arguments='{\"booking_reference\":\"PQR567\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_j5OG05P8Q80wB2SbdYUmq18B'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_302WxoZBNB2cn5aNGEQLg9os', function=Function(arguments='{\"booking_reference\":\"PQR567\"}', name='offer_flight_credit'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need a refund for my non-refundable flight.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'Sure, my booking reference is PQR567, my full name is David White, and the flight number is XYZ456.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"PQR567\",\"full_name\":\"David White\",\"flight_number\":\"XYZ456\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_pOwgDoDQm0UfxAzVa95he6y8'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking. However, I understand that it is a non-refundable flight.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"PQR567\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_j5OG05P8Q80wB2SbdYUmq18B'}, {'role': 'assistant', 'tool_call': {'name': 'offer_flight_credit', 'arguments': '{\"booking_reference\":\"PQR567\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>I need a refund for a flight I booked a few hours ago.</td>\n",
       "      <td>I want to get a refund for a recently booked flight and I have provided the necessary details to verify my booking.</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'STU678'}</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'STU678'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need a refund for a flight I booked a few hours ago.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking and request a refund for the flight I booked a few hours ago. My booking reference is STU678, and the flight number is ABC345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_y8zRnY265Teu6lVMESfSZ2HM', function=Function(arguments='{\"booking_reference\":\"STU678\",\"full_name\":\"\",\"flight_number\":\"ABC345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_y8zRnY265Teu6lVMESfSZ2HM'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_CBcBdznkGC6lIJlROE3eds8s', function=Function(arguments='{\"booking_reference\":\"STU678\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: full_refund', 'tool_call_id': 'call_CBcBdznkGC6lIJlROE3eds8s'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_KjQKODoF4cbkehk5yYUOLhGY', function=Function(arguments='{\"booking_reference\":\"STU678\"}', name='process_full_refund'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need a refund for a flight I booked a few hours ago.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking and request a refund for the flight I booked a few hours ago. My booking reference is STU678, and the flight number is ABC345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"STU678\",\"full_name\":\"\",\"flight_number\":\"ABC345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_y8zRnY265Teu6lVMESfSZ2HM'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"STU678\"}'}}, {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: full_refund', 'tool_call_id': 'call_CBcBdznkGC6lIJlROE3eds8s'}, {'role': 'assistant', 'tool_call': {'name': 'process_full_refund', 'arguments': '{\"booking_reference\":\"STU678\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>I missed my connecting flight because of a delay. Can I get rebooked?</td>\n",
       "      <td>I want to get rebooked on another flight because I missed my connecting flight due to a delay.</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'XYZ901'}</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'XYZ901'}</td>\n",
       "      <td>True</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of a delay. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'd like to get rebooked on another flight, please. My booking reference is XYZ901, and my flight number is LMN456.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_SjxNkJ43NsInWZNAUwXnCM6z', function=Function(arguments='{\"booking_reference\":\"XYZ901\",\"full_name\":\"\",\"flight_number\":\"LMN456\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_SjxNkJ43NsInWZNAUwXnCM6z'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_E7mnxyka1PlPUPICFZDP2jTH', function=Function(arguments='{\"booking_reference\":\"XYZ901\"}', name='prioritize_missed_connections'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of a delay. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'd like to get rebooked on another flight, please. My booking reference is XYZ901, and my flight number is LMN456.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"XYZ901\",\"full_name\":\"\",\"flight_number\":\"LMN456\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_SjxNkJ43NsInWZNAUwXnCM6z'}, {'role': 'assistant', 'tool_call': {'name': 'prioritize_missed_connections', 'arguments': '{\"booking_reference\":\"XYZ901\"}'}}]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                      request  \\\n",
       "0                                   I would like to cancel my flight, please.   \n",
       "1                I'd like to initiate a change to my flight for a later date.   \n",
       "2                 My flight was canceled by the airline, and I want a refund.   \n",
       "3                                  I want to cancel my non-refundable ticket.   \n",
       "4                                      I want to cancel my refundable ticket.   \n",
       "5                 I missed my connecting flight because of the airline delay.   \n",
       "6   I want to know if I am eligible for compensation because of a long delay.   \n",
       "7                                           I would like to rebook my flight.   \n",
       "8                  I want to cancel my flight because of a medical emergency.   \n",
       "9              I would like to change the passenger name on my group booking.   \n",
       "10          The airline canceled my flight, and I need a hotel for the night.   \n",
       "11                         I need to change my flight within the next 7 days.   \n",
       "12               I want to change my non-flexible flight to a different date.   \n",
       "13         The airline canceled my flight due to weather. Can I get rebooked?   \n",
       "14                              I need a refund for my non-refundable flight.   \n",
       "15                     I need a refund for a flight I booked a few hours ago.   \n",
       "16      I missed my connecting flight because of a delay. Can I get rebooked?   \n",
       "\n",
       "                                                                                                                context  \\\n",
       "0                                                I want to cancel my flight as I am initiating the cancellation myself.   \n",
       "1                                                                       I want to change my flight to a different date.   \n",
       "2                              I want to request a refund because my flight was canceled by the airline due to weather.   \n",
       "3                                                          I want to cancel my flight but I think it is non-refundable.   \n",
       "4                                                            I want to cancel my flight and I believe it is refundable.   \n",
       "5                     I want to address the issue of missing my connecting flight due to a delay caused by the airline.   \n",
       "6                                        I want to know if I can receive compensation due to a long delay on my flight.   \n",
       "7                                          I want to rebook my flight, and I have provided my details for verification.   \n",
       "8                                                                I want to cancel my flight due to a medical emergency.   \n",
       "9                                               I want to change the name of one of the passengers in my group booking.   \n",
       "10                                                I want to find a hotel because my flight was canceled by the airline.   \n",
       "11          I want to change my flight because it's within the next week and I hope I can do it without any extra fees.   \n",
       "12                       I want to change my flight date, but I know my ticket is non-flexible which might incur a fee.   \n",
       "13                             I want to get rebooked on a flight since my original flight was canceled due to weather.   \n",
       "14                                                         I want to cancel my flight but I think it is non-refundable.   \n",
       "15  I want to get a refund for a recently booked flight and I have provided the necessary details to verify my booking.   \n",
       "16                       I want to get rebooked on another flight because I missed my connecting flight due to a delay.   \n",
       "\n",
       "                 expected_function  \\\n",
       "0           process_partial_refund   \n",
       "1                 apply_change_fee   \n",
       "2              process_full_refund   \n",
       "3              offer_flight_credit   \n",
       "4           process_partial_refund   \n",
       "5    prioritize_missed_connections   \n",
       "6   check_compensation_eligibility   \n",
       "7            process_change_no_fee   \n",
       "8    process_flexible_cancellation   \n",
       "9               permit_name_change   \n",
       "10             offer_accommodation   \n",
       "11           process_change_no_fee   \n",
       "12                apply_change_fee   \n",
       "13              rebook_without_fee   \n",
       "14             offer_flight_credit   \n",
       "15             process_full_refund   \n",
       "16   prioritize_missed_connections   \n",
       "\n",
       "                                                     expected_inputs  \\\n",
       "0              {'booking_reference': 'ABC123', 'refund_amount': 198}   \n",
       "1                                    {'booking_reference': 'DEF456'}   \n",
       "2                                    {'booking_reference': 'GHI789'}   \n",
       "3                                    {'booking_reference': 'JKL012'}   \n",
       "4               {'booking_reference': 'MNO345', 'refund_amount': 65}   \n",
       "5                                    {'booking_reference': 'PQR678'}   \n",
       "6                                    {'booking_reference': 'STU901'}   \n",
       "7            {'booking_reference': 'VWX234', 'new_flight': 'LMG091'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}   \n",
       "9                                    {'booking_reference': 'BCD890'}   \n",
       "10                                   {'booking_reference': 'EFG123'}   \n",
       "11           {'booking_reference': 'GHI123', 'new_flight': 'LMG091'}   \n",
       "12                                   {'booking_reference': 'JKL234'}   \n",
       "13                                   {'booking_reference': 'MNO345'}   \n",
       "14                                   {'booking_reference': 'PQR567'}   \n",
       "15                                   {'booking_reference': 'STU678'}   \n",
       "16                                   {'booking_reference': 'XYZ901'}   \n",
       "\n",
       "                   actual_function  \\\n",
       "0              process_full_refund   \n",
       "1                 apply_change_fee   \n",
       "2               rebook_without_fee   \n",
       "3              offer_flight_credit   \n",
       "4              process_full_refund   \n",
       "5   check_compensation_eligibility   \n",
       "6   check_compensation_eligibility   \n",
       "7               rebook_without_fee   \n",
       "8    process_flexible_cancellation   \n",
       "9               permit_name_change   \n",
       "10             offer_accommodation   \n",
       "11           process_change_no_fee   \n",
       "12                apply_change_fee   \n",
       "13              rebook_without_fee   \n",
       "14             offer_flight_credit   \n",
       "15             process_full_refund   \n",
       "16   prioritize_missed_connections   \n",
       "\n",
       "                                                                  actual_inputs  \\\n",
       "0                                               {'booking_reference': 'ABC123'}   \n",
       "1                                               {'booking_reference': 'DEF456'}   \n",
       "2                                               {'booking_reference': 'GHI789'}   \n",
       "3                                               {'booking_reference': 'JKL012'}   \n",
       "4                                               {'booking_reference': 'MNO345'}   \n",
       "5                                               {'booking_reference': 'PQR678'}   \n",
       "6                                               {'booking_reference': 'STU901'}   \n",
       "7                                               {'booking_reference': 'VWX234'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'Medical emergency'}   \n",
       "9                                               {'booking_reference': 'BCD890'}   \n",
       "10                                              {'booking_reference': 'EFG123'}   \n",
       "11                                              {'booking_reference': 'GHI123'}   \n",
       "12                                              {'booking_reference': 'JKL234'}   \n",
       "13                                              {'booking_reference': 'MNO345'}   \n",
       "14                                              {'booking_reference': 'PQR567'}   \n",
       "15                                              {'booking_reference': 'STU678'}   \n",
       "16                                              {'booking_reference': 'XYZ901'}   \n",
       "\n",
       "    is_correct  \\\n",
       "0        False   \n",
       "1         True   \n",
       "2        False   \n",
       "3         True   \n",
       "4        False   \n",
       "5        False   \n",
       "6         True   \n",
       "7        False   \n",
       "8        False   \n",
       "9         True   \n",
       "10        True   \n",
       "11       False   \n",
       "12        True   \n",
       "13        True   \n",
       "14        True   \n",
       "15        True   \n",
       "16        True   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 transcript  \\\n",
       "0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  [{'role': 'user', 'content': 'I would like to cancel my flight, please.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight, please. My booking reference is ABC123, and the flight number is XYZ789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ansPc0MDF53FmKJBcELp4EZD', function=Function(arguments='{\"booking_reference\":\"ABC123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ansPc0MDF53FmKJBcELp4EZD'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_NQvw6o5ufa6LyNHMutEd8D7H', function=Function(arguments='{\"booking_reference\":\"ABC123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_NQvw6o5ufa6LyNHMutEd8D7H'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_0wVEPVZqo0Uy8tLD6e3XTdk4', function=Function(arguments='{\"booking_reference\":\"ABC123\"}', name='process_full_refund'), type='function')])]   \n",
       "1   [{'role': 'user', 'content': 'I'd like to initiate a change to my flight for a later date.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is DEF456, my name is Jane Smith, and my flight number is XYZ123.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ABMvPrO3nO7J5oPVVnp3HgrW', function=Function(arguments='{\"booking_reference\":\"DEF456\",\"full_name\":\"Jane Smith\",\"flight_number\":\"XYZ123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ABMvPrO3nO7J5oPVVnp3HgrW'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight to a later date.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_kwIlEJ5TwFobKnGeosQlFteA', function=Function(arguments='{\"booking_reference\":\"DEF456\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kwIlEJ5TwFobKnGeosQlFteA'}, {'role': 'assistant', 'content': 'Since your ticket is non-refundable, I will need to apply a change fee for your flight change. \n",
       "\n",
       "Could you please provide me with the new date you would like to change your flight to?'}, {'role': 'user', 'content': 'I would like to change my flight to May 15, 2024.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_qRjsvdTsTJgvjcoSSNF2wY4Q', function=Function(arguments='{\"booking_reference\":\"DEF456\"}', name='apply_change_fee'), type='function')])]   \n",
       "2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     [{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ZtpsjhHYT4UHiwwP63vNyPHl', function=Function(arguments='{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ZtpsjhHYT4UHiwwP63vNyPHl'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_64iGmmS4z3YCUQZrOpvoLN5R', function=Function(arguments='{\"booking_reference\":\"GHI789\"}', name='rebook_without_fee'), type='function')])]   \n",
       "3                                                                                                                                                                                                                                                                                                                     [{'role': 'user', 'content': 'I want to cancel my non-refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL012, my full name is Michael Brown, and my flight number is DEF456.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_sBbFdoQpFxfX0E17h9WVB7YP', function=Function(arguments='{\"booking_reference\":\"JKL012\",\"full_name\":\"Michael Brown\",\"flight_number\":\"DEF456\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_sBbFdoQpFxfX0E17h9WVB7YP'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_UuyTDOLHnEldsZxQNGttudvv', function=Function(arguments='{\"booking_reference\":\"JKL012\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_UuyTDOLHnEldsZxQNGttudvv'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_0sNtdz6t4wmS6fZwwUpvX0tT', function=Function(arguments='{\"booking_reference\":\"JKL012\"}', name='offer_flight_credit'), type='function')])]   \n",
       "4                                                                                                                                                                                                                    [{'role': 'user', 'content': 'I want to cancel my refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is MNO345, my name is Sarah Brown, and my flight number is STU678.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_b2cK3XE6VqPh0b5X47mTyrLH', function=Function(arguments='{\"booking_reference\":\"MNO345\",\"full_name\":\"Sarah Brown\",\"flight_number\":\"STU678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_b2cK3XE6VqPh0b5X47mTyrLH'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking. Since my ticket is refundable, I would like to know the steps and whether I will receive a refund.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_pABr2YlVt3gyTj3JBG3qmUud', function=Function(arguments='{\"booking_reference\":\"MNO345\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_pABr2YlVt3gyTj3JBG3qmUud'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_a61u81PgoIi6uMIod9wrjC4h', function=Function(arguments='{\"booking_reference\":\"MNO345\"}', name='process_full_refund'), type='function')])]   \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YvAyvvsrTy7eWUioyGJuArdH', function=Function(arguments='{\"booking_reference\":\"PQR678\"}', name='check_compensation_eligibility'), type='function')])]   \n",
       "6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               [{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is STU901, my name is Jessica Green, and my flight number is ABC789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_DVnnLtN4WVeiMWg7Zg7oYtkj', function=Function(arguments='{\"booking_reference\":\"STU901\",\"full_name\":\"Jessica Green\",\"flight_number\":\"ABC789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_DVnnLtN4WVeiMWg7Zg7oYtkj'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MT6vdLGV0IIQRsFJ6ObxgMqd', function=Function(arguments='{\"booking_reference\":\"STU901\"}', name='check_compensation_eligibility'), type='function')])]   \n",
       "7                                                                                                                                                                                                                                                                                                      [{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_vnkfocr8WZxG7PztJiHEPpmI', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_vnkfocr8WZxG7PztJiHEPpmI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight, please. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_B79fEFFzAxQna5tYlH2L7bnT', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_B79fEFFzAxQna5tYlH2L7bnT'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_WpW2EWdTx0WjIqmnyjbowXjV', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='rebook_without_fee'), type='function')])]   \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Please provide your booking reference, full name, and flight number so I can assist you with the cancellation.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_agAAAJPqFKkUEnT9tqiz5RRb', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_agAAAJPqFKkUEnT9tqiz5RRb'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_H0dvq31uvvNG1frGKccT9sxL', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}', name='process_flexible_cancellation'), type='function')])]   \n",
       "9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        [{'role': 'user', 'content': 'I would like to change the passenger name on my group booking.'}, {'role': 'assistant', 'content': 'I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change the name of one of the passengers in my group booking. The booking reference is BCD890, and the flight number is QRS123. What further information do you need from me to process this change?'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_7tpa7khnA1DEFjSGJPodNsLL', function=Function(arguments='{\"booking_reference\":\"BCD890\",\"full_name\":\"\",\"flight_number\":\"QRS123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_7tpa7khnA1DEFjSGJPodNsLL'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_xYTyqq9bcfoIMEUOKr58uQDs', function=Function(arguments='{\"booking_reference\":\"BCD890\"}', name='permit_name_change'), type='function')])]   \n",
       "10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          [{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_bjK14w3IYl3RMOaaWKLhZjmc', function=Function(arguments='{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_bjK14w3IYl3RMOaaWKLhZjmc'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_sJ3k9HY3ZJD9GC8ry4IlXtf6', function=Function(arguments='{\"booking_reference\":\"EFG123\"}', name='offer_accommodation'), type='function')])]   \n",
       "11                                                                                                                                                                                                                                       [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_e6L21PF8AfUJVQzeEJsSBbYI', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_e6L21PF8AfUJVQzeEJsSBbYI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. Since it's within the next week, I hope to do this without incurring any extra fees.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_TjqWp3bbENSTK5S4s0AmFMzq', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_TjqWp3bbENSTK5S4s0AmFMzq'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_zapiO6a2QuRdi0GuZRoLD2du', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='process_change_no_fee'), type='function')])]   \n",
       "12                                                                                                                                                     [{'role': 'user', 'content': 'I want to change my non-flexible flight to a different date.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL234, my flight number is STU567, and my name is Robert Williams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_HougMQ5lxJAeKxi1Zxzxn7N7', function=Function(arguments='{\"booking_reference\":\"JKL234\",\"full_name\":\"Robert Williams\",\"flight_number\":\"STU567\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_HougMQ5lxJAeKxi1Zxzxn7N7'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight date. I understand that my ticket is non-flexible, so I am aware that this might incur a fee. Can you provide the options for new dates and any associated fees?'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_kDkNNGC3ldu1WJOEbUiLLyqv', function=Function(arguments='{\"booking_reference\":\"JKL234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kDkNNGC3ldu1WJOEbUiLLyqv'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Y1p2T3dNe1oQnFB2xY3SnHKQ', function=Function(arguments='{\"booking_reference\":\"JKL234\"}', name='apply_change_fee'), type='function')])]   \n",
       "13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          [{'role': 'user', 'content': 'The airline canceled my flight due to weather. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to get rebooked on a flight, please. My original booking reference is MNO345, and the flight number was XYZ123.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Cv32WzCx8MnJ9vndtxPRVYOE', function=Function(arguments='{\"booking_reference\":\"MNO345\",\"full_name\":\"\",\"flight_number\":\"XYZ123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Cv32WzCx8MnJ9vndtxPRVYOE'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_h3xRdAF2T6AuN6RU43sh8ayI', function=Function(arguments='{\"booking_reference\":\"MNO345\"}', name='rebook_without_fee'), type='function')])]   \n",
       "14                                                                                                                                                                                                                                                                  [{'role': 'user', 'content': 'I need a refund for my non-refundable flight.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'Sure, my booking reference is PQR567, my full name is David White, and the flight number is XYZ456.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_pOwgDoDQm0UfxAzVa95he6y8', function=Function(arguments='{\"booking_reference\":\"PQR567\",\"full_name\":\"David White\",\"flight_number\":\"XYZ456\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_pOwgDoDQm0UfxAzVa95he6y8'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking. However, I understand that it is a non-refundable flight.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_j5OG05P8Q80wB2SbdYUmq18B', function=Function(arguments='{\"booking_reference\":\"PQR567\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_j5OG05P8Q80wB2SbdYUmq18B'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_302WxoZBNB2cn5aNGEQLg9os', function=Function(arguments='{\"booking_reference\":\"PQR567\"}', name='offer_flight_credit'), type='function')])]   \n",
       "15                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 [{'role': 'user', 'content': 'I need a refund for a flight I booked a few hours ago.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking and request a refund for the flight I booked a few hours ago. My booking reference is STU678, and the flight number is ABC345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_y8zRnY265Teu6lVMESfSZ2HM', function=Function(arguments='{\"booking_reference\":\"STU678\",\"full_name\":\"\",\"flight_number\":\"ABC345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_y8zRnY265Teu6lVMESfSZ2HM'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_CBcBdznkGC6lIJlROE3eds8s', function=Function(arguments='{\"booking_reference\":\"STU678\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: full_refund', 'tool_call_id': 'call_CBcBdznkGC6lIJlROE3eds8s'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_KjQKODoF4cbkehk5yYUOLhGY', function=Function(arguments='{\"booking_reference\":\"STU678\"}', name='process_full_refund'), type='function')])]   \n",
       "16                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     [{'role': 'user', 'content': 'I missed my connecting flight because of a delay. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'd like to get rebooked on another flight, please. My booking reference is XYZ901, and my flight number is LMN456.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_SjxNkJ43NsInWZNAUwXnCM6z', function=Function(arguments='{\"booking_reference\":\"XYZ901\",\"full_name\":\"\",\"flight_number\":\"LMN456\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_SjxNkJ43NsInWZNAUwXnCM6z'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_E7mnxyka1PlPUPICFZDP2jTH', function=Function(arguments='{\"booking_reference\":\"XYZ901\"}', name='prioritize_missed_connections'), type='function')])]   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  cleaned_transcript  \n",
       "0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  [{'role': 'user', 'content': 'I would like to cancel my flight, please.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight, please. My booking reference is ABC123, and the flight number is XYZ789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"ABC123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ansPc0MDF53FmKJBcELp4EZD'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_NQvw6o5ufa6LyNHMutEd8D7H'}, {'role': 'assistant', 'tool_call': {'name': 'process_full_refund', 'arguments': '{\"booking_reference\":\"ABC123\"}'}}]  \n",
       "1   [{'role': 'user', 'content': 'I'd like to initiate a change to my flight for a later date.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is DEF456, my name is Jane Smith, and my flight number is XYZ123.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"DEF456\",\"full_name\":\"Jane Smith\",\"flight_number\":\"XYZ123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ABMvPrO3nO7J5oPVVnp3HgrW'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight to a later date.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"DEF456\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kwIlEJ5TwFobKnGeosQlFteA'}, {'role': 'assistant', 'content': 'Since your ticket is non-refundable, I will need to apply a change fee for your flight change. \n",
       "\n",
       "Could you please provide me with the new date you would like to change your flight to?'}, {'role': 'user', 'content': 'I would like to change my flight to May 15, 2024.'}, {'role': 'assistant', 'tool_call': {'name': 'apply_change_fee', 'arguments': '{\"booking_reference\":\"DEF456\"}'}}]  \n",
       "2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        [{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_ZtpsjhHYT4UHiwwP63vNyPHl'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"GHI789\"}'}}]  \n",
       "3                                                                                                                                                                                                                                                                                                                     [{'role': 'user', 'content': 'I want to cancel my non-refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL012, my full name is Michael Brown, and my flight number is DEF456.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"JKL012\",\"full_name\":\"Michael Brown\",\"flight_number\":\"DEF456\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_sBbFdoQpFxfX0E17h9WVB7YP'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"JKL012\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_UuyTDOLHnEldsZxQNGttudvv'}, {'role': 'assistant', 'tool_call': {'name': 'offer_flight_credit', 'arguments': '{\"booking_reference\":\"JKL012\"}'}}]  \n",
       "4                                                                                                                                                                                                                    [{'role': 'user', 'content': 'I want to cancel my refundable ticket.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is MNO345, my name is Sarah Brown, and my flight number is STU678.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"MNO345\",\"full_name\":\"Sarah Brown\",\"flight_number\":\"STU678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_b2cK3XE6VqPh0b5X47mTyrLH'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to proceed with canceling my booking. Since my ticket is refundable, I would like to know the steps and whether I will receive a refund.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"MNO345\"}'}}, {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: partial_refund', 'tool_call_id': 'call_pABr2YlVt3gyTj3JBG3qmUud'}, {'role': 'assistant', 'tool_call': {'name': 'process_full_refund', 'arguments': '{\"booking_reference\":\"MNO345\"}'}}]  \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"PQR678\"}'}}]  \n",
       "6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  [{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is STU901, my name is Jessica Green, and my flight number is ABC789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"STU901\",\"full_name\":\"Jessica Green\",\"flight_number\":\"ABC789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_DVnnLtN4WVeiMWg7Zg7oYtkj'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"STU901\"}'}}]  \n",
       "7                                                                                                                                                                                                                                                                                                      [{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_vnkfocr8WZxG7PztJiHEPpmI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight, please. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_B79fEFFzAxQna5tYlH2L7bnT'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}]  \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Please provide your booking reference, full name, and flight number so I can assist you with the cancellation.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_agAAAJPqFKkUEnT9tqiz5RRb'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}'}}]  \n",
       "9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [{'role': 'user', 'content': 'I would like to change the passenger name on my group booking.'}, {'role': 'assistant', 'content': 'I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change the name of one of the passengers in my group booking. The booking reference is BCD890, and the flight number is QRS123. What further information do you need from me to process this change?'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"BCD890\",\"full_name\":\"\",\"flight_number\":\"QRS123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_7tpa7khnA1DEFjSGJPodNsLL'}, {'role': 'assistant', 'tool_call': {'name': 'permit_name_change', 'arguments': '{\"booking_reference\":\"BCD890\"}'}}]  \n",
       "10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             [{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_bjK14w3IYl3RMOaaWKLhZjmc'}, {'role': 'assistant', 'tool_call': {'name': 'offer_accommodation', 'arguments': '{\"booking_reference\":\"EFG123\"}'}}]  \n",
       "11                                                                                                                                                                                                                                       [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_e6L21PF8AfUJVQzeEJsSBbYI'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. Since it's within the next week, I hope to do this without incurring any extra fees.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_TjqWp3bbENSTK5S4s0AmFMzq'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}]  \n",
       "12                                                                                                                                                     [{'role': 'user', 'content': 'I want to change my non-flexible flight to a different date.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is JKL234, my flight number is STU567, and my name is Robert Williams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"JKL234\",\"full_name\":\"Robert Williams\",\"flight_number\":\"STU567\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_HougMQ5lxJAeKxi1Zxzxn7N7'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight date. I understand that my ticket is non-flexible, so I am aware that this might incur a fee. Can you provide the options for new dates and any associated fees?'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"JKL234\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_kDkNNGC3ldu1WJOEbUiLLyqv'}, {'role': 'assistant', 'tool_call': {'name': 'apply_change_fee', 'arguments': '{\"booking_reference\":\"JKL234\"}'}}]  \n",
       "13                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             [{'role': 'user', 'content': 'The airline canceled my flight due to weather. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to get rebooked on a flight, please. My original booking reference is MNO345, and the flight number was XYZ123.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"MNO345\",\"full_name\":\"\",\"flight_number\":\"XYZ123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Cv32WzCx8MnJ9vndtxPRVYOE'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"MNO345\"}'}}]  \n",
       "14                                                                                                                                                                                                                                                                  [{'role': 'user', 'content': 'I need a refund for my non-refundable flight.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'Sure, my booking reference is PQR567, my full name is David White, and the flight number is XYZ456.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"PQR567\",\"full_name\":\"David White\",\"flight_number\":\"XYZ456\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_pOwgDoDQm0UfxAzVa95he6y8'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking. However, I understand that it is a non-refundable flight.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"PQR567\"}'}}, {'role': 'tool', 'content': 'Ticket type: nonrefundable\n",
       "Fare rules: ', 'tool_call_id': 'call_j5OG05P8Q80wB2SbdYUmq18B'}, {'role': 'assistant', 'tool_call': {'name': 'offer_flight_credit', 'arguments': '{\"booking_reference\":\"PQR567\"}'}}]  \n",
       "15                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 [{'role': 'user', 'content': 'I need a refund for a flight I booked a few hours ago.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking and request a refund for the flight I booked a few hours ago. My booking reference is STU678, and the flight number is ABC345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"STU678\",\"full_name\":\"\",\"flight_number\":\"ABC345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_y8zRnY265Teu6lVMESfSZ2HM'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"STU678\"}'}}, {'role': 'tool', 'content': 'Ticket type: refundable\n",
       "Fare rules: full_refund', 'tool_call_id': 'call_CBcBdznkGC6lIJlROE3eds8s'}, {'role': 'assistant', 'tool_call': {'name': 'process_full_refund', 'arguments': '{\"booking_reference\":\"STU678\"}'}}]  \n",
       "16                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        [{'role': 'user', 'content': 'I missed my connecting flight because of a delay. Can I get rebooked?'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'd like to get rebooked on another flight, please. My booking reference is XYZ901, and my flight number is LMN456.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"XYZ901\",\"full_name\":\"\",\"flight_number\":\"LMN456\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_SjxNkJ43NsInWZNAUwXnCM6z'}, {'role': 'assistant', 'tool_call': {'name': 'prioritize_missed_connections', 'arguments': '{\"booking_reference\":\"XYZ901\"}'}}]  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df, accuracy = evaluate_function_calls(eval_df, flight_cancellation_routine, GPT_MODEL)\n",
    "\n",
    "# Display the accuracy as a mini header\n",
    "display(Markdown(f\"### Accuracy: {accuracy:.2%}\"))\n",
    "\n",
    "display(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eval DataFrame Row:\n",
      "ID                                                                                 8\n",
      "context                       I want to cancel my flight due to a medical emergency.\n",
      "request                   I want to cancel my flight because of a medical emergency.\n",
      "full_name                                                                Laura Adams\n",
      "booking_ref                                                                   YZA567\n",
      "flight_no                                                                     PQR678\n",
      "expected_function                                      process_flexible_cancellation\n",
      "expected_inputs        {\"booking_reference\":\"YZA567\",\"medical_certificate\":\"MC1234\"}\n",
      "ticket_type                                                                 flexible\n",
      "fare_rules                                                                       NaN\n",
      "medical_certificate                                                           MC1234\n",
      "refund_amount                                                                    NaN\n",
      "Name: 8, dtype: object\n",
      "\n",
      "DataFrame Columns:\n",
      "actual_function: process_flexible_cancellation\n",
      "actual_inputs: {'booking_reference': 'YZA567', 'medical_certificate': 'Medical emergency'}\n",
      "is_correct: False\n",
      "transcript:\n",
      "\u001b[34m{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}\u001b[0m\n",
      "\u001b[31m{'role': 'assistant', 'content': 'Please provide your booking reference, full name, and flight number so I can assist you with the cancellation.'}\u001b[0m\n",
      "\u001b[34m{'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}\u001b[0m\n",
      "\u001b[31m{'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}\u001b[0m\n",
      "\u001b[31m{'role': 'tool', 'content': 'True', 'tool_call_id': 'call_agAAAJPqFKkUEnT9tqiz5RRb'}\u001b[0m\n",
      "\u001b[31m{'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}'}}\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "display_row(8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Improve 4o routine\n",
    "\n",
    "Let's now leverage o1 again to add in a metaprompting loop to see if we can improve the quality of our evals.\n",
    "\n",
    "We'll take the following multi-step approach:\n",
    "- We'll pass in the current routine + eval results to o1 and ask it analyze the results and update the routine accordingly\n",
    "- Since o1 does not currently support structured outputs, we'll chain with output with a 4o to enforce a schema we can parse\n",
    "- Finally, we take the new routine and run it back through our eval to generate new results\n",
    "\n",
    "We'll run this loop a fixed number of times and see what improvements we can make"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "height": 540
   },
   "outputs": [],
   "source": [
    "def get_openai_response(messages, model, response_format=None):\n",
    "    try:\n",
    "        if response_format:\n",
    "            response = client.chat.completions.create(\n",
    "                model=model,\n",
    "                messages=messages,\n",
    "                response_format={\"type\": \"json_schema\", \"json_schema\": response_format}\n",
    "            )\n",
    "        else:\n",
    "            response = client.chat.completions.create(\n",
    "                model=model,\n",
    "                messages=messages\n",
    "            )\n",
    "        return response.choices[0].message.content\n",
    "    except Exception as e:\n",
    "        print(f\"An error occurred: {e}\")\n",
    "\n",
    "\n",
    "OUTPUT_SCHEMA = {\n",
    "                    \"name\": \"policy_output\",\n",
    "                    \"schema\": {\n",
    "                    \"type\": \"object\",\n",
    "                    \"properties\": {\n",
    "                        \"final_answer\": { \"type\": \"string\" }\n",
    "                    },\n",
    "                    \"required\": [\"final_answer\"],\n",
    "                    \"additionalProperties\": False\n",
    "                    },\n",
    "                    \"strict\": True\n",
    "                }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "height": 795
   },
   "outputs": [],
   "source": [
    "start_o1_message = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": f\"\"\"# Instructions\n",
    "You are an agent that is responsible for improving the quality of instructions that are provided to a customer service LLM agent. \n",
    "Your task is to improve the instructions that are provided to the LLM agent in order to increase accuracy on a test set while adhering to the initial policy. \n",
    "\n",
    "## Criteria\n",
    "- Analyze the existing instructions and the results of the eval. Understand which behaviors lead to failures.\n",
    "- For example, if the LLM agent is not asking for the booking reference when it is needed, you should add a step to ask for the booking reference.\n",
    "- Improve the instructions to address the gaps in the eval results.\n",
    "- Ensure changes made are compliant with the original policy.\n",
    "- Only use the tools provided.\n",
    "- Use the functions provided to the best of your ability to ensure the LLM agent can handle the customer service requests effectively. Including gathering data where necessary.\n",
    "- Try changing the format if this formatting doesn't work well - consider basic XML (e.g. <step> <substep> <if> <case>) or markdown as alternatives.\n",
    "\n",
    "You will be provided with 4 items:\n",
    "1) The ground-truth policy for the customer service agent containing detailed instructions on how to handle flight cancellations and changes.\n",
    "2) Full list of available functions.\n",
    "3) A routine instruction set.\n",
    "4) A results that shows the LLMs performance on a test set using this routine instruction set. This dataset contains columns showing:\n",
    "    - request: This is the initial user request. \n",
    "    - expected_function: This is the function we expect the LLM to call at the end of the conversation. \n",
    "    - expected_input: This is the input we expect the LLM to provide to the function at the end of the conversation.\n",
    "    - actual_function: This is the final function the LLM called using the current instructions.\n",
    "    - actual_input: These are the function parameters the LLM provided based on the current instructions.\n",
    "    - transcript: This is the conversation transcript between the user and the LLM agent. \n",
    "    - is_correct: True/False value depending on if the model responded correctly\n",
    "\n",
    "You may be provided with a history of edits and evaluations. You can use this information to understand what has been tried before and what has worked or not worked.\n",
    "\n",
    "# Data\n",
    "\n",
    "## 1. Original policy\n",
    "{flight_cancellation_policy}\n",
    "\n",
    "## 2. Functions\n",
    "{TOOLS}\n",
    "\n",
    "# Conclusion\n",
    "\n",
    "Return the improved policy exactly as written within the defined JSON. Remove all parts from the LLM's answer that are not part of the policy, and do not create additional keys.\n",
    "\"\"\"\n",
    "     }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "height": 183
   },
   "outputs": [],
   "source": [
    "import tiktoken \n",
    "\n",
    "encoding = tiktoken.get_encoding(\"o200k_base\")\n",
    "\n",
    "def num_tokens_from_messages(messages):\n",
    "    entire_input = \"\"\n",
    "    for message in messages:\n",
    "        entire_input += message[\"content\"] + \" \"\n",
    "    tokens = encoding.encode(entire_input)\n",
    "    return len(tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When improving the routine, consider these approaches:\n",
    "- Adding synthetic examples to show the model how to choose between two commonly mistaken functions\n",
    "- Given it general guidance on what attributes to look out for which it commonly mislabels\n",
    "- Being more specific with your guidance on how to follow the routine."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"background-color:#f7fff8; padding:15px; border-width:3px; border-color:#e0f0e0; border-style:solid; border-radius:6px\"> 🚨\n",
    "&nbsp; <b>High Token Warning!:</b> Running the following cell consumes many tokens. Running multiple times may result in a message regarding exceeding a monthy quota.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "height": 1407
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Iteration 1"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "num tokens 12315\n",
      "len messages 2\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "### Accuracy: 70.59%"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>request</th>\n",
       "      <th>context</th>\n",
       "      <th>expected_function</th>\n",
       "      <th>expected_inputs</th>\n",
       "      <th>actual_function</th>\n",
       "      <th>actual_inputs</th>\n",
       "      <th>is_correct</th>\n",
       "      <th>transcript</th>\n",
       "      <th>cleaned_transcript</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>My flight was canceled by the airline, and I want a refund.</td>\n",
       "      <td>I want to request a refund because my flight was canceled by the airline due to weather.</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'GHI789'}</td>\n",
       "      <td>rebook_without_fee</td>\n",
       "      <td>{'booking_reference': 'GHI789'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_BJoHEmiqzngbOnaSZUiJTe05', function=Function(arguments='{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_BJoHEmiqzngbOnaSZUiJTe05'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_6J2fcpokHAjVVcu94PmepNs6', function=Function(arguments='{\"booking_reference\":\"GHI789\"}', name='rebook_without_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_BJoHEmiqzngbOnaSZUiJTe05'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"GHI789\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>I missed my connecting flight because of the airline delay.</td>\n",
       "      <td>I want to address the issue of missing my connecting flight due to a delay caused by the airline.</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>check_compensation_eligibility</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Fj0av1tpOSzRvCytvMNeJuIt', function=Function(arguments='{\"booking_reference\":\"PQR678\",\"full_name\":\"\",\"flight_number\":\"LMN789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Fj0av1tpOSzRvCytvMNeJuIt'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YE7vTLMsxLWFiBKE1gaOErT7', function=Function(arguments='{\"booking_reference\":\"PQR678\"}', name='check_compensation_eligibility'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"PQR678\",\"full_name\":\"\",\"flight_number\":\"LMN789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Fj0av1tpOSzRvCytvMNeJuIt'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"PQR678\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>I would like to rebook my flight.</td>\n",
       "      <td>I want to rebook my flight, and I have provided my details for verification.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'VWX234', 'new_flight': 'LMG091'}</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'VWX234', 'new_flight': 'XYZ345'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_wMQ2fhYeyG8nWy6feztejb0G', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wMQ2fhYeyG8nWy6feztejb0G'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MZirIbTW35q7VjPLFTn5Z4ur', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_MZirIbTW35q7VjPLFTn5Z4ur'}, {'role': 'assistant', 'content': 'Please provide me with the new flight details, such as the flight code and date, so I can assist you with rebooking your flight.'}, {'role': 'user', 'content': 'I would like to rebook my flight for the same route, but on a different date. I am looking to change my flight to next Thursday.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_48x49PELzMKccsHA2OxdLzod', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}', name='process_change_no_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wMQ2fhYeyG8nWy6feztejb0G'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_MZirIbTW35q7VjPLFTn5Z4ur'}, {'role': 'assistant', 'content': 'Please provide me with the new flight details, such as the flight code and date, so I can assist you with rebooking your flight.'}, {'role': 'user', 'content': 'I would like to rebook my flight for the same route, but on a different date. I am looking to change my flight to next Thursday.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>I want to cancel my flight because of a medical emergency.</td>\n",
       "      <td>I want to cancel my flight due to a medical emergency.</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'Provided'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Could you please provide your booking reference, full name, and flight number to verify your identity?'}, {'role': 'user', 'content': 'My booking reference is YZA567, and my flight number is PQR678. My name is Laura Adams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_wiGANOSSO7O8uMUgVzodsz0s', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wiGANOSSO7O8uMUgVzodsz0s'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking due to the medical emergency. Additionally, if you could help me with any details on the cancellation policy or refund options, that would be great.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MK9KRD1PxuNYDOPQZnMccwEf', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Provided\"}', name='process_flexible_cancellation'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Could you please provide your booking reference, full name, and flight number to verify your identity?'}, {'role': 'user', 'content': 'My booking reference is YZA567, and my flight number is PQR678. My name is Laura Adams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wiGANOSSO7O8uMUgVzodsz0s'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking due to the medical emergency. Additionally, if you could help me with any details on the cancellation policy or refund options, that would be great.'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Provided\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>I need to change my flight within the next 7 days.</td>\n",
       "      <td>I want to change my flight because it's within the next week and I hope I can do it without any extra fees.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'LMG091'}</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'ABC123'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_tvvVZYLcz6IvgC3d9h8JM7d4', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_tvvVZYLcz6IvgC3d9h8JM7d4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. I'm hoping to do this without incurring any extra fees since it's within the next week.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ClxMz3dwmbJdotoAKTEUUQ3c', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_ClxMz3dwmbJdotoAKTEUUQ3c'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without incurring any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can process the change for you.'}, {'role': 'user', 'content': 'I’m considering changing to flight number ABC123. The new date I’m looking at is next Tuesday.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_aikR5D4Kb2L4ZlafY4S5Tqxb', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}', name='process_change_no_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_tvvVZYLcz6IvgC3d9h8JM7d4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. I'm hoping to do this without incurring any extra fees since it's within the next week.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_ClxMz3dwmbJdotoAKTEUUQ3c'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without incurring any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can process the change for you.'}, {'role': 'user', 'content': 'I’m considering changing to flight number ABC123. The new date I’m looking at is next Tuesday.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}'}}]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                        request  \\\n",
       "2   My flight was canceled by the airline, and I want a refund.   \n",
       "5   I missed my connecting flight because of the airline delay.   \n",
       "7                             I would like to rebook my flight.   \n",
       "8    I want to cancel my flight because of a medical emergency.   \n",
       "11           I need to change my flight within the next 7 days.   \n",
       "\n",
       "                                                                                                        context  \\\n",
       "2                      I want to request a refund because my flight was canceled by the airline due to weather.   \n",
       "5             I want to address the issue of missing my connecting flight due to a delay caused by the airline.   \n",
       "7                                  I want to rebook my flight, and I have provided my details for verification.   \n",
       "8                                                        I want to cancel my flight due to a medical emergency.   \n",
       "11  I want to change my flight because it's within the next week and I hope I can do it without any extra fees.   \n",
       "\n",
       "                expected_function  \\\n",
       "2             process_full_refund   \n",
       "5   prioritize_missed_connections   \n",
       "7           process_change_no_fee   \n",
       "8   process_flexible_cancellation   \n",
       "11          process_change_no_fee   \n",
       "\n",
       "                                                     expected_inputs  \\\n",
       "2                                    {'booking_reference': 'GHI789'}   \n",
       "5                                    {'booking_reference': 'PQR678'}   \n",
       "7            {'booking_reference': 'VWX234', 'new_flight': 'LMG091'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}   \n",
       "11           {'booking_reference': 'GHI123', 'new_flight': 'LMG091'}   \n",
       "\n",
       "                   actual_function  \\\n",
       "2               rebook_without_fee   \n",
       "5   check_compensation_eligibility   \n",
       "7            process_change_no_fee   \n",
       "8    process_flexible_cancellation   \n",
       "11           process_change_no_fee   \n",
       "\n",
       "                                                         actual_inputs  \\\n",
       "2                                      {'booking_reference': 'GHI789'}   \n",
       "5                                      {'booking_reference': 'PQR678'}   \n",
       "7              {'booking_reference': 'VWX234', 'new_flight': 'XYZ345'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'Provided'}   \n",
       "11             {'booking_reference': 'GHI123', 'new_flight': 'ABC123'}   \n",
       "\n",
       "    is_correct  \\\n",
       "2        False   \n",
       "5        False   \n",
       "7        False   \n",
       "8        False   \n",
       "11       False   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              transcript  \\\n",
       "2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  [{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_BJoHEmiqzngbOnaSZUiJTe05', function=Function(arguments='{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_BJoHEmiqzngbOnaSZUiJTe05'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_6J2fcpokHAjVVcu94PmepNs6', function=Function(arguments='{\"booking_reference\":\"GHI789\"}', name='rebook_without_fee'), type='function')])]   \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Fj0av1tpOSzRvCytvMNeJuIt', function=Function(arguments='{\"booking_reference\":\"PQR678\",\"full_name\":\"\",\"flight_number\":\"LMN789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Fj0av1tpOSzRvCytvMNeJuIt'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YE7vTLMsxLWFiBKE1gaOErT7', function=Function(arguments='{\"booking_reference\":\"PQR678\"}', name='check_compensation_eligibility'), type='function')])]   \n",
       "7                                                                                                                                                                                                                                                                                                                                                                                                                                 [{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_wMQ2fhYeyG8nWy6feztejb0G', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wMQ2fhYeyG8nWy6feztejb0G'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MZirIbTW35q7VjPLFTn5Z4ur', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_MZirIbTW35q7VjPLFTn5Z4ur'}, {'role': 'assistant', 'content': 'Please provide me with the new flight details, such as the flight code and date, so I can assist you with rebooking your flight.'}, {'role': 'user', 'content': 'I would like to rebook my flight for the same route, but on a different date. I am looking to change my flight to next Thursday.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_48x49PELzMKccsHA2OxdLzod', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}', name='process_change_no_fee'), type='function')])]   \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Could you please provide your booking reference, full name, and flight number to verify your identity?'}, {'role': 'user', 'content': 'My booking reference is YZA567, and my flight number is PQR678. My name is Laura Adams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_wiGANOSSO7O8uMUgVzodsz0s', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wiGANOSSO7O8uMUgVzodsz0s'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking due to the medical emergency. Additionally, if you could help me with any details on the cancellation policy or refund options, that would be great.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_MK9KRD1PxuNYDOPQZnMccwEf', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Provided\"}', name='process_flexible_cancellation'), type='function')])]   \n",
       "11  [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_tvvVZYLcz6IvgC3d9h8JM7d4', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_tvvVZYLcz6IvgC3d9h8JM7d4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. I'm hoping to do this without incurring any extra fees since it's within the next week.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_ClxMz3dwmbJdotoAKTEUUQ3c', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_ClxMz3dwmbJdotoAKTEUUQ3c'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without incurring any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can process the change for you.'}, {'role': 'user', 'content': 'I’m considering changing to flight number ABC123. The new date I’m looking at is next Tuesday.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_aikR5D4Kb2L4ZlafY4S5Tqxb', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}', name='process_change_no_fee'), type='function')])]   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               cleaned_transcript  \n",
       "2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     [{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to request a refund for my canceled flight. My booking reference is GHI789, and the flight number is ABC123. The cancellation was due to weather conditions.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI789\",\"full_name\":\"\",\"flight_number\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_BJoHEmiqzngbOnaSZUiJTe05'}, {'role': 'assistant', 'tool_call': {'name': 'rebook_without_fee', 'arguments': '{\"booking_reference\":\"GHI789\"}'}}]  \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"PQR678\",\"full_name\":\"\",\"flight_number\":\"LMN789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Fj0av1tpOSzRvCytvMNeJuIt'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"PQR678\"}'}}]  \n",
       "7                                                                                                                                                                                                                                                                                                                                                                                                                                 [{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wMQ2fhYeyG8nWy6feztejb0G'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_MZirIbTW35q7VjPLFTn5Z4ur'}, {'role': 'assistant', 'content': 'Please provide me with the new flight details, such as the flight code and date, so I can assist you with rebooking your flight.'}, {'role': 'user', 'content': 'I would like to rebook my flight for the same route, but on a different date. I am looking to change my flight to next Thursday.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}'}}]  \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'Could you please provide your booking reference, full name, and flight number to verify your identity?'}, {'role': 'user', 'content': 'My booking reference is YZA567, and my flight number is PQR678. My name is Laura Adams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_wiGANOSSO7O8uMUgVzodsz0s'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my booking due to the medical emergency. Additionally, if you could help me with any details on the cancellation policy or refund options, that would be great.'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Provided\"}'}}]  \n",
       "11  [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_tvvVZYLcz6IvgC3d9h8JM7d4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight, please. I'm hoping to do this without incurring any extra fees since it's within the next week.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_ClxMz3dwmbJdotoAKTEUUQ3c'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without incurring any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can process the change for you.'}, {'role': 'user', 'content': 'I’m considering changing to flight number ABC123. The new date I’m looking at is next Tuesday.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}'}}]  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Iteration 2"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "num tokens 14625\n",
      "len messages 3\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "### Accuracy: 58.82%"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>request</th>\n",
       "      <th>context</th>\n",
       "      <th>expected_function</th>\n",
       "      <th>expected_inputs</th>\n",
       "      <th>actual_function</th>\n",
       "      <th>actual_inputs</th>\n",
       "      <th>is_correct</th>\n",
       "      <th>transcript</th>\n",
       "      <th>cleaned_transcript</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>My flight was canceled by the airline, and I want a refund.</td>\n",
       "      <td>I want to request a refund because my flight was canceled by the airline due to weather.</td>\n",
       "      <td>process_full_refund</td>\n",
       "      <td>{'booking_reference': 'GHI789'}</td>\n",
       "      <td>offer_accommodation</td>\n",
       "      <td>{'booking_reference': 'GHI789'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'To assist you with your request for a refund due to the airline-initiated cancellation, I first need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI789, and my flight number is ABC123. My name is Emma Johnson.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_PdJm0Wt87JYErwT3gDphExg4', function=Function(arguments='{\"booking_reference\":\"GHI789\",\"full_name\":\"Emma Johnson\",\"flight_number\":\"ABC123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_PdJm0Wt87JYErwT3gDphExg4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'm looking to request a refund since my flight was canceled.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_C6bchdfB0r5zyIhe32MxFUUL', function=Function(arguments='{\"booking_reference\":\"GHI789\"}', name='offer_accommodation'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'To assist you with your request for a refund due to the airline-initiated cancellation, I first need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI789, and my flight number is ABC123. My name is Emma Johnson.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI789\",\"full_name\":\"Emma Johnson\",\"flight_number\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_PdJm0Wt87JYErwT3gDphExg4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'm looking to request a refund since my flight was canceled.'}, {'role': 'assistant', 'tool_call': {'name': 'offer_accommodation', 'arguments': '{\"booking_reference\":\"GHI789\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>I missed my connecting flight because of the airline delay.</td>\n",
       "      <td>I want to address the issue of missing my connecting flight due to a delay caused by the airline.</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>check_compensation_eligibility</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_C0UJqJXn4VUdUHGKTetcY1EP', function=Function(arguments='{\"booking_reference\":\"PQR678\"}', name='check_compensation_eligibility'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"PQR678\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>I want to know if I am eligible for compensation because of a long delay.</td>\n",
       "      <td>I want to know if I can receive compensation due to a long delay on my flight.</td>\n",
       "      <td>check_compensation_eligibility</td>\n",
       "      <td>{'booking_reference': 'STU901'}</td>\n",
       "      <td>assess_compensation_request</td>\n",
       "      <td>{'booking_reference': 'STU901'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for a long delay on my flight. My booking reference is STU901, and the flight number is ABC789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_m6gQTGG2DpDlWoOQeEK8z9GK', function=Function(arguments='{\"booking_reference\":\"STU901\",\"full_name\":\"\",\"flight_number\":\"ABC789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_m6gQTGG2DpDlWoOQeEK8z9GK'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_geTYUVFQB3LEyKF5Efa3LL0g', function=Function(arguments='{\"booking_reference\":\"STU901\"}', name='assess_compensation_request'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for a long delay on my flight. My booking reference is STU901, and the flight number is ABC789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"STU901\",\"full_name\":\"\",\"flight_number\":\"ABC789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_m6gQTGG2DpDlWoOQeEK8z9GK'}, {'role': 'assistant', 'tool_call': {'name': 'assess_compensation_request', 'arguments': '{\"booking_reference\":\"STU901\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>I would like to rebook my flight.</td>\n",
       "      <td>I want to rebook my flight, and I have provided my details for verification.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'VWX234', 'new_flight': 'LMG091'}</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'VWX234', 'new_flight': 'XYZ345'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_OaHEZYWazYER2oVF6tyupfHS', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_OaHEZYWazYER2oVF6tyupfHS'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_IKZLO0AzTykdOcKXvTL5QXeC', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_IKZLO0AzTykdOcKXvTL5QXeC'}, {'role': 'assistant', 'content': 'Please provide the new flight details, such as the flight code and date.'}, {'role': 'user', 'content': 'I would like to rebook for the same flight number, XYZ345, but for the date of October 15th.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_v2RXxsfwtQEPaJCVT3CKJZ4w', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}', name='process_change_no_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_OaHEZYWazYER2oVF6tyupfHS'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_IKZLO0AzTykdOcKXvTL5QXeC'}, {'role': 'assistant', 'content': 'Please provide the new flight details, such as the flight code and date.'}, {'role': 'user', 'content': 'I would like to rebook for the same flight number, XYZ345, but for the date of October 15th.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>I want to cancel my flight because of a medical emergency.</td>\n",
       "      <td>I want to cancel my flight due to a medical emergency.</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'Medical certificate ID'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight due to a medical emergency. My booking reference is YZA567.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_gCps2dnSoXK7pV2rJWOLHneQ', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"\",\"flight_number\":\"\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_gCps2dnSoXK7pV2rJWOLHneQ'}, {'role': 'assistant', 'content': 'Could you please provide your full name and flight number to proceed with the cancellation due to your medical emergency?'}, {'role': 'user', 'content': 'My name is Laura Adams, and my flight number is PQR678.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_U82KGIlPlIH7efWY8Fyfs7bh', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical certificate ID\"}', name='process_flexible_cancellation'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight due to a medical emergency. My booking reference is YZA567.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"\",\"flight_number\":\"\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_gCps2dnSoXK7pV2rJWOLHneQ'}, {'role': 'assistant', 'content': 'Could you please provide your full name and flight number to proceed with the cancellation due to your medical emergency?'}, {'role': 'user', 'content': 'My name is Laura Adams, and my flight number is PQR678.'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical certificate ID\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>The airline canceled my flight, and I need a hotel for the night.</td>\n",
       "      <td>I want to find a hotel because my flight was canceled by the airline.</td>\n",
       "      <td>offer_accommodation</td>\n",
       "      <td>{'booking_reference': 'EFG123'}</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'EFG123'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YkWOC3fi7rqhEaMC1xheUSC2', function=Function(arguments='{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YkWOC3fi7rqhEaMC1xheUSC2'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_rQC5XEcuqKDXYciVNlTi7Any', function=Function(arguments='{\"booking_reference\":\"EFG123\"}', name='prioritize_missed_connections'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YkWOC3fi7rqhEaMC1xheUSC2'}, {'role': 'assistant', 'tool_call': {'name': 'prioritize_missed_connections', 'arguments': '{\"booking_reference\":\"EFG123\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>I need to change my flight within the next 7 days.</td>\n",
       "      <td>I want to change my flight because it's within the next week and I hope I can do it without any extra fees.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'LMG091'}</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'ABC123'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my flight number is PQR789, and my full name is Lisa Brown.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Y0NVKADwsB39Nto6gXr5EsJF', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Y0NVKADwsB39Nto6gXr5EsJF'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight if possible. I hope to do it without any extra fees since it’s within the next week.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_OgcXnd050q2QBXFKAbGGx3yV', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_OgcXnd050q2QBXFKAbGGx3yV'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to flight number ABC123 on the date of November 15, 2023.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_IZUaWkx44BhpkIk2vBFi2MNV', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}', name='process_change_no_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my flight number is PQR789, and my full name is Lisa Brown.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Y0NVKADwsB39Nto6gXr5EsJF'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight if possible. I hope to do it without any extra fees since it’s within the next week.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_OgcXnd050q2QBXFKAbGGx3yV'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to flight number ABC123 on the date of November 15, 2023.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}'}}]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                      request  \\\n",
       "2                 My flight was canceled by the airline, and I want a refund.   \n",
       "5                 I missed my connecting flight because of the airline delay.   \n",
       "6   I want to know if I am eligible for compensation because of a long delay.   \n",
       "7                                           I would like to rebook my flight.   \n",
       "8                  I want to cancel my flight because of a medical emergency.   \n",
       "10          The airline canceled my flight, and I need a hotel for the night.   \n",
       "11                         I need to change my flight within the next 7 days.   \n",
       "\n",
       "                                                                                                        context  \\\n",
       "2                      I want to request a refund because my flight was canceled by the airline due to weather.   \n",
       "5             I want to address the issue of missing my connecting flight due to a delay caused by the airline.   \n",
       "6                                I want to know if I can receive compensation due to a long delay on my flight.   \n",
       "7                                  I want to rebook my flight, and I have provided my details for verification.   \n",
       "8                                                        I want to cancel my flight due to a medical emergency.   \n",
       "10                                        I want to find a hotel because my flight was canceled by the airline.   \n",
       "11  I want to change my flight because it's within the next week and I hope I can do it without any extra fees.   \n",
       "\n",
       "                 expected_function  \\\n",
       "2              process_full_refund   \n",
       "5    prioritize_missed_connections   \n",
       "6   check_compensation_eligibility   \n",
       "7            process_change_no_fee   \n",
       "8    process_flexible_cancellation   \n",
       "10             offer_accommodation   \n",
       "11           process_change_no_fee   \n",
       "\n",
       "                                                     expected_inputs  \\\n",
       "2                                    {'booking_reference': 'GHI789'}   \n",
       "5                                    {'booking_reference': 'PQR678'}   \n",
       "6                                    {'booking_reference': 'STU901'}   \n",
       "7            {'booking_reference': 'VWX234', 'new_flight': 'LMG091'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}   \n",
       "10                                   {'booking_reference': 'EFG123'}   \n",
       "11           {'booking_reference': 'GHI123', 'new_flight': 'LMG091'}   \n",
       "\n",
       "                   actual_function  \\\n",
       "2              offer_accommodation   \n",
       "5   check_compensation_eligibility   \n",
       "6      assess_compensation_request   \n",
       "7            process_change_no_fee   \n",
       "8    process_flexible_cancellation   \n",
       "10   prioritize_missed_connections   \n",
       "11           process_change_no_fee   \n",
       "\n",
       "                                                                       actual_inputs  \\\n",
       "2                                                    {'booking_reference': 'GHI789'}   \n",
       "5                                                    {'booking_reference': 'PQR678'}   \n",
       "6                                                    {'booking_reference': 'STU901'}   \n",
       "7                            {'booking_reference': 'VWX234', 'new_flight': 'XYZ345'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'Medical certificate ID'}   \n",
       "10                                                   {'booking_reference': 'EFG123'}   \n",
       "11                           {'booking_reference': 'GHI123', 'new_flight': 'ABC123'}   \n",
       "\n",
       "    is_correct  \\\n",
       "2        False   \n",
       "5        False   \n",
       "6        False   \n",
       "7        False   \n",
       "8        False   \n",
       "10       False   \n",
       "11       False   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               transcript  \\\n",
       "2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         [{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'To assist you with your request for a refund due to the airline-initiated cancellation, I first need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI789, and my flight number is ABC123. My name is Emma Johnson.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_PdJm0Wt87JYErwT3gDphExg4', function=Function(arguments='{\"booking_reference\":\"GHI789\",\"full_name\":\"Emma Johnson\",\"flight_number\":\"ABC123\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_PdJm0Wt87JYErwT3gDphExg4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'm looking to request a refund since my flight was canceled.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_C6bchdfB0r5zyIhe32MxFUUL', function=Function(arguments='{\"booking_reference\":\"GHI789\"}', name='offer_accommodation'), type='function')])]   \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_C0UJqJXn4VUdUHGKTetcY1EP', function=Function(arguments='{\"booking_reference\":\"PQR678\"}', name='check_compensation_eligibility'), type='function')])]   \n",
       "6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for a long delay on my flight. My booking reference is STU901, and the flight number is ABC789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_m6gQTGG2DpDlWoOQeEK8z9GK', function=Function(arguments='{\"booking_reference\":\"STU901\",\"full_name\":\"\",\"flight_number\":\"ABC789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_m6gQTGG2DpDlWoOQeEK8z9GK'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_geTYUVFQB3LEyKF5Efa3LL0g', function=Function(arguments='{\"booking_reference\":\"STU901\"}', name='assess_compensation_request'), type='function')])]   \n",
       "7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_OaHEZYWazYER2oVF6tyupfHS', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_OaHEZYWazYER2oVF6tyupfHS'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_IKZLO0AzTykdOcKXvTL5QXeC', function=Function(arguments='{\"booking_reference\":\"VWX234\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_IKZLO0AzTykdOcKXvTL5QXeC'}, {'role': 'assistant', 'content': 'Please provide the new flight details, such as the flight code and date.'}, {'role': 'user', 'content': 'I would like to rebook for the same flight number, XYZ345, but for the date of October 15th.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_v2RXxsfwtQEPaJCVT3CKJZ4w', function=Function(arguments='{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}', name='process_change_no_fee'), type='function')])]   \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight due to a medical emergency. My booking reference is YZA567.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_gCps2dnSoXK7pV2rJWOLHneQ', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"\",\"flight_number\":\"\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_gCps2dnSoXK7pV2rJWOLHneQ'}, {'role': 'assistant', 'content': 'Could you please provide your full name and flight number to proceed with the cancellation due to your medical emergency?'}, {'role': 'user', 'content': 'My name is Laura Adams, and my flight number is PQR678.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_U82KGIlPlIH7efWY8Fyfs7bh', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical certificate ID\"}', name='process_flexible_cancellation'), type='function')])]   \n",
       "10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YkWOC3fi7rqhEaMC1xheUSC2', function=Function(arguments='{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YkWOC3fi7rqhEaMC1xheUSC2'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_rQC5XEcuqKDXYciVNlTi7Any', function=Function(arguments='{\"booking_reference\":\"EFG123\"}', name='prioritize_missed_connections'), type='function')])]   \n",
       "11  [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my flight number is PQR789, and my full name is Lisa Brown.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Y0NVKADwsB39Nto6gXr5EsJF', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Y0NVKADwsB39Nto6gXr5EsJF'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight if possible. I hope to do it without any extra fees since it’s within the next week.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_OgcXnd050q2QBXFKAbGGx3yV', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_OgcXnd050q2QBXFKAbGGx3yV'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to flight number ABC123 on the date of November 15, 2023.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_IZUaWkx44BhpkIk2vBFi2MNV', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}', name='process_change_no_fee'), type='function')])]   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                cleaned_transcript  \n",
       "2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            [{'role': 'user', 'content': 'My flight was canceled by the airline, and I want a refund.'}, {'role': 'assistant', 'content': 'To assist you with your request for a refund due to the airline-initiated cancellation, I first need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI789, and my flight number is ABC123. My name is Emma Johnson.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI789\",\"full_name\":\"Emma Johnson\",\"flight_number\":\"ABC123\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_PdJm0Wt87JYErwT3gDphExg4'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I'm looking to request a refund since my flight was canceled.'}, {'role': 'assistant', 'tool_call': {'name': 'offer_accommodation', 'arguments': '{\"booking_reference\":\"GHI789\"}'}}]  \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for missing my connecting flight due to the airline delay. My booking reference is PQR678 and my flight number is LMN789.'}, {'role': 'assistant', 'tool_call': {'name': 'check_compensation_eligibility', 'arguments': '{\"booking_reference\":\"PQR678\"}'}}]  \n",
       "6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [{'role': 'user', 'content': 'I want to know if I am eligible for compensation because of a long delay.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to inquire about compensation for a long delay on my flight. My booking reference is STU901, and the flight number is ABC789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"STU901\",\"full_name\":\"\",\"flight_number\":\"ABC789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_m6gQTGG2DpDlWoOQeEK8z9GK'}, {'role': 'assistant', 'tool_call': {'name': 'assess_compensation_request', 'arguments': '{\"booking_reference\":\"STU901\"}'}}]  \n",
       "7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              [{'role': 'user', 'content': 'I would like to rebook my flight.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to rebook my flight. My booking reference is VWX234, and my flight number is XYZ345.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"VWX234\",\"full_name\":\"\",\"flight_number\":\"XYZ345\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_OaHEZYWazYER2oVF6tyupfHS'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"VWX234\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_IKZLO0AzTykdOcKXvTL5QXeC'}, {'role': 'assistant', 'content': 'Please provide the new flight details, such as the flight code and date.'}, {'role': 'user', 'content': 'I would like to rebook for the same flight number, XYZ345, but for the date of October 15th.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"VWX234\",\"new_flight\":\"XYZ345\"}'}}]  \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to cancel my flight due to a medical emergency. My booking reference is YZA567.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"\",\"flight_number\":\"\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_gCps2dnSoXK7pV2rJWOLHneQ'}, {'role': 'assistant', 'content': 'Could you please provide your full name and flight number to proceed with the cancellation due to your medical emergency?'}, {'role': 'user', 'content': 'My name is Laura Adams, and my flight number is PQR678.'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical certificate ID\"}'}}]  \n",
       "10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 [{'role': 'user', 'content': 'The airline canceled my flight, and I need a hotel for the night.'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I need a hotel for the night because my flight was canceled. My booking reference is EFG123, and my flight number is XYZ789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"EFG123\",\"full_name\":\"\",\"flight_number\":\"XYZ789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YkWOC3fi7rqhEaMC1xheUSC2'}, {'role': 'assistant', 'tool_call': {'name': 'prioritize_missed_connections', 'arguments': '{\"booking_reference\":\"EFG123\"}'}}]  \n",
       "11  [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can help you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my flight number is PQR789, and my full name is Lisa Brown.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_Y0NVKADwsB39Nto6gXr5EsJF'}, {'role': 'assistant', 'content': 'How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?'}, {'role': 'user', 'content': 'I would like to change my flight if possible. I hope to do it without any extra fees since it’s within the next week.'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_OgcXnd050q2QBXFKAbGGx3yV'}, {'role': 'assistant', 'content': 'Since you have a flexible ticket, you can change your flight without any extra fees. Please provide me with the new flight details, such as the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to flight number ABC123 on the date of November 15, 2023.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}'}}]  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Iteration 3"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "num tokens 15602\n",
      "len messages 4\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "### Accuracy: 82.35%"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>request</th>\n",
       "      <th>context</th>\n",
       "      <th>expected_function</th>\n",
       "      <th>expected_inputs</th>\n",
       "      <th>actual_function</th>\n",
       "      <th>actual_inputs</th>\n",
       "      <th>is_correct</th>\n",
       "      <th>transcript</th>\n",
       "      <th>cleaned_transcript</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>I missed my connecting flight because of the airline delay.</td>\n",
       "      <td>I want to address the issue of missing my connecting flight due to a delay caused by the airline.</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'PQR678'}</td>\n",
       "      <td>prioritize_missed_connections</td>\n",
       "      <td>{'booking_reference': 'N/A'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_EWCjjkK3HFAuiFWvHnSsr2vj', function=Function(arguments='{\"booking_reference\":\"N/A\"}', name='prioritize_missed_connections'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'tool_call': {'name': 'prioritize_missed_connections', 'arguments': '{\"booking_reference\":\"N/A\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>I want to cancel my flight because of a medical emergency.</td>\n",
       "      <td>I want to cancel my flight due to a medical emergency.</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}</td>\n",
       "      <td>process_flexible_cancellation</td>\n",
       "      <td>{'booking_reference': 'YZA567', 'medical_certificate': 'Medical emergency'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_1pDHtAGAmvUoTE2WB9QliCfR', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_1pDHtAGAmvUoTE2WB9QliCfR'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Qq2CewjLfjoYIBiip9vPcyqo', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}', name='process_flexible_cancellation'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_1pDHtAGAmvUoTE2WB9QliCfR'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}'}}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>I need to change my flight within the next 7 days.</td>\n",
       "      <td>I want to change my flight because it's within the next week and I hope I can do it without any extra fees.</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'LMG091'}</td>\n",
       "      <td>process_change_no_fee</td>\n",
       "      <td>{'booking_reference': 'GHI123', 'new_flight': 'ABC123'}</td>\n",
       "      <td>False</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YCfHHxafHOLC4A7cpnXSxFFZ', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YCfHHxafHOLC4A7cpnXSxFFZ'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_vyOuABw59GDwFBUo5pZ0G8MU', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_vyOuABw59GDwFBUo5pZ0G8MU'}, {'role': 'assistant', 'content': 'Your ticket is flexible, which allows for changes without additional fees. Please provide me with the new flight details, including the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to Flight Number ABC123 on March 21, 2024.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_5tI9HIckbNnFcHS7uZBXEsa7', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}', name='process_change_no_fee'), type='function')])]</td>\n",
       "      <td>[{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YCfHHxafHOLC4A7cpnXSxFFZ'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_vyOuABw59GDwFBUo5pZ0G8MU'}, {'role': 'assistant', 'content': 'Your ticket is flexible, which allows for changes without additional fees. Please provide me with the new flight details, including the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to Flight Number ABC123 on March 21, 2024.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}'}}]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                        request  \\\n",
       "5   I missed my connecting flight because of the airline delay.   \n",
       "8    I want to cancel my flight because of a medical emergency.   \n",
       "11           I need to change my flight within the next 7 days.   \n",
       "\n",
       "                                                                                                        context  \\\n",
       "5             I want to address the issue of missing my connecting flight due to a delay caused by the airline.   \n",
       "8                                                        I want to cancel my flight due to a medical emergency.   \n",
       "11  I want to change my flight because it's within the next week and I hope I can do it without any extra fees.   \n",
       "\n",
       "                expected_function  \\\n",
       "5   prioritize_missed_connections   \n",
       "8   process_flexible_cancellation   \n",
       "11          process_change_no_fee   \n",
       "\n",
       "                                                     expected_inputs  \\\n",
       "5                                    {'booking_reference': 'PQR678'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'MC1234'}   \n",
       "11           {'booking_reference': 'GHI123', 'new_flight': 'LMG091'}   \n",
       "\n",
       "                  actual_function  \\\n",
       "5   prioritize_missed_connections   \n",
       "8   process_flexible_cancellation   \n",
       "11          process_change_no_fee   \n",
       "\n",
       "                                                                  actual_inputs  \\\n",
       "5                                                  {'booking_reference': 'N/A'}   \n",
       "8   {'booking_reference': 'YZA567', 'medical_certificate': 'Medical emergency'}   \n",
       "11                      {'booking_reference': 'GHI123', 'new_flight': 'ABC123'}   \n",
       "\n",
       "    is_correct  \\\n",
       "5        False   \n",
       "8        False   \n",
       "11       False   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                transcript  \\\n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_EWCjjkK3HFAuiFWvHnSsr2vj', function=Function(arguments='{\"booking_reference\":\"N/A\"}', name='prioritize_missed_connections'), type='function')])]   \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_1pDHtAGAmvUoTE2WB9QliCfR', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_1pDHtAGAmvUoTE2WB9QliCfR'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Qq2CewjLfjoYIBiip9vPcyqo', function=Function(arguments='{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}', name='process_flexible_cancellation'), type='function')])]   \n",
       "11  [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_YCfHHxafHOLC4A7cpnXSxFFZ', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}', name='verify_identity'), type='function')]), {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YCfHHxafHOLC4A7cpnXSxFFZ'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_vyOuABw59GDwFBUo5pZ0G8MU', function=Function(arguments='{\"booking_reference\":\"GHI123\"}', name='check_ticket_type'), type='function')]), {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_vyOuABw59GDwFBUo5pZ0G8MU'}, {'role': 'assistant', 'content': 'Your ticket is flexible, which allows for changes without additional fees. Please provide me with the new flight details, including the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to Flight Number ABC123 on March 21, 2024.'}, ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_5tI9HIckbNnFcHS7uZBXEsa7', function=Function(arguments='{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}', name='process_change_no_fee'), type='function')])]   \n",
       "\n",
       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 cleaned_transcript  \n",
       "5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           [{'role': 'user', 'content': 'I missed my connecting flight because of the airline delay.'}, {'role': 'assistant', 'tool_call': {'name': 'prioritize_missed_connections', 'arguments': '{\"booking_reference\":\"N/A\"}'}}]  \n",
       "8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         [{'role': 'user', 'content': 'I want to cancel my flight because of a medical emergency.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is YZA567, my flight number is PQR678, and my name is Laura Adams.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"YZA567\",\"full_name\":\"Laura Adams\",\"flight_number\":\"PQR678\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_1pDHtAGAmvUoTE2WB9QliCfR'}, {'role': 'assistant', 'tool_call': {'name': 'process_flexible_cancellation', 'arguments': '{\"booking_reference\":\"YZA567\",\"medical_certificate\":\"Medical emergency\"}'}}]  \n",
       "11  [{'role': 'user', 'content': 'I need to change my flight within the next 7 days.'}, {'role': 'assistant', 'content': 'I can assist you with that. First, I need to verify your identity. Please provide me with your booking reference, full name, and flight number.'}, {'role': 'user', 'content': 'My booking reference is GHI123, my full name is Lisa Brown, and my flight number is PQR789.'}, {'role': 'assistant', 'tool_call': {'name': 'verify_identity', 'arguments': '{\"booking_reference\":\"GHI123\",\"full_name\":\"Lisa Brown\",\"flight_number\":\"PQR789\"}'}}, {'role': 'tool', 'content': 'True', 'tool_call_id': 'call_YCfHHxafHOLC4A7cpnXSxFFZ'}, {'role': 'assistant', 'tool_call': {'name': 'check_ticket_type', 'arguments': '{\"booking_reference\":\"GHI123\"}'}}, {'role': 'tool', 'content': 'Ticket type: flexible\n",
       "Fare rules: ', 'tool_call_id': 'call_vyOuABw59GDwFBUo5pZ0G8MU'}, {'role': 'assistant', 'content': 'Your ticket is flexible, which allows for changes without additional fees. Please provide me with the new flight details, including the flight code and date, so I can proceed with the change.'}, {'role': 'user', 'content': 'I would like to change my flight to Flight Number ABC123 on March 21, 2024.'}, {'role': 'assistant', 'tool_call': {'name': 'process_change_no_fee', 'arguments': '{\"booking_reference\":\"GHI123\",\"new_flight\":\"ABC123\"}'}}]  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# start with data from first run\n",
    "routines = [flight_cancellation_routine]\n",
    "results = [df]\n",
    "accuracies = [round(accuracy * 100, 2)]\n",
    "o1_messages = start_o1_message.copy()\n",
    "\n",
    "MAX_TOKENS = 120000\n",
    "\n",
    "for i in range(3):\n",
    "    display(Markdown(f\"## Iteration {i+1}\"))\n",
    "    \n",
    "    # Build a candidate message with the most recent eval (full table)\n",
    "    prev_results = results[-1]\n",
    "    pf = prev_results.drop(columns=['transcript'])\n",
    "\n",
    "    new_eval_content = (\n",
    "        f\"## 3. Routine instructions:\\n\"\n",
    "        f\"{routines[-1]}\\n\"\n",
    "        f\"## 4. Results table:\\n\"\n",
    "        f\"{pf.to_json(orient='records')}\\n\"\n",
    "    )\n",
    "    \n",
    "    # Check token count before adding the newest eval\n",
    "    candidate_messages = o1_messages + [{\"role\": \"user\", \"content\": new_eval_content}]\n",
    "    if num_tokens_from_messages(candidate_messages) <= MAX_TOKENS:\n",
    "        o1_messages.append({\"role\":\"user\", \"content\": new_eval_content})\n",
    "    else:\n",
    "        # If we can't even fit the newest eval, consider truncation or error handling\n",
    "        # For simplicity, we skip truncation logic here\n",
    "        print(\"excluding eval content {num_tokens_from_messages(candidate_messages)}\")\n",
    "        pass\n",
    "\n",
    "    print(f\"num tokens {num_tokens_from_messages(o1_messages)}\")\n",
    "    print(f\"len messages {len(o1_messages)}\")\n",
    "\n",
    "\n",
    "    # Get the updated routine from the metaprompting assistant\n",
    "    temp_routine_json = get_openai_response(o1_messages\n",
    "                                            , O1_MODEL\n",
    "                                            ,response_format=OUTPUT_SCHEMA)\n",
    "    #temp_routine_str = temp_routine_json.strip(\"json```\").strip(\"```\")\n",
    "    new_routine = json.loads(temp_routine_json)[\"final_answer\"]\n",
    "    routines.append(new_routine)\n",
    "\n",
    "    eval_df = pd.read_csv('evals/policyEvals.csv')\n",
    "\n",
    "    # Evaluate the function calls with the current policy on the gpt-4o model\n",
    "    eval_df, accuracy = evaluate_function_calls(\n",
    "        eval_df,\n",
    "        new_routine,\n",
    "        GPT_MODEL,\n",
    "        i\n",
    "    )\n",
    "    accuracies.append(round(accuracy * 100, 2))\n",
    "    results.append(eval_df)\n",
    "    display(Markdown(f\"### Accuracy: {accuracy:.2%}\"))\n",
    "\n",
    "    #list the IDs of the failed rows\n",
    "    failed_ids = eval_df[eval_df['is_correct'] == False].index.tolist()\n",
    "    display(eval_df.loc[failed_ids])\n",
    "    eval_df.to_csv(f'evals/results_run_{i+1}.csv', index=False)\n",
    "\n",
    "    # Instantiate fresh o1 message\n",
    "    o1_messages = start_o1_message.copy()\n",
    "\n",
    "    # Attempt to include older examples (omitting 'transcript') within token limit\n",
    "    older_results = results[:-1][::-1]  # all except the newest, reversed for most recent first\n",
    "    for old_eval_df in older_results:\n",
    "        if 'transcript' in old_eval_df.columns:\n",
    "            truncated_df = old_eval_df.drop(columns=['transcript','cleaned_transcript'])\n",
    "        else:\n",
    "            truncated_df = old_eval_df\n",
    "\n",
    "        old_content = \"## Older eval (no transcript):\\n\" + truncated_df.to_json(orient='records')\n",
    "        candidate_messages = o1_messages + [{\"role\": \"user\", \"content\": old_content}]\n",
    "        if num_tokens_from_messages(candidate_messages) <= MAX_TOKENS:\n",
    "            o1_messages.append({\"role\": \"user\", \"content\": old_content})\n",
    "        else:\n",
    "            # If we exceed token limits, stop adding older examples\n",
    "            break\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "height": 149
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAHHCAYAAACle7JuAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAZPJJREFUeJzt3Xd4U2X/BvA7SdM9aOmGLgqUPUpbKFMZIiKCMgQZLWX9EEX0VV9wIaIivjheFRGFDmTLEkVRhsimhVJo2dANHZTSvZPn90chr6UU2pL2JOn9ua5clz05Se4eYvLtc57zfWRCCAEiIiIiPSSXOgARERFRfbGQISIiIr3FQoaIiIj0FgsZIiIi0lssZIiIiEhvsZAhIiIivcVChoiIiPQWCxkiIiLSWyxkiIiISG+xkCEi0gHh4eGQyWRITEyUOgqRXmEhQ6RDvv32W8hkMvTs2VPqKNRAPv74Y+zYsUPqGEQGQ8a1loh0R58+fXDjxg0kJibiypUraN26tdSRSMssLS0xZswYhIeHV9muUqlQXl4OExMTyGQyacIR6SGOyBDpiISEBBw9ehSff/45HBwcsG7dOqkj1aiwsFDqCDqroqICZWVldX6cQqGAqakpixiiOmIhQ6Qj1q1bB1tbWwwfPhxjxoypsZDJycnBq6++Ck9PT5iYmKBly5aYMmUKsrKyNPuUlJTg/fffR9u2bWFqagoXFxc899xzuHbtGgDgwIEDkMlkOHDgQJXnTkxMhEwmqzJaEBwcDEtLS1y7dg1PPfUUrKysMHHiRADAoUOHMHbsWLi7u8PExARubm549dVXUVxcXC33xYsXMW7cODg4OMDMzAw+Pj54++23AQB//fUXZDIZtm/fXu1x69evh0wmw7Fjxx54/OLj4zF27FjY2dnB3NwcvXr1wq5duzT3Z2RkwMjICIsWLar22EuXLkEmk+Gbb76pcpznzZsHNzc3mJiYoHXr1li6dCnUanW147Vs2TJ8+eWX8Pb2homJCc6fP3/fjDKZDIWFhYiIiIBMJoNMJkNwcDCA+8+R8fT0xNNPP40DBw7Az88PZmZm6Ny5s+bfbdu2bejcuTNMTU3Ro0cPnD59utprXrx4EWPGjIGdnR1MTU3h5+eHnTt3PvBYEukTI6kDEFGldevW4bnnnoOxsTEmTJiAFStWICoqCv7+/pp9CgoK0K9fP1y4cAEhISHw9fVFVlYWdu7cidTUVNjb20OlUuHpp5/Gvn37MH78eLzyyivIz8/Hnj17EBcXB29v7zpnq6iowNChQ9G3b18sW7YM5ubmAICffvoJRUVFmD17Npo3b47IyEh8/fXXSE1NxU8//aR5/NmzZ9GvXz8olUrMnDkTnp6euHbtGn755Rd89NFHeOyxx+Dm5oZ169bh2WefrXZcvL29ERgYWGO+jIwM9O7dG0VFRZg7dy6aN2+OiIgIPPPMM9iyZQueffZZODk5YcCAAdi8eTMWLlxY5fGbNm2CQqHA2LFjAQBFRUUYMGAArl+/jlmzZsHd3R1Hjx7FggULkJaWhi+//LLK48PCwlBSUoKZM2fCxMQEdnZ29835448/Yvr06QgICMDMmTMB4KH/HlevXsULL7yAWbNmYdKkSVi2bBlGjBiB7777Dm+99RZefPFFAMCSJUswbtw4XLp0CXJ55d+o586dQ58+fdCiRQvMnz8fFhYW2Lx5M0aNGoWtW7dWO9ZEekkQkeROnjwpAIg9e/YIIYRQq9WiZcuW4pVXXqmy33vvvScAiG3btlV7DrVaLYQQIjQ0VAAQn3/+eY37/PXXXwKA+Ouvv6rcn5CQIACIsLAwzbagoCABQMyfP7/a8xUVFVXbtmTJEiGTyURSUpJmW//+/YWVlVWVbf/MI4QQCxYsECYmJiInJ0ezLTMzUxgZGYmFCxdWe51/mjdvngAgDh06pNmWn58vvLy8hKenp1CpVEIIIVauXCkAiNjY2CqP79Chgxg4cKDm58WLFwsLCwtx+fLlKvvNnz9fKBQKkZycLIT43/GytrYWmZmZD8x4l4WFhQgKCqq2PSwsTAAQCQkJmm0eHh4CgDh69Khm2x9//CEACDMzsyrH8+7v9s9/00GDBonOnTuLkpISzTa1Wi169+4t2rRpU6u8RLqOp5aIdMC6devg5OSExx9/HEDlKYjnn38eGzduhEql0uy3detWdO3a9b5/Sd+dW7F161bY29vj5ZdfrnGf+pg9e3a1bWZmZpr/LiwsRFZWFnr37g0hhOY0x82bN3Hw4EGEhITA3d29xjxTpkxBaWkptmzZotm2adMmVFRUYNKkSQ/M9ttvvyEgIAB9+/bVbLO0tMTMmTORmJioOdXz3HPPwcjICJs2bdLsFxcXh/Pnz+P555/XbPvpp5/Qr18/2NraIisrS3MbPHgwVCoVDh48WOX1R48eDQcHhwdmrK8OHTpUGY26e0XbwIEDqxzPu9vj4+MBANnZ2di/fz/GjRuH/Px8ze9w69YtDB06FFeuXMH169cbJDNRY2IhQyQxlUqFjRs34vHHH0dCQgKuXr2Kq1evomfPnsjIyMC+ffs0+167dg2dOnV64PNdu3YNPj4+MDLS3pljIyMjtGzZstr25ORkBAcHw87ODpaWlnBwcMCAAQMAALm5uQD+98X6sNzt2rWDv79/lblB69atQ69evR569VZSUhJ8fHyqbW/fvr3mfgCwt7fHoEGDsHnzZs0+mzZtgpGREZ577jnNtitXrmD37t1wcHCochs8eDAAIDMzs8rreHl5PTDfo7i3+LOxsQEAuLm53Xf77du3AVSekhJC4N133632e9w9tXbv70GkjzhHhkhi+/fvR1paGjZu3IiNGzdWu3/dunV44okntPqaNY3M/HP0559MTEw08y7+ue+QIUOQnZ2Nf//732jXrh0sLCxw/fp1BAcHV5kUW1tTpkzBK6+8gtTUVJSWluL48eNVJuBqw/jx4zF16lTExMSgW7du2Lx5MwYNGgR7e3vNPmq1GkOGDMGbb7553+do27ZtlZ//OTKlbQqFok7bxZ2OGneP/+uvv46hQ4fed19e3k+GgIUMkcTWrVsHR0dHLF++vNp927Ztw/bt2/Hdd9/BzMwM3t7eiIuLe+DzeXt748SJEygvL4dSqbzvPra2tgAqr8z5p7sjF7URGxuLy5cvIyIiAlOmTNFs37NnT5X9WrVqBQAPzQ1UFhmvvfYaNmzYgOLiYiiVyiqnfGri4eGBS5cuVdt+8eJFzf13jRo1CrNmzdKcXrp8+TIWLFhQ5XHe3t4oKCjQjMBoU2NdXn33uCuVygb5PYh0BU8tEUmouLgY27Ztw9NPP40xY8ZUu7300kvIz8/XXC47evRonDlz5r6XKd/9S3z06NHIysq670jG3X08PDygUCiqzfX49ttva5397oiA+EdPTSEE/vvf/1bZz8HBAf3790doaCiSk5Pvm+cue3t7DBs2DGvXrsW6devw5JNPVhkpqclTTz2FyMjIKpdoFxYW4vvvv4enpyc6dOig2d6sWTMMHToUmzdvxsaNG2FsbIxRo0ZVeb5x48bh2LFj+OOPP6q9Vk5ODioqKh6aqSYWFhbVCsiG4OjoiMceewwrV65EWlpatftv3rzZ4BmIGgNHZIgktHPnTuTn5+OZZ5657/29evXSNMd7/vnn8cYbb2DLli0YO3YsQkJC0KNHD2RnZ2Pnzp347rvv0LVrV0yZMgVr1qzBa6+9hsjISPTr1w+FhYXYu3cvXnzxRYwcORI2NjYYO3Ysvv76a8hkMnh7e+PXX3+t05yJdu3awdvbG6+//jquX78Oa2trbN26VTNH45+++uor9O3bF76+vpg5cya8vLyQmJiIXbt2ISYmpsq+U6ZMwZgxYwAAixcvrlWW+fPnY8OGDRg2bBjmzp0LOzs7REREICEhAVu3bq12Wuz555/HpEmT8O2332Lo0KFo1qxZlfvfeOMN7Ny5E08//TSCg4PRo0cPFBYWIjY2Flu2bEFiYmKtCqz76dGjB/bu3YvPP/8crq6u8PLyarAlKZYvX46+ffuic+fOmDFjBlq1aoWMjAwcO3YMqampOHPmTIO8LlGjku6CKSIaMWKEMDU1FYWFhTXuExwcLJRKpcjKyhJCCHHr1i3x0ksviRYtWghjY2PRsmVLERQUpLlfiMrLot9++23h5eUllEqlcHZ2FmPGjBHXrl3T7HPz5k0xevRoYW5uLmxtbcWsWbNEXFzcfS+/trCwuG+28+fPi8GDBwtLS0thb28vZsyYIc6cOVPtOYQQIi4uTjz77LOiWbNmwtTUVPj4+Ih333232nOWlpYKW1tbYWNjI4qLi2tzGIUQQly7dk2MGTNG8/wBAQHi119/ve++eXl5wszMTAAQa9euve8++fn5YsGCBaJ169bC2NhY2Nvbi969e4tly5aJsrIyIcT/Lr/+z3/+U+ucFy9eFP3799e8/t1LsWu6/Hr48OHVngOAmDNnTpVtNWW5du2amDJlinB2dhZKpVK0aNFCPP3002LLli21zkyky7jWEhHplIqKCri6umLEiBFYvXq11HGISMdxjgwR6ZQdO3bg5s2bVSYQExHVhCMyRKQTTpw4gbNnz2Lx4sWwt7dHdHS01JGISA9wRIaIdMKKFSswe/ZsODo6Ys2aNVLHISI9wREZIiIi0lsckSEiIiK9xUKGiIiI9JbBN8RTq9W4ceMGrKysGq01OBERET0aIQTy8/Ph6uparanlPxl8IXPjxo1qq8QSERGRfkhJSUHLli1rvN/gCxkrKysAlQfC2tpa4jRERERUG3l5eXBzc9N8j9fE4AuZu6eTrK2tWcgQERHpmYdNC+FkXyIiItJbLGSIiIhIb7GQISIiIr3FQoaIiIj0FgsZIiIi0lssZIiIiEhvsZAhIiIivcVChoiIiPQWCxkiIiLSWwbf2ZeIiIi0T6UWiEzIRmZ+CRytTBHgZQeFvPEXZ2YhQ0RERHWyOy4Ni345j7TcEs02FxtTLBzRAU92cmnULDy1RERERLW2Oy4Ns9dGVyliACA9twSz10Zjd1xao+ZhIUNERES1olILLPrlPMR97ru7bdEv56FS32+PhsFChoiIiGolMiG72kjMPwkAabkliEzIbrRMLGSIiIioVjLzay5i6rOfNrCQISIiolpxtDLV6n7awEKGiIiIaiXAyw4uNqao6SJrGSqvXgrwsmu0TCxkiIiIqFYUchkWjuhw38m+d4ubhSM6NGo/GRYyREREVGtDOzrDxbr6qSNnG1OsmOTb6H1k2BCPiIiIau3vyzeRllcCC2MFvprQHQWlFezsS0RERPph1aEEAMD4AHcMau8kcRqeWiIiIqJaupCWh8NXsyCXAcG9PaWOA4CFDBEREdXS3dGYYZ1d4GZnLnGaSixkiIiI6KEy80qw88x1AMD0vl4Sp/kfFjJERET0UBHHElGuEvDzsEV3d1up42iwkCEiIqIHKiqrwLoTyQCA6f10ZzQGYCFDRERED7H1VCpyisrhbmeOIR2cpY5TBQsZIiIiqpFaLbD6cOUk35A+npL0inkQFjJERERUo70XMpB4qwjWpkYY6+cmdZxqWMgQERFRjVbdGY15oacHLEx0r48uCxkiIiK6r7OpOYhMyIaRXKYzDfDuxUKGiIiI7utuA7wRXV3hbFN9oUhdwEKGiIiIqrmeU4xdsWkAgGk61ADvXixkiIiIqJqIo4lQqQUCWzVHpxY2UsepEQsZIiIiqiK/pBwb7jTAm9Ffd0djABYyREREdI/NJ1ORX1qBVg4WeKyto9RxHoiFDBEREWlUqNQIvXPJ9fS+rSDXsQZ492IhQ0RERBp/nMvA9Zxi2FkY4znfFlLHeSgWMkRERAQAEELgh0PxAIBJvTxgqlRInOjhWMgQERERACA6+TZiUnJgbCTH5F4eUsepFRYyREREBAD44WDl3Jhnu7WAg5WJxGlqh4UMERERIelWIf44nw4AmNZPty+5/icWMkRERISwI4kQAhjQ1gFtnaykjlNrkhYyKpUK7777Lry8vGBmZgZvb28sXrwYQgjNPkIIvPfee3BxcYGZmRkGDx6MK1euSJiaiIjIsOQWlWPzyRQAwHQ9Go0BJC5kli5dihUrVuCbb77BhQsXsHTpUnz66af4+uuvNft8+umn+Oqrr/Ddd9/hxIkTsLCwwNChQ1FSUiJhciIiIsOxPjIZRWUqtHO2Qt/W9lLHqRMjKV/86NGjGDlyJIYPHw4A8PT0xIYNGxAZGQmgcjTmyy+/xDvvvIORI0cCANasWQMnJyfs2LED48ePlyw7ERGRISirUCP8aOUk32l9vSCT6XYDvHtJOiLTu3dv7Nu3D5cvXwYAnDlzBocPH8awYcMAAAkJCUhPT8fgwYM1j7GxsUHPnj1x7Nix+z5naWkp8vLyqtyIiIjo/nbF3kBGXikcrEzwTDdXqePUmaQjMvPnz0deXh7atWsHhUIBlUqFjz76CBMnTgQApKdXzp52cnKq8jgnJyfNffdasmQJFi1a1LDBiYiIDIAQQnPJdXBvT5gY6X4DvHtJOiKzefNmrFu3DuvXr0d0dDQiIiKwbNkyRERE1Ps5FyxYgNzcXM0tJSVFi4mJiIgMx7H4WziflgdTpRwvBLhLHadeJB2ReeONNzB//nzNXJfOnTsjKSkJS5YsQVBQEJydnQEAGRkZcHFx0TwuIyMD3bp1u+9zmpiYwMREP5r4EBERSWnVocrRmLE93GBrYSxxmvqRdESmqKgIcnnVCAqFAmq1GgDg5eUFZ2dn7Nu3T3N/Xl4eTpw4gcDAwEbNSkREZEiuZhZg/8VMyGRASF/9uuT6nyQdkRkxYgQ++ugjuLu7o2PHjjh9+jQ+//xzhISEAABkMhnmzZuHDz/8EG3atIGXlxfeffdduLq6YtSoUVJGJyIi0murD1eOxgxu7wQvewuJ09SfpIXM119/jXfffRcvvvgiMjMz4erqilmzZuG9997T7PPmm2+isLAQM2fORE5ODvr27Yvdu3fD1NRUwuRERET661ZBKbZFpwIApuvxaAwAyMQ/2+gaoLy8PNjY2CA3NxfW1tZSxyEiIpLcf/dewRd7L6NLSxv8PKePTvaOqe33N9daIiIiakJKylX48XgiAP1sgHcvFjJERERNyM8x15FVUAZXG1M81dnl4Q/QcSxkiIiImgghhOaS6+A+nlAq9L8M0P/fgIiIiGrl78s3cSWzABbGCozX0wZ492IhQ0RE1ETcveT6eX93WJsqJU6jHSxkiIiImoALaXk4dCULchkwtY+n1HG0hoUMERFRE3B3NGZYJxe42ZlLnEZ7WMgQEREZuMy8Evwccx0AML2ffjfAuxcLGSIiIgO35lgSylUCPTxs0d3dVuo4WsVChoiIyIAVlVVg7YkkAMAMAxuNAVjIEBERGbSt0deRU1QOdztzDOngLHUcrWMhQ0REZKDUaoHQO5N8Q/p4QiHX7+UI7oeFDBERkYHadzETCVmFsDY1wlg/N6njNAgWMkRERAbqh0PxAIAXenrAwsRI4jQNg4UMERGRATqbmoPIhGwYyWUI6u0hdZwGw0KGiIjIAN1dHHJEV1e42JhJnKbhsJAhIiIyMDdyirErNg0AMK2v4V1y/U8sZIiIiAxM+NFEqNQCga2ao1MLG6njNCgWMkRERAakoLQCG04kAzC85Qjuh4UMERGRAdkUlYL80gq0crDA4z6OUsdpcCxkiIiIDESFSq1pgDe9byvIDbAB3r1YyBARERmIP85l4HpOMewsjPGcbwup4zQKFjJEREQGQAihaYA3qZcHTJUKiRM1DhYyREREBiA6+TZiUnJgbCTH5F6G2wDvXixkiIiIDMAPByvnxjzbrQUcrEwkTtN4WMgQERHpuaRbhfjjfDoAYFoTuOT6n1jIEBER6bmwI4kQAhjQ1gFtnaykjtOoWMgQERHpsdyicmw+mQKgaTTAuxcLGSIiIj22PjIZRWUqtHO2Qt/W9lLHaXQsZIiIiPRUWYUa4UcrJ/lO6+sFmczwG+Ddi4UMERGRntoVewMZeaVwsDLBM91cpY4jCRYyREREekgIgVWHKkdjggI9YGLUNBrg3YuFDBERkR46Fn8L527kwVQpx8SeTacB3r1YyBAREemh1XdGY8b0aAlbC2OJ00iHhQwREZGeuZpZgH0XMyGTASF9mt4l1//EQoaIiEjPhB6pHI0Z1M4JrRwsJU4jLRYyREREeuRWQSm2nkoFAMxogg3w7sVChoiISI+sO5GM0go1OrewQYCXndRxJMdChoiISE+UlKuw5lgigMrlCJpiA7x7sZAhIiLSEztjbiCroAwuNqZ4qrOL1HF0AgsZIiIiPSCEwKrD8QCAqX08oVTwKxxgIUNERKQXDl7JwuWMAlgYK/C8v7vUcXQGCxkiIiI9sOpQ5WjM8/7usDFTSpxGd7CQISIi0nEX0/Nw6EoW5LLK00r0PyxkiIiIdNzdxSGHdXKBm525xGl0CwsZIiIiHZaZV4KfY64DAKaxAV41LGSIiIh02JpjSShXCfTwsIWvu63UcXQOCxkiIiIdVVymwtoTSQC4HEFNWMgQERHpqC3RqcgpKoe7nTmGdHCWOo5OYiFDRESkg9RqgdDDlZN8Q/p4QiHncgT3w0KGiIhIB+27mImErEJYmxphrJ+b1HF0FgsZIiIiHfTDnQZ4L/T0gIWJkcRpdBcLGSIiIh1zNjUHkQnZMJLLENTbQ+o4Oo2FDBERkY652wBvRFdXuNiYSZxGt7GQISIi0iE3coqxKzYNADCtLy+5fhgWMkRERDok/GgiVGqBwFbN0amFjdRxdB4LGSIiIh1RUFqBDSeSAQDT2QCvVljIEBER6YhNUSnIL61AKwcLPO7jKHUcvcBChoiISAdUqNQIO1I5yXdaXy/I2QCvVljIEBER6YA/zmUg9XYxbM2VGO3bUuo4eoOFDBERkQ5YdbiyAd7kXh4wVSokTqM/JC1kPD09IZPJqt3mzJkDAHjssceq3fd///d/UkYmIiLSulNJ2TidnANjhRyTAz2ljqNXJO15HBUVBZVKpfk5Li4OQ4YMwdixYzXbZsyYgQ8++EDzs7m5eaNmJCIiamh3G+CN6u4KBysTidPoF0kLGQcHhyo/f/LJJ/D29saAAQM028zNzeHszKXLiYjIMCXfKsIf59IBANP7tZI4jf7RmTkyZWVlWLt2LUJCQiCT/W+m9rp162Bvb49OnTphwYIFKCoqeuDzlJaWIi8vr8qNiIhIV4UeSYBaAP3bOqCtk5XUcfSOziynuWPHDuTk5CA4OFiz7YUXXoCHhwdcXV1x9uxZ/Pvf/8alS5ewbdu2Gp9nyZIlWLRoUSMkJiIiejS5ReXYfDIFADCDDfDqRSaEEFKHAIChQ4fC2NgYv/zyS4377N+/H4MGDcLVq1fh7e19331KS0tRWlqq+TkvLw9ubm7Izc2FtbW11nMTERHV13d/X8Mnv19EO2cr/P5KvypnJJq6vLw82NjYPPT7WydGZJKSkrB3794HjrQAQM+ePQHggYWMiYkJTEw4UYqIiHRbWYUa4UcSAVQ2wGMRUz86MUcmLCwMjo6OGD58+AP3i4mJAQC4uLg0QioiIqKG81tsGtLzSuBgZYJnurlKHUdvST4io1arERYWhqCgIBgZ/S/OtWvXsH79ejz11FNo3rw5zp49i1dffRX9+/dHly5dJExMRET0aIQQ+OFQZQO8oEAPmBixAV59SV7I7N27F8nJyQgJCamy3djYGHv37sWXX36JwsJCuLm5YfTo0XjnnXckSkpERKQdx+Ozce5GHkyVckzs6SF1HL0meSHzxBNP4H7zjd3c3PD3339LkIiIiKhhrbozGjOmR0vYWhhLnEa/6cQcGSIioqbi2s0C7LuYCZkMCOnDS64fFQsZIiKiRrT6cOVyBIPaOaGVg6XEafQfCxkiIqJGkl1Yhq2nUgGwAZ62sJAhIiJqJGuPJ6G0Qo3OLWwQ4GUndRyDwEKGiIioEZSUq7DmWCIAYHo/NsDTFhYyREREjWBnzA1kFZTBxcYUT3VmY1dtYSFDRETUwIQQWHW48pLrqX08oVTw61dbeCSJiIga2MErWbicUQALYwWe93eXOo5BYSFDRETUwO42wHve3x02ZkqJ0xgWFjJEREQN6GJ6Hg5dyYJcVnlaibSLhQwREVEDWnWosgHesE4ucLMzlziN4WEhQ0RE1EAy80rwc8x1AMA0NsBrECxkiIiIGsiaY0koVwn08LCFr7ut1HEMEgsZIiKiBlBcpsLaE0kAgOl9ORrTUFjIEBERNYAt0anIKSqHm50ZnujoLHUcg8VChoiISMvUaoHQO6tch/TxgkLO5QgaCgsZIiIiLdt3MRMJWYWwMjXCOD83qeMYNBYyREREWna3Ad4LPd1hYWIkcRrDxkKGiIhIi2JTc3EiIRtGchmCe3tKHcfgsZAhIiLSoruLQz7dxQUuNmYSpzF8LGSIiIi05EZOMX49mwYAmN6vlcRpmgYWMkRERFoScTQRKrVAr1Z26NTCRuo4TQILGSIiIi0oKK3A+shkAMAMjsY0GhYyREREWrA5KgX5JRVo5WCBx30cpY7TZLCQISIiekQVKjVCj1Q2wJvW1wtyNsBrNCxkiIiIHtGf5zOQersYtuZKPNe9pdRxmhQWMkRERI/ohzsN8Cb38oCZsULiNE1LnQsZT09PfPDBB0hOTm6IPERERHrlVNJtnE7OgbFCjkmBHlLHaXLqXMjMmzcP27ZtQ6tWrTBkyBBs3LgRpaWlDZGNiIhI591djmBUd1c4WplKnKbpqVchExMTg8jISLRv3x4vv/wyXFxc8NJLLyE6OrohMhIREemk5FtF+ONcOgA2wJNKvefI+Pr64quvvsKNGzewcOFCrFq1Cv7+/ujWrRtCQ0MhhNBmTiIiIp0TeiQBagH0b+uAtk5WUsdpkuq9JGd5eTm2b9+OsLAw7NmzB7169cK0adOQmpqKt956C3v37sX69eu1mZWIiEhn5BaVY/PJFADAjH5eEqdpuupcyERHRyMsLAwbNmyAXC7HlClT8MUXX6Bdu3aafZ599ln4+/trNSgREZEu2RCVjKIyFdo5W6Fva3up4zRZdS5k/P39MWTIEKxYsQKjRo2CUqmsto+XlxfGjx+vlYBERES6pqxCjfAjiQAqG+DJZGyAJ5U6FzLx8fHw8Hjw5WUWFhYICwurdygiIiJd9ltsGtLzSuBgZYJnurlKHadJq/Nk38zMTJw4caLa9hMnTuDkyZNaCUVERKSrhBCaBnhBgR4wMWIDPCnVuZCZM2cOUlJSqm2/fv065syZo5VQREREuup4fDbO3ciDqVKOiT3ZAE9qdS5kzp8/D19f32rbu3fvjvPnz2slFBERka662wBvTI+WsLUwljgN1bmQMTExQUZGRrXtaWlpMDKq99XcREREOu/azQLsu5gJmQwI6cNLrnVBnQuZJ554AgsWLEBubq5mW05ODt566y0MGTJEq+GIiIh0yerDCQCAQe2c0MrBUuI0BNTjqqVly5ahf//+8PDwQPfu3QEAMTExcHJywo8//qj1gERERLogu7AMW0+lAgCmswGezqhzIdOiRQucPXsW69atw5kzZ2BmZoapU6diwoQJ9+0pQ0REZAjWHk9CaYUanVvYoKeXndRx6I56TWqxsLDAzJkztZ2FiIhIJ5WUq7DmWCKAytEYNsDTHfWenXv+/HkkJyejrKysyvZnnnnmkUMRERHpkp0xN5BVUAYXG1M81dlF6jj0D/Xq7Pvss88iNjYWMplMs8r13epUpVJpNyEREZGEhBBYdbjykuvg3p5QKup8nQw1oDr/a7zyyivw8vJCZmYmzM3Nce7cORw8eBB+fn44cOBAA0QkIiKSzsErWbicUQALYwXGB7hLHYfuUecRmWPHjmH//v2wt7eHXC6HXC5H3759sWTJEsydOxenT59uiJxERESSuNsAb5y/G2zMeFGLrqnziIxKpYKVlRUAwN7eHjdu3AAAeHh44NKlS9pNR0REJKGL6Xk4dCULcjbA01l1HpHp1KkTzpw5Ay8vL/Ts2ROffvopjI2N8f3336NVq1YNkZGIiEgSqw9VNsB7spMz3OzMJU5D91PnQuadd95BYWEhAOCDDz7A008/jX79+qF58+bYtGmT1gMSERFJITO/BD/HVJ51mN6Pf6jrqjoXMkOHDtX8d+vWrXHx4kVkZ2fD1taW19UTEZHB+PFYEspUavi6N4Ovu63UcagGdZojU15eDiMjI8TFxVXZbmdnxyKGSAtUaoFj127h55jrOHbtFlRqIXUkoiapuEyFtceTAAAzOBqj0+o0IqNUKuHu7s5eMUQNYHdcGhb9ch5puSWabS42plg4ogOe7MQGXESNaWt0Km4XlcPNzgxPdHSWOg49QJ2vWnr77bfx1ltvITs7uyHyEDVJu+PSMHttdJUiBgDSc0swe200dselSZSMqOlRqwVC76xyHdLHCwo5zzjosjrPkfnmm29w9epVuLq6wsPDAxYWFlXuj46O1lo4oqZApRZY9Mt53O8kkgAgA7Dol/MY0sGZH6hEjWD/xUzEZxXCytQI4/zcpI5DD1HnQmbUqFENEIOo6YpMyK42EvNPAkBabgkiE7IR6N288YIRNVE/3GmA90JPd1iY1HtJQmokdf4XWrhwYUPkIGqyMvNrLmLqsx8R1V9sai5OJGTDSC5DcG9PqeNQLXDlKyKJOVqZanU/Iqq/u4tDPt3FBS42ZhKnodqo84iMXC5/4KXWvKKJqG5cm5lCLgMedKW1i40pArzsGi8UURN0I6cYu85WTqxnAzz9UedCZvv27VV+Li8vx+nTpxEREYFFixZpLRhRU3C7sAwh4VGaIkYG3HfS7wAfB070JWpgEUcTUaEW6NXKDp1a2Egdh2qpzqeWRo4cWeU2ZswYfPTRR/j000+xc+fOOj2Xp6cnZDJZtducOXMAACUlJZgzZw6aN28OS0tLjB49GhkZGXWNTKSTSspVmL7mJK7dLISrjSmWPNcZzjZVTx9Z3plouCkqBbvj0qWISdQkFJRWYH1kMgA2wNM3MiGEVlqHxsfHo0uXLigoKKj1Y27evFnlVFRcXByGDBmCv/76C4899hhmz56NXbt2ITw8HDY2NnjppZcgl8tx5MiRWr9GXl4ebGxskJubC2tr6zr9TkQNpUKlxux10dhzPgPWpkbYMrs32jpZQaUWiEzIRmZ+CRytTOHvaYt3f47DhsgUGBvJsW56T/h78hQTkbaFHk7AB7+eRysHC+x9dQDkHAGVXG2/v7VyXVlxcTG++uortGjRok6Pc3BwqPLzJ598Am9vbwwYMAC5ublYvXo11q9fj4EDBwIAwsLC0L59exw/fhy9evXSRnSiRieEwHs7z2HP+QwYG8mxOtgfbZ2sAAAKuazaJdaLR3bCzfwy7L2QgWnhUdg6uzfa3NmfiB5dhUqN0COVDfCm9fViEaNn6nxqydbWFnZ2dpqbra0trKysEBoaiv/85z/1DlJWVoa1a9ciJCQEMpkMp06dQnl5OQYPHqzZp127dnB3d8exY8dqfJ7S0lLk5eVVuRHpkq/3X8X6E8mQyYCvxnd76AiLkUKOryd0h697M+SVVCAoNBJpucWNlJbI8P15PgOpt4tha67Ec91bSh2H6qjOIzJffPFFlauW5HI5HBwc0LNnT9ja1n910B07diAnJwfBwcEAgPT0dBgbG6NZs2ZV9nNyckJ6es1zBZYsWcJJx6SzNkUl4/M9lwEAHzzTsdZrKJkZK7A6yB+jvzuK+JuFCA6Nwub/C4SNmbIh4xI1CXcb4E3u5QEzY4XEaaiu6lzI3C00tG316tUYNmwYXF1dH+l5FixYgNdee03zc15eHtzc2GKapLf/Ygbe2l65cvycx70xOdCzTo+3tTBGxNQAPLfiKC5l5GPmmpOICAmAqZIfvET1dSrpNk4n58BYIcekQA+p41A91PnUUlhYGH766adq23/66SdERETUK0RSUhL27t2L6dOna7Y5OzujrKwMOTk5VfbNyMiAs3PNK5GamJjA2tq6yo1IaqeTb+PFddFQqQXG9GiJ15/wqdfzuNmZI3yqPyxNjHAiIRv/2nwG6gc1oCGiB1p1ZzRmVHdXNp3UU3UuZJYsWQJ7e/tq2x0dHfHxxx/XK0RYWBgcHR0xfPhwzbYePXpAqVRi3759mm2XLl1CcnIyAgMD6/U6RFKIv1mAkPAolJSr8ZiPA5Y81/mBTSUfpqOrDb6f3ANKhQy7YtPwwa/noaWLD4malORbRfjjXOVUhWl9ecm1vqpzIZOcnAwvL69q2z08PJCcnFznAGq1GmFhYQgKCoKR0f/OdNnY2GDatGl47bXX8Ndff+HUqVOYOnUqAgMDecUS6Y3M/BJMCY3E7aJydGlpg+Uv+EKpePSVQXq3tsdn47oBAMKPJmLlwfhHfk6ipib0SALUAujf1gE+zrwSUF/V+RPV0dERZ8+erbb9zJkzaN687ivz7t27F8nJyQgJCal23xdffIGnn34ao0ePRv/+/eHs7Ixt27bV+TWIpJBfUo6pYVFIvV0Mz+bmCA321+pKus90dcU7w9sDAD75/SK2Radq7bmJDF1ucTk2n0wBAEzvW/2Pc9Ifdf5UnTBhAubOnQsrKyv0798fAPD333/jlVdewfjx4+sc4IknnqhxWNzU1BTLly/H8uXL6/y8RFIqq1Bj9tponLuRB3tLY0SEBMDe0kTrrzO9Xyuk55Zg1eEEvLnlLOwtTdC/rcPDH0jUxG2ITEZRmQo+Tlbo16b6dAnSH3UekVm8eDF69uyJQYMGwczMDGZmZnjiiScwcODAes+RITIkarXAm1vO4PDVLJgbKxAa7A+P5hYN9npvPdUez3R1RYVaYPbaU4i7nttgr0VkCMpVaoQfSQQATOvn9Uhz1kh69V6i4MqVK4iJiYGZmRk6d+4MDw/dvGyNSxRQY1vy2wWsPBgPI7kMq4P9MaARRkhKK1SYGhaFo9duwd7SGNtm94F7c/MGf10ifbTj9HXM2xQDe0sTHJn/OEyM2MJAFzX4EgVt2rRBmzZt6vtwIoMUejhBM/F26egujVLEAICJkQIrJ/fAuJXHcSEtD0Fhkdjyf4Fo3gCns4j0mRACqw5X/j8aFOjBIsYA1PnU0ujRo7F06dJq2z/99FOMHTtWK6GI9NGvZ29g8a7zAIA3n/TB6B6N2+rcylSJiKn+aNHMDAlZhQiJOImisopGzUCk647HZyPueh5MlXJM7KWbZxKobupcyBw8eBBPPfVUte3Dhg3DwYMHtRKKSN8cu3YLr206AyGAKYEemD3AW5IcjtamWDMtAM3MlTiTkoOX1p9GhUotSRYiXbT6zmjMaN+WsLMwljgNaUOdC5mCggIYG1f/x1cqlVygkZqki+l5mPnjSZSp1BjWyRkLR3SUdPKgt4MlVgf5w8RIjv0XM/HW9lg2zCMCcO1mAfZeyARQuco1GYY6FzKdO3fGpk2bqm3fuHEjOnTooJVQRPriek4xgkIjkV9SgQBPO3zxfDco5NJfAdHDwxbfvOALuQzYfDIVX9xZqJKoKQs9nAAAGNzeEa0cLCVOQ9pS58m+7777Lp577jlcu3YNAwcOBADs27cP69evx5YtW7QekEhX5RSVISg0Ehl5pWjrZIkfpvjp1AKOQzo44cNRnfHW9lh8tf8qnGxMMbEn5wRQ05RdWIYtpyqbRk7vx+UIDEmdC5kRI0Zgx44d+Pjjj7FlyxaYmZmha9eu2L9/P+zs7BoiI5HOKSlXYXrESVzNLICztSnCpwbAxlwpdaxqXujpjvS8Eny17wre3REHe0sTDO1Y86KrRIZq3fEklFao0amFNXp68bvKkNRr0Zfhw4fjyJEjKCwsRHx8PMaNG4fXX38dXbt21XY+Ip2jUgvM3XAaJ5Nuw9rUCBEhAXBtZiZ1rBq9OrgNxvu7QS1QmTsxW+pIRI2qpFyFiGNJAIAZ/VqxAZ6BqffqdQcPHkRQUBBcXV3x2WefYeDAgTh+/Lg2sxHpHCEEFu6Mw5/nM2BsJMcPU/x0frE5mUyGD0d1wqB2jiitUGNaxElczcyXOhZRo9l55gayCkrhYmOKpzq7SB2HtKxOhUx6ejo++eQTtGnTBmPHjoW1tTVKS0uxY8cOfPLJJ/D392+onEQ6YflfV7H2eDJkMuC/z3dDz1Z1XyhVCkYKOb5+oTu6uTVDbnE5gkKjkJFXInUsogYnhMDqQ5WTfIN7e2pl9XnSLbX+Fx0xYgR8fHxw9uxZfPnll7hx4wa+/vrrhsxGpFM2n0zBsj8rr/55f0RHDNOzv+zMjY0QGuyPVvYWmqut8krKpY5F1KAOXcnCpYx8WBgrMD7AXeo41ABqXcj8/vvvmDZtGhYtWoThw4dDodCdqzOIGtpfFzOxYFssAGD2Y94I6u0pbaB6srOoXInbwcoEF9PzMXPNSZRWqKSORdRgfjhU2QBvnL8bbMx0b0I+PbpaFzKHDx9Gfn4+evTogZ49e+Kbb75BVlZWQ2Yj0gkxKTl4cV00VGqB53xb4M2hPlJHeiRuduYIC/aHpYkRjsdn47XNZ6BWs2EeGZ5L6fk4dCULchkQ0ocN8AxVrQuZXr164YcffkBaWhpmzZqFjRs3wtXVFWq1Gnv27EF+PicPkuFJyCpESHgUistV6N/WAUtHdzGIKx46tbDBd5N6QKmQYdfZNHy46wK7/5LBWXVnNObJTs5ws+Nq8IaqzrOeLCwsEBISgsOHDyM2Nhb/+te/8Mknn8DR0RHPPPNMQ2QkksTN/FJMCT2B7MIydG5hgxUTfQ1qomDfNvZYNrayZULokQSsujMhksgQZOaX4OeYGwDYAM/QPdKnso+PDz799FOkpqZiw4YN2spEJLmC0gpMDY9ESnYxPJqbIzTYHxYmde4fqfNGdmuBt55qBwD46LcL+DnmusSJiLTjx2NJKFOp4eveDL7utlLHoQaklT8vFQoFRo0ahZ07d2rj6YgkVVahxuy1pxB3PQ/NLYwRMbVycqyhmtGvlWb+wOs/ncGRq5z7RvqtuEyFtcf/1wCPDJvhjJMTaYEQAvO3nsWhK1kwUyqwOtgfnvYWUsdqUDKZDO8Mb4+nu7igXCUw68dTOHcjV+pYRPW2NToVt4vK4WZnhie4JIfBYyFD9A9Ld1/CttPXoZDL8O0kX3RzayZ1pEYhl8vw2biuCGzVHAWlFQgOi0JKdpHUsYjqTK0WmlWuQ/p46cRq9NSwWMgQ3RF+JAHf/X0NAPDJc53xuI+jxIkal4mRAiun9EA7ZyvczC9FUGgksgvLpI5FVCf7L2YiPqsQVqZGGOvnJnUcagQsZIgA/BabhkW/ngcAvP5E2yb7AWhtqkT41AC0aGaG+KxCTIuIQnEZG+aR/rjbAO+Fnu6wNMAJ+lQdCxlq8k7E38K8TTEQApjcywNzHm8tdSRJOduYIiLEHzZmSpxOzsFL66NRoVJLHYvooWJTc3EiIRtGchmC9bT7NtUdCxlq0i6l52P6mpMoq1BjaEcnvP9MR4NoePeoWjtaYXWQH0yM5Nh3MRPv7IhjwzzSeasOV47GPN3FBS42ZhKnocbCQoaarBt3Fk7ML6mAn4ct/ju+OycG/oOfpx2+mtAdchmwMSoFX+69InUkohrdyCnGrrNpANgAr6lhIUNNUm5ROYJCI5GeV4LWjpZYFeQHUyUXQr3X0I7O+GBkJwDAf/ddwfoTyRInIrq/iKOJqFAL9Gplh04tbKSOQ42IhQw1OSXlKsxYcxJXMgvgbG2KiJAANDM3ljqWzprUywMvD6ycN/TOjljsOZ8hcSKiqgpKK7A+srLInt6XozFNDQsZalJUaoF5G2MQmZgNK1MjhIf4o0Uznkt/mNeGtMU4v5ZQC+DlDdE4lXRb6khEGpujUpBfUoFW9hYY2K5ptU0gFjLUhAghsOiXc9h9Lh3GCjm+n+yHds7WUsfSCzKZDB892xmP+zigpFyNaRFRuJpZIHUsIqjUAqFH7jTA6+sFOee5NTksZKjJ+PbANaw5lgSZDPji+W4I9G4udSS9olTIsXyiL7q6NUPOnTlGGXklUseiJu6Pc+lIvV0MW3MlRvu2lDoOSYCFDDUJW06l4j9/XAIAvPd0Bwzv4iJxIv1kbmyE0CA/eNlb4HpOMYLDopBfUi51LGrCVt1pgDeplwfMjDlhvyliIUMG78ClTPx761kAwKwBrTD1zkrPVD/NLU0QMTUA9pYmuJCWh/9bewplFWyYR43vVNJtRCfnwFghx+RAD6njkERYyJBBO5OSgxfXRUOlFhjVzRX/HtpO6kgGwb25OcKn+sPCWIEjV2/h9Z/OQK1mwzxqXKvvNMAb2c0VjlamEqchqbCQIYOVmFWIkPAoFJWp0K+NPT4d05UTAbWoUwsbfDe5B4zkMuw8cwNLfr8gdSRqQlKyi7A7Lh0AG+A1dSxkyCBlFZQiKCwStwrL0NHVGism9YCxEd/u2tavjQP+M7YLAOCHQwma+QpEDS30SALUAujXxh4+zlZSxyEJ8ZOdDE5haQVCwqOQdKsIbnZmCJvqz1VwG9Cz3Vti/rDKU3Yf7rqAnWduSJyIDF1ucTk2R6UAAGZwNKbJYyFDBqVcpcaL66JxNjUXdhbGiJgawHPnjWBW/1aa1Yb/tTkGR69mSRuIDNrGyGQUlqng42SFfm3spY5DEmMhQwZDCIH5W2Px9+WbMFMqsDrID60cLKWO1STIZDK8+3QHPNXZGeUqgZk/nsL5G3lSxyIDVK5SI/xoIgBgWj8vrlZPLGTIcCz78xK2RqdCIZdh+cTu6O5uK3WkJkUhl+Hzcd3Q08sOBaUVCA6LREp2kdSxyMD8FpuGtNwS2FuaYGQ3V6njkA5gIUMG4cdjiVj+1zUAwJJnO2NgOyeJEzVNpkoFvp/iBx8nK2TmV064vl1YJnUsMhBCCPxwZ0J5UKAHTIzYAI9YyJAB2B2Xhvd2ngNwZ3FDfzeJEzVtNmZKhIf4w8XGFPE3CzEtIgrFZSqpY5EBOJGQjbjreTBVyjGxFxvgUSUWMqTXIhOyMXdjDIQAXujpjpcHtpY6EgFwsTFDREgArE2NEJ2cg5c3nEaFit1/6dHcvbx/tG9L2FkYS5yGdAULGdJblzPyMT0iCmUVagzp4ITFIztx4p8OaetkhdXB/jA2kmPvhQy8+/M5CMHuv1Q/8TcLsPdCJgBgWl8uM0L/w0KG9FJabjGCQiORV1KBHh62+HpCdyjYtVfn+Hva4avx3SCTARsik/HVvqtSRyI9tfpwAgBgcHtHXo1IVbCQIb2TW1yO4NAopOWWwNvBAquD/GCq5KQ/XfVkJxd88ExHAMAXey9jY2SyxIlI32QXlmHLqVQAXI6AqmMhQ3qlpFyFGWtO4lJGPpysTRAREoBm5jxXrusmB3pizuPeAIC3d8Rh34UMiRORPll3PAmlFWp0amGNnl52UschHcNChvSGSi3w2uYYRCZkw8rECOFTA9DS1lzqWFRLrz/hgzE9WkKlFpizPhqnk29LHYn0QEm5ChHHkgBULkfAeXB0LxYypBeEEFj863n8FpsOpUKGlZN7oL2LtdSxqA5kMhmWPNcZj/k4oKRcjZDwKMTfLJA6Fum4nWduIKugFC42pniqs4vUcUgHsZAhvfDd3/GatuSfjeuG3q25voo+UirkWP6CL7q0tMHtonJMCY1EZn6J1LFIRwkhsPpQ5STf4N6eUCr4lUXV8V1BOm9bdCqW7r4IAHhneHs805VtyfWZhYkRQoP94dncHKm3izE1LAr5JeVSxyIddOhKFi5l5MPCWIHxAe5SxyEdxUKGdNrfl2/izS1nAQAz+nnxigUDYW9ZOVHb3tIY527kYfbaaJRVsGEeVXV3OYJx/m6wMVNKnIZ0FQsZ0lmxqbmYvfYUKtQCI7u5YsGw9lJHIi3yaG6B0GB/mBsrcPhqFt7ccgZqNRvmUaVL6fk4dCULchkQ0ocN8KhmLGRIJyXdKsTU8EgUlanQp3Vz/GdMV8jZ8M7gdGnZDN9O9IWRXIYdMTc0pxCJ7i5H8GQnZ7jZ8epEqhkLGdI5twpKERQaiayCMnRwscZ3k3rA2IhvVUP1mI8jPhndBQCw8mA8Qu90cKWmKzO/BD/H3AAATOvL08n0YPx2IJ1SVFaBkPAoJN4qQktbM4RP9YeVKc+NG7oxPVrijaE+AIDFu87j17M3JE5EUvrxWBLKVGr4ujdDDw9bqeOQjmMhQzqjXKXGnHXROJOaC1tzJSJCAuBobSp1LGokLz7mjSmBHhACeG3TGRy9liV1JJJAcZkKa49XNsDj5H6qDRYypBOEEHh7eyz+unQTpko5Vgf7w5sLwzUpMpkMC0d0xLBOzihTqTFrzSlcSMuTOhY1sq3RqbhdVA43OzMM7egsdRzSAyxkSCd8vucyNp9MhVwGfDPBF77uHE5uihRyGb54vhsCPO2QX1qB4LBIXM8pljoWNRK1WmjmSE3t7cUV7alWWMiQ5NYeT8LX+68CAD5+tjMGd3CSOBFJyVSpwA9T/NDWyRIZeZUTv3OKyqSORY1g/8VMxGcVwsrUCOP83aSOQ3pC8kLm+vXrmDRpEpo3bw4zMzN07twZJ0+e1NwfHBwMmUxW5fbkk09KmJi06Y9z6Xjv5zgAwLzBbdi9kwAANuZKhE8NgLO1Ka5mFmB6xEmUlKukjkUNbNXhykuuXwhwh6WJkcRpSF9IWsjcvn0bffr0gVKpxO+//47z58/js88+g61t1dMKTz75JNLS0jS3DRs2SJSYtOlkYjbmbjgNtQAmBLjhlUFtpI5EOsS1mRkiQgJgbWqEk0m3MXfDaajYMM9gxV3PxfH4bBjJZQju4yl1HNIjkpa8S5cuhZubG8LCwjTbvLyqd3A0MTGBszMnfRmSKxn5mBZxEqUVagxu74jFIztBJuP5cKrKx9kKP0zxw+TQSPx5PgMLd8bxvWKg7jbAG97FBS42ZhKnIX0i6YjMzp074efnh7Fjx8LR0RHdu3fHDz/8UG2/AwcOwNHRET4+Ppg9ezZu3bolQVrSlvTcEgSFRiK3uBzd3Zvh6wm+MOKqtlSDnq2a47/Pd4NMBqw9nozlf12VOhJpWVpuMX49mwYAmMFLrqmOJP32iI+Px4oVK9CmTRv88ccfmD17NubOnYuIiAjNPk8++STWrFmDffv2YenSpfj7778xbNgwqFT3P19eWlqKvLy8KjfSHbnF5QgOi8SN3BK0srfA6iB/mBkrpI5FOm5YZxe8P6IjAGDZn5ex+WSKxIlIm8KPJqJCLdCrlR06tbCROg7pGZkQQrKTzsbGxvDz88PRo0c12+bOnYuoqCgcO3bsvo+Jj4+Ht7c39u7di0GDBlW7//3338eiRYuqbc/NzYW1tbX2wlOdlVaoEBQaiePx2XCwMsG22b25hgrVydLdF7HiwDUo5DKsmuKHx9s5Sh2JHlFBaQUCl+xDfkkFVk3x41WLpJGXlwcbG5uHfn9LOiLj4uKCDh06VNnWvn17JCcn1/iYVq1awd7eHlev3n94ecGCBcjNzdXcUlL4l5suUKsFXtt8Bsfjs2FpYoTwqf4sYqjO3hzqg+d8W0ClFnhxXTRiUnKkjkSP6KeTKcgvqUArewsMZGFK9SBpIdOnTx9cunSpyrbLly/Dw8Ojxsekpqbi1q1bcHFxue/9JiYmsLa2rnIjaQkhsHjXeew6mwalQoaVk3ugoyuHj6nuZDIZlo7ugv5tHVBcrkJIeBQSsgqljkX1pFILhB6pbIAX0teLK9xTvUhayLz66qs4fvw4Pv74Y1y9ehXr16/H999/jzlz5gAACgoK8MYbb+D48eNITEzEvn37MHLkSLRu3RpDhw6VMjrVwfcH4xF2JBEAsGxsV/RpbS9tINJrSoUcKyb6onMLG2QXlmFK6AnczC+VOhbVw5/n0pGSXQxbcyVG+7aUOg7pKUkLGX9/f2zfvh0bNmxAp06dsHjxYnz55ZeYOHEiAEChUODs2bN45pln0LZtW0ybNg09evTAoUOHYGJiImV0qqXtp1Ox5PeLAIC3n2qPkd1aSJyIDIGFiRFCg/3hbmeOlOxiTA2PREFphdSxqI5+uHPJ9aReHpz0T/Um6WTfxlDbyUKkfYeu3MTUsChUqAWm9fXCu093ePiDiOogIasQo1ccRXZhGfq1scfqIH8YG/FSfn1wKuk2Rq84CmOFHIfnPw5HK650T1XpxWRfMlxx13Pxfz+eQoVaYERXV7z9VHupI5EB8rK3QGiwP8yUChy6koX5W8/CwP82Mxir7yxHMLKbK4sYeiQsZEjrUrKLEBwWhcIyFQJbNceysV04iY8aTDe3Zvh2ki8Uchm2nb6OpbsvPfxBJKmU7CLsjksHAExnAzx6RCxkSKsqJ19GIqugFO2crbBySg+YGPHcNzWsx30c8clznQEA3/19DeF3roQh3RR6JAFqAfRrYw8fZyup45CeYyFDWlNUVqG5HLaFZsE/pdSxqIkY6+eG159oCwBY9Gvl5f6ke3KLy7E5qrK/F5cjIG1gIUNaUaFS4+X1pxGTkoNm5kpEhATAyZrnvalxzXm8NSb38oAQwKubYnA8nuuy6ZqNkckoLFPBx8kK/dqwFQM9OhYy9MiEEHh7exz2XcyEiZEcq4P80NrRUupY1ATJZDK8/0xHDO3ohDKVGjPWnMSl9HypY9Ed5So1wo8mAgCm9fPiKuakFSxk6JF9sfcKNp1MgVwGfD2hO3p42EkdiZowhVyG/47vDj8PW+SXVCAoNBI3coqljkUAfotNQ1puCewtTTCym6vUcchAsJChR7LuRBK+2ncFALB4VCc80dFZ4kREgKlSgVV3RgbT80oQFBqJ3KJyqWM1aUIITQO8oEAPXgRAWsNChurtz3PpeHdHHABg7sDWmNiz5jWyiBpbM3NjRIQEwNnaFFcyCzBjzUmUlKukjtVknUjIRtz1PJgq5ZjYi58VpD0sZKheTiVl4+UNp6EWwPN+bnh1SFupIxFV06KZGcJD/GFlaoTIxGzM2xgDlZoN86Sw6s5ozGjflrCzMJY4DRkSFjJUZ1czCzAt4iRKK9QY2M4RHz3biZP2SGe1c7bG95P9YKyQY/e5dCz65Ry7/zay+JsF2HshE0DlKtdE2sRChuok4858g5yicnR1a4ZvXugOIwXfRqTbAr2b44vnu0EmA9YcS8K3B65JHalJWX24skHh4PaO8HbgFY2kXfwGolrLKylHUGgkrucUV65xE+QHc2MjqWMR1crwLi54787Cpf/54xK2nEqVOFHTkF1Yhq3Rlcd6Wl82wCPtYyFDtVJaocKsNadwMT0f9pYmWBMSgOaWJlLHIqqTqX28MGtA5Zfpv7eexYFLmRInMnzrjiehpFyNTi2s0asVWzOQ9rGQoYdSqwX+tfkMjsXfgoWxAuFT/eFmZy51LKJ6+ffQdhjVzRUqtcCL66JxJiVH6kgGq7RChYhjSQCA6X1bcS4dNQgWMvRQH/12Ab+eTYORXIbvJvdApxY2Ukciqje5XIZPx3RFvzb2KCpTISQ8ColZhVLHMkg/x9xAVkEpnK1NMbyLi9RxyECxkKEHWnUoXjNRb9nYrujXxkHiRESPzthIjhWTeqCjqzVuFZYhKKxyxXbSHiEEVh+q/OwI7uMJJS8KoAbCdxbV6OeY6/hw1wUAwIJh7TCqewuJExFpj6WJEcKm+sPNzgxJt4oQEh6FwtIKqWMZjENXsnApIx/mxgpMCHCXOg4ZMBYydF9Hrmbh9Z/OAACm9vHEzP682oAMj6OVKSKmBsDOwhhnU3Mxe100ylVqqWMZhFV3RnLH+bnBxkwpcRoyZCxkqJpzN3Ix68dTKFcJDO/igneHd+AkPTJYrRwssTrID2ZKBQ5evon5W2PZMO8RXUrPx8HLNyGXAdPYAI8aGAsZqiIluwjBYVEoKK1Ar1Z2+HxcV8jlLGLIsHV3t8Xyid2hkMuwNToVy/68JHUkvbb6cOVyBE92cuYVjtTgWMiQxu07kx5v5peinbMVVk724wq11GQMbOeEJc92BgAs/+safjyWKG0gPZWZX4Idp28AYAM8ahwsZAgAUFymQkhEFOJvFsLVxhThUwN4XpuanHH+bnjtzgKo7+08h91xaRIn0j9rjyWhTKWGr3sz9PCwlToONQEsZAgVKjVe3hCN08k5sDFTIiIkAM42plLHIpLEywNb44We7hACmLsxBpEJ2VJH0hvFZSr8ePxOA7x+HI2hxsFCpokTQuDdn+Ow90ImTIzkWBXkhzZOVlLHIpKMTCbD4pGdMKSDE8oq1JgeEYXLGflSx9IL206n4nZROdzszDC0o7PUcaiJYCHTxP133xVsiEyBXAb8d3x3+HtyLRQihVyGryd0Rw8PW+SVVCAoNBJpucVSx9JpavX/GuBN7e0FBS8SoEbCQqYJ2xCZjC/3XgEALBrZCU924l9QRHeZKhVYHeQHbwcLpOWWIDg0CrnF5VLH0ll/XcpEfFYhrEyNMM7fTeo41ISwkGmi9p7PwNvbYwEALz3eGpN7eUiciEj3NDM3RkRIAJysTXApIx8z1pxESblK6lg66YdDlZdcvxDgDksTI4nTUFPCQqYJik6+jZc2REMtgLE9WuJfT7SVOhKRzmppa47wqQGwMjFCZEI2XtscA5WaDfP+Ke56Lo7HZ8NILkNwH0+p41ATw0Kmibl2swDTwqNQUq7GYz4O+Pi5zuzaS/QQ7V2ssXJKDygVMvwWm47Fv55n999/WHVnNGZ4Fxe42JhJnIaaGhYyTUhmXgmmrI7E7aJydG1pg28n+nJFWqJa6u1tj8/GdQMAhB9NxHd/x0sbSEek5Rbj17OV/XamswEeSYDfYk1Efkk5gsKicD2nGJ7NzREa7A9zY57HJqqLZ7q64p3h7QEAS3dfxLboVIkTSS/8aCIq1AI9vezQuaWN1HGoCWIh0wSUVajxf2tP4UJaHuwtjbEmpCeaW5pIHYtIL03v1woz+lUuhPjmlrP4+/JNiRNJp6C0AutPJAMAZrABHkmEhYyBU6sF3thyBkeu3oKFsQJhwQFwb85F3IgexYJh7TGymysq1AKz155CbGqu1JEk8dPJFOSXVKCVvQUGtnOUOg41USxkDNwnuy/i55gbMJLLsGJSDw79EmmBXC7Df8Z0RZ/WzVFUpsLU8Egk3yqSOlajUqkFQo9UNsAL6esFORvgkURYyBiw1YcT8P3BygmJS0d3Qf+2DhInIjIcxkZyfDepBzq4WCOroAxTQk/gVkGp1LEazZ/n0pGSXQxbcyVG+7aUOg41YSxkDNQvZ25g8a/nAQBvPumD0T34QUOkbVamSoRP9UdLWzMk3ipCSHgUisoqpI7VKO42wJvUywNmxgqJ01BTxkLGAB29loV/bT4DAAgK9MDsAd4SJyIyXI7WpogICYCtuRJnUnMxZ100ylVqqWM1qFNJtxGdnANjhRyTA9kVnKTFQsbAnL+Rh1lrTqFMpcZTnZ3x3oiObHhH1MC8HSyxOtgfpko5/rp0E29vjzXohnmrD1eOxozs5gpHK1OJ01BTx0LGgKTeLkJwWCTySysQ4GWHz8d14wq0RI3E190W30zwhVwGbD6Zis/3XJY6UoNIyS7C7rh0AMC0O5ehE0mJhYyBuF1YhqDQSGTml6KtkyV+mOwHUyXPWxM1psEdnPDxs50BAF/vv4q1x5MkTqR9oUcSoBZAvzb2aOdsLXUcIhYyhqCkXIXpa07i2s1CuNhUnq+3MVdKHYuoSRof4I55g9sAAN77OQ5/nEuXOJH25BaXY3NUCoDKxoBEuoCFjJ6rUKnx8obTOJV0G9amRogICeCibUQSe2VQG0wIcINaAHM3nMbJxGypI2nFxshkFJap0NbJEv3b2EsdhwgACxm9JoTAezvPYc/5DBgbybEqyB9tnaykjkXU5MlkMiwe2QmD2zuhtEKNaREncSUjX+pYj6RcpUb40UQAlYtD8iIC0hUsZPTY1/uvYv2JZMhkwFfjuyHAy07qSER0h5FCjq8ndIevezPkFpcjKDQS6bklUseqt99i05CWWwJ7SxOM7O4qdRwiDRYyempTVLLmqohFz3TEk51cJE5ERPcyM1ZgdZA/WjlY4EZuCYLDIpFbXC51rDoTQmga4E0J9ICJES8kIN3BQkYP7b+Ygbe2xwEAXnzMG1MCPaUNREQ1srUwRsTUADhYmeBiej5m/XgSpRUqqWPVyYmEbMRdz4OpUo5JvdgAj3QLCxk9czr5Nl5cFw2VWmC0b0u8MdRH6khE9BBuduYIn+oPSxMjHI/Pxmubz0Ct1p+GeasOVS4OOdq3JewsjCVOQ1QVCxk9En+zACHhUSgpV2NAWwd8MrozJ9wR6YmOrjZYObkHlAoZdp1Nw+Jd5/Wi+2/8zQLsu5gBoHKVayJdw0JGT2Tml2BKaCRuF5WjcwsbfDvRF0oF//mI9Emf1vZYNrYrACDsSKJm3okuCz2SACGAwe0d4e1gKXUcomr4TagHCkorMDUsCqm3i+HR3Byhwf6wMDGSOhYR1cPIbi3w9lPtAQAf/3YRO05flzhRzW4XlmHLqVQAwLS+bIBHuomFjI4rq1Bj9tpTOHcjD83/MWmQiPTXjP6tMO3OaZo3tpzB4StZEie6v3UnklBSrkanFtbo1YrtHUg3sZDRYWq1wL+3nsWhK1kwUyoQGuwPT3sLqWMRkRa8/VR7jOjqinKVwKwfTyLueq7UkaoorVAh4ljlWlFsgEe6jIWMDlv6x0VsP30dCrkM307yRVe3ZlJHIiItkctlWDa2CwJbNUdhmQrBYVFIyS6SOpbGzpgbuJlfCmdrUwzvwj5VpLtYyOiosCMJWPl35UTAT57rjMd9HCVORETaZmKkwMopPdDO2QpZBaWYEhqJ7MIyqWNBCIHVhysvuQ7u48kLC0in8d2pg349ewMf/HoeAPDGUB+M9XOTOBERNRRrUyUiQgLQopkZErIKERIehaKyCkkzHb6ahYvp+TA3VmBCgLukWYgehoWMjjl27RZe23QGQgCTe3ngxce8pY5ERA3MydoUESEBaGauRExKDl5efxoVKrVkeX640wBvnJ8bbMyUkuUgqg0WMjrkYnoeZv54EmUqNZ7s6Iz3n+nICXZETURrR0usDvKDiZEc+y5m4u3tcZI0zLuUno+Dl29CLgNC+rABHuk+FjI64npOMYJCI5FfUgF/T1t8Ob4bFHIWMURNSQ8PO3zzgi/kMmDTyRR8sfdKo2dYfbhybt7Qjs5wb27e6K9PVFcsZHRATlEZgkIjkZFXijaOllg1xR+mSq4uS9QUDenghA9HdQYAfLXvCtadSGq0187ML8GO0zcAANP7sQEe6QcWMhIrKVdhesRJXM0sgPOd8+Q25jwnTdSUvdDTHXMHtQEAvLsjDn+eS2+U1117LAllKjW6uzdDDw/bRnlNokcleSFz/fp1TJo0Cc2bN4eZmRk6d+6MkydPau4XQuC9996Di4sLzMzMMHjwYFy50vjDrQ1BpRaYu+E0TibdhpWpESJCAuDazEzqWESkA14d3AbP+7lBLYCXN5zGqaTsBn294jIVfjxeOfozg6MxpEckLWRu376NPn36QKlU4vfff8f58+fx2Wefwdb2f38JfPrpp/jqq6/w3Xff4cSJE7CwsMDQoUNRUlIiYfJHJ4TAwp1x+PN8BowVcvwwxQ8+zlZSxyIiHSGTyfDRs50wsJ0jSivUmHZn5LahbDudittF5Whpa4YnOjg12OsQaZtMSLiO/Pz583HkyBEcOnTovvcLIeDq6op//etfeP311wEAubm5cHJyQnh4OMaPH//Q18jLy4ONjQ1yc3NhbW2t1fyP4pv9V7Dsz8uQyYBvJviycyYR3VdRWQUm/HACZ1Jy0KKZGba92BtO1qZafQ21WmDw538jPqsQ7z3dASF9ebUSSa+239+Sjsjs3LkTfn5+GDt2LBwdHdG9e3f88MMPmvsTEhKQnp6OwYMHa7bZ2NigZ8+eOHbs2H2fs7S0FHl5eVVuumbzyRQs+/MyAOC9pzuwiCGiGpkbGyE0yA9e9ha4nlOM4LAo5JWUa/U1/rqUifisQliZGmGcPxtwkn6RtJCJj4/HihUr0KZNG/zxxx+YPXs25s6di4iICABAenrlBDcnp6rDnE5OTpr77rVkyRLY2Nhobm5uuvU/5V8XM7FgWywAYNaAVpjKPg1E9BDNLU2wJiQA9pYmuJCWh//78RRKK1Rae/4fDlVecv1CgDssTYy09rxEjUHSQkatVsPX1xcff/wxunfvjpkzZ2LGjBn47rvv6v2cCxYsQG5uruaWkpKixcSPJiYlBy+ui4ZKLfBs9xb499B2UkciIj3hZmeO8Kn+sDBW4Oi1W3j9p7NQqx99ZkDc9Vwcj8+GkVyGoN6ejx6UqJFJWsi4uLigQ4cOVba1b98eycnJAABnZ2cAQEZGRpV9MjIyNPfdy8TEBNbW1lVuuiDxzhoqxeUq9Gtjj6Wju0DOhndEVAedWtjgu8k9YCSX4ZczN/Dxbxce+TlX3RmNGd7FhVdNkl6StJDp06cPLl26VGXb5cuX4eHhAQDw8vKCs7Mz9u3bp7k/Ly8PJ06cQGBgYKNmfRQ38/+3qm2nFtZYMakHjI0kv/KdiPRQvzYOWDa2KwBg1eEETSFSH2m5xfj1bBoAYHpfXnJN+knSb9NXX30Vx48fx8cff4yrV69i/fr1+P777zFnzhwAlZcfzps3Dx9++CF27tyJ2NhYTJkyBa6urhg1apSU0WutsLQCIeFRSM4ugpudGUKD/XkOmogeyajuLbBgWOWp6Q93XcDPMdfr9TzhRxNRoRbo6WWHzi1ttBmRqNFI+o3q7++P7du3Y8GCBfjggw/g5eWFL7/8EhMnTtTs8+abb6KwsBAzZ85ETk4O+vbti927d8PUVLuXHzaEcpUas9dFI/Z6LuwsjLEmpCccrXQ/NxHpvpn9WyE9rwRhRxLx+k9nYG9pgj6t7Wv9+MLSCqw/UXkan8sRkD6TtI9MY5Cqj4wQAv/66Qy2RV+HmVKBDTN7oZtbs0Z7fSIyfGq1wMsbT2PX2TRYmhhh06xe6Ohau5GVsCMJWPTLeXjZW2DfawM4Z490jl70kTFkn/5xCduir0Mhl+Hbib4sYohI6+RyGT4f1xW9WtmhoLQCwWFRSMkueujjVGqB0CMJAICQvl4sYkivsZBpABFHE7HiwDUAwJLnOuPxdo4SJyIiQ2VipMD3U/zQztkKN/NLERQWiduFZQ98zJ/n0pGSXQxbcyXG+LZspKREDYOFjJb9FpuG9385BwD415C2GOenWw35iMjwWJsqET41AC2amSH+ZiFCIqJQXFZzw7xVhytHYyb18oCZsaKxYhI1CBYyWnQi/hbmbYqBEMDEnu54aWBrqSMRURPhbGOKiBB/2JgpcTo5By9viEaFSl1tv+jk2ziVdBvGCjkmB3pIkJRIu1jIaMml9HxMX3MSZRVqPNHBCR+M7ASZjOediajxtHa0wuogP5gYybH3Qibe/TkO917PsfpQ5WjMyG6uvIqSDAIbmtSDSi0QmZCNzPwSOFqZoqWtGYJCI5FfUgE/D1t8NaE7FJw8R0QS8PO0w1cTumP22lPYEJkCZ2szvDSwNSITsnExLQ+/xVY2wJvWj+u8kWFgIVNHu+PSsOiX80jLLdFsM5LLUKEW8HawwKogP5gqec6ZiKQztKMzFo3shHd3xOGLvZcReiQBucX/WzHb2EiOxKxCtHPWjSVciB4FTy3Vwe64NMxeG12liAGAijsLt4X08UIzc2MpohERVTG5lweGdapck+6fRQwAlFWoMXttNHbHpUkRjUirWMjUkkotsOiX83hQ98Bv/roKlRZWoyUielQqtcDplJwH7rPol/P8zCK9x0KmliITsquNxNwrLbcEkQnZjZSIiKhmkQnZSH/AZ5YAP7PIMLCQqaXM/AcXMXXdj4ioIfEzi5oKFjK1VNvLFHk5IxHpAn5mUVPBQqaWArzs4GJjipouqpYBcLExRYCXXWPGIiK6L35mUVPBQqaWFHIZFo7oAADVPhju/rxwRAf2jyEincDPLGoqWMjUwZOdXLBiki+cbaoOxTrbmGLFJF882clFomRERNXxM4uaApm4t3+1gcnLy4ONjQ1yc3Nhba2d5k/3dvYN8LLjXzVEpLP4mUX6qLbf3+zsWw8KuQyB3s2ljkFEVCv8zCJDxlNLREREpLdYyBAREZHeYiFDREREeouFDBEREektFjJERESkt1jIEBERkd5iIUNERER6i4UMERER6S0WMkRERKS3DL6z790VGPLy8iROQkRERLV193v7YSspGXwhk5+fDwBwc3OTOAkRERHVVX5+PmxsbGq83+AXjVSr1bhx4wasrKwgk2lvkbS8vDy4ubkhJSVFa4tRGjIer9rjsao9Hqva47GqPR6r2mvIYyWEQH5+PlxdXSGX1zwTxuBHZORyOVq2bNlgz29tbc03eh3weNUej1Xt8VjVHo9V7fFY1V5DHasHjcTcxcm+REREpLdYyBAREZHeYiFTTyYmJli4cCFMTEykjqIXeLxqj8eq9nisao/HqvZ4rGpPF46VwU/2JSIiIsPFERkiIiLSWyxkiIiISG+xkCEiIiK9xUKGiIiI9BYLmRocPHgQI0aMgKurK2QyGXbs2PHQxxw4cAC+vr4wMTFB69atER4e3uA5dUFdj9WBAwcgk8mq3dLT0xsnsISWLFkCf39/WFlZwdHREaNGjcKlS5ce+riffvoJ7dq1g6mpKTp37ozffvutEdJKqz7HKjw8vNr7ytTUtJESS2fFihXo0qWLpilZYGAgfv/99wc+pim+p4C6H6um+p66n08++QQymQzz5s174H6N/d5iIVODwsJCdO3aFcuXL6/V/gkJCRg+fDgef/xxxMTEYN68eZg+fTr++OOPBk4qvboeq7suXbqEtLQ0zc3R0bGBEuqOv//+G3PmzMHx48exZ88elJeX44knnkBhYWGNjzl69CgmTJiAadOm4fTp0xg1ahRGjRqFuLi4Rkze+OpzrIDKDqP/fF8lJSU1UmLptGzZEp988glOnTqFkydPYuDAgRg5ciTOnTt33/2b6nsKqPuxAprme+peUVFRWLlyJbp06fLA/SR5bwl6KABi+/btD9znzTffFB07dqyy7fnnnxdDhw5twGS6pzbH6q+//hIAxO3btxslky7LzMwUAMTff/9d4z7jxo0Tw4cPr7KtZ8+eYtasWQ0dT6fU5liFhYUJGxubxgulw2xtbcWqVavuex/fU1U96FjxPSVEfn6+aNOmjdizZ48YMGCAeOWVV2rcV4r3FkdktOTYsWMYPHhwlW1Dhw7FsWPHJEqk+7p16wYXFxcMGTIER44ckTqOJHJzcwEAdnZ2Ne7D91al2hwrACgoKICHhwfc3Nwe+pe2IVKpVNi4cSMKCwsRGBh43334nqpUm2MF8D01Z84cDB8+vNp75n6keG8Z/KKRjSU9PR1OTk5Vtjk5OSEvLw/FxcUwMzOTKJnucXFxwXfffQc/Pz+UlpZi1apVeOyxx3DixAn4+vpKHa/RqNVqzJs3D3369EGnTp1q3K+m91ZTmFN0V22PlY+PD0JDQ9GlSxfk5uZi2bJl6N27N86dO9egi8fqgtjYWAQGBqKkpASWlpbYvn07OnTocN99m/p7qi7Hqim/pwBg48aNiI6ORlRUVK32l+K9xUKGGp2Pjw98fHw0P/fu3RvXrl3DF198gR9//FHCZI1rzpw5iIuLw+HDh6WOovNqe6wCAwOr/GXdu3dvtG/fHitXrsTixYsbOqakfHx8EBMTg9zcXGzZsgVBQUH4+++/a/yCbsrqcqya8nsqJSUFr7zyCvbs2aPTE5xZyGiJs7MzMjIyqmzLyMiAtbU1R2NqISAgoEl9ob/00kv49ddfcfDgwYf+VVfTe8vZ2bkhI+qMuhyreymVSnTv3h1Xr15toHS6w9jYGK1btwYA9OjRA1FRUfjvf/+LlStXVtu3qb+n6nKs7tWU3lOnTp1CZmZmlZFylUqFgwcP4ptvvkFpaSkUCkWVx0jx3uIcGS0JDAzEvn37qmzbs2fPA8+70v/ExMTAxcVF6hgNTgiBl156Cdu3b8f+/fvh5eX10Mc01fdWfY7VvVQqFWJjY5vEe+tearUapaWl972vqb6navKgY3WvpvSeGjRoEGJjYxETE6O5+fn5YeLEiYiJialWxAASvbcabBqxnsvPzxenT58Wp0+fFgDE559/Lk6fPi2SkpKEEELMnz9fTJ48WbN/fHy8MDc3F2+88Ya4cOGCWL58uVAoFGL37t1S/QqNpq7H6osvvhA7duwQV65cEbGxseKVV14Rcrlc7N27V6pfodHMnj1b2NjYiAMHDoi0tDTNraioSLPP5MmTxfz58zU/HzlyRBgZGYlly5aJCxcuiIULFwqlUiliY2Ol+BUaTX2O1aJFi8Qff/whrl27Jk6dOiXGjx8vTE1Nxblz56T4FRrN/Pnzxd9//y0SEhLE2bNnxfz584VMJhN//vmnEILvqX+q67Fqqu+pmtx71ZIuvLdYyNTg7iXC996CgoKEEEIEBQWJAQMGVHtMt27dhLGxsWjVqpUICwtr9NxSqOuxWrp0qfD29hampqbCzs5OPPbYY2L//v3ShG9k9ztOAKq8VwYMGKA5dndt3rxZtG3bVhgbG4uOHTuKXbt2NW5wCdTnWM2bN0+4u7sLY2Nj4eTkJJ566ikRHR3d+OEbWUhIiPDw8BDGxsbCwcFBDBo0SPPFLATfU/9U12PVVN9TNbm3kNGF95ZMCCEabryHiIiIqOFwjgwRERHpLRYyREREpLdYyBAREZHeYiFDREREeouFDBEREektFjJERESkt1jIEBERkd5iIUNE9A+enp748ssvpY5BRLXEQoaI7is4OBgymQwymQxKpRJeXl548803UVJSInU0IiINrn5NRDV68sknERYWhvLycpw6dQpBQUGQyWRYunSp1NH0SllZGYyNjaWOQWSQOCJDRDUyMTGBs7Mz3NzcMGrUKAwePBh79uzR3H+/0zDdunXD+++/r/lZJpNh1apVePbZZ2Fubo42bdpg586dD3xdT09PfPzxxwgJCYGVlRXc3d3x/fffa+4/cOAAZDIZcnJyNNtiYmIgk8mQmJgIAAgPD0ezZs3w66+/wsfHB+bm5hgzZgyKiooQEREBT09P2NraYu7cuVCpVFVePz8/HxMmTICFhQVatGiB5cuXV7k/JycH06dPh4ODA6ytrTFw4ECcOXNGc//777+Pbt26YdWqVfDy8oKpqekDf18iqj8WMkRUK3FxcTh69Gi9RhYWLVqEcePG4ezZs3jqqacwceJEZGdnP/Axn332Gfz8/HD69Gm8+OKLmD17Ni5dulSn1y0qKsJXX32FjRs3Yvfu3Thw4ACeffZZ/Pbbb/jtt9/w448/YuXKldiyZUuVx/3nP/9B165dcfr0acyfPx+vvPJKlQJu7NixyMzMxO+//45Tp07B19cXgwYNqvI7Xb16FVu3bsW2bdsQExNTp9xEVAcNuiQlEemtoKAgoVAohIWFhTAxMREAhFwuF1u2bNHs4+HhIb744osqj+vatatYuHCh5mcA4p133tH8XFBQIACI33//vcbX9vDwEJMmTdL8rFarhaOjo1ixYoUQ4n8rrt++fVuzz+nTpwUAkZCQIIQQIiwsTAAQV69e1ewza9YsYW5uLvLz8zXbhg4dKmbNmlXltZ988skqeZ5//nkxbNgwIYQQhw4dEtbW1qKkpKTKPt7e3mLlypVCCCEWLlwolEqlyMzMrPF3JCLt4BwZIqrR448/jhUrVqCwsBBffPEFjIyMMHr06Do/T5cuXTT/bWFhAWtra2RmZtb6MTKZDM7Ozg99zL3Mzc3h7e2t+dnJyQmenp6wtLSssu3e5w0MDKz2891TaGfOnEFBQQGaN29eZZ/i4mJcu3ZN87OHhwccHBzqlJeI6o6FDBHVyMLCAq1btwYAhIaGomvXrli9ejWmTZsGAJDL5RBCVHlMeXl5tedRKpVVfpbJZFCr1Q987Qc9Ri6vPCv+z9eu7evWJ8s/FRQUwMXFBQcOHKh2X7NmzTT/bWFhUevnJKL6YyFDRLUil8vx1ltv4bXXXsMLL7wAMzMzODg4IC0tTbNPXl4eEhISGjzL3ZGOtLQ02NraAoBW56EcP3682s/t27cHAPj6+iI9PR1GRkbw9PTU2msSUf1wsi8R1drYsWOhUCg0V/EMHDgQP/74Iw4dOoTY2FgEBQVBoVA0eI7WrVvDzc0N77//Pq5cuYJdu3bhs88+09rzHzlyBJ9++ikuX76M5cuX46effsIrr7wCABg8eDACAwMxatQo/Pnnn0hMTMTRo0fx9ttv4+TJk1rLQES1w0KGiGrNyMgIL730Ej799FMUFhZiwYIFGDBgAJ5++mkMHz4co0aNqjInpaEolUps2LABFy9eRJcuXbB06VJ8+OGHWnv+f/3rXzh58iS6d++ODz/8EJ9//jmGDh0KoPJU1G+//Yb+/ftj6tSpaNu2LcaPH4+kpCQ4OTlpLQMR1Y5M3HuCm4iIiEhPcESGiIiI9BYLGSIiItJbLGSIiIhIb7GQISIiIr3FQoaIiIj0FgsZIiIi0lssZIiIiEhvsZAhIiIivcVChoiIiPQWCxkiIiLSWyxkiIiISG+xkCEiIiK99f85nQDsCGP3IwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot accuracy over runs\n",
    "plt.plot(range(1,len(accuracies)+1), accuracies, marker='o')\n",
    "plt.title('Accuracy over time')\n",
    "plt.xlabel('Run number')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.savefig('metaprompt_accuracy.png')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "height": 64
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Best Routine\n",
       "```markdown\n",
       "1. Verify Customer Identity\n",
       "   a. Call `verify_identity`.\n",
       "      - Prompt the customer to provide their booking reference, full name, and flight number.\n",
       "\n",
       "2. Listen and Understand Customer Request\n",
       "   a. Call `ask_clarification` with prompt: \"How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?\"\n",
       "      - If the customer has already stated their specific request (e.g., \"My flight was canceled by the airline and I want a refund\"), you may skip this step or confirm that you understood them correctly.\n",
       "\n",
       "3. Determine Customer Request Type\n",
       "   a. If the customer wants to **cancel** or states the flight was canceled by the airline:\n",
       "      i. If it is **airline-initiated** (customer states airline canceled, or internal data confirms it): proceed to step 5.\n",
       "      ii. Else if it is **customer-initiated**: proceed to step 4.\n",
       "   b. Else if the customer wants to **change**:\n",
       "      i. Proceed to step 6.\n",
       "   c. Else if the customer wants to **inquire about compensation**:\n",
       "      i. Proceed to step 9.\n",
       "   d. Else:\n",
       "      i. Call `ask_clarification` with prompt: \"I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?\"\n",
       "\n",
       "4. Handle Customer-Initiated Cancellations\n",
       "   a. Call `check_ticket_type` to determine if the ticket is refundable, non-refundable, or flexible.\n",
       "   b. If ticket is **refundable**:\n",
       "      i. If cancellation is **within 24 hours of booking**:\n",
       "         - Call `process_full_refund`.\n",
       "      ii. Else:\n",
       "         - Call `check_fare_rules`.\n",
       "         - Parse the fare rules to see if a partial refund is indicated and the refund amount.\n",
       "             * If a partial refund is applicable:\n",
       "               - Call `process_partial_refund` using the amount returned by `check_fare_rules`.\n",
       "             * Else if a full refund is indicated:\n",
       "               - Call `process_full_refund`.\n",
       "   c. Else if ticket is **non-refundable**:\n",
       "      i. Call `offer_flight_credit`.\n",
       "      ii. Inform the customer about the administration fee.\n",
       "      iii. Call `determine_refund_method`.\n",
       "      iv. If customer opts for travel credit:\n",
       "         - Call `apply_travel_credit`.\n",
       "      v. Else:\n",
       "         - Inform about penalty fees.\n",
       "   d. Else if ticket is **flexible**:\n",
       "      i. Treat as a refundable ticket and follow the same steps in 4(b).\n",
       "   e. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   f. Call `case_resolution`.\n",
       "\n",
       "5. Handle Airline-Initiated Cancellations\n",
       "   a. Determine reason (weather-related or mechanical/operational). If the user explicitly states they want a refund, proceed with `process_full_refund`. If the user wants to be rebooked, proceed with `rebook_without_fee`. If the user requests a hotel due to an overnight delay, proceed with `offer_accommodation`.\n",
       "   b. If cancellation is **weather-related**:\n",
       "      i. Ask if the customer would like to rebook or receive a refund.\n",
       "         * If the customer wants to be rebooked:\n",
       "           - Call `rebook_without_fee`.\n",
       "         * If the customer explicitly asks for a refund:\n",
       "           - Call `process_full_refund`.\n",
       "   c. Else if cancellation is due to **mechanical issues/operational changes**:\n",
       "      i. If the customer needs rebooking:\n",
       "         - Call `prioritize_missed_connections`.\n",
       "      ii. If overnight stay is required:\n",
       "         - Call `offer_accommodation`.\n",
       "   d. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   e. Call `case_resolution`.\n",
       "\n",
       "6. Handle Changes\n",
       "   a. Call `check_ticket_type` to determine if the ticket is flexible or non-flexible.\n",
       "   b. Gather the new flight details from the customer (e.g., flight code and date). Confirm you have the correct details before proceeding.\n",
       "   c. If the change is **customer-initiated**:\n",
       "      i. If **same-day change**:\n",
       "         - If ticket is **flexible**:\n",
       "            * Call `process_change_no_fee` with the exact `new_flight` details provided by the customer.\n",
       "         - Else:\n",
       "            * Call `apply_change_fee`.\n",
       "            * Call `check_fare_rules`.\n",
       "            * If a fare difference exists, inform the customer about any additional fare.\n",
       "      ii. Else if **changes in advance**:\n",
       "         - If **within 7 days of departure**:\n",
       "            * Call `apply_change_fee`.\n",
       "         - Else:\n",
       "            * Call `apply_lesser_change_fee`.\n",
       "   d. Else if the change is **airline-initiated**:\n",
       "      i. If schedule change is **minor (less than 2 hours)**:\n",
       "         - Inform the customer of the new schedule.\n",
       "         - If customer requests an alternative:\n",
       "            * Call `provide_alternative_options`.\n",
       "      ii. Else if schedule change is **major (2+ hours)**:\n",
       "         - Offer options to `rebook_without_fee`, `process_full_refund`, or `offer_flight_credit`.\n",
       "   e. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   f. Call `case_resolution`.\n",
       "\n",
       "7. Rebooking Guidelines\n",
       "   a. Call `check_next_available_flight`.\n",
       "   b. If no suitable flight is available:\n",
       "      i. Call `check_interline_partners`.\n",
       "      ii. If interline partners have available flights:\n",
       "         - Call `book_interline_flight`.\n",
       "      iii. Else:\n",
       "         - Inform the customer and offer alternative solutions.\n",
       "   c. If a higher class is available:\n",
       "      i. Call `offer_available_upgrades`.\n",
       "   d. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   e. Call `case_resolution`.\n",
       "\n",
       "8. Compensation and Refund Processing\n",
       "   a. Inform the customer about the refund timeline based on payment method.\n",
       "   b. Call `determine_refund_method`.\n",
       "      i. If refund to **original payment method**:\n",
       "         - Call `process_refund`.\n",
       "      ii. Else if offering **travel credits**:\n",
       "         - Call `apply_travel_credit`.\n",
       "   c. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   d. Call `case_resolution`.\n",
       "\n",
       "9. Handle Compensation Requests\n",
       "   a. If the customer wants to check if they are eligible for compensation (\"Am I eligible?\"), call `check_compensation_eligibility`.\n",
       "   b. If the customer explicitly wants to request or proceed with compensation, call `assess_compensation_request`.\n",
       "   c. If eligible for compensation:\n",
       "      i. Offer meal vouchers or hotel accommodation if delay exceeds four hours.\n",
       "      ii. Inform about monetary compensation if applicable.\n",
       "   d. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "   e. Call `case_resolution`.\n",
       "\n",
       "10. Handle Special Cases\n",
       "    a. If **medical emergency**:\n",
       "       i. Ask the customer for the exact medical certificate ID.\n",
       "       ii. Call `process_flexible_cancellation` with the precise medical certificate ID.\n",
       "       iii. Call `waive_change_fees` if applicable.\n",
       "    b. If **bereavement case**:\n",
       "       i. Call `apply_bereavement_flexibility` with documentation.\n",
       "    c. If **group booking**:\n",
       "       i. Call `process_partial_group_cancellation` for individual cancellations.\n",
       "       ii. If name change is requested:\n",
       "          - If made 7+ days before departure:\n",
       "             * Call `permit_name_change`.\n",
       "    d. If **unaccompanied minor**:\n",
       "       i. Call `arrange_supervision` for flight changes.\n",
       "       ii. Call `prioritize_minors_rebooking`.\n",
       "    e. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "    f. Call `case_resolution`.\n",
       "\n",
       "11. Handle FAQs\n",
       "    a. For **connecting flight affected**:\n",
       "       i. If customer missed a connecting flight due to airline delay and requires rebooking:\n",
       "          - Call `prioritize_missed_connections`.\n",
       "       ii. If customer requests compensation for the delay:\n",
       "          - Call `check_compensation_eligibility` or `assess_compensation_request` depending on their request.\n",
       "    b. For **upgrade requests after cancellation**:\n",
       "       i. Call `offer_available_upgrades`.\n",
       "    c. For **destination changes**:\n",
       "       i. Call `process_destination_change`.\n",
       "    d. For **third-party bookings**:\n",
       "       i. Call `assist_third_party_booking`.\n",
       "    e. For **compensation requests**:\n",
       "       i. If the customer specifically asks about eligibility: call `check_compensation_eligibility`.\n",
       "       ii. If requesting compensation: call `assess_compensation_request`.\n",
       "    f. If **preferred flight is fully booked**:\n",
       "       i. Offer the next available option.\n",
       "       ii. If possible, place on waitlist or offer upgrade.\n",
       "    g. Ask the customer: \"Is there anything else I can assist you with?\"\n",
       "    h. Call `case_resolution`.\n",
       "\n",
       "12. Finalize Case\n",
       "    a. Ensure all customer requests have been addressed.\n",
       "    b. Call `case_resolution`.\n",
       "```"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Print best routine\n",
    "best_routine = routines[accuracies.index(max(accuracies))]\n",
    "display(Markdown(f\"## Best Routine\\n{best_routine}\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'```markdown\\n1. Verify Customer Identity\\n   a. Call `verify_identity`.\\n      - Prompt the customer to provide their booking reference, full name, and flight number.\\n\\n2. Listen and Understand Customer Request\\n   a. Call `ask_clarification` with prompt: \"How may I assist you today? Would you like to cancel your booking, make changes to your flight, or inquire about compensation?\"\\n      - If the customer has already stated their specific request (e.g., \"My flight was canceled by the airline and I want a refund\"), you may skip this step or confirm that you understood them correctly.\\n\\n3. Determine Customer Request Type\\n   a. If the customer wants to **cancel** or states the flight was canceled by the airline:\\n      i. If it is **airline-initiated** (customer states airline canceled, or internal data confirms it): proceed to step 5.\\n      ii. Else if it is **customer-initiated**: proceed to step 4.\\n   b. Else if the customer wants to **change**:\\n      i. Proceed to step 6.\\n   c. Else if the customer wants to **inquire about compensation**:\\n      i. Proceed to step 9.\\n   d. Else:\\n      i. Call `ask_clarification` with prompt: \"I’m sorry, I didn’t quite understand that. Could you please clarify if you want to cancel, change your flight, or inquire about compensation?\"\\n\\n4. Handle Customer-Initiated Cancellations\\n   a. Call `check_ticket_type` to determine if the ticket is refundable, non-refundable, or flexible.\\n   b. If ticket is **refundable**:\\n      i. If cancellation is **within 24 hours of booking**:\\n         - Call `process_full_refund`.\\n      ii. Else:\\n         - Call `check_fare_rules`.\\n         - Parse the fare rules to see if a partial refund is indicated and the refund amount.\\n             * If a partial refund is applicable:\\n               - Call `process_partial_refund` using the amount returned by `check_fare_rules`.\\n             * Else if a full refund is indicated:\\n               - Call `process_full_refund`.\\n   c. Else if ticket is **non-refundable**:\\n      i. Call `offer_flight_credit`.\\n      ii. Inform the customer about the administration fee.\\n      iii. Call `determine_refund_method`.\\n      iv. If customer opts for travel credit:\\n         - Call `apply_travel_credit`.\\n      v. Else:\\n         - Inform about penalty fees.\\n   d. Else if ticket is **flexible**:\\n      i. Treat as a refundable ticket and follow the same steps in 4(b).\\n   e. Ask the customer: \"Is there anything else I can assist you with?\"\\n   f. Call `case_resolution`.\\n\\n5. Handle Airline-Initiated Cancellations\\n   a. Determine reason (weather-related or mechanical/operational). If the user explicitly states they want a refund, proceed with `process_full_refund`. If the user wants to be rebooked, proceed with `rebook_without_fee`. If the user requests a hotel due to an overnight delay, proceed with `offer_accommodation`.\\n   b. If cancellation is **weather-related**:\\n      i. Ask if the customer would like to rebook or receive a refund.\\n         * If the customer wants to be rebooked:\\n           - Call `rebook_without_fee`.\\n         * If the customer explicitly asks for a refund:\\n           - Call `process_full_refund`.\\n   c. Else if cancellation is due to **mechanical issues/operational changes**:\\n      i. If the customer needs rebooking:\\n         - Call `prioritize_missed_connections`.\\n      ii. If overnight stay is required:\\n         - Call `offer_accommodation`.\\n   d. Ask the customer: \"Is there anything else I can assist you with?\"\\n   e. Call `case_resolution`.\\n\\n6. Handle Changes\\n   a. Call `check_ticket_type` to determine if the ticket is flexible or non-flexible.\\n   b. Gather the new flight details from the customer (e.g., flight code and date). Confirm you have the correct details before proceeding.\\n   c. If the change is **customer-initiated**:\\n      i. If **same-day change**:\\n         - If ticket is **flexible**:\\n            * Call `process_change_no_fee` with the exact `new_flight` details provided by the customer.\\n         - Else:\\n            * Call `apply_change_fee`.\\n            * Call `check_fare_rules`.\\n            * If a fare difference exists, inform the customer about any additional fare.\\n      ii. Else if **changes in advance**:\\n         - If **within 7 days of departure**:\\n            * Call `apply_change_fee`.\\n         - Else:\\n            * Call `apply_lesser_change_fee`.\\n   d. Else if the change is **airline-initiated**:\\n      i. If schedule change is **minor (less than 2 hours)**:\\n         - Inform the customer of the new schedule.\\n         - If customer requests an alternative:\\n            * Call `provide_alternative_options`.\\n      ii. Else if schedule change is **major (2+ hours)**:\\n         - Offer options to `rebook_without_fee`, `process_full_refund`, or `offer_flight_credit`.\\n   e. Ask the customer: \"Is there anything else I can assist you with?\"\\n   f. Call `case_resolution`.\\n\\n7. Rebooking Guidelines\\n   a. Call `check_next_available_flight`.\\n   b. If no suitable flight is available:\\n      i. Call `check_interline_partners`.\\n      ii. If interline partners have available flights:\\n         - Call `book_interline_flight`.\\n      iii. Else:\\n         - Inform the customer and offer alternative solutions.\\n   c. If a higher class is available:\\n      i. Call `offer_available_upgrades`.\\n   d. Ask the customer: \"Is there anything else I can assist you with?\"\\n   e. Call `case_resolution`.\\n\\n8. Compensation and Refund Processing\\n   a. Inform the customer about the refund timeline based on payment method.\\n   b. Call `determine_refund_method`.\\n      i. If refund to **original payment method**:\\n         - Call `process_refund`.\\n      ii. Else if offering **travel credits**:\\n         - Call `apply_travel_credit`.\\n   c. Ask the customer: \"Is there anything else I can assist you with?\"\\n   d. Call `case_resolution`.\\n\\n9. Handle Compensation Requests\\n   a. If the customer wants to check if they are eligible for compensation (\"Am I eligible?\"), call `check_compensation_eligibility`.\\n   b. If the customer explicitly wants to request or proceed with compensation, call `assess_compensation_request`.\\n   c. If eligible for compensation:\\n      i. Offer meal vouchers or hotel accommodation if delay exceeds four hours.\\n      ii. Inform about monetary compensation if applicable.\\n   d. Ask the customer: \"Is there anything else I can assist you with?\"\\n   e. Call `case_resolution`.\\n\\n10. Handle Special Cases\\n    a. If **medical emergency**:\\n       i. Ask the customer for the exact medical certificate ID.\\n       ii. Call `process_flexible_cancellation` with the precise medical certificate ID.\\n       iii. Call `waive_change_fees` if applicable.\\n    b. If **bereavement case**:\\n       i. Call `apply_bereavement_flexibility` with documentation.\\n    c. If **group booking**:\\n       i. Call `process_partial_group_cancellation` for individual cancellations.\\n       ii. If name change is requested:\\n          - If made 7+ days before departure:\\n             * Call `permit_name_change`.\\n    d. If **unaccompanied minor**:\\n       i. Call `arrange_supervision` for flight changes.\\n       ii. Call `prioritize_minors_rebooking`.\\n    e. Ask the customer: \"Is there anything else I can assist you with?\"\\n    f. Call `case_resolution`.\\n\\n11. Handle FAQs\\n    a. For **connecting flight affected**:\\n       i. If customer missed a connecting flight due to airline delay and requires rebooking:\\n          - Call `prioritize_missed_connections`.\\n       ii. If customer requests compensation for the delay:\\n          - Call `check_compensation_eligibility` or `assess_compensation_request` depending on their request.\\n    b. For **upgrade requests after cancellation**:\\n       i. Call `offer_available_upgrades`.\\n    c. For **destination changes**:\\n       i. Call `process_destination_change`.\\n    d. For **third-party bookings**:\\n       i. Call `assist_third_party_booking`.\\n    e. For **compensation requests**:\\n       i. If the customer specifically asks about eligibility: call `check_compensation_eligibility`.\\n       ii. If requesting compensation: call `assess_compensation_request`.\\n    f. If **preferred flight is fully booked**:\\n       i. Offer the next available option.\\n       ii. If possible, place on waitlist or offer upgrade.\\n    g. Ask the customer: \"Is there anything else I can assist you with?\"\\n    h. Call `case_resolution`.\\n\\n12. Finalize Case\\n    a. Ensure all customer requests have been addressed.\\n    b. Call `case_resolution`.\\n```'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "routines[accuracies.index(max(accuracies))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "02dd1466359040b2b9abf121fe6afc21": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "047fdf3d270f4bf88a7f4118da18c77f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "049fddffb97047a1a0ee50b50fb14819": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "07ba1e18e0144d40983bffe1de10dd0a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "19577df9b7334a99bb3f6160c1b5e94a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e539b644861d43298f7e532a314ae793",
      "placeholder": "​",
      "style": "IPY_MODEL_a117e78c14a34feba1279c804afe18e7",
      "value": "Downloading readme: 100%"
     }
    },
    "19cecde4bacd4a5988fa63096022f064": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1cadd43bcf0d485abc8d1cdbb2d5133d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_590b561a6c3d49dd8ba9137b65a9e8f7",
      "placeholder": "​",
      "style": "IPY_MODEL_78b8098d991442a3be84d1b6aec1973f",
      "value": "Downloading data: 100%"
     }
    },
    "1efd7409c79a4e21a45dee261fd7d892": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8fc91035bd4e493da805823bfbc6bfe4",
      "placeholder": "​",
      "style": "IPY_MODEL_51d7117eada64d55b8cc39df1fc35ebc",
      "value": "Downloading readme: 100%"
     }
    },
    "24179d73b2d84bff89a8918f366bf049": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2943d456d9884f7fb532d1cf545eaf45": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "29d3c367dab9400f817331d768ef3e9f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2be38f341a5e42eda84ee8a46dd11ea1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1cadd43bcf0d485abc8d1cdbb2d5133d",
       "IPY_MODEL_cfabaca567c04ec5bf74a49a73e4adb2",
       "IPY_MODEL_9a1de3dd22b3433690a97ebe951201f3"
      ],
      "layout": "IPY_MODEL_c3fb20b52379441cb506afebe4a11d50"
     }
    },
    "300d75910ebc49f7b2e13a5381b6f5a0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "323f452344ae4b5b9c95a29b37ae7a4d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "386e9e99a6a040abac9b3b7d106aec6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "394020bf24064870a1eaa0ceaac4bb7c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1efd7409c79a4e21a45dee261fd7d892",
       "IPY_MODEL_8b559554d99741baa15099170cb88032",
       "IPY_MODEL_890882a4891942548135ae361ad731c2"
      ],
      "layout": "IPY_MODEL_a2ceaa67922d445aad0bf021c3a4afd1"
     }
    },
    "397d08517afd42d6ab48c9ebddb2dd55": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3d74b3de38074d87938cece35eb967b8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3d80170b57c4401fbe7710f373e2a4fd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d253b03ae47443d5a7d261611179780b",
      "placeholder": "​",
      "style": "IPY_MODEL_e18fb366517a451c94624d9c8b865533",
      "value": " 1260/1260 [00:00&lt;00:00, 3245.45 examples/s]"
     }
    },
    "43cbdd84aac249c4a338332fcdb47ff4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4522ee8aed0c4529a2240a9c8148af5b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "47927954d79d4c4f89ca85bdcc17f803": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5a16ff12c8ea4869bbde146b88cd89c2",
       "IPY_MODEL_bcc52307c76e453b8888b818e898dd48",
       "IPY_MODEL_b4c762cf527f494dabcb4ffe4643fef8"
      ],
      "layout": "IPY_MODEL_98c2b36767184aa69eb1335932e342e0"
     }
    },
    "4ae4a21026c94828a2722f071b7fc4aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e1e581eebe824f03a091502158de280d",
      "max": 140,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b1ad6430c4c7494790ea13d48a6859b2",
      "value": 140
     }
    },
    "4ce8885841844ba68b3dc2e7b0f0a7e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3d74b3de38074d87938cece35eb967b8",
      "placeholder": "​",
      "style": "IPY_MODEL_4522ee8aed0c4529a2240a9c8148af5b",
      "value": "Generating test split: 100%"
     }
    },
    "4dfd53e3963e4f7f868b2074c08f6f00": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7ee1abd40d7e471ea8e0d3e9aef59a0d",
      "placeholder": "​",
      "style": "IPY_MODEL_bff34a8843494bdebe0fa50f4afdc469",
      "value": "Downloading data: 100%"
     }
    },
    "4e9002860aba496b8ec3a531dfb930ef": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "51d7117eada64d55b8cc39df1fc35ebc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "547559a5534945a994a78581da8d227c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "54d7f9b6319c4ff59f3296c4a18af352": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "590b561a6c3d49dd8ba9137b65a9e8f7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5a16ff12c8ea4869bbde146b88cd89c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_83b7bfc6a35d4d65bc6cf43b26ddc6d5",
      "placeholder": "​",
      "style": "IPY_MODEL_6c42afe7ecb2447b8536d2b8d9f855e2",
      "value": "Downloading data files: 100%"
     }
    },
    "5f400dd761f541fbad68a80fbfdbdfcb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "6750b0c252aa4a5880469512a64808eb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_19577df9b7334a99bb3f6160c1b5e94a",
       "IPY_MODEL_ed23e5abd0874ed08370f8ca2f67ac86",
       "IPY_MODEL_b9840c6b5845445fbfc4cdbca5c23be8"
      ],
      "layout": "IPY_MODEL_24179d73b2d84bff89a8918f366bf049"
     }
    },
    "6bf62902aa91468ea7912ed174ba9244": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6c42afe7ecb2447b8536d2b8d9f855e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6fcb5098379b49738048131f0e4e309f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "735f73487e3f4b7c94a31b068c195bb7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "73c02cc2024e499dabbbc14e790e9806": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b0d71144c9a04c5b9c50fa603fd4fe2a",
       "IPY_MODEL_9976fdc2d6cc4e4cba69ade071de958f",
       "IPY_MODEL_ff7be293b0914be8b5cfc40ea4831b7c"
      ],
      "layout": "IPY_MODEL_a8ea2c9ffb3a41f58b50a5c6891f2861"
     }
    },
    "73efe6f9ec6341e3b4fa1a5d72b6089e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "77ecb79e1e514c81897e4e7cae0bf7d6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b5e33283acf64301bfdfa96d3840d474",
       "IPY_MODEL_f3181fc8fe41466a99e5389554e95113",
       "IPY_MODEL_3d80170b57c4401fbe7710f373e2a4fd"
      ],
      "layout": "IPY_MODEL_dfa935dfcc52466eae0ce086bb4be9b4"
     }
    },
    "78b8098d991442a3be84d1b6aec1973f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7b6accd51fe04209a54fddb054d32c8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_851dd72892df4f57a0966d108281a0a8",
      "placeholder": "​",
      "style": "IPY_MODEL_ed0345a208a142dd9b077b31cd890478",
      "value": " 140/140 [00:00&lt;00:00, 3367.28 examples/s]"
     }
    },
    "7ee1abd40d7e471ea8e0d3e9aef59a0d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "801483308cca4385bdfda0af53a3c2dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bce78a67b1ed4cd1ac83252eaa20ba96",
      "placeholder": "​",
      "style": "IPY_MODEL_f98efab2cb64414c8823ad6a3e4a942b",
      "value": " 2/2 [00:00&lt;00:00, 65.47it/s]"
     }
    },
    "822cb0eac16d49ba9b7755e7785403db": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "82674221ba934264a771f33400f48355": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_54d7f9b6319c4ff59f3296c4a18af352",
      "placeholder": "​",
      "style": "IPY_MODEL_8575b19028a74f8a8b7f7a9b7b64693e",
      "value": "Extracting data files: 100%"
     }
    },
    "83b7bfc6a35d4d65bc6cf43b26ddc6d5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "851dd72892df4f57a0966d108281a0a8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8575b19028a74f8a8b7f7a9b7b64693e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "88ab1bf634ef48c1a7c062530783eb2e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "890882a4891942548135ae361ad731c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e6b565f0b93a4ca98b0851050c8bb298",
      "placeholder": "​",
      "style": "IPY_MODEL_386e9e99a6a040abac9b3b7d106aec6b",
      "value": " 14.2k/14.2k [00:00&lt;00:00, 8.78kB/s]"
     }
    },
    "8b559554d99741baa15099170cb88032": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b2992394d8804669930b256a846c4869",
      "max": 14225,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_047fdf3d270f4bf88a7f4118da18c77f",
      "value": 14225
     }
    },
    "8fc91035bd4e493da805823bfbc6bfe4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "98c2b36767184aa69eb1335932e342e0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9976fdc2d6cc4e4cba69ade071de958f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_547559a5534945a994a78581da8d227c",
      "max": 9528,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_aba53e2fc4b64fd984abf96a474447a1",
      "value": 9528
     }
    },
    "9a1de3dd22b3433690a97ebe951201f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4e9002860aba496b8ec3a531dfb930ef",
      "placeholder": "​",
      "style": "IPY_MODEL_43cbdd84aac249c4a338332fcdb47ff4",
      "value": " 83.7k/83.7k [00:00&lt;00:00, 248kB/s]"
     }
    },
    "9b74c1d6cc9347e78527d55252dc1bf7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4dfd53e3963e4f7f868b2074c08f6f00",
       "IPY_MODEL_ead64f9d0cc44ebaa9924c5fa5792e9e",
       "IPY_MODEL_bcba2836350e4770ae7566d4ef93d810"
      ],
      "layout": "IPY_MODEL_fe9b186a194b4c409337e5731c9ccd2b"
     }
    },
    "a117e78c14a34feba1279c804afe18e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a2ceaa67922d445aad0bf021c3a4afd1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a33edde3e06a4e1ab9f5292d9828bb88": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a4740a1b11d34096ac87bec0abc50b1f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a7c7693e253a45f5b9355f538ca738c1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4ce8885841844ba68b3dc2e7b0f0a7e8",
       "IPY_MODEL_4ae4a21026c94828a2722f071b7fc4aa",
       "IPY_MODEL_7b6accd51fe04209a54fddb054d32c8f"
      ],
      "layout": "IPY_MODEL_049fddffb97047a1a0ee50b50fb14819"
     }
    },
    "a8ea2c9ffb3a41f58b50a5c6891f2861": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "aba53e2fc4b64fd984abf96a474447a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b0d71144c9a04c5b9c50fa603fd4fe2a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d42bdbcc10724273b5ec89175f4d6826",
      "placeholder": "​",
      "style": "IPY_MODEL_d3396bdc2fbc4bfeb80dce05c9817897",
      "value": "Downloading builder script: 100%"
     }
    },
    "b1ad6430c4c7494790ea13d48a6859b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b2992394d8804669930b256a846c4869": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b3f76afc0b154f09b50332641caad632": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b4c762cf527f494dabcb4ffe4643fef8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_29d3c367dab9400f817331d768ef3e9f",
      "placeholder": "​",
      "style": "IPY_MODEL_07ba1e18e0144d40983bffe1de10dd0a",
      "value": " 2/2 [00:00&lt;00:00,  2.27it/s]"
     }
    },
    "b5e33283acf64301bfdfa96d3840d474": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d9a0803cde9d41289d4562ca591c3e62",
      "placeholder": "​",
      "style": "IPY_MODEL_19cecde4bacd4a5988fa63096022f064",
      "value": "Generating train split: 100%"
     }
    },
    "b9840c6b5845445fbfc4cdbca5c23be8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_397d08517afd42d6ab48c9ebddb2dd55",
      "placeholder": "​",
      "style": "IPY_MODEL_6bf62902aa91468ea7912ed174ba9244",
      "value": " 577/577 [00:00&lt;00:00, 24.6kB/s]"
     }
    },
    "bcba2836350e4770ae7566d4ef93d810": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_300d75910ebc49f7b2e13a5381b6f5a0",
      "placeholder": "​",
      "style": "IPY_MODEL_c5e892714f0941109f12c739d5c4fe0f",
      "value": " 615k/615k [00:00&lt;00:00, 1.15MB/s]"
     }
    },
    "bcc52307c76e453b8888b818e898dd48": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_02dd1466359040b2b9abf121fe6afc21",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_822cb0eac16d49ba9b7755e7785403db",
      "value": 2
     }
    },
    "bce78a67b1ed4cd1ac83252eaa20ba96": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bff34a8843494bdebe0fa50f4afdc469": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c2555aa5eb274eb985e6f254fc1f526d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c3fb20b52379441cb506afebe4a11d50": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c5e892714f0941109f12c739d5c4fe0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cfabaca567c04ec5bf74a49a73e4adb2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f14a394f21e146a3bacf3ac5107bafb4",
      "max": 83671,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_323f452344ae4b5b9c95a29b37ae7a4d",
      "value": 83671
     }
    },
    "d253b03ae47443d5a7d261611179780b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d3396bdc2fbc4bfeb80dce05c9817897": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d42bdbcc10724273b5ec89175f4d6826": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d55cfdb129a04217b236426077e59d5a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_88ab1bf634ef48c1a7c062530783eb2e",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5f400dd761f541fbad68a80fbfdbdfcb",
      "value": 2
     }
    },
    "d9a0803cde9d41289d4562ca591c3e62": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d9f68e0cf073460a9057a15265462832": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_82674221ba934264a771f33400f48355",
       "IPY_MODEL_d55cfdb129a04217b236426077e59d5a",
       "IPY_MODEL_801483308cca4385bdfda0af53a3c2dd"
      ],
      "layout": "IPY_MODEL_73efe6f9ec6341e3b4fa1a5d72b6089e"
     }
    },
    "dfa935dfcc52466eae0ce086bb4be9b4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e18fb366517a451c94624d9c8b865533": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e1e581eebe824f03a091502158de280d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e539b644861d43298f7e532a314ae793": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e6b565f0b93a4ca98b0851050c8bb298": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ead64f9d0cc44ebaa9924c5fa5792e9e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a33edde3e06a4e1ab9f5292d9828bb88",
      "max": 614936,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6fcb5098379b49738048131f0e4e309f",
      "value": 614936
     }
    },
    "ed0345a208a142dd9b077b31cd890478": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ed23e5abd0874ed08370f8ca2f67ac86": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f1dc2afff2af44229be04dc43db5ab50",
      "max": 577,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c2555aa5eb274eb985e6f254fc1f526d",
      "value": 577
     }
    },
    "f14a394f21e146a3bacf3ac5107bafb4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f1dc2afff2af44229be04dc43db5ab50": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f3181fc8fe41466a99e5389554e95113": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2943d456d9884f7fb532d1cf545eaf45",
      "max": 1260,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_735f73487e3f4b7c94a31b068c195bb7",
      "value": 1260
     }
    },
    "f98efab2cb64414c8823ad6a3e4a942b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fe9b186a194b4c409337e5731c9ccd2b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ff7be293b0914be8b5cfc40ea4831b7c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b3f76afc0b154f09b50332641caad632",
      "placeholder": "​",
      "style": "IPY_MODEL_a4740a1b11d34096ac87bec0abc50b1f",
      "value": " 9.53k/9.53k [00:00&lt;00:00, 145kB/s]"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}