Hugging Face & Raspberry Pi with Modzy Edge.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d1bcf708",
   "metadata": {},
   "source": [
    "# Running a Hugging Face Model on Raspberry Pi\n",
    "In this notebook, we will download a model from **[Hugging Face](https://huggingface.co)** using the `transformers` library, save it locally to desktop, and automatically containerize it using **[Chassis.ml](https://chassis.ml)**. \n",
    "\n",
    "Then, we will use **[Modzy's](https://modzy.com)** edge feature to deploy the model to the Pi and leverage the Modzy [Python SDK](https://pypi.org/project/modzy-sdk/) to make inference calls to the model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "175cb037",
   "metadata": {},
   "source": [
    "### Environment Set Up\n",
    "\n",
    "1. Create a virtual environment (venv, conda, or other preferred virtual environment) with Python 3.6 or newer.\n",
    "2. Pip install at a minimum the following requirements:\n",
    "\n",
    "```pip install torch transformers[torch] numpy chassisml modzy-sdk```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b148eaf",
   "metadata": {},
   "source": [
    "### Hardware\n",
    "\n",
    "This tutorial is broken up into two parts: (1) containerizing a Hugging Face model, and (2) making API calls to the container. For part 1, all you will need to do is run the top half of this notebook, which you can do right on your laptop, Google colab, cloud environment, or your other preferred environment. In the second part, we will download the model container directly to our Raspberry Pi. To follow along in part 2, you can use your own Raspberry Pi or you can run your model container on your laptop or other infrastructure of choice if no Pi or edge device is available.    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71b2e349",
   "metadata": {},
   "source": [
    "###  Additional Resources\n",
    "\n",
    "Below are links to the several Python libraries used, Hugging Face model, built Docker container, ....\n",
    "* **[Chassis.ml](https://chassis.ml)**\n",
    "* **[Transformers](https://huggingface.co/docs/transformers/main/en/installation)**\n",
    "* **[TinyBERT HF Model](https://huggingface.co/gokuls/BERT-tiny-emotion-intent?text=I+like+you.+I+love+you)**\n",
    "* **[TinyBERT Docker Container](https://hub.docker.com/repository/docker/modzy/tinybert-arm)**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fda7c475",
   "metadata": {},
   "source": [
    "# 1. Containerize Hugging Face Model\n",
    "In the first section of this notebook, we will download a [TinyBERT](https://huggingface.co/gokuls/BERT-tiny-emotion-intent?text=I+like+you.+I+love+you) model from Hugging Face and use the Chassis Python library to automatically build a machine learning model container."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90e0d9ff",
   "metadata": {},
   "source": [
    "## Hugging Face\n",
    "Download and test model from transformers library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "873cbc0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import packages\n",
    "import os\n",
    "import torch\n",
    "import chassisml\n",
    "import numpy as np\n",
    "from transformers import BertTokenizer, BertForSequenceClassification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "267615e2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f50fdefe9b184e269b84f5399aa673df",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/226k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d194228c4a5246538a969607fbacfa8e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/125 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3e5318f4c606482d95c48a1fba58013e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/389 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2e4c90c9138549dfa84d99577469268b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/973 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a6e2acd794df4e078eaafa2dbb0d640a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/16.8M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# download TinyBERT model and tokenizer\n",
    "tokenizer = BertTokenizer.from_pretrained(\"gokuls/BERT-tiny-emotion-intent\")\n",
    "model = BertForSequenceClassification.from_pretrained(\"gokuls/BERT-tiny-emotion-intent\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bc663cd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# save model locally so we can use/access it with Chassisml package\n",
    "tokenizer.save_pretrained(\"./tiny-bert-model\")\n",
    "model.save_pretrained(\"./tiny-bert-model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9c284d80",
   "metadata": {},
   "outputs": [],
   "source": [
    "# define labels from model config\n",
    "labels = model.config.id2label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "37e4d90a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'result': {'classPredictions': [{'class': 'LABEL_1', 'score': 0.99906284}, {'class': 'LABEL_2', 'score': 0.00026847195}, {'class': 'LABEL_0', 'score': 0.0002184094}, {'class': 'LABEL_3', 'score': 0.00019786197}, {'class': 'LABEL_5', 'score': 0.00013340256}, {'class': 'LABEL_4', 'score': 0.00011909042}]}}}\n"
     ]
    }
   ],
   "source": [
    "# run sample inference on TinyBERT model\n",
    "text = \"I cannot wait to go skiing this winter!\"\n",
    "inputs = tokenizer(text, return_tensors=\"pt\")\n",
    "with torch.no_grad():\n",
    "    logits = model(**inputs).logits\n",
    "    softmax = torch.nn.functional.softmax(logits, dim=1).detach().cpu().numpy()\n",
    "indices = np.argsort(softmax)[0][::-1]\n",
    "results = {\n",
    "    \"data\": {\n",
    "        \"result\": {\n",
    "            \"classPredictions\": [{\"class\": labels[i], \"score\": softmax[0][i]} for i in indices]\n",
    "        }\n",
    "    }\n",
    "}\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f35848d",
   "metadata": {},
   "source": [
    "## Chassisml\n",
    "Automatically build a container from this model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7f0b5f47",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load model to memory\n",
    "tinybert_tokenizer = BertTokenizer.from_pretrained(\"./tiny-bert-model\")\n",
    "tinybert_model = BertForSequenceClassification.from_pretrained(\"./tiny-bert-model\")\n",
    "mapped_labels = {\"LABEL_0\": 'sadness',\"LABEL_1\": 'joy',\"LABEL_2\": 'love',\"LABEL_3\": 'anger',\"LABEL_4\": 'fear',\"LABEL_5\": 'surprise'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5d02d21f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# define process function that will serve as our inference function\n",
    "def process(input_bytes):\n",
    "    # decode and preprocess data bytes\n",
    "    text = input_bytes.decode()\n",
    "    inputs = tinybert_tokenizer(text, return_tensors=\"pt\")\n",
    "    \n",
    "    # run preprocessed data through model\n",
    "    with torch.no_grad():\n",
    "        logits = tinybert_model(**inputs).logits\n",
    "        softmax = torch.nn.functional.softmax(logits, dim=1).detach().cpu().numpy()\n",
    "        \n",
    "    # postprocess \n",
    "    indices = np.argsort(softmax)[0][::-1]\n",
    "    results = {\n",
    "        \"data\": {\n",
    "            \"result\": {\n",
    "                \"classPredictions\": [{\"class\": mapped_labels[labels[i]], \"score\": softmax[0][i]} for i in indices]\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    return results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "d60edd10",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize Chassis client\n",
    "chassis_client = chassisml.ChassisClient(os.getenv(\"CHASSIS_URL\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e7626bb8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'{\"data\":{\"result\":{\"classPredictions\":[{\"class\":\"joy\",\"score\":0.9988540410995483},{\"class\":\"sadness\",\"score\":0.0006223577074706554},{\"class\":\"love\",\"score\":0.00022895698202773929},{\"class\":\"surprise\",\"score\":0.0001073237945092842},{\"class\":\"anger\",\"score\":0.0001029252671287395},{\"class\":\"fear\",\"score\":8.438550867140293e-05}]}}}'\n"
     ]
    }
   ],
   "source": [
    "# create Chassis model\n",
    "chassis_model = chassis_client.create_model(process_fn=process)\n",
    "\n",
    "# test Chassis model locally (can pass filepath, bufferedreader, bytes, or text here):\n",
    "sample_filepath = './input.text'\n",
    "results = chassis_model.test(sample_filepath)\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "45059e87",
   "metadata": {},
   "outputs": [],
   "source": [
    "# manually construct conda environment to pass to Chassis job. This step is optional, but we will define this env variable to minimize requirements installed in container\n",
    "env = {\n",
    "    \"name\": \"huggingface-chassis\",\n",
    "    \"channels\": ['conda-forge'],\n",
    "    \"dependencies\": [\n",
    "        \"python=3.8.5\",\n",
    "        {\n",
    "            \"pip\": [\n",
    "                \"chassisml\",\n",
    "                \"torch\",\n",
    "                \"transformers[torch]\",\n",
    "                \"numpy\"\n",
    "            ] \n",
    "        }\n",
    "    ]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f8e4a1b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting build job... Ok!\n"
     ]
    }
   ],
   "source": [
    "# publish model to Dockerhub\n",
    "response = chassis_model.publish(\n",
    "    model_name=\"TinyBERT ARM\",\n",
    "    model_version=\"1.0.0\",\n",
    "    registry_user=os.getenv(\"DOCKER_USER\"),\n",
    "    registry_pass=os.getenv(\"DOCKER_PASS\"),\n",
    "    conda_env=env,\n",
    "    arm64=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "527fb47b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# wait for job to complete and print out final status\n",
    "job_id = response.get('job_id')\n",
    "final_status = chassis_client.block_until_complete(job_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b625d49c",
   "metadata": {},
   "source": [
    "# 2. Make gRPC API Calls to Container\n",
    "In this section, we will leverage **[Modzy's](https://modzy.com)** Edge feature and APIs to deploy our model to our Raspberry Pi and make inference calls."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "056a68cd",
   "metadata": {},
   "source": [
    "## Modzy Edge APIs\n",
    "Leverage Modzy's Edge feature and inference APIs to scale production inferencing across multiple Pi devices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "3481db7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 reviews pulled\n",
      "Randomly sampled 10 reviews\n",
      "Job job-2HMlhHIfsvYVOocLzr1ScFLAXhh complete\n",
      "{'jobIdentifier': 'job-2HMlhHIfsvYVOocLzr1ScFLAXhh', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:50.267273022Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:50.267273022Z', 'updateTime': '2022-11-10T17:55:50.933817805Z', 'endTime': '2022-11-10T17:55:50.931753650Z', 'results.json': {'data': {'drift': None, 'explanation': None, 'result': {'classPredictions': [{'class': 'joy', 'score': 0.9976446032524109}, {'class': 'love', 'score': 0.0014795303577557206}, {'class': 'surprise', 'score': 0.0004638918035198003}, {'class': 'sadness', 'score': 0.0001844268263084814}, {'score': 0.0001166945367003791, 'class': 'fear'}, {'score': 0.0001107103016693145, 'class': 'anger'}]}}}}}}\n",
      "Job job-2HMlhMivGYFiSJN0RFVS9IwVvR8 complete\n",
      "{'jobIdentifier': 'job-2HMlhMivGYFiSJN0RFVS9IwVvR8', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:51.042811210Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:51.042811210Z', 'updateTime': '2022-11-10T17:55:51.094591314Z', 'endTime': '2022-11-10T17:55:51.092808513Z', 'results.json': {'data': {'result': {'classPredictions': [{'score': 0.9993231296539307, 'class': 'joy'}, {'score': 0.0002355954347876832, 'class': 'love'}, {'class': 'sadness', 'score': 0.00020303689234424382}, {'class': 'anger', 'score': 9.609204425942153e-05}, {'class': 'surprise', 'score': 8.383918611798435e-05}, {'score': 5.829294241266325e-05, 'class': 'fear'}]}, 'drift': None, 'explanation': None}}}}}\n",
      "Job job-2HMlhMNqPpmcnt94OesurjLbzRn complete\n",
      "{'jobIdentifier': 'job-2HMlhMNqPpmcnt94OesurjLbzRn', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:51.173286048Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:51.173286048Z', 'updateTime': '2022-11-10T17:55:51.246969770Z', 'endTime': '2022-11-10T17:55:51.245080251Z', 'results.json': {'data': {'explanation': None, 'result': {'classPredictions': [{'score': 0.9932616353034973, 'class': 'joy'}, {'class': 'love', 'score': 0.004308647476136684}, {'score': 0.0012882580049335957, 'class': 'sadness'}, {'class': 'surprise', 'score': 0.0007650450570508838}, {'class': 'anger', 'score': 0.0002663525810930878}, {'class': 'fear', 'score': 0.00011007327702827752}]}, 'drift': None}}}}}\n",
      "Job job-2HMlhPZPJHRWxH3w3BI4ZuSMo9q complete\n",
      "{'jobIdentifier': 'job-2HMlhPZPJHRWxH3w3BI4ZuSMo9q', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:51.355762604Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:51.355762604Z', 'updateTime': '2022-11-10T17:55:51.506791433Z', 'endTime': '2022-11-10T17:55:51.504825038Z', 'results.json': {'data': {'result': {'classPredictions': [{'class': 'joy', 'score': 0.985401451587677}, {'class': 'sadness', 'score': 0.010587078519165516}, {'score': 0.0021021743305027485, 'class': 'fear'}, {'class': 'anger', 'score': 0.0013567139394581318}, {'score': 0.00031111398129723966, 'class': 'love'}, {'class': 'surprise', 'score': 0.00024138731532730162}]}, 'explanation': None, 'drift': None}}}}}\n",
      "Job job-2HMlhLJcLWHSqipmmtpiFpiI353 complete\n",
      "{'jobIdentifier': 'job-2HMlhLJcLWHSqipmmtpiFpiI353', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:51.599922017Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:51.599922017Z', 'updateTime': '2022-11-10T17:55:51.656766831Z', 'endTime': '2022-11-10T17:55:51.654576479Z', 'results.json': {'data': {'result': {'classPredictions': [{'score': 0.9989933371543884, 'class': 'joy'}, {'score': 0.00033517638803459704, 'class': 'love'}, {'score': 0.000276383594609797, 'class': 'sadness'}, {'score': 0.00023546558804810047, 'class': 'anger'}, {'score': 9.12137984414585e-05, 'class': 'surprise'}, {'class': 'fear', 'score': 6.844829476904124e-05}]}, 'drift': None, 'explanation': None}}}}}\n",
      "Job job-2HMlhPS3hDFESFMQqF7oaHFOeIo complete\n",
      "{'jobIdentifier': 'job-2HMlhPS3hDFESFMQqF7oaHFOeIo', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:51.830540470Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:51.830540470Z', 'updateTime': '2022-11-10T17:55:51.899917552Z', 'endTime': '2022-11-10T17:55:51.898226312Z', 'results.json': {'data': {'explanation': None, 'result': {'classPredictions': [{'score': 0.9915585517883301, 'class': 'love'}, {'score': 0.003124780487269163, 'class': 'surprise'}, {'class': 'sadness', 'score': 0.002370655769482255}, {'score': 0.0019722934812307358, 'class': 'joy'}, {'score': 0.0005037139053456485, 'class': 'fear'}, {'score': 0.00047009275294840336, 'class': 'anger'}]}, 'drift': None}}}}}\n",
      "Job job-2HMlhZPFC2krGQxIm5kMG1pQrlW complete\n",
      "{'jobIdentifier': 'job-2HMlhZPFC2krGQxIm5kMG1pQrlW', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:52.020758802Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:52.020758802Z', 'updateTime': '2022-11-10T17:55:52.097619745Z', 'endTime': '2022-11-10T17:55:52.095626163Z', 'results.json': {'data': {'result': {'classPredictions': [{'score': 0.9985620379447937, 'class': 'sadness'}, {'class': 'joy', 'score': 0.0007896428578533232}, {'class': 'anger', 'score': 0.00029310458921827376}, {'class': 'fear', 'score': 0.0002052598720183596}, {'class': 'love', 'score': 0.000110613189463038}, {'class': 'surprise', 'score': 3.938205190934241e-05}]}, 'drift': None, 'explanation': None}}}}}\n",
      "Job job-2HMlhSyP5u3cczbEdDPkA4yfHJm complete\n",
      "{'jobIdentifier': 'job-2HMlhSyP5u3cczbEdDPkA4yfHJm', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:52.213029049Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:52.213029049Z', 'updateTime': '2022-11-10T17:55:52.325931910Z', 'endTime': '2022-11-10T17:55:52.324136816Z', 'results.json': {'data': {'result': {'classPredictions': [{'class': 'sadness', 'score': 0.9986981153488159}, {'score': 0.00047383105265907943, 'class': 'anger'}, {'score': 0.00036283937515690923, 'class': 'joy'}, {'class': 'fear', 'score': 0.00032776634907349944}, {'score': 8.706829248694703e-05, 'class': 'surprise'}, {'class': 'love', 'score': 5.0409136747475713e-05}]}, 'drift': None, 'explanation': None}}}}}\n",
      "Job job-2HMlhV9qb8RPQITlNtWop2yfZan complete\n",
      "{'jobIdentifier': 'job-2HMlhV9qb8RPQITlNtWop2yfZan', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:52.450198765Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:52.450198765Z', 'updateTime': '2022-11-10T17:55:52.629866119Z', 'endTime': '2022-11-10T17:55:52.627496081Z', 'results.json': {'data': {'explanation': None, 'result': {'classPredictions': [{'score': 0.9937891364097595, 'class': 'joy'}, {'class': 'anger', 'score': 0.0018118483712896705}, {'class': 'fear', 'score': 0.0017218805151060224}, {'class': 'surprise', 'score': 0.0012009210186079144}, {'class': 'love', 'score': 0.0011244256747886539}, {'class': 'sadness', 'score': 0.00035184461739845574}]}, 'drift': None}}}}}\n",
      "Job job-2HMlhYYYCgvjOKFJ3Aiv8sDszcM complete\n",
      "{'jobIdentifier': 'job-2HMlhYYYCgvjOKFJ3Aiv8sDszcM', 'accountIdentifier': 'local', 'submittedAt': '2022-11-10T17:55:52.847907095Z', 'total': 1, 'completed': 1, 'finished': True, 'results': {'job': {'status': 'SUCCESSFUL', 'startTime': '2022-11-10T17:55:52.847907095Z', 'updateTime': '2022-11-10T17:55:53.122880229Z', 'endTime': '2022-11-10T17:55:53.121572424Z', 'results.json': {'data': {'result': {'classPredictions': [{'class': 'love', 'score': 0.9732000827789307}, {'score': 0.014863110147416592, 'class': 'anger'}, {'score': 0.006234226282685995, 'class': 'surprise'}, {'class': 'joy', 'score': 0.004161038901656866}, {'score': 0.0012793490896001458, 'class': 'fear'}, {'class': 'sadness', 'score': 0.00026221826556138694}]}, 'drift': None, 'explanation': None}}}}}\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "from modzy.edge.client import EdgeClient\n",
    "client = EdgeClient(\"localhost\",55000)\n",
    "\n",
    "MODEL_ID = \"uxchm260wz\"\n",
    "MODEL_VERSION = \"1.0.0\"\n",
    "\n",
    "with open(\"test.ft.txt\", \"r\", encoding=\"utf-8\") as f:\n",
    "    text = f.readlines()[:2500] \n",
    "    \n",
    "# clean reviews before feeding to model\n",
    "text_cleaned = [t.split(\"__label__\")[-1][2:].replace(\"\\n\", \"\") for t in text]\n",
    "print(f\"{len(text_cleaned)} reviews pulled\")\n",
    "\n",
    "# randomly select n number of reviews\n",
    "reviews = random.choices(text_cleaned, k=10)\n",
    "print(f\"Randomly sampled {len(reviews)} reviews\")\n",
    "\n",
    "# submit n jobs for random baseline\n",
    "for review in reviews:\n",
    "    job = client.submit_text(MODEL_ID, MODEL_VERSION, {\"input.txt\": review})\n",
    "    final_job_details = client.block_until_complete(job)\n",
    "    results = client.get_results(job)\n",
    "    print(f\"Job {job} complete\")\n",
    "    print(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2808285b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "vscode": {
   "interpreter": {
    "hash": "d7abee4167f60b771422116448c3a76487982b8235f5dfe9b322751d0cef7d0c"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}