06-smart-device-automation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2144c632",
   "metadata": {},
   "source": [
    "# Smart Device Automation\n",
    "Now that we have generated a complete set of APIs and web pages for PhotovoltaicMeasurement data, now we will build an OpenShift CustomResourceDefinition (CRD), Ansible Role, and Ansible Playbooks to easily deploy PhotovoltaicMeasurement smart devices, register the devices in the IoT Agent, update the entity data in the Context Broker, and send the same entity data to the Smart Village Platform so that we can visualize it. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75a00608",
   "metadata": {},
   "source": [
    "## Install prerequisite operator-sdk\n",
    "\n",
    "Run the commands below to deploy the Operator SDK, for building new CRDs like PhotovoltaicMeasurement into our `smartvillage-operator``. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "07b2f6fa",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac)\n",
    "export OS=$(uname | awk '{print tolower($0)}')\n",
    "export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/v1.33.0\n",
    "curl -o /tmp/operator-sdk_${OS}_${ARCH} -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH}\n",
    "gpg --keyserver keyserver.ubuntu.com --recv-keys 052996E2A20B5C7E\n",
    "curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt\n",
    "curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt.asc\n",
    "gpg -u \"Operator SDK (release) <cncf-operator-sdk@cncf.io>\" --verify checksums.txt.asc\n",
    "grep operator-sdk_${OS}_${ARCH} checksums.txt | sha256sum -c -\n",
    "chmod +x /tmp/operator-sdk_${OS}_${ARCH} && mv /tmp/operator-sdk_${OS}_${ARCH} ~/.local/bin/operator-sdk\n",
    "echo DONE\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d923025",
   "metadata": {},
   "source": [
    "## Install python dependencies\n",
    "An IoT Agent expects Entity Data from all Smart Data Models to be sent through a message broker like RabbitMQ. We install `pika` so that messages can be sent to RabbitMQ to register new PhotovoltaicMeasurement Smart Devices and update Smart Device entity data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6cfdf7d4",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "pip install pika"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce4835fb",
   "metadata": {},
   "source": [
    "## Reconfigure NGSI-LD Smart Village Sync\n",
    "When an IoT Agent receives a message from RabbitMQ with identifiers that match a registered smart device, it sends the data to the Context Broker in NGSI-LD format. After the Context Broker updates the entity data for the device in PostgreSQL, it looks if any services are subscribed to the entity. We configure all smart devices with a Context Broker subscription (webhook) so that the Context Broker sends all entity data updates to the NGSI-LD Smart Village Sync application we deployed in the project. The NGSI-LD Smart Village Sync application is configured to send subscription data straight to the Smart Village Platform API. We need to update our NGSI-LD Smart Village Sync application to point to the new Smart Village application we started running in our OpenShift AI Workbench for development. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "448e33ca",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "oc -n $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) \\\n",
    "  set env deployment/ngsild-smartvillage-sync SMARTVILLAGE_HOST_NAME=$(oc get route/vscode-devel -o jsonpath={.spec.host})\n",
    "oc -n $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) \\\n",
    "  set env deployment/ngsild-smartvillage-sync SMARTVILLAGE_PORT=443\n",
    "oc -n $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) \\\n",
    "  set env deployment/ngsild-smartvillage-sync SMARTVILLAGE_SSL=true\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f620263d",
   "metadata": {},
   "source": [
    "## Define new automation for PhotovoltaicMeasurement in the smartvillage-operator\n",
    "\n",
    "To make it easy to deploy PhotovoltaicMeasurement data at the Edge like a real smart device, we create a new CustomResourceDefinition in our smartvillage-operator and some Ansible tasks, roles, and playbooks to make it easy. The smartvillage-operator gives you the choice to deploy smart devices as CRDs in OpenShift, or simply run ansible playbooks pointed to the same yaml definition in a file, instead of being managed by a pod in OpenShift. Use the Operator SDK to create a new CRD for PhotovoltaicMeasurement. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad93f6bc",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "(cd ~/smartvillage-operator && operator-sdk create api --group smartvillage --version v1 --kind PhotovoltaicMeasurement --generate-role)\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36952b80",
   "metadata": {},
   "source": [
    "During the code generation, we already generated the CRD schema for PhotovoltaicMeasurement automatically. Run the command below to observe the operator schema for PhotovoltaicMeasurement. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9eb3d352",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "cat ~/smartvillage-operator/roles/smart-data-model-vars/vars/PhotovoltaicMeasurement.yaml\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "312b973d",
   "metadata": {},
   "source": [
    "With the operator schema for PhotovoltaicMeasurement already defined, run the command below to generate the Ansible Playbooks, Roles, Defaults, and more to complete integration of PhotovoltaicMeasurement data into the smartvillage-operator. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06cf6583",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "(cd ~/smartvillage-operator && ansible-playbook write-smart-data-model-templates.yaml -e ENTITY_TYPE=PhotovoltaicMeasurement)\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e203fd6",
   "metadata": {},
   "source": [
    "## Deploy new PhotovoltaicMeasurement device shaped like a star on the map\n",
    "\n",
    "Our Smart Village Platform is ready to receive new PhotovoltaicMeasurement data in development, and our smartvillage-operator is ready to configure new PhotovoltaicMeasurement smart devices in the NGSI-LD supported FIWARE platform. Now it's time to create new PhotovoltaicMeasurement devices and update their entity data. Run the command below to deploy a new PhotovoltaicMeasurement smart device named `star` in the shape of a star on the map. Notice the `areaServed` and `name` attributes below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dfdfc41",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "echo \"\n",
    "apiVersion: smartvillage.computate.org/v1\n",
    "kind: PhotovoltaicMeasurement\n",
    "metadata:\n",
    "  name: star\n",
    "  namespace: $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)\n",
    "spec:\n",
    "  iotagent:\n",
    "    base_url: http://iotagent-json:4041\n",
    "  context_broker:\n",
    "    base_url: http://scorpiobroker:9090\n",
    "  ngsi_ld:\n",
    "    service: smarttrafficlights\n",
    "    service_path: /Sweden/Veberod/CityCenter\n",
    "    context: https://raw.githubusercontent.com/computate-org/smartabyar-smartvillage-static/main/fiware/context.jsonld\n",
    "  device:\n",
    "    id: star\n",
    "    subscription_url: http://ngsild-smartvillage-sync:8080\n",
    "  message_broker:\n",
    "    transport: AMQP\n",
    "    host: default-rabbitmq\n",
    "    port: 5672\n",
    "    user: user\n",
    "    secret:\n",
    "      name: rabbitmq-password\n",
    "      key: rabbitmq-password\n",
    "  smartvillage:\n",
    "    auth_secret_name: smartvillage\n",
    "    auth_token_url: https://sso.smartabyarsmartvillage.org/auth/realms/SMARTVILLAGE/protocol/openid-connect/token\n",
    "    site_base_url: https://$(oc get route/vscode-devel -o jsonpath={.spec.host})\n",
    "    smart_model_api: /api/PhotovoltaicMeasurement\n",
    "  attributes:\n",
    "    areaServed: {\n",
    "        \"coordinates\": [\n",
    "          [\n",
    "            [\n",
    "              13.492267031569327,\n",
    "              55.63382607412092\n",
    "            ],\n",
    "            [\n",
    "              13.492145868594037,\n",
    "              55.63382081304013\n",
    "            ],\n",
    "            [\n",
    "              13.492234410768901,\n",
    "              55.633773463278374\n",
    "            ],\n",
    "            [\n",
    "              13.492201789967083,\n",
    "              55.63370506907751\n",
    "            ],\n",
    "            [\n",
    "              13.492304312485658,\n",
    "              55.63373926619286\n",
    "            ],\n",
    "            [\n",
    "              13.492397514773728,\n",
    "              55.63370506907751\n",
    "            ],\n",
    "            [\n",
    "              13.492369554087844,\n",
    "              55.63377609382226\n",
    "            ],\n",
    "            [\n",
    "              13.492439455803208,\n",
    "              55.63382081304013\n",
    "            ],\n",
    "            [\n",
    "              13.492350913629707,\n",
    "              55.63382081304013\n",
    "            ],\n",
    "            [\n",
    "              13.492304312485658,\n",
    "              55.633878684892835\n",
    "            ],\n",
    "            [\n",
    "              13.492267031569327,\n",
    "              55.63382607412092\n",
    "            ]\n",
    "          ]\n",
    "        ],\n",
    "        \"type\": \"Polygon\"\n",
    "      }\n",
    "    name: star\n",
    "\" | tee /tmp/PhotovoltaicMeasurement-star.yaml >/dev/null\n",
    "\n",
    "ansible-playbook ~/smartvillage-operator/apply-photovoltaicmeasurement.yaml \\\n",
    "  -e ansible_operator_meta_namespace=$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) \\\n",
    "  -e crd_path=/tmp/PhotovoltaicMeasurement-star.yaml\n",
    "\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "698f908f",
   "metadata": {},
   "source": [
    "## Deploy new PhotovoltaicMeasurement device shaped like a building on the map\n",
    "\n",
    "Run the command below to deploy a new PhotovoltaicMeasurement smart device named `building` in the shape of a building on the map. Notice the `areaServed` and `name` attributes below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d108865d",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "echo \"\n",
    "apiVersion: smartvillage.computate.org/v1\n",
    "kind: PhotovoltaicMeasurement\n",
    "metadata:\n",
    "  name: building\n",
    "  namespace: $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)\n",
    "spec:\n",
    "  iotagent:\n",
    "    base_url: http://iotagent-json:4041\n",
    "  context_broker:\n",
    "    base_url: http://scorpiobroker:9090\n",
    "  ngsi_ld:\n",
    "    service: smarttrafficlights\n",
    "    service_path: /Sweden/Veberod/CityCenter\n",
    "    context: https://raw.githubusercontent.com/computate-org/smartabyar-smartvillage-static/main/fiware/context.jsonld\n",
    "  device:\n",
    "    id: building\n",
    "    subscription_url: http://ngsild-smartvillage-sync:8080\n",
    "  message_broker:\n",
    "    transport: AMQP\n",
    "    host: default-rabbitmq\n",
    "    port: 5672\n",
    "    user: user\n",
    "    secret:\n",
    "      name: rabbitmq-password\n",
    "      key: rabbitmq-password\n",
    "  smartvillage:\n",
    "    auth_secret_name: smartvillage\n",
    "    auth_token_url: https://sso.smartabyarsmartvillage.org/auth/realms/SMARTVILLAGE/protocol/openid-connect/token\n",
    "    site_base_url: https://$(oc get route/vscode-devel -o jsonpath={.spec.host})\n",
    "    smart_model_api: /api/PhotovoltaicMeasurement\n",
    "  attributes:\n",
    "    areaServed: {\n",
    "        \"coordinates\": [\n",
    "          [\n",
    "            [\n",
    "              13.492518677749047,\n",
    "              55.63403125573214\n",
    "            ],\n",
    "            [\n",
    "              13.492630520495311,\n",
    "              55.633954970386725\n",
    "            ],\n",
    "            [\n",
    "              13.492733043013828,\n",
    "              55.63400758098555\n",
    "            ],\n",
    "            [\n",
    "              13.492616540153023,\n",
    "              55.63407860518194\n",
    "            ],\n",
    "            [\n",
    "              13.492518677749047,\n",
    "              55.63403125573214\n",
    "            ]\n",
    "          ]\n",
    "        ],\n",
    "        \"type\": \"Polygon\"\n",
    "      }\n",
    "    name: building\n",
    "\" | tee /tmp/PhotovoltaicMeasurement-building.yaml >/dev/null\n",
    "\n",
    "ansible-playbook ~/smartvillage-operator/apply-photovoltaicmeasurement.yaml \\\n",
    "  -e ansible_operator_meta_namespace=$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) \\\n",
    "  -e crd_path=/tmp/PhotovoltaicMeasurement-building.yaml\n",
    "\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c34281d",
   "metadata": {},
   "source": [
    "## Verify the PhotovoltaicMeasurement data was received\n",
    "\n",
    "Check the logs of the IoT Agent to see that new PhotovoltaicMeasurement data was received"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54199648",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "oc -n $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) logs -l app.kubernetes.io/name=iotagent-json \\\n",
    "  | grep PhotovoltaicMeasurement | grep updated | fold -s -w150\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec0d6efc",
   "metadata": {},
   "source": [
    "See how the devices are registered by the IoT Agent in MongoDB. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64112bb5",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "oc -n $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) exec $(oc get pod -l app.kubernetes.io/component=mongodb -o name) \\\n",
    "  -- bash -c 'mongo -u root -p $MONGODB_ROOT_PASSWORD --eval \"db.devices.find()\" iotagentjson'\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4944d81d",
   "metadata": {},
   "source": [
    "See how the PhotovoltaicMeasurement entity data is persisted in PostgreSQL by the ScorpioBroker Context Broker. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a74baa3b",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "oc -n $(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace) exec $(oc get pod -l app=postgres -o name) \\\n",
    "  -- bash -c 'DB=$(psql -t -U scorpiobroker scorpiobroker -c \"select database_name from tenant where tenant_id='\"'\"'smarttrafficlights'\"'\"'\") && psql -U scorpiobroker $DB -c \"select * from entity\"'\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdd120e0",
   "metadata": {},
   "source": [
    "## View the new PhotovoltaicMeasurement in the Smart Village Platform\n",
    "\n",
    "To see the PhotovoltaicMeasurement data on a user friendly dashboard and map, run the command below to generate the link to the data in the Smart Village Platform. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0f54c5c",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "echo \"https://$(oc get route/vscode-devel -o jsonpath={.spec.host})/PhotovoltaicMeasurement\"\n",
    "echo DONE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5e798fb",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}