JSONSchemaNotes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# JSON Schema Parser\n",
    "\n",
    "Notes on the JSON schema / Traitlets package\n",
    "\n",
    "**Goal**: write a function that, given a JSON Schema, will generate code for traitlets objects which provide equivalent validation.\n",
    "\n",
    "## Links\n",
    "\n",
    "- [JSON Schema Validation Information](http://json-schema.org/latest/json-schema-validation.html)\n",
    "- [``jsonschema`` Python package](https://pypi.python.org/pypi/jsonschema)\n",
    "- [Altair 1.0 parsing code](https://github.com/altair-viz/altair/blob/master/tools/generate_schema_interface.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## By-Hand Example\n",
    "\n",
    "First confirm that we're doing things correctly with the ``jsonschema`` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import jsonschema\n",
    "\n",
    "simple_schema = {\n",
    "    \"type\": \"object\",\n",
    "    \"properties\": {\n",
    "        \"foo\": {\"type\": \"string\"},\n",
    "        \"bar\": {\"type\": \"number\"}\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "good_instance = {\n",
    "    \"foo\": \"hello world\",\n",
    "    \"bar\": 3.141592653,\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "bad_instance = {\n",
    "    \"foo\" : 42,\n",
    "    \"bar\" : \"string\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Should succeed\n",
    "jsonschema.validate(good_instance, simple_schema)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42 is not of type 'string'\n",
      "\n",
      "Failed validating 'type' in schema['properties']['foo']:\n",
      "    {'type': 'string'}\n",
      "\n",
      "On instance['foo']:\n",
      "    42\n"
     ]
    }
   ],
   "source": [
    "# Should fail\n",
    "try:\n",
    "    jsonschema.validate(bad_instance, simple_schema)\n",
    "except jsonschema.ValidationError as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "OK, now let's write a traitlets class that does the same thing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import traitlets as T\n",
    "\n",
    "class SimpleInstance(T.HasTraits):\n",
    "    foo = T.Unicode()\n",
    "    bar = T.Float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.SimpleInstance at 0x10b06c828>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Should succeed\n",
    "\n",
    "SimpleInstance(**good_instance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The 'foo' trait of a SimpleInstance instance must be a unicode string, but a value of 42 <class 'int'> was specified.\n"
     ]
    }
   ],
   "source": [
    "# Should fail\n",
    "\n",
    "try:\n",
    "    SimpleInstance(**bad_instance)\n",
    "except T.TraitError as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Roadmap\n",
    "\n",
    "1. Start by recognizing all simple JSON types in the schema (\"string\", \"number\", \"integer\", \"boolean\", \"null\")\n",
    "\n",
    "2. Next recognize objects containing simple types\n",
    "\n",
    "2. Next recognize compound simple types (i.e. where type is a list of simple types)\n",
    "\n",
    "3. Next recognize arrays & enums\n",
    "\n",
    "4. Next recognize \"$ref\" definitions\n",
    "\n",
    "5. Next recognize \"anyOf\", \"oneOf\", \"allOf\" definitions... first is essentially a traitlets Union, second is a Union where only one must match, and \"allOf\" is essentially a composite object (not sure if traitlets has that...) Note that among these, Vega-Lite only uses \"anyOf\"\n",
    "\n",
    "6. Catalog all validation keywords... Implement custom traitlets that support all the various validation keywords for each type. (Validation keywords listed [here](http://json-schema.org/latest/json-schema-validation.html#rfc.section.6))\n",
    "\n",
    "7. Use [hypothesis](https://hypothesis.readthedocs.io) for testing?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Challenges & Questions to think about\n",
    "\n",
    "- JSONSchema ignores any keys/properties which are not listed in the schema. Traitlets warns, and in the future will raise an error for undefined keys/properties\n",
    "\n",
    "  - *this may be OK... we can just document the fact that traitlets is more prescriptive than JSONschema*\n",
    "\n",
    "- JSON allows undefined values as well as explicit nulls, which map to None. Traitlets treats ``None`` as undefined. How to resolve this?\n",
    "\n",
    "  - *Best option is probably to use an ``undefined`` sentinel within the traitlets structure, such that the code knows when to ignore keys & produces dicts which translate directly to the correct JSON*\n",
    "\n",
    "- Will probably need to define some custom trait types, e.g. ``Null``, and also extend simple trait types to allow for the more extensive validations allowed in JSON Schema.\n",
    "\n",
    "  - *Generate subclasses with the code*\n",
    "\n",
    "- What version of the schema should we target? Perhaps try to target multiple versions?\n",
    "\n",
    "  - *start with 04 because this is what's supported by jsonschema and used by Vega(-Lite)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Ideas\n",
    "\n",
    "Easiest way: validate everything with a single ``HasTraits`` class via ``jsonschema``, splitting out properties into traitlets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Interface\n",
    "\n",
    "- root schema and all definitions should become their own ``T.HasTraits`` class\n",
    "- Objects defined inline should also have their own class with a generated anonymous name\n",
    "- Use Jinja templating; allow output to one file or multiple files with relative imports\n",
    "- root object *must* have type=\"object\"... this differs from jsonschema"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Testing\n",
    "\n",
    "- test cases should be an increasingly complicated set of jsonschema objects, with test cases that should pass and fail. Perhaps store these in a JSON structure? (With a schema?)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## An initial prototype\n",
    "\n",
    "Let's try generating some traitlets classes for simple cases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import jinja2\n",
    "\n",
    "\n",
    "OBJECT_TEMPLATE = \"\"\"\n",
    "{%- for import in cls.imports %}\n",
    "{{ import }}\n",
    "{%- endfor %}\n",
    "\n",
    "class {{ cls.classname }}({{ cls.baseclass }}):\n",
    "    {%- for (name, prop) in cls.properties.items() %}\n",
    "    {{ name }} = {{ prop.trait_code }}\n",
    "    {%- endfor %}\n",
    "\"\"\"\n",
    "\n",
    "class JSONSchema(object):\n",
    "    \"\"\"A class to wrap JSON Schema objects and reason about their contents\"\"\"\n",
    "    object_template = OBJECT_TEMPLATE\n",
    "    \n",
    "    def __init__(self, schema, root=None):\n",
    "        self.schema = schema\n",
    "        self.root = root or schema\n",
    "        \n",
    "    @property\n",
    "    def type(self):\n",
    "        # TODO: should the default type be considered object?\n",
    "        return self.schema.get('type', 'object')\n",
    "    \n",
    "    @property\n",
    "    def trait_code(self):\n",
    "        type_dict = {'string': 'T.Unicode()',\n",
    "                     'number': 'T.Float()',\n",
    "                     'integer': 'T.Integer()',\n",
    "                     'boolean': 'T.Bool()'}\n",
    "        if self.type not in type_dict:\n",
    "            raise NotImplementedError()\n",
    "        return type_dict[self.type]\n",
    "    \n",
    "    @property\n",
    "    def classname(self):\n",
    "        # TODO: deal with non-root schemas somehow...\n",
    "        if self.schema is self.root:\n",
    "            return \"RootInstance\"\n",
    "        else:\n",
    "            raise NotImplementedError(\"Non-root object schema\")\n",
    "            \n",
    "    @property\n",
    "    def baseclass(self):\n",
    "        return \"T.HasTraits\"\n",
    "    \n",
    "    @property\n",
    "    def imports(self):\n",
    "        return [\"import traitlets as T\"]\n",
    "    \n",
    "    @property\n",
    "    def properties(self):\n",
    "        return {key: JSONSchema(val) for key, val in self.schema.get('properties', {}).items()}\n",
    "    \n",
    "    def object_code(self):\n",
    "        return jinja2.Template(self.object_template).render(cls=self)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Trying it out..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "import traitlets as T\n",
      "\n",
      "class RootInstance(T.HasTraits):\n",
      "    foo = T.Unicode()\n",
      "    bar = T.Float()\n"
     ]
    }
   ],
   "source": [
    "code = JSONSchema(simple_schema).object_code()\n",
    "print(code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Testing the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "exec(code)  # defines RootInstance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.RootInstance at 0x10b7fefd0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Good instance should validate correctly\n",
    "RootInstance(**good_instance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The 'foo' trait of a RootInstance instance must be a unicode string, but a value of 42 <class 'int'> was specified.\n"
     ]
    }
   ],
   "source": [
    "# Bad instance should raise a TraitError\n",
    "try:\n",
    "    RootInstance(**bad_instance)\n",
    "except T.TraitError as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Seems to work 😀\n",
    "\n",
    "We'll start with something like this in the package, and then build from there."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "python3.6"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}