Spark + Python + Jupyter for web-based modelling facility

[mgalas]

We want to allow users to run Python models on the hosted Hadoop infrastructure and data. Consequently we need to investigate validity of integration of Spark + Python and Jupyter to enable multiple users to deploy python scripts on the infrastructure.

A starting point may be to investigate the following: http://blog.insightdatalabs.com/jupyter-on-apache-spark-step-by-step/

[ulince]

I have found the following official project that seems to have multi-user support for Jupyter: https://github.com/jupyterhub/jupyterhub

[mgalas]

It looks like a viable option but I need some info on the way it integrates with Spark. Let's discuss on today's meeting

[ulince]

Example of setting up JupyterHub in Cloudera: jupyterhub/jupyterhub#564