|
1 |
| -**Kaggle Kernels** allow users to run scripts against our competitions and datasets without having to download data or set up their environment. Here's [an example](https://www.kaggle.com/devinanzelmo/d/devinanzelmo/dota-2-matches/setting-up-a-prediction-problem-dota-2): |
| 1 | +[Kaggle Kernels](https://www.kaggle.com/kernels) allow users to run a Python Notebook in the cloud against our competitions and datasets without having to download data or set up their environment. |
2 | 2 |
|
3 |
| - |
| 3 | +This repository includes our Dockerfiles for building the [CPU-only](Dockerfile) and [GPU](gpu.Dockerfile) image that runs Python Kernels on Kaggle. |
4 | 4 |
|
5 |
| -This is the Dockerfile (etc.) used for building the image that runs python scripts on Kaggle. [Here's](https://gcr.io/kaggle-images/python) the Docker image on Google Container Registry. |
| 5 | +Our Python Docker images are stored on Google Container Registry at: |
| 6 | + |
| 7 | +* CPU-only: [gcr.io/kaggle-images/python](https://gcr.io/kaggle-images/python) |
| 8 | +* GPU: private for now, we will make it public soon. |
| 9 | + |
| 10 | +Note: The base image for the GPU image is our CPU-only image. The [gpu.Dockerfile](gpu.Dockerfile) adds a few extra layers to install GPU related libraries and packages (cuda, libcudnn, pycuda etc.) and reinstall packages with specific GPU builds (torch, tensorflow and a few mores). |
6 | 11 |
|
7 | 12 | ## Getting started
|
8 | 13 |
|
9 | 14 | To get started with this image, read our [guide](http://blog.kaggle.com/2016/02/05/how-to-get-started-with-data-science-in-containers/) to using it yourself, or browse [Kaggle Kernels](https://www.kaggle.com/kernels) for ideas.
|
10 | 15 |
|
11 |
| -## Requesting new features |
| 16 | +## Requesting new packages |
12 | 17 |
|
13 |
| -**We welcome pull requests** if there are any packages you'd like to add! |
| 18 | +First, evaluate whether installing the package yourself in your own Kernels suits your needs. See [guide](wiki/Missing-Packages). |
14 | 19 |
|
15 |
| -We can merge your request quickly if you check that it builds correctly. Here's how to do that. |
| 20 | +If you the first step above doesn't work for your use case, [open an issue](https://github.com/Kaggle/docker-python/issues/new) or a [pull request](https://github.com/Kaggle/docker-python/pulls). |
16 | 21 |
|
17 |
| -Start by running this image on your system: |
| 22 | +## Opening a pull request |
18 | 23 |
|
19 |
| -``` |
20 |
| -me@my-computer:/home$ docker run --rm -it gcr.io/kaggle-images/python |
21 |
| -root@d72b81a003e1:/# |
22 |
| -``` |
| 24 | +1. Update the *Dockerfile* |
| 25 | + 1. For changes specific to the GPU image, update the [gpu.Dockerfile](gpu.Dockerfile). |
| 26 | + 2. Otherwise, update the [Dockerfile](Dockerfile). |
| 27 | +2. Follow the instructions below to build a new image. |
| 28 | +3. Add tests for your new package. See this [example](https://github.com/Kaggle/docker-python/blob/master/tests/test_fastai.py). |
| 29 | +4. Follow the instructions below to test the new image. |
| 30 | +5. Open a PR on this repo and you are all set! |
23 | 31 |
|
24 |
| -Then follow the package's installation instructions for a Linux system. It could be as simple as installing via Pip: |
| 32 | +## Building a new image |
25 | 33 |
|
26 |
| -``` |
27 |
| -root@d72b81a003e1:/# pip install coolpackage |
28 |
| -Collecting coolpackage |
29 |
| -[...etc...] |
| 34 | +```sh |
| 35 | +./build |
30 | 36 | ```
|
31 | 37 |
|
32 |
| -Once that's done, check that you can import it correctly. (Sometimes, if a package is missing a dependency, it throws an error when you try to import it.) |
| 38 | +Flags: |
33 | 39 |
|
34 |
| -``` |
35 |
| -root@d72b81a003e1:/# python |
36 |
| -Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:53:06) |
37 |
| -[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux |
38 |
| -Type "help", "copyright", "credits" or "license" for more information. |
39 |
| ->>> import coolpackage |
40 |
| ->>> |
41 |
| -``` |
| 40 | +* `--gpu` to build an image for GPU. |
| 41 | +* `--use-cache` for faster iterative builds. |
42 | 42 |
|
43 |
| -Once that's working, add the necessary lines to our [Dockerfile](https://github.com/Kaggle/docker-python/blob/master/Dockerfile). (In this case, that would mean adding `pip install coolpackage` to the last section.) |
| 43 | +## Testing a new image |
44 | 44 |
|
45 |
| -Next run the build: |
| 45 | +A suite of tests can be found under the `/tests` folder. You can run the test using this command: |
46 | 46 |
|
47 |
| -``` |
48 |
| -./build |
| 47 | +```sh |
| 48 | +./test |
49 | 49 | ```
|
50 | 50 |
|
51 |
| -Finally run the tests: |
| 51 | +Flags: |
52 | 52 |
|
53 |
| -``` |
54 |
| -./test |
55 |
| -``` |
| 53 | +* `--gpu` to test the GPU image. |
56 | 54 |
|
57 |
| -Then submit your pull request, and you're all set! |
| 55 | +## Tensorflow custom pre-built wheels |
58 | 56 |
|
| 57 | +We are building Tensorflow from sources mainly for: |
59 | 58 |
|
| 59 | +* Better performance. When building from sources, we can leverage CPU specific optimizations |
| 60 | +* Tensorflow with GPU support must be built from sources |
60 | 61 |
|
| 62 | +The [Dockerfile](tensorflow-whl/Dockerfile) and the [instructions](tensorflow-whl/README.md) can be found in the [tensorflow-whl folder/](tensorflow/). |
0 commit comments