|
5 | 5 | TensorFlow I/O is a collection of file systems and file formats that are not |
6 | 6 | available in TensorFlow's built-in support. |
7 | 7 |
|
8 | | -## Developing |
9 | | - |
10 | | -The TensorFlow I/O package (`tensorflow-io`) could be built from source: |
11 | | -```sh |
12 | | -$ docker run -it -v ${PWD}:/working_dir -w /working_dir tensorflow/tensorflow:custom-op |
13 | | -$ # In docker |
14 | | -$ ./configure.sh |
15 | | -$ bazel build build_pip_pkg |
16 | | -$ bazel-bin/build_pip_pkg artifacts |
17 | | -``` |
18 | | - |
19 | | -A package file `artifacts/tensorflow_io-*.whl` will be generated after a build is successful. |
| 8 | +At the moment TensorFlow I/O supports 4 data sources: |
| 9 | +- `tensorflow_io.ignite`: Data source for Apache Ignite and Ignite File System (IGFS). |
| 10 | +- `tensorflow_io.kafka`: Apache Kafka stream-processing support. |
| 11 | +- `tensorflow_io.kinesis`: Amazon Kinesis data streams support. |
| 12 | +- `tensorflow_io.hadoop`: Hadoop SequenceFile format support. |
20 | 13 |
|
21 | 14 | ## Installation |
22 | 15 |
|
23 | | -Once a package file `artifacts/tensorflow_io-*.whl` is ready, installation could be done through: |
| 16 | +The `tensorflow-io` package could be installed with pip directly: |
24 | 17 | ``` |
25 | | -$ pip install working_dir/artifacts/tensorflow_io-*.whl |
| 18 | +$ pip install tensorflow-io |
26 | 19 | ``` |
27 | 20 |
|
28 | | -The related module could be imported with python: |
| 21 | +The related module such as Kafka could be imported with python: |
29 | 22 | ``` |
30 | 23 | $ python |
31 | 24 | Python 2.7.6 (default, Nov 13 2018, 12:45:42) |
32 | 25 | [GCC 4.8.4] on linux2 |
33 | 26 | Type "help", "copyright", "credits" or "license" for more information. |
34 | | ->>> from tensorflow_io.kafka import KafkaDataset |
| 27 | +>>> import tensorflow as tf |
| 28 | +>>> import tensorflow_io.kafka as kafka |
| 29 | +>>> |
| 30 | +>>> dataset = kafka.KafkaDataset(["test:0:0:4"], group="test", eof=True) |
| 31 | +>>> iterator = dataset.make_initializable_iterator() |
| 32 | +>>> init_op = iterator.initializer |
| 33 | +>>> get_next = iterator.get_next() |
| 34 | +>>> |
| 35 | +>>> with tf.Session() as sess: |
| 36 | +... print(sess.run(init_op)) |
| 37 | +... for i in range(5): |
| 38 | +... print(sess.run(get_next)) |
35 | 39 | >>> |
36 | 40 | ``` |
37 | 41 |
|
38 | 42 | Note that python has to run outside of repo directory itself, otherwise python may not |
39 | 43 | be able to find the correct path to the module. |
40 | 44 |
|
| 45 | +## Developing |
| 46 | + |
| 47 | +The TensorFlow I/O package (`tensorflow-io`) could be built from source: |
| 48 | +```sh |
| 49 | +$ docker run -it -v ${PWD}:/working_dir -w /working_dir tensorflow/tensorflow:custom-op |
| 50 | +$ # In docker |
| 51 | +$ ./configure.sh |
| 52 | +$ bazel build build_pip_pkg |
| 53 | +$ bazel-bin/build_pip_pkg artifacts |
| 54 | +``` |
| 55 | + |
| 56 | +A package file `artifacts/tensorflow_io-*.whl` will be generated after a build is successful. |
| 57 | + |
41 | 58 | ## License |
42 | 59 |
|
43 | 60 | [Apache License 2.0](LICENSE) |
0 commit comments