Skip to content

Commit 7b3be01

Browse files
authored
Merge branch 'master' into transformer
2 parents e0e81f0 + ca882d6 commit 7b3be01

File tree

10 files changed

+352
-157
lines changed

10 files changed

+352
-157
lines changed

.travis.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ env:
2626
# Backward Compatibility in insured for release less than 1 year old.
2727
# https://pypi.org/project/tensorflow/#history
2828
matrix:
29-
- _TF_VERSION=2.0.0b1
29+
- _TF_VERSION=2.0.0-rc1
3030
# - _TF_VERSION=1.12.0 # Remove on Oct 22, 2019
3131
# - _TF_VERSION=1.11.0 # Remove on Sep 28, 2019
3232
# - _TF_VERSION=1.10.1 # Remove on Aug 24, 2019
@@ -63,7 +63,7 @@ matrix:
6363
install:
6464
- |
6565
if [[ -v _DOC_AND_YAPF_TEST ]]; then
66-
pip install tensorflow==2.0.0b1
66+
pip install tensorflow==2.0.0-rc1
6767
pip install yapf
6868
pip install -e .[doc]
6969
else
@@ -101,7 +101,7 @@ deploy:
101101
on:
102102
tags: true
103103
python: '3.6'
104-
condition: '$_TF_VERSION = 2.0.0b1'
104+
condition: '$_TF_VERSION = 2.0.0-rc1'
105105
# condition: '$_TF_VERSION = 1.11.0'
106106

107107
# Documentation: https://docs.travis-ci.com/user/deployment/releases/
@@ -115,5 +115,5 @@ deploy:
115115
on:
116116
tags: true
117117
python: '3.6'
118-
condition: '$_TF_VERSION = 2.0.0b1'
118+
condition: '$_TF_VERSION = 2.0.0-rc1'
119119
# condition: '$_TF_VERSION = 1.11.0'

CHANGELOG.md

+14-7
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,6 @@ To release a new version, please update the changelog as followed:
7979
### Deprecated
8080

8181
### Fixed
82-
- RNN updates: remove warnings, fix if seq_len=0, unitest (#PR 1033)
8382

8483
### Removed
8584

@@ -88,21 +87,30 @@ To release a new version, please update the changelog as followed:
8887
### Contributors
8988

9089

91-
## [2.2.1]
90+
## [2.2.0] - 2019-09-13
91+
92+
TensorLayer 2.2.0 is a maintenance release.
93+
It contains numerous API improvement and bug fixes.
94+
This release is compatible with TensorFlow 2 RC1.
9295

9396
### Added
9497
- Support nested layer customization (#PR 1015)
9598
- Support string dtype in InputLayer (#PR 1017)
9699
- Support Dynamic RNN in RNN (#PR 1023)
97100
- Add ResNet50 static model (#PR 1030)
98101
- Add Transformer model (#PR 1027)
102+
- Add performance test code in static model (#PR 1041)
99103

100104
### Changed
101105

102106
- `SpatialTransform2dAffine` auto `in_channels`
103-
- support TensorFlow 2.0.0-beta1
107+
- support TensorFlow 2.0.0-rc1
104108
- Update model weights property, now returns its copy (#PR 1010)
105109

110+
### Fixed
111+
- RNN updates: remove warnings, fix if seq_len=0, unitest (#PR 1033)
112+
- BN updates: fix BatchNorm1d for 2D data, refactored (#PR 1040)
113+
106114
### Dependencies Update
107115

108116
### Deprecated
@@ -116,6 +124,7 @@ To release a new version, please update the changelog as followed:
116124
- Copy original model's `trainable_weights` and `nontrainable_weights` when initializing `LayerList` (#PR 1029)
117125
- Remove redundant parts in `model.all_layers` (#PR 1029)
118126
- Replace `tf.image.resize_image_with_crop_or_pad` with `tf.image.resize_with_crop_or_pad` (#PR 1032)
127+
- Fix a bug in `ResNet50` static model (#PR 1041)
119128

120129
### Removed
121130

@@ -129,6 +138,7 @@ To release a new version, please update the changelog as followed:
129138
- @ArnoldLIULJ: #1023 #1027
130139
- @JingqingZ: #1023 #1027
131140

141+
132142
## [2.1.0]
133143

134144
### Changed
@@ -199,15 +209,12 @@ A maintain release.
199209
- @warshallrho: #PR966
200210
- @zsdonghao: #931
201211
- @yd-yin: #963
202-
<<<<<<< HEAD
203212
- @Tokarev-TT-33: # 995
204213
- @initial-h: # 995
205214
- @quantumiracle: #995
206215
- @Officium: #995
207-
=======
208216
- @1FengL: #958
209217
- @dvklopfenstein: #971
210-
>>>>>>> 560dbb8a17963023a3b1d59a79e1c2752530114a
211218

212219

213220
## [2.0.0] - 2019-05-04
@@ -560,7 +567,7 @@ To many PR for this update, please check [here](https://github.com/tensorlayer/t
560567
@zsdonghao @luomai @DEKHTIARJonathan
561568

562569
[Unreleased]: https://github.com/tensorlayer/tensorlayer/compare/2.0....master
563-
[2.1.1]: https://github.com/tensorlayer/tensorlayer/compare/2.1.1...2.1.1
570+
[2.2.0]: https://github.com/tensorlayer/tensorlayer/compare/2.2.0...2.2.0
564571
[2.1.0]: https://github.com/tensorlayer/tensorlayer/compare/2.1.0...2.1.0
565572
[2.0.2]: https://github.com/tensorlayer/tensorlayer/compare/2.0.2...2.0.2
566573
[2.0.1]: https://github.com/tensorlayer/tensorlayer/compare/2.0.1...2.0.1

README.md

+45-66
Original file line numberDiff line numberDiff line change
@@ -34,45 +34,43 @@
3434

3535
<br/>
3636

37-
TensorLayer is a novel TensorFlow-based deep learning and reinforcement learning library designed for researchers and engineers. It provides a large collection of customizable neural layers / functions that are key to build real-world AI applications. TensorLayer is awarded the 2017 Best Open Source Software by the [ACM Multimedia Society](https://twitter.com/ImperialDSI/status/923928895325442049).
37+
TensorLayer is a novel TensorFlow-based deep learning and reinforcement learning library designed for researchers and engineers. It provides an extensive collection of customizable neural layers to build complex AI models. TensorLayer is awarded the 2017 Best Open Source Software by the [ACM Multimedia Society](https://twitter.com/ImperialDSI/status/923928895325442049).
38+
TensorLayer can also be found at [iHub](https://code.ihub.org.cn/projects/328) and [Gitee](https://gitee.com/organizations/TensorLayer).
39+
40+
# News
3841

3942
🔥📰🔥 Reinforcement Learning Model Zoos: [Low-level APIs for Research](https://github.com/tensorlayer/tensorlayer/tree/master/examples/reinforcement_learning) and [High-level APIs for Production](https://github.com/tensorlayer/RLzoo)
4043

4144
🔥📰🔥 [Sipeed Maxi-EMC](https://github.com/sipeed/Maix-EMC): Run TensorLayer models on the **low-cost AI chip** (e.g., K210) (Alpha Version)
4245

4346
🔥📰🔥 [NNoM](https://github.com/majianjia/nnom): Run TensorLayer quantized models on the **MCU** (e.g., STM32) (Coming Soon)
4447

45-
4648
# Features
4749

48-
As deep learning practitioners, we have been looking for a library that can address various development
49-
purposes. This library is easy to adopt by providing diverse examples, tutorials and pre-trained models.
50-
Also, it allow users to easily fine-tune TensorFlow; while being suitable for production deployment. TensorLayer aims to satisfy all these purposes. It has three key features:
50+
TensorLayer is a new deep learning library designed with simplicity, flexibility and high-performance in mind.
5151

52-
- ***Simplicity*** : TensorLayer lifts the low-level dataflow interface of TensorFlow to *high-level* layers / models. It is very easy to learn through the rich [example codes](https://github.com/tensorlayer/awesome-tensorlayer) contributed by a wide community.
53-
- ***Flexibility*** : TensorLayer APIs are transparent: it does not mask TensorFlow from users; but leaving massive hooks that help *low-level tuning* and *deep customization*.
54-
- ***Zero-cost Abstraction*** : TensorLayer can achieve the *full power* of TensorFlow. The following table shows the training speeds of [VGG16](http://www.robots.ox.ac.uk/~vgg/research/very_deep/) using TensorLayer and native TensorFlow on a TITAN Xp.
52+
- ***Simplicity*** : TensorLayer has a high-level layer/model abstraction which is effortless to learn. You can learn how deep learning can benefit your AI tasks in minutes through the massive [examples](https://github.com/tensorlayer/awesome-tensorlayer).
53+
- ***Flexibility*** : TensorLayer APIs are transparent and flexible, inspired by the emerging PyTorch library. Compared to the Keras abstraction, TensorLayer makes it much easier to build and train complex AI models.
54+
- ***Zero-cost Abstraction*** : Though simple to use, TensorLayer does not require you to make any compromise in the performance of TensorFlow (Check the following benchmark section for more details).
5555

56-
| Mode | Lib | Data Format | Max GPU Memory Usage(MB) |Max CPU Memory Usage(MB) | Avg CPU Memory Usage(MB) | Runtime (sec) |
57-
| :-------: | :-------------: | :-----------: | :-----------------: | :-----------------: | :-----------------: | :-----------: |
58-
| AutoGraph | TensorFlow 2.0 | channel last | 11833 | 2161 | 2136 | 74 |
59-
| | Tensorlayer 2.0 | channel last | 11833 | 2187 | 2169 | 76 |
60-
| Graph | Keras | channel last | 8677 | 2580 | 2576 | 101 |
61-
| Eager | TensorFlow 2.0 | channel last | 8723 | 2052 | 2024 | 97 |
62-
| | TensorLayer 2.0 | channel last | 8723 | 2010 | 2007 | 95 |
56+
TensorLayer is NOT yet another library in the TensorFlow world. Other wrappers like Keras and TFLearn
57+
hide many powerful features of TensorFlow and provide little support for writing custom, complex AI models. Inspired by PyTorch, TensorLayer APIs are simple, flexible and most importantly, pythonic.
58+
TensorLayer has a fast-growing community. It has been used by researchers and engineers all over the world, including those from Peking University,
59+
Imperial College London, UC Berkeley, Carnegie Mellon University, Stanford University, and companies like Google, Microsoft, Alibaba, Tencent, Xiaomi, and Bloomberg.
6360

61+
# Multilingual documents
6462

65-
TensorLayer stands at a unique spot in the library landscape. Other wrapper libraries like Keras and TFLearn also provide high-level abstractions. They, however, often
66-
hide the underlying engine from users, which make them hard to customize
67-
and fine-tune. On the contrary, TensorLayer APIs are generally lightweight, flexible and transparent.
68-
Users often find it easy to start with the examples and tutorials, and then dive
69-
into TensorFlow seamlessly. In addition, TensorLayer does not create library lock-in through native supports for importing components from Keras.
63+
TensorLayer has extensive documentation for both beginners and professionals. The documentation is available in
64+
both English and Chinese.
7065

71-
TensorLayer has a fast growing usage among top researchers and engineers, from universities like Peking University,
72-
Imperial College London, UC Berkeley, Carnegie Mellon University, Stanford University, and
73-
University of Technology of Compiegne (UTC), and companies like Google, Microsoft, Alibaba, Tencent, Xiaomi, and Bloomberg.
66+
[![English Documentation](https://img.shields.io/badge/documentation-english-blue.svg)](https://tensorlayer.readthedocs.io/)
67+
[![Chinese Documentation](https://img.shields.io/badge/documentation-%E4%B8%AD%E6%96%87-blue.svg)](https://tensorlayercn.readthedocs.io/)
68+
[![Chinese Book](https://img.shields.io/badge/book-%E4%B8%AD%E6%96%87-blue.svg)](http://www.broadview.com.cn/book/5059/)
69+
70+
If you want to try the experimental features on the the master branch, you can find the latest document
71+
[here](https://tensorlayer.readthedocs.io/en/latest/).
7472

75-
# Tutorials and Real-World Applications
73+
# Extensive examples
7674

7775
You can find a large collection of tutorials, examples and real-world applications using TensorLayer within [examples](examples/) or through the following space:
7876

@@ -82,73 +80,42 @@ You can find a large collection of tutorials, examples and real-world applicatio
8280
</div>
8381
</a>
8482

85-
# Documentation
86-
87-
TensorLayer has extensive documentation for both beginners and professionals. The documentation is available in
88-
both English and Chinese. Please click the following icons to find the documents you need:
89-
90-
[![English Documentation](https://img.shields.io/badge/documentation-english-blue.svg)](https://tensorlayer.readthedocs.io/)
91-
[![Chinese Documentation](https://img.shields.io/badge/documentation-%E4%B8%AD%E6%96%87-blue.svg)](https://tensorlayercn.readthedocs.io/)
92-
[![Chinese Book](https://img.shields.io/badge/book-%E4%B8%AD%E6%96%87-blue.svg)](http://www.broadview.com.cn/book/5059/)
93-
94-
If you want to try the experimental features on the the master branch, you can find the latest document
95-
[here](https://tensorlayer.readthedocs.io/en/latest/).
96-
97-
# Install
83+
# Installing TensorLayer is easy
9884

99-
For latest code for TensorLayer 2.0, please build from the source. TensorLayer 2.0 has pre-requisites including TensorFlow 2, numpy, and others. For GPU support, CUDA and cuDNN are required.
85+
TensorLayer 2.0 relies on TensorFlow, numpy, and others. To use GPUs, CUDA and cuDNN are required.
10086

10187
Install TensorFlow:
10288

10389
```bash
104-
pip3 install tensorflow-gpu==2.0.0-beta1 # specific version (YOU SHOULD INSTALL THIS ONE NOW)
105-
pip3 install tensorflow-gpu # GPU version
90+
pip3 install tensorflow-gpu==2.0.0-rc1 # TensorFlow GPU (version 2.0 RC1)
10691
pip3 install tensorflow # CPU version
10792
```
10893

109-
Install the stable version of TensorLayer:
94+
Install the stable release of TensorLayer:
11095

11196
```bash
11297
pip3 install tensorlayer
11398
```
11499

115-
Install the latest version of TensorLayer:
100+
Install the unstable development version of TensorLayer:
116101

117102
```bash
118103
pip3 install git+https://github.com/tensorlayer/tensorlayer.git
119-
or
120-
pip3 install https://github.com/tensorlayer/tensorlayer/archive/master.zip
121104
```
122105

123-
For developers, you should clone the folder to your local machine and put it along with your project scripts.
124-
106+
If you want to install the additional dependencies, you can also run
125107
```bash
126-
git clone https://github.com/tensorlayer/tensorlayer.git
127-
```
128-
129-
If you want install TensorLayer 1.X, the simplest way to install TensorLayer 1.X is to use the **Py**thon **P**ackage **I**ndex (PyPI):
130-
131-
```bash
132-
# for last stable version of TensorLayer 1.X
133-
pip3 install --upgrade tensorlayer==1.X
134-
135-
# for latest release candidate of TensorLayer 1.X
136-
pip3 install --upgrade --pre tensorlayer
137-
138-
# if you want to install the additional dependencies, you can also run
139108
pip3 install --upgrade tensorlayer[all] # all additional dependencies
140109
pip3 install --upgrade tensorlayer[extra] # only the `extra` dependencies
141110
pip3 install --upgrade tensorlayer[contrib_loggers] # only the `contrib_loggers` dependencies
142111
```
143-
<!---
144-
Alternatively, you can install the latest or development version by directly pulling from github:
112+
113+
If you are TensorFlow 1.X users, you can use TensorLayer 1.X:
145114

146115
```bash
147-
pip3 install https://github.com/tensorlayer/tensorlayer/archive/master.zip
148-
# or
149-
# pip3 install https://github.com/tensorlayer/tensorlayer/archive/<branch-name>.zip
116+
# For last stable version of TensorLayer 1.X
117+
pip3 install --upgrade tensorlayer==1.X
150118
```
151-
--->
152119

153120
<!---
154121
## Using Docker
@@ -182,6 +149,18 @@ nvidia-docker run -it --rm -p 8888:8888 -p 6006:6006 -e PASSWORD=JUPYTER_NB_PASS
182149
```
183150
--->
184151

152+
# Benchmark
153+
154+
The following table shows the training speeds of [VGG16](http://www.robots.ox.ac.uk/~vgg/research/very_deep/) using TensorLayer and native TensorFlow on a TITAN Xp.
155+
156+
| Mode | Lib | Data Format | Max GPU Memory Usage(MB) |Max CPU Memory Usage(MB) | Avg CPU Memory Usage(MB) | Runtime (sec) |
157+
| :-------: | :-------------: | :-----------: | :-----------------: | :-----------------: | :-----------------: | :-----------: |
158+
| AutoGraph | TensorFlow 2.0 | channel last | 11833 | 2161 | 2136 | 74 |
159+
| | Tensorlayer 2.0 | channel last | 11833 | 2187 | 2169 | 76 |
160+
| Graph | Keras | channel last | 8677 | 2580 | 2576 | 101 |
161+
| Eager | TensorFlow 2.0 | channel last | 8723 | 2052 | 2024 | 97 |
162+
| | TensorLayer 2.0 | channel last | 8723 | 2010 | 2007 | 95 |
163+
185164
# Contribute
186165

187166
Please read the [Contributor Guideline](CONTRIBUTING.md) before submitting your PRs.
@@ -201,4 +180,4 @@ If you use TensorLayer for any projects, please cite this paper:
201180

202181
# License
203182

204-
TensorLayer is released under the Apache 2.0 license. We also host TensorLayer on [iHub](https://code.ihub.org.cn/projects/328) and [Gitee](https://gitee.com/organizations/TensorLayer).
183+
TensorLayer is released under the Apache 2.0 license.

tensorlayer/files/utils.py

+4
Original file line numberDiff line numberDiff line change
@@ -2666,6 +2666,10 @@ def _load_weights_from_hdf5_group(f, layers, skip=False):
26662666
elif isinstance(layer, tl.layers.Layer):
26672667
weight_names = [n.decode('utf8') for n in g.attrs['weight_names']]
26682668
for iid, w_name in enumerate(weight_names):
2669+
# FIXME : this is only for compatibility
2670+
if isinstance(layer, tl.layers.BatchNorm) and np.asarray(g[w_name]).ndim > 1:
2671+
assign_tf_variable(layer.all_weights[iid], np.asarray(g[w_name]).squeeze())
2672+
continue
26692673
assign_tf_variable(layer.all_weights[iid], np.asarray(g[w_name]))
26702674
else:
26712675
raise Exception("Only layer or model can be saved into hdf5.")

0 commit comments

Comments
 (0)