Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Commit d9bb9f1

Browse files
BradLarsonBradLarson
authored
Documenting models and components (#730)
* Expanding the documentation for swift-models. * Minor wording edits, completing the image processing section.
1 parent 243bfb4 commit d9bb9f1

File tree

1 file changed

+241
-22
lines changed

1 file changed

+241
-22
lines changed

‎README.md

+241-22
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,263 @@
11
# Swift for TensorFlow Models
22

3-
This repository contains TensorFlow models written in Swift.
3+
This repository contains many examples of how Swift for TensorFlow can be used to build machine
4+
learning applications, as well as the models, datasets, and other components required to build them.
5+
These examples are intended to demonstrate best practices for the use of
6+
[Swift for TensorFlow APIs](https://github.com/tensorflow/swift-apis) and act as end-to-end tests
7+
to validate the function and performance of those APIs.
48

5-
Active development occurs on the `main` branch.
9+
Active development occurs on the `main` branch, and that is kept current against the `main` branch
10+
of [the Swift compiler](https://github.com/apple/swift) and the `main` branch of [the Swift for TensorFlow APIs](https://github.com/tensorflow/swift-apis).
611

7-
Use the ```tensorflow-xx``` branch that corresponds to the release you are using from [Swift for TensorFlow releases](https://github.com/tensorflow/swift/blob/master/Installation.md#releases). For example, for the 0.6 release, use the ```tensorflow-0.6``` branch.
12+
For stable snapshots, use the ```tensorflow-xx``` branch that corresponds to the toolchain you are using from the [Swift for TensorFlow releases](https://github.com/tensorflow/swift/blob/master/Installation.md#releases). For example, for the 0.12 release, use the ```tensorflow-0.12``` branch.
813

9-
For general information about Swift for TensorFlow development, please visit
14+
To learn more about Swift for TensorFlow development, please visit
1015
[tensorflow/swift](https://github.com/tensorflow/swift).
1116

12-
## Development
17+
## Examples
1318

14-
### macOS and Linux
19+
The examples within this repository are all designed to be run as standalone applications. The easiest way to do this is to use Swift Package Manager to build and run individual examples. This
20+
can be accomplished by changing to the root directory of the project and typing something like
1521

16-
Use Swift Package Manager to develop Swift for TensorFlow models.
22+
```bash
23+
swift run -c release [Example] [Options]
24+
```
1725

18-
#### Build
26+
For Windows, an additional flag may be required:
1927

20-
```bash
21-
swift build
28+
```cmd
29+
swift run -Xswiftc -use-ld=lld -c release [Example] [Options]
2230
```
2331

24-
#### Test
32+
This will build and run a specific example in the release configuration. Due to significant
33+
performance differences between debug and release builds in Swift, we highly recommend running the
34+
examples from a release build. Some examples have additional command-line options, and those will
35+
be described in the example's README.
2536

26-
```bash
27-
swift test
37+
The following is a catalog of the current examples, grouped by subject area, with links to their
38+
location within the project. Each example should have documentation for what it is demonstrating
39+
and how to use it.
40+
41+
### Image classification
42+
43+
- [A custom model training against CIFAR-10](Examples/Custom-CIFAR10)
44+
- [LeNet-5 training against MNIST](Examples/LeNet-MNIST)
45+
- [MobileNet V1 training against Imagenette](Examples/MobileNetV1-Imagenette)
46+
- [MobileNet V2 training against Imagenette](Examples/MobileNetV2-Imagenette)
47+
- [ResNet-56 training against CIFAR-10](Examples/ResNet-CIFAR10)
48+
- [ResNet-50 training against ImageNet](Examples/ResNet50-ImageNet)
49+
- [VGG-16 training against Imagewoof](Examples/VGG-Imagewoof)
50+
51+
### Text
52+
53+
- [BERT training against CoLA](Examples/BERT-CoLA)
54+
- [Pretrained GPT-2 performing text generation](Examples/GPT2-Inference)
55+
- [GPT-2 training against WikiText-2](Examples/GPT2-WikiText2)
56+
- [WordSeg](Examples/WordSeg)
57+
58+
### Generative models
59+
60+
- [1-D Autoencoder](Autoencoder/Autoencoder1D)
61+
- [2-D Autoencoder](Autoencoder/Autoencoder2D)
62+
- [1-D Variational Autoencoder](Autoencoder/VAE1D)
63+
- [CycleGAN](CycleGAN)
64+
- [GAN](GAN)
65+
- [DCGAN](DCGAN)
66+
- [pix2pix](pix2pix)
67+
68+
### Reinforcement learning
69+
70+
- [Blackjack](Gym)
71+
- [CartPole](Gym)
72+
- [Catch](Catch)
73+
- [FrozenLake](Gym)
74+
- [MiniGo](MiniGo)
75+
76+
### Standalone
77+
78+
- [Differentiable Shallow Water PDE Solver](Examples/Shallow-Water-PDE)
79+
- [Fast Style Transfer](FastStyleTransfer)
80+
- [Fractals](Examples/Fractals)
81+
- [Growing Neural Cellular Automata](Examples/GrowingNeuralCellularAutomata)
82+
- [Neural Collaborative Filtering using MovieLens](Examples/NeuMF-MovieLens)
83+
- [PersonLab Human Pose Estimator](PersonLab)
84+
- [Regression using BostonHousing](Examples/Regression-BostonHousing)
85+
86+
## Components
87+
88+
Beyond examples that use Swift for TensorFlow, this repository also contains reusable components
89+
for constructing machine learning applications. These components reside in modules that can be
90+
imported into separate Swift projects and used by themselves.
91+
92+
These components provide standalone machine learning models, datasets, image loading and saving,
93+
TensorBoard integration, and a training loop abstraction, among other capabilities.
94+
95+
The Swift for TensorFlow models repository has acted as a staging ground for experimental
96+
capabilities, letting us evaluate new components and interfaces before elevating them into the core
97+
Swift for TensorFlow APIs. As a result, the design and interfaces of these components may change
98+
regularly.
99+
100+
### Models
101+
102+
Several modules are provided that contain reusable Swift models for image classification, text
103+
processing, and more. These modules are used within the example applications to demonstrate the
104+
capabilities of these models, but they can also be imported into many other projects.
105+
106+
#### Image classification
107+
108+
Many common image classification models are present within
109+
[the ImageClassificationModels module](Models/ImageClassification). To use them within a Swift
110+
project, add ImageClassificationModels as a dependency and import the module:
111+
112+
```swift
113+
import ImageClassificationModels
28114
```
29115

30-
### Windows
116+
These models include:
117+
118+
- DenseNet121
119+
- EfficientNet
120+
- LeNet-5
121+
- MobileNetV1
122+
- MobileNetV2
123+
- MobileNetV3
124+
- ResNet
125+
- ResNetV2
126+
- ShuffleNetV2
127+
- SqueezeNet
128+
- VGG
129+
- WideResNet
130+
- Xception
131+
132+
#### Recommendation
133+
134+
Several recommendation models are present within
135+
[the RecommendationModels module](Models/Recommendation). To use them within a Swift
136+
project, add RecommendationModels as a dependency and import the module:
137+
138+
```swift
139+
import RecommendationModels
140+
```
31141

32-
#### Build
142+
These models include:
33143

34-
```cmd
35-
swift build -Xswiftc -use-ld=lld -c release
144+
- DLRM
145+
- MLP
146+
- NeuMF
147+
148+
#### Text
149+
150+
Several text models are present within
151+
[the TextModels module](Models/Text). To use them within a Swift
152+
project, add TextModels as a dependency and import the module:
153+
154+
```swift
155+
import TextModels
156+
```
157+
158+
These models include:
159+
160+
- [BERT](Models/Text/BERT)
161+
- [GPT-2](Models/Text/GPT2)
162+
- [WordSeg](Models/Text/WordSeg)
163+
164+
### Datasets
165+
166+
In addition to the machine learning model itself, a dataset is usually required when training.
167+
Swift wrappers have been built for many common datasets to ease their use within machine learning
168+
applications. Most of these use the
169+
[Epochs](https://github.com/tensorflow/swift-apis/tree/main/Sources/TensorFlow/Epochs) API that
170+
provides a generalized abstraction of common dataset operations.
171+
172+
The [Datasets](Datasets) module provides these wrappers. To use them within a Swift
173+
project, add Datasets as a dependency and import the module:
174+
175+
```swift
176+
import Datasets
177+
```
178+
179+
These are the currently provided dataset wrappers:
180+
181+
- [BostonHousing](Datasets/BostonHousing)
182+
- [CIFAR-10](Datasets/CIFAR10)
183+
- [MS COCO](Datasets/COCO)
184+
- [CoLA](Datasets/CoLA)
185+
- [ImageNet](Datasets/Imagenette)
186+
- [Imagenette](Datasets/Imagenette)
187+
- [Imagewoof](Datasets/Imagenette)
188+
- [FashionMNIST](Datasets/MNIST)
189+
- [KuzushijiMNIST](Datasets/MNIST)
190+
- [MNIST](Datasets/MNIST)
191+
- [MovieLens](Datasets/MovieLens)
192+
- [Oxford-IIIT Pet](Datasets/OxfordIIITPets)
193+
- [WordSeg](Datasets/WordSeg)
194+
195+
### Model checkpoints
196+
197+
Model saving and loading is provided by the [Checkpoints](Checkpoints) module. To use the model
198+
checkpointing functionality, add Checkpoints as a dependency and import the module:
199+
200+
```swift
201+
import Checkpoints
202+
```
203+
204+
### Image loading and saving
205+
206+
The [ModelSupport](Support) module contains many shared utilites that are needed within the Swift
207+
machine learning examples. This includes the loading, saving, and processing of
208+
[still images](Support/Image.swift) via the
209+
[stb_image](https://github.com/nothings/stb) library.
210+
[Animated images](Support/AnimatedImage.swift) can also be written out as GIF files from multiple
211+
tensors.
212+
213+
Experimental support for libjpeg-turbo as an accelerated image loader [is present](ImageLoader),
214+
but has not yet been incorporated into the main image loading capabilities.
215+
216+
### Generalized training loop
217+
218+
A generalized training loop that can be customized via callbacks is provided within the
219+
[TrainingLoop](TrainingLoop) module. All of the image classification examples use this training
220+
loop, with the exception of the Custom-CIFAR10 example that demonstrates how to define your own
221+
training loop from scratch. Other examples are being gradually converted to use this training loop.
222+
223+
### TensorBoard integration
224+
225+
[TensorBoard](https://www.tensorflow.org/tensorboard) integration is provided in the
226+
[TensorBoard module](TensorBoard) as a callback for the generalized training loop. TensorBoard
227+
lets you visualize the progress of your model as it trains by plotting model statistics as they
228+
update, or to review the training process afterward.
229+
230+
The [GPT2-WikiText2](Examples/GPT2-WikiText2) example demonstrates how this can be used when
231+
training your own models.
232+
233+
## Benchmarks and tests
234+
235+
A core goal of this repository is to validate the proper function of the Swift for TensorFlow APIs.
236+
In addition to the models and end-to-end applications present within this project, a suite of
237+
benchmarks and unit tests reside here.
238+
239+
The benchmarks are split into a core of functionality, the
240+
[SwiftModelsBenchmarksCore](SwiftModelsBenchmarksCore) module, and a
241+
[Benchmarks](SwiftModelsBenchmarks) command-line application for running these benchmarks. Refer to
242+
the [documentation](SwiftModelsBenchmarks) for how to run the benchmarks on your system.
243+
244+
The [unit tests](Tests) verify functionality within models, datasets and other components. To run
245+
them using Swift Package Manager on macOS or Linux:
246+
247+
```bash
248+
swift test
36249
```
37250

38-
#### Test
251+
and to run them on Windows:
39252

40253
```cmd
41254
swift test -Xswiftc -use-ld=lld -c debug
42255
```
43256

44-
### Using CMake for Development
257+
## Using CMake for Development
45258

46-
Use CMake to develop Swift for TensorFlow models.
259+
In addition to Swift Package Manager, CMake can be used to build and run Swift for TensorFlow
260+
models.
47261

48262
### *Experimental* CMake Support
49263

@@ -124,8 +338,13 @@ mailing list.
124338
## Contributing
125339

126340
We welcome contributions: please read the [Contributor Guide](CONTRIBUTING.md)
127-
to get started. It's always a good idea to discuss your plans on the mailing
128-
list before making any major submissions.
341+
to get started. It's always a good idea to discuss your plans on
342+
[the mailing list](https://groups.google.com/a/tensorflow.org/d/forum/swift) before making
343+
any major submissions.
344+
345+
We have labeled some issues as ["good first issue"](https://github.com/tensorflow/swift-models/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
346+
or ["help wanted"](https://github.com/tensorflow/swift-models/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22)
347+
to provide some suggestions for where new contributors might be able to start.
129348

130349
## Code of Conduct
131350

0 commit comments

Comments
 (0)