Skip to content

Commit c58ea0c

Browse files
esonghoricopybara-github
esonghori
authored and
copybara-github
committed
keep using tf-keras rather than keras-3 which is not compatible.
Also, moving tensorflow from [reverb] requirement to main requirments. PiperOrigin-RevId: 592609539 Change-Id: Ife593ab7960dddbd545d33eb9a49dce60e27f1f6
1 parent e5219db commit c58ea0c

17 files changed

+245
-21
lines changed

‎README.md

+7
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,8 @@ documentation for the release is on
108108
```shell
109109
$ pip install --user tf-agents[reverb]
110110

111+
# Use keras-2
112+
$ export TF_USE_LEGACY_KERAS=1
111113
# Use this tag get the matching examples and colabs.
112114
$ git clone https://github.com/tensorflow/agents.git
113115
$ cd agents
@@ -120,6 +122,7 @@ by the pip dependency check, use the following pattern below at your own risk.
120122

121123
```shell
122124
$ pip install --user tensorflow
125+
$ pip install --user tf-keras
123126
$ pip install --user dm-reverb
124127
$ pip install --user tf-agents
125128
```
@@ -143,8 +146,12 @@ tested against.
143146
To install the nightly build version, run the following:
144147

145148
```shell
149+
# Use keras-2
150+
$ export TF_USE_LEGACY_KERAS=1
151+
146152
# `--force-reinstall helps guarantee the right versions.
147153
$ pip install --user --force-reinstall tf-nightly
154+
$ pip install --user --force-reinstall tf-keras-nightly
148155
$ pip install --user --force-reinstall tfp-nightly
149156
$ pip install --user --force-reinstall dm-reverb-nightly
150157

‎docs/overview.md

+6
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ documentation for the release is on
2929
```shell
3030
$ pip install --user tf-agents[reverb]
3131

32+
# Use keras-2
33+
$ export TF_USE_LEGACY_KERAS=1
3234
# Use this tag get the matching examples and colabs.
3335
$ git clone https://github.com/tensorflow/agents.git
3436
$ cd agents
@@ -41,6 +43,7 @@ by the pip dependency check, use the following pattern below at your own risk.
4143

4244
```shell
4345
$ pip install --user tensorflow
46+
$ pip install --user tf-keras
4447
$ pip install --user dm-reverb
4548
$ pip install --user tf-agents
4649
```
@@ -64,8 +67,11 @@ tested against.
6467
To install the nightly build version, run the following:
6568

6669
```shell
70+
# Use keras-2
71+
$ export TF_USE_LEGACY_KERAS=1
6772
# `--force-reinstall helps guarantee the right versions.
6873
$ pip install --user --force-reinstall tf-nightly
74+
$ pip install --user --force-reinstall tf-keras-nightly
6975
$ pip install --user --force-reinstall tfp-nightly
7076
$ pip install --user --force-reinstall dm-reverb-nightly
7177

‎docs/tutorials/10_checkpointer_policysaver_tutorial.ipynb

+13
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,19 @@
111111
"!pip install tf-agents[reverb]"
112112
]
113113
},
114+
{
115+
"cell_type": "code",
116+
"execution_count": null,
117+
"metadata": {
118+
"id": "WPuD0bMEY9Iz"
119+
},
120+
"outputs": [],
121+
"source": [
122+
"import os\n",
123+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
124+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
125+
]
126+
},
114127
{
115128
"cell_type": "code",
116129
"execution_count": null,

‎docs/tutorials/1_dqn_tutorial.ipynb

+20-8
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,19 @@
120120
"!pip install pyglet"
121121
]
122122
},
123+
{
124+
"cell_type": "code",
125+
"execution_count": null,
126+
"metadata": {
127+
"id": "UX0aSKBCYmj2"
128+
},
129+
"outputs": [],
130+
"source": [
131+
"import os\n",
132+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
133+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
134+
]
135+
},
123136
{
124137
"cell_type": "code",
125138
"execution_count": null,
@@ -222,7 +235,7 @@
222235
"\n",
223236
"In Reinforcement Learning (RL), an environment represents the task or problem to be solved. Standard environments can be created in TF-Agents using `tf_agents.environments` suites. TF-Agents has suites for loading environments from sources such as the OpenAI Gym, Atari, and DM Control.\n",
224237
"\n",
225-
"Load the CartPole environment from the OpenAI Gym suite. "
238+
"Load the CartPole environment from the OpenAI Gym suite."
226239
]
227240
},
228241
{
@@ -323,9 +336,9 @@
323336
"source": [
324337
"In the Cartpole environment:\n",
325338
"\n",
326-
"- `observation` is an array of 4 floats: \n",
339+
"- `observation` is an array of 4 floats:\n",
327340
" - the position and velocity of the cart\n",
328-
" - the angular position and velocity of the pole \n",
341+
" - the angular position and velocity of the pole\n",
329342
"- `reward` is a scalar float value\n",
330343
"- `action` is a scalar integer with only two possible values:\n",
331344
" - `0` — \"move left\"\n",
@@ -357,7 +370,7 @@
357370
"id": "4JSc9GviWUBK"
358371
},
359372
"source": [
360-
"Usually two environments are instantiated: one for training and one for evaluation. "
373+
"Usually two environments are instantiated: one for training and one for evaluation."
361374
]
362375
},
363376
{
@@ -500,7 +513,7 @@
500513
"- The desired outcome is keeping the pole balanced upright over the cart.\n",
501514
"- The policy returns an action (left or right) for each `time_step` observation.\n",
502515
"\n",
503-
"Agents contain two policies: \n",
516+
"Agents contain two policies:\n",
504517
"\n",
505518
"- `agent.policy` — The main policy that is used for evaluation and deployment.\n",
506519
"- `agent.collect_policy` — A second policy that is used for data collection.\n"
@@ -834,7 +847,7 @@
834847
"source": [
835848
"# For the curious:\n",
836849
"# Uncomment to see what the dataset iterator is feeding to the agent.\n",
837-
"# Compare this representation of replay data \n",
850+
"# Compare this representation of replay data\n",
838851
"# to the collection of individual trajectories shown earlier.\n",
839852
"\n",
840853
"# iterator.next()"
@@ -967,7 +980,7 @@
967980
"id": "9pGfGxSH32gn"
968981
},
969982
"source": [
970-
"Charts are nice. But more exciting is seeing an agent actually performing a task in an environment. \n",
983+
"Charts are nice. But more exciting is seeing an agent actually performing a task in an environment.\n",
971984
"\n",
972985
"First, create a function to embed videos in the notebook."
973986
]
@@ -1048,7 +1061,6 @@
10481061
],
10491062
"metadata": {
10501063
"colab": {
1051-
"collapsed_sections": [],
10521064
"name": "DQN Tutorial.ipynb",
10531065
"private_outputs": true,
10541066
"provenance": [],

‎docs/tutorials/2_environments_tutorial.ipynb

+14-2
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,19 @@
112112
"!pip install tf-agents[reverb]\n"
113113
]
114114
},
115+
{
116+
"cell_type": "code",
117+
"execution_count": null,
118+
"metadata": {
119+
"id": "WPuD0bMEY9Iz"
120+
},
121+
"outputs": [],
122+
"source": [
123+
"import os\n",
124+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
125+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
126+
]
127+
},
115128
{
116129
"cell_type": "code",
117130
"execution_count": null,
@@ -489,7 +502,7 @@
489502
"The interface for TF environments is defined in `environments/tf_environment.TFEnvironment` and looks very similar to the Python environments. TF Environments differ from Python envs in a couple of ways:\n",
490503
"\n",
491504
"* They generate tensor objects instead of arrays\n",
492-
"* TF environments add a batch dimension to the tensors generated when compared to the specs. \n",
505+
"* TF environments add a batch dimension to the tensors generated when compared to the specs.\n",
493506
"\n",
494507
"Converting the Python environments into TFEnvs allows tensorflow to parallelize operations. For example, one could define a `collect_experience_op` that collects data from the environment and adds to a `replay_buffer`, and a `train_op` that reads from the `replay_buffer` and trains the agent, and run them in parallel naturally in TensorFlow."
495508
]
@@ -702,7 +715,6 @@
702715
],
703716
"metadata": {
704717
"colab": {
705-
"collapsed_sections": [],
706718
"name": "TF-Agents Environments Tutorial.ipynb",
707719
"private_outputs": true,
708720
"provenance": [],

‎docs/tutorials/3_policies_tutorial.ipynb

+17-5
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,19 @@
114114
"!pip install tf-agents"
115115
]
116116
},
117+
{
118+
"cell_type": "code",
119+
"execution_count": null,
120+
"metadata": {
121+
"id": "h3dkwi09ZQeJ"
122+
},
123+
"outputs": [],
124+
"source": [
125+
"import os\n",
126+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
127+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
128+
]
129+
},
117130
{
118131
"cell_type": "code",
119132
"execution_count": null,
@@ -295,9 +308,9 @@
295308
"outputs": [],
296309
"source": [
297310
"action_spec = array_spec.BoundedArraySpec((2,), np.int32, -10, 10)\n",
298-
"action_script = [(1, np.array([5, 2], dtype=np.int32)), \n",
311+
"action_script = [(1, np.array([5, 2], dtype=np.int32)),\n",
299312
" (0, np.array([0, 0], dtype=np.int32)), # Setting `num_repeats` to 0 will skip this action.\n",
300-
" (2, np.array([1, 2], dtype=np.int32)), \n",
313+
" (2, np.array([1, 2], dtype=np.int32)),\n",
301314
" (1, np.array([3, 4], dtype=np.int32))]\n",
302315
"\n",
303316
"my_scripted_py_policy = scripted_py_policy.ScriptedPyPolicy(\n",
@@ -555,7 +568,7 @@
555568
"id": "BzoNGJnlibtz"
556569
},
557570
"source": [
558-
"Note that in the above, actions are clipped to the range of the given action spec [-1, 1]. This is because a constructor argument of ActorPolicy clip=True by default. Setting this to false will return unclipped actions produced by the network. "
571+
"Note that in the above, actions are clipped to the range of the given action spec [-1, 1]. This is because a constructor argument of ActorPolicy clip=True by default. Setting this to false will return unclipped actions produced by the network."
559572
]
560573
},
561574
{
@@ -655,7 +668,7 @@
655668
"id": "OfaUrqRAoigk"
656669
},
657670
"source": [
658-
"A policy wrapper can be used to wrap and modify a given policy, e.g. add noise. Policy wrappers are a subclass of Policy (Python/TensorFlow) and can therefore be used just like any other policy. "
671+
"A policy wrapper can be used to wrap and modify a given policy, e.g. add noise. Policy wrappers are a subclass of Policy (Python/TensorFlow) and can therefore be used just like any other policy."
659672
]
660673
},
661674
{
@@ -692,7 +705,6 @@
692705
],
693706
"metadata": {
694707
"colab": {
695-
"collapsed_sections": [],
696708
"name": "TF-Agents Policies Tutorial.ipynb",
697709
"private_outputs": true,
698710
"provenance": [],

‎docs/tutorials/4_drivers_tutorial.ipynb

+13
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,19 @@
105105
"!pip install tf-agents\n"
106106
]
107107
},
108+
{
109+
"cell_type": "code",
110+
"execution_count": null,
111+
"metadata": {
112+
"id": "WPuD0bMEY9Iz"
113+
},
114+
"outputs": [],
115+
"source": [
116+
"import os\n",
117+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
118+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
119+
]
120+
},
108121
{
109122
"cell_type": "code",
110123
"execution_count": null,

‎docs/tutorials/5_replay_buffers_tutorial.ipynb

+13
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,19 @@
103103
"!pip install tf-agents\n"
104104
]
105105
},
106+
{
107+
"cell_type": "code",
108+
"execution_count": null,
109+
"metadata": {
110+
"id": "WPuD0bMEY9Iz"
111+
},
112+
"outputs": [],
113+
"source": [
114+
"import os\n",
115+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
116+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
117+
]
118+
},
106119
{
107120
"cell_type": "code",
108121
"execution_count": null,

‎docs/tutorials/6_reinforce_tutorial.ipynb

+14-2
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,19 @@
117117
"!pip install pyglet xvfbwrapper\n"
118118
]
119119
},
120+
{
121+
"cell_type": "code",
122+
"execution_count": null,
123+
"metadata": {
124+
"id": "WPuD0bMEY9Iz"
125+
},
126+
"outputs": [],
127+
"source": [
128+
"import os\n",
129+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
130+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
131+
]
132+
},
120133
{
121134
"cell_type": "code",
122135
"execution_count": null,
@@ -571,7 +584,7 @@
571584
" # Use data from the buffer and update the agent's network.\n",
572585
" iterator = iter(replay_buffer.as_dataset(sample_batch_size=1))\n",
573586
" trajectories, _ = next(iterator)\n",
574-
" train_loss = tf_agent.train(experience=trajectories) \n",
587+
" train_loss = tf_agent.train(experience=trajectories)\n",
575588
"\n",
576589
" replay_buffer.clear()\n",
577590
"\n",
@@ -697,7 +710,6 @@
697710
"metadata": {
698711
"accelerator": "GPU",
699712
"colab": {
700-
"collapsed_sections": [],
701713
"name": "6_reinforce_tutorial.ipynb",
702714
"provenance": [],
703715
"toc_visible": true

‎docs/tutorials/7_SAC_minitaur_tutorial.ipynb

+13
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,19 @@
112112
"!pip install pybullet"
113113
]
114114
},
115+
{
116+
"cell_type": "code",
117+
"execution_count": null,
118+
"metadata": {
119+
"id": "WPuD0bMEY9Iz"
120+
},
121+
"outputs": [],
122+
"source": [
123+
"import os\n",
124+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
125+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
126+
]
127+
},
115128
{
116129
"cell_type": "markdown",
117130
"metadata": {

‎docs/tutorials/8_networks_tutorial.ipynb

+13
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,19 @@
117117
"!pip install tf-agents"
118118
]
119119
},
120+
{
121+
"cell_type": "code",
122+
"execution_count": null,
123+
"metadata": {
124+
"id": "WPuD0bMEY9Iz"
125+
},
126+
"outputs": [],
127+
"source": [
128+
"import os\n",
129+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
130+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
131+
]
132+
},
120133
{
121134
"cell_type": "code",
122135
"execution_count": null,

‎docs/tutorials/9_c51_tutorial.ipynb

+13
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,19 @@
117117
"!pip install pyglet"
118118
]
119119
},
120+
{
121+
"cell_type": "code",
122+
"execution_count": null,
123+
"metadata": {
124+
"id": "WPuD0bMEY9Iz"
125+
},
126+
"outputs": [],
127+
"source": [
128+
"import os\n",
129+
"# Keep using keras-2 (tf-keras) rather than keras-3 (keras).\n",
130+
"os.environ['TF_USE_LEGACY_KERAS'] = '1'"
131+
]
132+
},
120133
{
121134
"cell_type": "code",
122135
"execution_count": null,

0 commit comments

Comments
 (0)