This repository was archived by the owner on Aug 7, 2024. It is now read-only.
Commit 31fba04
Fix an issue in sync_amax (#169)
Summary:
To fix this error
```
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor []] is at version 1; expected version 0 instead.
```
----
Also tried
```
torch.no_grad()
def sync_float8_amax_and_scale_history(
```
which didn't work.
----
We can look into if there are any better ways to fix this.
Pull Request resolved: #169
Test Plan: ./test/test_fsdp.sh
Reviewed By: vkuzo
Differential Revision: D52373985
Pulled By: y-sq
fbshipit-source-id: a25f4b0fee21dd5801c444b28f8a2f878bbafa351 parent f4812ee commit 31fba04
1 file changed
Lines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
166 | | - | |
167 | | - | |
168 | | - | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
169 | 169 | | |
170 | 170 | | |
171 | 171 | | |
| |||
0 commit comments