Skip to content

Commit 600b67e

Browse files
add loss curves
1 parent 105f1ad commit 600b67e

File tree

2 files changed

+8
-1
lines changed

2 files changed

+8
-1
lines changed

torchao/prototype/float8nocompile/README.md

+8-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# float8nocompile
22

33

4-
A prototype API for high performance eager mode float8 training via handwritten Triton kernels for quantization.
4+
A prototype API for high performance eager mode float8 training that uses handwritten Triton kernels for quantization.
55

66
### Usage
77

@@ -63,3 +63,10 @@ The results indicate a solid 6-10% tokens/sec speedup with relatively flat memor
6363
| bfloat16, eager | 4502.0 | 28.07 | 0% | 0.00% |
6464
| float8nocompile prototype | 4773.4 | 28.07 | 6.03% | 0.00% |
6565
| float8 + torch.compile | 5775.2 | 28.03 | 28.28% | -0.14% |
66+
67+
68+
## Numerical accuracy
69+
70+
Numerical accuracy has been verified via unit tests as well as manually verifying that the training loss curves maintain fidelity with the loss curves for bf16 eager and production float8 + torch.compile:
71+
72+
![loss curves](float8nocompile_loss_curves.png "Loss curves")
Loading

0 commit comments

Comments
 (0)