Skip to content

Commit f33c722

Browse files
author
rocky Cloud User
committed
Results on mlperf-reproducibility-hw added by ClusDur
1 parent b9c4f2d commit f33c722

21 files changed

+983
-0
lines changed

open/MLCommons/code/bert-99/README.md

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
TBD
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
2+
|---------|------------|------------|--------------|-------------------|
3+
| bert-99 | offline | 90.8749 | 48.312 | - |
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
This experiment is generated using the [MLCommons Collective Mind automation framework (CM)](https://github.com/mlcommons/cm4mlops).
2+
3+
*Check [CM MLPerf docs](https://docs.mlcommons.org/inference) for more details.*
4+
5+
## Host platform
6+
7+
* OS version: Linux-6.1.110-1.el9.elrepo.x86_64-x86_64-with-glibc2.34
8+
* CPU version: x86_64
9+
* Python version: 3.9.18 (main, Aug 23 2024, 00:00:00)
10+
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)]
11+
* MLCommons CM version: 3.2.6
12+
13+
## CM Run Command
14+
15+
See [CM installation guide](https://docs.mlcommons.org/inference/install/).
16+
17+
```bash
18+
pip install -U cmind
19+
20+
cm rm cache -f
21+
22+
cm pull repo mlcommons@cm4mlops --checkout=98fbefa641c777312599cc18f241001355c78e5e
23+
24+
cm run script \
25+
- \
26+
- \
27+
a \
28+
d \
29+
r \
30+
. \
31+
p \
32+
y \
33+
t \
34+
h \
35+
o \
36+
n \
37+
. \
38+
n \
39+
a \
40+
m \
41+
e \
42+
= \
43+
m \
44+
l \
45+
p \
46+
e \
47+
r \
48+
f
49+
```
50+
*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf (CM scripts),
51+
you should simply reload mlcommons@cm4mlops without checkout and clean CM cache as follows:*
52+
53+
```bash
54+
cm rm repo mlcommons@cm4mlops
55+
cm pull repo mlcommons@cm4mlops
56+
cm rm cache -f
57+
58+
```
59+
60+
## Results
61+
62+
Platform: scc124_gpu0.novalocal-reference-gpu-pytorch-default_config
63+
64+
Model Precision: fp32
65+
66+
### Accuracy Results
67+
`F1`: `90.87487`, Required accuracy for closed division `>= 89.96526`
68+
69+
### Performance Results
70+
`Samples per second`: `48.3116`

open/MLCommons/measurements/scc124_gpu0.novalocal-reference-gpu-pytorch-default_config/bert-99/offline/accuracy_console.out

Whitespace-only changes.

0 commit comments

Comments
 (0)