Skip to content

Commit 9dc1e10

Browse files
authored
Merge pull request #68 from whdduq52/mlperf-inference-results-scc24
Results on system <JetStream>
2 parents fc6710a + a05d8a3 commit 9dc1e10

File tree

21 files changed

+921
-0
lines changed

21 files changed

+921
-0
lines changed
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
TBD
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
| Model | Scenario | Accuracy | Throughput | Latency (in ms) | Power Efficiency (in samples/J) | TEST01 | TEST05 | TEST04 |
2+
|---------|------------|------------|--------------|-------------------|-----------------------------------|----------|----------|----------|
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
This experiment is generated using the [MLCommons Collective Mind automation framework (CM)](https://github.com/mlcommons/cm4mlops).
2+
3+
*Check [CM MLPerf docs](https://docs.mlcommons.org/inference) for more details.*
4+
5+
## Host platform
6+
7+
* OS version: Linux-6.1.110-1.el9.elrepo.x86_64-x86_64-with-glibc2.34
8+
* CPU version: x86_64
9+
* Python version: 3.9.18 (main, Aug 23 2024, 00:00:00)
10+
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)]
11+
* MLCommons CM version: 3.2.7
12+
13+
## CM Run Command
14+
15+
See [CM installation guide](https://docs.mlcommons.org/inference/install/).
16+
17+
```bash
18+
pip install -U cmind
19+
20+
cm rm cache -f
21+
22+
cm pull repo mlcommons@cm4mlops --checkout=358347c4a4876921fca310d9c54ed6495d50b7d1
23+
24+
cm run script \
25+
- \
26+
- \
27+
a \
28+
d \
29+
r \
30+
. \
31+
p \
32+
y \
33+
t \
34+
h \
35+
o \
36+
n \
37+
. \
38+
n \
39+
a \
40+
m \
41+
e \
42+
= \
43+
m \
44+
l \
45+
p \
46+
e \
47+
r \
48+
f
49+
```
50+
*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf (CM scripts),
51+
you should simply reload mlcommons@cm4mlops without checkout and clean CM cache as follows:*
52+
53+
```bash
54+
cm rm repo mlcommons@cm4mlops
55+
cm pull repo mlcommons@cm4mlops
56+
cm rm cache -f
57+
58+
```
59+
60+
## Results
61+
62+
Platform: scc122_cpu0.novalocal-reference-cpu-deepsparse_v1.8.0-default_config
63+
64+
Model Precision: int8
65+
66+
### Accuracy Results
67+
`F1`: `90.81269`, Required accuracy for closed division `>= 89.96526`
68+
69+
### Performance Results
70+
`Samples per second`: `16.1242`

open/scc122/measurements/scc122_cpu0.novalocal-reference-cpu-deepsparse_v1.8.0-default_config/mobilebert-base_quant-none-bert-99/offline/accuracy_console.out

Whitespace-only changes.

0 commit comments

Comments
 (0)