Skip to content

Commit 6c26924

Browse files
gateoverflow-admingateoverflow-admin
committed
Updated results summary
1 parent 4936f06 commit 6c26924

File tree

2 files changed

+89
-15
lines changed

2 files changed

+89
-15
lines changed

‎README.md

+17-15
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,25 @@
11
```
2-
[2024-11-18 17:40:43,833 submission_checker1.py:3109 INFO] Results=1, NoResults=0, Power Results=0
3-
[2024-11-18 17:40:43,833 submission_checker1.py:3116 INFO] ---
4-
[2024-11-18 17:40:43,833 submission_checker1.py:3117 INFO] Closed Results=0, Closed Power Results=0
2+
[2024-11-18 21:51:28,038 submission_checker1.py:3109 INFO] Results=3, NoResults=0, Power Results=0
3+
[2024-11-18 21:51:28,038 submission_checker1.py:3116 INFO] ---
4+
[2024-11-18 21:51:28,038 submission_checker1.py:3117 INFO] Closed Results=0, Closed Power Results=0
55
6-
[2024-11-18 17:40:43,833 submission_checker1.py:3122 INFO] Open Results=1, Open Power Results=0
6+
[2024-11-18 21:51:28,038 submission_checker1.py:3122 INFO] Open Results=3, Open Power Results=0
77
8-
[2024-11-18 17:40:43,833 submission_checker1.py:3127 INFO] Network Results=0, Network Power Results=0
8+
[2024-11-18 21:51:28,038 submission_checker1.py:3127 INFO] Network Results=0, Network Power Results=0
99
10-
[2024-11-18 17:40:43,833 submission_checker1.py:3132 INFO] ---
11-
[2024-11-18 17:40:43,833 submission_checker1.py:3134 INFO] Systems=1, Power Systems=0
12-
[2024-11-18 17:40:43,833 submission_checker1.py:3138 INFO] Closed Systems=0, Closed Power Systems=0
13-
[2024-11-18 17:40:43,833 submission_checker1.py:3143 INFO] Open Systems=1, Open Power Systems=0
14-
[2024-11-18 17:40:43,833 submission_checker1.py:3148 INFO] Network Systems=0, Network Power Systems=0
15-
[2024-11-18 17:40:43,833 submission_checker1.py:3153 INFO] ---
16-
[2024-11-18 17:40:43,833 submission_checker1.py:3158 INFO] SUMMARY: submission looks OK
10+
[2024-11-18 21:51:28,038 submission_checker1.py:3132 INFO] ---
11+
[2024-11-18 21:51:28,038 submission_checker1.py:3134 INFO] Systems=3, Power Systems=0
12+
[2024-11-18 21:51:28,038 submission_checker1.py:3138 INFO] Closed Systems=0, Closed Power Systems=0
13+
[2024-11-18 21:51:28,038 submission_checker1.py:3143 INFO] Open Systems=3, Open Power Systems=0
14+
[2024-11-18 21:51:28,038 submission_checker1.py:3148 INFO] Network Systems=0, Network Power Systems=0
15+
[2024-11-18 21:51:28,038 submission_checker1.py:3153 INFO] ---
16+
[2024-11-18 21:51:28,038 submission_checker1.py:3158 INFO] SUMMARY: submission looks OK
1717
INFO:root: ! call "postprocess" from /home/runner/CM/repos/mlcommons@cm4mlops/script/run-mlperf-inference-submission-checker/customize.py
1818
1919
```
2020

21-
| | Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types |
22-
|---:|:---------------|:---------------|:-----------|:-------------|:-------------|:--------------------------------------------------------------|:--------------------|:--------------------|:-----------|---------:|:-------------------------------------------------------------|------------------:|:---------------------------------|---------------------------:|----------------------------:|:-------------------------|------------------------:|:------------------------------------------------------------------------------------------------------------|:------------|:----------------------------------------------|:----------------------------------|-------------:|---------:|:----------|-----------:|:------------|:----------|:--------------------|
23-
| 0 | scc2 | available | open | datacenter | f2ea47b0a016 | f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.46497 | CLIP_SCORE: 16.63159880042076 FID_SCORE: 234.62921308614995 | 1 | AMD EPYC 9754 128-Core Processor | 2 | 128 | NVIDIA H100 80GB HBM3 | 6 | open/scc2/results/f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-6.1.0-27-amd64-glibc2.31) | Automated by MLCommons CM v3.3.3. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
21+
| | Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types |
22+
|---:|:---------------|:---------------|:-----------|:-------------|:-------------|:--------------------------------------------------------------|:--------------------|:--------------------|:-----------|---------:|:-------------------------------------------------------------|------------------:|:---------------------------------|---------------------------:|----------------------------:|:-------------------------|------------------------:|:------------------------------------------------------------------------------------------------------------|:---------------|:------------------------------------------------|:----------------------------------|-------------:|---------:|:----------|-----------:|:------------|:----------|:--------------------|
23+
| 0 | NTHU | available | open | datacenter | 334907abd4f4 | Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.528528 | CLIP_SCORE: 16.4035627245903 FID_SCORE: 235.76925470141796 | 1 | INTEL(R) XEON(R) PLATINUM 8592+ | 2 | 64 | NVIDIA H100 PCIe | 4 | open/NTHU/results/Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124/stable-diffusion-xl/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35) | Automated by MLCommons CM v3.1.0. | 1 | 0 | v4.1 | 0 | False | Samples/s | fp32 |
24+
| 1 | NTHU | available | open | datacenter | ba7f5c94d10f | ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base | stable-diffusion-xl | stable-diffusion-xl | Offline | 4.2742 | CLIP_SCORE: 31.234263841211796 FID_SCORE: 23.67815832489498 | 1 | INTEL(R) XEON(R) PLATINUM 8592+ | 2 | 64 | NVIDIA H100 PCIe | 4 | open/NTHU/results/ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline | TensorRT | Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
25+
| 2 | scc2 | available | open | datacenter | f2ea47b0a016 | f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.46497 | CLIP_SCORE: 16.63159880042076 FID_SCORE: 234.62921308614995 | 1 | AMD EPYC 9754 128-Core Processor | 2 | 128 | NVIDIA H100 80GB HBM3 | 6 | open/scc2/results/f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-6.1.0-27-amd64-glibc2.31) | Automated by MLCommons CM v3.3.3. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |

‎summary_results.json

+72
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,78 @@
11
[
22
{
33
"ID": "4.1-0001",
4+
"Submitter": "NTHU",
5+
"Availability": "available",
6+
"Category": "open",
7+
"Suite": "datacenter",
8+
"System": "334907abd4f4",
9+
"Platform": "Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124",
10+
"UsedModel": "stable-diffusion-xl",
11+
"Model": "stable-diffusion-xl",
12+
"Scenario": "Offline",
13+
"Accuracy": "CLIP_SCORE: 16.4035627245903 FID_SCORE: 235.76925470141796",
14+
"Nodes": 1,
15+
"Processor": "INTEL(R) XEON(R) PLATINUM 8592+",
16+
"host_processors_per_node": 2,
17+
"host_processor_core_count": 64,
18+
"Accelerator": "NVIDIA H100 PCIe",
19+
"a#": 4,
20+
"Location": "open/NTHU/results/Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124/stable-diffusion-xl/offline",
21+
"Software": "pytorch v2.5.1",
22+
"operating_system": "Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35)",
23+
"Notes": "Automated by MLCommons CM v3.1.0. ",
24+
"compliance": 1,
25+
"errors": 0,
26+
"version": "v4.1",
27+
"inferred": 0,
28+
"has_power": false,
29+
"weight_data_types": "fp32",
30+
"p#": 2,
31+
"Details": "https://github.com/mlcommons/cm4mlperf-inference/tree/mlperf-inference-results-scc24/open/NTHU/results/Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124",
32+
"Code": "https://github.com/mlcommons/cm4mlperf-inference/tree/mlperf-inference-results-scc24/open/NTHU/code",
33+
"Unique ID (e.g. for Audit)": "datacenter/open/NTHU/Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124/stable-diffusion-xl",
34+
"ColorKey": "availableNTHU",
35+
"Performance_Result": 0.528528,
36+
"Performance_Units": "Samples/s"
37+
},
38+
{
39+
"ID": "4.1-0002",
40+
"Submitter": "NTHU",
41+
"Availability": "available",
42+
"Category": "open",
43+
"Suite": "datacenter",
44+
"System": "ba7f5c94d10f",
45+
"Platform": "ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base",
46+
"UsedModel": "stable-diffusion-xl",
47+
"Model": "stable-diffusion-xl",
48+
"Scenario": "Offline",
49+
"Accuracy": "CLIP_SCORE: 31.234263841211796 FID_SCORE: 23.67815832489498",
50+
"Nodes": 1,
51+
"Processor": "INTEL(R) XEON(R) PLATINUM 8592+",
52+
"host_processors_per_node": 2,
53+
"host_processor_core_count": 64,
54+
"Accelerator": "NVIDIA H100 PCIe",
55+
"a#": 4,
56+
"Location": "open/NTHU/results/ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline",
57+
"Software": "TensorRT",
58+
"operating_system": "Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35)",
59+
"Notes": "Automated by MLCommons CM v3.4.1. ",
60+
"compliance": 1,
61+
"errors": 0,
62+
"version": "v4.1",
63+
"inferred": 0,
64+
"has_power": false,
65+
"weight_data_types": "int8",
66+
"p#": 2,
67+
"Details": "https://github.com/mlcommons/cm4mlperf-inference/tree/mlperf-inference-results-scc24/open/NTHU/results/ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base",
68+
"Code": "https://github.com/mlcommons/cm4mlperf-inference/tree/mlperf-inference-results-scc24/open/NTHU/code",
69+
"Unique ID (e.g. for Audit)": "datacenter/open/NTHU/ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl",
70+
"ColorKey": "availableNTHU",
71+
"Performance_Result": 4.2742,
72+
"Performance_Units": "Samples/s"
73+
},
74+
{
75+
"ID": "4.1-0003",
476
"Submitter": "scc2",
577
"Availability": "available",
678
"Category": "open",

0 commit comments

Comments
 (0)