Skip to content

Commit 6af1d5f

Browse files
gateoverflow-admingateoverflow-admin
committed
Updated results summary
1 parent 874a0b3 commit 6af1d5f

File tree

2 files changed

+96
-22
lines changed

2 files changed

+96
-22
lines changed

‎README.md

+22-20
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,30 @@
11
```
2-
[2024-11-18 22:34:35,397 submission_checker1.py:3109 INFO] Results=6, NoResults=0, Power Results=0
3-
[2024-11-18 22:34:35,397 submission_checker1.py:3116 INFO] ---
4-
[2024-11-18 22:34:35,397 submission_checker1.py:3117 INFO] Closed Results=0, Closed Power Results=0
2+
[2024-11-18 22:42:37,628 submission_checker1.py:3109 INFO] Results=8, NoResults=0, Power Results=0
3+
[2024-11-18 22:42:37,628 submission_checker1.py:3116 INFO] ---
4+
[2024-11-18 22:42:37,628 submission_checker1.py:3117 INFO] Closed Results=0, Closed Power Results=0
55
6-
[2024-11-18 22:34:35,397 submission_checker1.py:3122 INFO] Open Results=6, Open Power Results=0
6+
[2024-11-18 22:42:37,628 submission_checker1.py:3122 INFO] Open Results=8, Open Power Results=0
77
8-
[2024-11-18 22:34:35,397 submission_checker1.py:3127 INFO] Network Results=0, Network Power Results=0
8+
[2024-11-18 22:42:37,628 submission_checker1.py:3127 INFO] Network Results=0, Network Power Results=0
99
10-
[2024-11-18 22:34:35,397 submission_checker1.py:3132 INFO] ---
11-
[2024-11-18 22:34:35,397 submission_checker1.py:3134 INFO] Systems=6, Power Systems=0
12-
[2024-11-18 22:34:35,397 submission_checker1.py:3138 INFO] Closed Systems=0, Closed Power Systems=0
13-
[2024-11-18 22:34:35,397 submission_checker1.py:3143 INFO] Open Systems=6, Open Power Systems=0
14-
[2024-11-18 22:34:35,397 submission_checker1.py:3148 INFO] Network Systems=0, Network Power Systems=0
15-
[2024-11-18 22:34:35,397 submission_checker1.py:3153 INFO] ---
16-
[2024-11-18 22:34:35,397 submission_checker1.py:3158 INFO] SUMMARY: submission looks OK
10+
[2024-11-18 22:42:37,628 submission_checker1.py:3132 INFO] ---
11+
[2024-11-18 22:42:37,628 submission_checker1.py:3134 INFO] Systems=8, Power Systems=0
12+
[2024-11-18 22:42:37,628 submission_checker1.py:3138 INFO] Closed Systems=0, Closed Power Systems=0
13+
[2024-11-18 22:42:37,628 submission_checker1.py:3143 INFO] Open Systems=8, Open Power Systems=0
14+
[2024-11-18 22:42:37,628 submission_checker1.py:3148 INFO] Network Systems=0, Network Power Systems=0
15+
[2024-11-18 22:42:37,628 submission_checker1.py:3153 INFO] ---
16+
[2024-11-18 22:42:37,628 submission_checker1.py:3158 INFO] SUMMARY: submission looks OK
1717
INFO:root: ! call "postprocess" from /home/runner/CM/repos/mlcommons@cm4mlops/script/run-mlperf-inference-submission-checker/customize.py
1818
1919
```
2020

21-
| | Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types |
22-
|---:|:---------------|:---------------|:-----------|:-------------|:------------------------|:-------------------------------------------------------------------------|:--------------------|:--------------------|:-----------|---------:|:--------------------------------------------------------------|------------------:|:---------------------------------|---------------------------:|----------------------------:|:-------------------------|------------------------:|:--------------------------------------------------------------------------------------------------------------------------|:---------------|:------------------------------------------------------------|:----------------------------------|-------------:|---------:|:----------|-----------:|:------------|:----------|:--------------------|
23-
| 0 | NTHU | available | open | datacenter | 334907abd4f4 | Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.528528 | CLIP_SCORE: 16.4035627245903 FID_SCORE: 235.76925470141796 | 1 | INTEL(R) XEON(R) PLATINUM 8592+ | 2 | 64 | NVIDIA H100 PCIe | 4 | open/NTHU/results/Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124/stable-diffusion-xl/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35) | Automated by MLCommons CM v3.1.0. | 1 | 0 | v4.1 | 0 | False | Samples/s | fp32 |
24-
| 1 | NTHU | available | open | datacenter | ba7f5c94d10f | ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base | stable-diffusion-xl | stable-diffusion-xl | Offline | 4.2742 | CLIP_SCORE: 31.234263841211796 FID_SCORE: 23.67815832489498 | 1 | INTEL(R) XEON(R) PLATINUM 8592+ | 2 | 64 | NVIDIA H100 PCIe | 4 | open/NTHU/results/ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline | TensorRT | Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
25-
| 2 | peking1 | available | open | datacenter | mlperf_inference_lry_40 | mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.47046 | CLIP_SCORE: 14.028269715607166 FID_SCORE: 84.33061840229146 | 1 | AMD EPYC 9684X 96-Core Processor | 2 | 96 | NVIDIA H100 80GB HBM3 | 5 | open/peking1/results/mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-5.14.0-427.33.1.el9_4.x86_64-glibc2.31) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
26-
| 3 | peking | available | open | datacenter | mlperf_inference_lry_40 | mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.64431 | CLIP_SCORE: 14.028269715607166 FID_SCORE: 84.33061840229146 | 1 | AMD EPYC 9684X 96-Core Processor | 2 | 96 | NVIDIA H100 80GB HBM3 | 5 | open/peking/results/mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-5.14.0-427.33.1.el9_4.x86_64-glibc2.31) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
27-
| 4 | NTUHPC | available | open | datacenter | 1b41d1041a1b | Coffeepot-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 4.18764 | CLIP_SCORE: 16.503750428557396 FID_SCORE: 232.23581854554644 | 1 | AMD EPYC 9654 96-Core Processor | 1 | 96 | NVIDIA H100 PCIe | 4 | open/NTUHPC/results/Coffeepot-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-6.5.0-27-generic-glibc2.31) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
28-
| 5 | scc2 | available | open | datacenter | f2ea47b0a016 | f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.46497 | CLIP_SCORE: 16.63159880042076 FID_SCORE: 234.62921308614995 | 1 | AMD EPYC 9754 128-Core Processor | 2 | 128 | NVIDIA H100 80GB HBM3 | 6 | open/scc2/results/f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-6.1.0-27-amd64-glibc2.31) | Automated by MLCommons CM v3.3.3. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
21+
| | Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types |
22+
|---:|:---------------|:---------------|:-----------|:-------------|:------------------------|:-------------------------------------------------------------------------|:--------------------|:--------------------|:-----------|---------:|:--------------------------------------------------------------|------------------:|:---------------------------------|---------------------------:|----------------------------:|:-------------------------|------------------------:|:--------------------------------------------------------------------------------------------------------------------------|:---------------------------|:------------------------------------------------------------|:----------------------------------|-------------:|---------:|:----------|-----------:|:------------|:----------|:--------------------|
23+
| 0 | NTHU | available | open | datacenter | 334907abd4f4 | Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.528528 | CLIP_SCORE: 16.4035627245903 FID_SCORE: 235.76925470141796 | 1 | INTEL(R) XEON(R) PLATINUM 8592+ | 2 | 64 | NVIDIA H100 PCIe | 4 | open/NTHU/results/Kuai-Kuai_702-reference-gpu-pytorch-v2.5.1-scc24-base_cu124/stable-diffusion-xl/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35) | Automated by MLCommons CM v3.1.0. | 1 | 0 | v4.1 | 0 | False | Samples/s | fp32 |
24+
| 1 | NTHU | available | open | datacenter | ba7f5c94d10f | ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base | stable-diffusion-xl | stable-diffusion-xl | Offline | 4.2742 | CLIP_SCORE: 31.234263841211796 FID_SCORE: 23.67815832489498 | 1 | INTEL(R) XEON(R) PLATINUM 8592+ | 2 | 64 | NVIDIA H100 PCIe | 4 | open/NTHU/results/ba7f5c94d10f-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline | TensorRT | Ubuntu 22.04 (linux-6.8.0-47-generic-glibc2.35) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
25+
| 2 | peking1 | available | open | datacenter | mlperf_inference_lry_40 | mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.47046 | CLIP_SCORE: 14.028269715607166 FID_SCORE: 84.33061840229146 | 1 | AMD EPYC 9684X 96-Core Processor | 2 | 96 | NVIDIA H100 80GB HBM3 | 5 | open/peking1/results/mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-5.14.0-427.33.1.el9_4.x86_64-glibc2.31) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
26+
| 3 | UCSD | available | open | datacenter | aqua | aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.209132 | CLIP_SCORE: 15.227862536907196 FID_SCORE: 236.96183273783743 | 1 | AMD EPYC 9354 32-Core Processor | 2 | 32 | nan | 0 | open/UCSD/results/aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base/stable-diffusion-xl/offline | pytorch v2.6.0.dev20241109 | Rocky 9.4 (linux-5.14.0-427.42.1.el9_4.x86_64-glibc2.34) | Automated by MLCommons CM v3.1.0. | 1 | 0 | v4.1 | 0 | False | Samples/s | fp32 |
27+
| 4 | UCSD | available | open | datacenter | aqua | aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-base | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.20835 | CLIP_SCORE: 16.381833255290985 FID_SCORE: 236.8570675351765 | 1 | AMD EPYC 9354 32-Core Processor | 2 | 32 | nan | 0 | open/UCSD/results/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-base/stable-diffusion-xl/offline | pytorch v2.6.0.dev20241118 | Ubuntu 22.04 (linux-5.14.0-427.42.1.el9_4.x86_64-glibc2.35) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | fp32 |
28+
| 5 | peking | available | open | datacenter | mlperf_inference_lry_40 | mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.64431 | CLIP_SCORE: 14.028269715607166 FID_SCORE: 84.33061840229146 | 1 | AMD EPYC 9684X 96-Core Processor | 2 | 96 | NVIDIA H100 80GB HBM3 | 5 | open/peking/results/mlperf_inference_lry_40-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-5.14.0-427.33.1.el9_4.x86_64-glibc2.31) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
29+
| 6 | NTUHPC | available | open | datacenter | 1b41d1041a1b | Coffeepot-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 4.18764 | CLIP_SCORE: 16.503750428557396 FID_SCORE: 232.23581854554644 | 1 | AMD EPYC 9654 96-Core Processor | 1 | 96 | NVIDIA H100 PCIe | 4 | open/NTUHPC/results/Coffeepot-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-6.5.0-27-generic-glibc2.31) | Automated by MLCommons CM v3.4.1. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |
30+
| 7 | scc2 | available | open | datacenter | f2ea47b0a016 | f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main | stable-diffusion-xl | stable-diffusion-xl | Offline | 7.46497 | CLIP_SCORE: 16.63159880042076 FID_SCORE: 234.62921308614995 | 1 | AMD EPYC 9754 128-Core Processor | 2 | 128 | NVIDIA H100 80GB HBM3 | 6 | open/scc2/results/f2ea47b0a016-nvidia_original-gpu-tensorrt-vdefault-scc24-main/stable-diffusion-xl/offline | TensorRT | Ubuntu 20.04 (linux-6.1.0-27-amd64-glibc2.31) | Automated by MLCommons CM v3.3.3. | 1 | 0 | v4.1 | 0 | False | Samples/s | int8 |

0 commit comments

Comments
 (0)