diff --git a/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/README.md b/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/README.md new file mode 100644 index 0000000..7feae32 --- /dev/null +++ b/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/README.md @@ -0,0 +1,3 @@ +| Model | Scenario | Accuracy | Throughput | Latency (in ms) | +|---------------------|------------|----------------------|--------------|-------------------| +| stable-diffusion-xl | offline | (15.22477, 84.24318) | 0.848 | - | \ No newline at end of file diff --git a/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/README.md b/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/README.md new file mode 100644 index 0000000..a52d873 --- /dev/null +++ b/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/README.md @@ -0,0 +1,59 @@ +This experiment is generated using the [MLCommons Collective Mind automation framework (CM)](https://github.com/mlcommons/cm4mlops). + +*Check [CM MLPerf docs](https://docs.mlcommons.org/inference) for more details.* + +## Host platform + +* OS version: Linux-5.14.0-427.42.1.el9_4.x86_64-x86_64-with-glibc2.35 +* CPU version: x86_64 +* Python version: 3.10.15 (main, Oct 3 2024, 07:27:34) [GCC 11.2.0] +* MLCommons CM version: 3.4.1 + +## CM Run Command + +See [CM installation guide](https://docs.mlcommons.org/inference/install/). + +```bash +pip install -U cmind + +cm rm cache -f + +cm pull repo mlcommons@cm4mlops --checkout=b32ded2a4c3039ad16dadc734bee03dd1a97f228 + +cm run script \ + --tags=run-mlperf,inference,_r4.1-dev,_scc24-main \ + --model=sdxl \ + --framework=pytorch \ + --category=datacenter \ + --scenario=Offline \ + --execution_mode=test \ + --device=rocm \ + --quiet \ + --precision=float16 \ + --adr.mlperf-implementation.tags=_branch.multinode-test,_repo.https://github.com/zixianwang2022/mlperf-scc24 \ + --adr.mlperf-implementation.version=custom \ + --env.CM_GET_PLATFORM_DETAILS=no \ + --target_qps=1.8 +``` +*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf (CM scripts), + you should simply reload mlcommons@cm4mlops without checkout and clean CM cache as follows:* + +```bash +cm rm repo mlcommons@cm4mlops +cm pull repo mlcommons@cm4mlops +cm rm cache -f + +``` + +## Results + +Platform: aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main + +Model Precision: fp32 + +### Accuracy Results +`CLIP_SCORE`: `15.22477`, Required accuracy for closed division `>= 31.68632` and `<= 31.81332` +`FID_SCORE`: `84.24318`, Required accuracy for closed division `>= 23.01086` and `<= 23.95008` + +### Performance Results +`Samples per second`: `0.847525` diff --git a/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/accuracy_console.out b/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/accuracy_console.out new file mode 100644 index 0000000..7882180 --- /dev/null +++ b/open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/accuracy_console.out @@ -0,0 +1,12 @@ +INFO:main:Namespace(sut_server=['http://10.0.0.14:8008', 'http://10.0.0.12:8008'], dataset='coco-1024', dataset_path='/root/CM/repos/local/cache/61dd835801c542a3/install', profile='stable-diffusion-xl-pytorch', scenario='Offline', max_batchsize=1, threads=1, accuracy=True, find_peak_performance=False, backend='pytorch', model_name='stable-diffusion-xl', output='/root/CM/repos/local/cache/d549713c4a534705/test_results/aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-main/stable-diffusion-xl/offline/accuracy', qps=None, model_path='/root/CM/repos/local/cache/c4b6bbbebe504f28/stable_diffusion_fp16', dtype='fp16', device='cuda', latent_framework='torch', mlperf_conf='mlperf.conf', user_conf='/root/CM/repos/mlcommons@cm4mlops/script/generate-mlperf-inference-user-conf/tmp/6626c9658bff4d2291e3121038a4cfca.conf', audit_conf='audit.config', ids_path='/root/CM/repos/local/cache/61dd835801c542a3/install/sample_ids.txt', time=None, count=10, debug=False, performance_sample_count=5000, max_latency=None, samples_per_query=8) +WARNING:backend-pytorch:Model path not provided, running with default hugging face weights +This may not be valid for official submissions +Keyword arguments {'safety_checker': None} are not expected by StableDiffusionXLPipeline and will be ignored. + Loading pipeline components...: 0%| | 0/7 [00:00