The below tables lists the actively tests being carried out by MLCommons for the currently active MLPerf inference benchmarks. Please contact Support if you would like to add any new tests here. The tests are run using GitHub actions and the test results go to the below two repositories.
- Short Runs: https://github.com/mlcommons/mlperf_inference_test_submissions_v5.0/
- Full Runs: https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.0/
These reporitories are per inference round.
| Benchmark | CPU/GPU | Test Type |
|---|---|---|
| resnet50 | CPU | Short |
| retinanet | CPU | Short |
| 3d-unet | - | TBD |
| bert | CPU | Short |
| gptj | Both | Short |
| llama2-70b | CPU | Short |
| mixtral-8x7b | CPU | Short |
| llama3.1-405b | - | - |
| rgat | CPU | Short |
| pointpainting | GPU | - |
| stable-diffusion-xl | GPU | Short |
| dlrm_v2 | CPU | Short |
| Benchmark | Test Type |
|---|---|
| resnet50 | Full |
| retinanet | Full |
| 3d-unet | Full |
| bert | Full |
| gptj | Full |
| llama2-70b | TBD |
| mixtral-8x7b | TBD |
| llama3.1-405b | - |
| rgat | NA |
| pointpainting | NA |
| stable-diffusion-xl | Full |
| dlrm_v2 | TBD |