Skip to content

Performance

Jump to bottom
Vasiliev Evgenii edited this page Feb 28, 2024 · 59 revisions

Performance

Framework Version Archive Mode HTML XLSX Comments
Apache TVM 0.14.dev264 ZIP
TAR.GZ 		Host — XLSX Python API
Metrics:
  • FPS
Parameters:
Apache TVM 0.14.dev264 ZIP
TAR.GZ 		Docker — XLSX Python API
Metrics:
  • FPS
Parameters:
Intel® Distribution
of OpenVINO™ Toolkit		 2023.0 ZIP
TAR.GZ 		Host LINK XLSX Python API
Metrics:
  • FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
MXNet 1.9.1 ZIP
TAR.GZ 		Host — XLSX Python API
Metrics:
  • FPS
MXNet 1.9.1 ZIP
TAR.GZ 		Docker — XLSX Python API
Metrics:
  • FPS
ONNX Runtime 1.14.0 ZIP
TAR.GZ 		Host — XLSX Python API
Metrics:
  • FPS
ONNX Runtime 1.14.0 ZIP
TAR.GZ 		Docker — XLSX Python API
Metrics:
  • FPS
Intel® Distribution
of OpenVINO™ Toolkit		 2022.3 ZIP
TAR.GZ 		Docker LINK XLSX Python API
Metrics:
  • FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 2022.2 ZIP
TAR.GZ 		Docker LINK XLSX Python API
Metrics:
  • Batch FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 2022.2 ZIP
TAR.GZ 		Host LINK XLSX Python API
Metrics:
  • Batch FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 2022.1 ZIP
TAR.GZ 		Docker LINK XLSX Python API
Metrics:
  • Batch FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 2021.4 ZIP
TAR.GZ 		Docker LINK XLSX Python API
Metrics:
  • Batch FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 2020.2 ZIP
TAR.GZ 		Docker Public Models
Intel Models 		— Python API
Metrics:
  • Batch FPS
Parameters:
  • Default number of threads in the latency mode
  • Optimal number of streams in the throughput mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 R3 ZIP
TAR.GZ 		Docker LINK — Python API
Metrics:
  • Batch FPS
Parameters:
  • Default number of threads in the sync mode
  • Optimal number of streams in the async mode, number of streams equals number of requests
Intel® Distribution
of OpenVINO™ Toolkit		 R5.01 ZIP
TAR.GZ 		Host LINK — Python API
Metrics:
  • Latency (for the sync mode) and average time of the single pass (for the async mode)
  • Batch FPS
Parameters:
  • Number of threads in the sync mode is written in the table header ("Thread count")
  • Number of requests and streams in the async mode is written in the table header ("Async request count" and "Stream count")

Note: DLI supports two execution modes. The first one is an execution in Docker container (it is named Docker in the table), and the second one is a direct execution on the host (it is named Host in the table).

Clone this wiki locally