v0.3.0 Linux GPU: TensorRT

New Features: NVIDIA GPU Inference support via TensorRT

🎉🎉TensorRT: Boost inference performance with NVIDIA GPU via TensorRT.

Run bash ./build.sh tensorrt to build lite.ai.toolkit with TensorRT support, and then test yolov5 with the codes below. NOTE: lite.ai.toolkit need TensorRT 10.x (or later) and CUDA 12.x (or later). Please check build.sh, tensorrt-linux-x86_64-install.zh.md, test_lite_yolov5.cpp and NVIDIA/TensorRT for more details.

// trtexec --onnx=yolov5s.onnx --saveEngine=yolov5s.engine
auto *yolov5 = new lite::trt::cv::detection::YOLOV5(engine_path);

Class	Class	Class	Class	Class	System	Engine
✅YOLOv5	✅YOLOv6	✅YOLOv8	✅YOLOv8Face	✅YOLOv5Face	Linux	TensorRT
✅YOLOX	✅YOLOv5BlazeFace	✅StableDiffusion	/	/	Linux	TensorRT

What's Changed

• Add Yoloface-8n by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/412
• [README] Update authors by @DefTruth in https://github.com/DefTruth/lite.ai.toolkit/pull/413
• 增加Linux下的TensorRT的安装文档 by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/415
• update first tensortt inference example by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/416
• Update TensorRT BasicHandler and add yolov5 cpp file by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/417
• add yolox implement by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/418
• [TensorRT] support YOLOv8 with TensorRT backend by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/419
• [TensorRT] support YOLOv6 with TensorRT backend by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/420
• [TensorRT] support YOLOV5Face with TensorRT backend by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/421
• [TensorRT] fix undefine bug in yolov5face trt version by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/422
• [onnxruntime] Clip onnx update by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/424
• [Misc] Remove un-need comments by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/425
• [TensorRT] Clip TensorRT code update by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/426
• Update README.md by @DefTruth in https://github.com/DefTruth/lite.ai.toolkit/pull/430
• [TensorRT] [ORT] support Stable Diffusion text2img Pipeline by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/432
• [Bugfix] Fix some build errors for SD by @DefTruth in https://github.com/DefTruth/lite.ai.toolkit/pull/434
• [Bugfix] Fix OrtCompatiableGetInputName errors by @DefTruth in https://github.com/DefTruth/lite.ai.toolkit/pull/435
• [Fix Bug] remove duplicate code by @wangzijian1010 in https://github.com/DefTruth/lite.ai.toolkit/pull/436
• Bump up to v0.3.0 by @DefTruth in https://github.com/DefTruth/lite.ai.toolkit/pull/438

New Contributors

• @wangzijian1010 made their first contribution in https://github.com/DefTruth/lite.ai.toolkit/pull/412

Full Changelog: DefTruth/lite.ai.toolkit@v0.2.0...v0.3.0-rc1