System Info

Jetson Orin AGX 64G
SM 8.7
cargo 1.75.0 (1d8b05cdd 2023-11-20)

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

1. curl https://sh.rustup.rs -sSf | sh
2. sudo apt-get install libssl-dev gcc -y
3. git clone https://github.com/huggingface/text-embeddings-inference.git
4. cd text-embeddings-inference
5. cargo install --path router -F candle-cuda -F http --no-default-features
6. model=BAAI/bge-reranker-large
7. text-embeddings-router --model-id $model --port 8080

Expected behavior

Could not start Candle backend: Could not start backend: Runtime compute cap 87 is not compatible with compile time compute cap 87
Error: Could not create backend

Caused by:
    Could not start backend: Could not start a suitable backend