Onnxruntime Initialization Issue with silero-vad on Jetson

[wskurniawan]

Hello everyone,

I'm encountering an issue while trying to run silero-vad using onnxruntime on my Jetson device. I hope to get some assistance or advice from this community.

Problem

When running the code, I get the following error:

/opt/rh/gcc-toolset-12/root/usr/include/c++/12/bits/stl_vector.h:1123: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned int; _Alloc = std::allocator<unsigned int>; reference = unsigned int&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.

This error occurs during the initialization of the InferenceSession.

Code Used

Here's the snippet of code I'm using to initialize the InferenceSession:

opts = onnxruntime.SessionOptions()
opts.inter_op_num_threads = 1
opts.intra_op_num_threads = 1
opts.log_severity_level = 4

providers = ["CPUExecutionProvider"]

self.session = onnxruntime.InferenceSession(
    path,
    providers=providers,
    sess_options=opts,
) 

Environment

• Jetson AGX Orin 64GB
• JetPack: 6.0
• Python: 3.10
• onnxruntime: 1.18.0 (installed from https://elinux.org/Jetson_Zoo#ONNX_Runtime)
• silero-vad: https://github.com/snakers4/silero-vad/blob/master/src/silero_vad/data/silero_vad.onnx

What I've Tried

I've attempted to use different versions of onnxruntime:

1. Built from source for version 1.18.1
2. Built from source for version 1.20.0

However, the results remain the same, and the error persists.

Questions

1. Has anyone experienced a similar issue?
2. Is there a configuration error that I might have made?
3. Are there any suggestions or solutions I could try?

Thank you for your help and time!

[wskurniawan]

Solved, thank you.

[wen152]

I met the same problems.How to solve?

[brushknight]

I have the same problem, @wskurniawan can you share the solution please?

[altair199797]

I had the same problem and solved it by uninstalling all onnxruntime-gpu verions I had in pip. And then reinstall onnxruntime-gpu (with wheel from this website)
Reason:
I had a onnxruntime-gpu 1.19 version, which was probably used and i needed 1.11. After uninstalling 1.19 the onnxruntime-gpu 1.11 wasn't working, so I uninstalled it also and then after reinstallation everything worked.

[SteinsHead]

Solved, thank you.
I have the same problem, @wskurniawan can you share the solution please?

[yejianfengblue]

@brushknight @SteinsHead @wen152 The reason may be some cpu core is offline when initialize onnxruntime session. My machine is Ubuntu on an old Andoird phone instead of Jetson.
My temp workaround is

1. Install stress with sudo apt install stress
2. run stress --cpu 8 --timeout 500s to force all cpu cores online. Replace the cpu core number 8 with yours. Be aware that command stress is a benchmark command and make your cpu load 100%.
3. At the same time, initialize onnxruntime session.
4. Ctrl+c terminates the stress command.

Use command watch -n 0.1 "cat /sys/devices/system/cpu/cpu*/online" to see whether a cpu core is online.
Also try

1. Add write permission with command sudo chmod +w /sys/devices/system/cpu/cpu*/online
2. Use chcpu command to set cpu cores online, for example chcpu --enable 0.
   chcpu is useless on Android in my case because it would be overwritten by Android kernel later.