Larod & Parallel processing

[keesschollaart81]

Hi! Question about the way Larod works...

We use Larod both for image pre-processing and for AI Inference. We run these jobs in parallel, aiming to utilize the available hardware best. Now we notice though that Larod is only working on one job at the time, even though it regards multiple larod-devices. The ‘other’ jobs are waiting/being blocked.

Is this expected behavior? Are there any specifics we need to do to make parallel processing working?

Looking forward to your views here!

[isak-jakobsson]

Hi,

In general, it is possible to run jobs in parallel, but there are some platform specific limitations. Which larod device are you using for the pre-processing and inference respectively?

[keesschollaart81]

We are using axis-a8-dlpu-tflite and axis-a8-gpu-proc. That should be 2 different pieces of 'hardware' right? But still we see them processing sequentially.

[isak-jakobsson]

The ARTPEC-8 DLPU and GPU can't be used at the same time. However, if you use the cpu-proc or axis-ace-proc instead of axis-a8-gpu-proc for the pre-processing, it is possible to run the jobs in parallel.

Also, to get the jobs running in parallel you should use the larodRunJobAsync (which is not blocking) instead of larodRunJob.