[Mobile] QNN GPU backend crashes.

[monokim]

Describe the issue

I am trying to use ONNXruntime with QNN execution provider on a Qualcomm Android device Qualcomm® QCS610 which supports GPU for QNN.
I have tested platform validators from the device and no problem with QNN with GPU backend.

QNN is supported for backend GPU on the device.
*********** Results Summary ***********
Backend = GPU
{
  Backend Hardware  : Supported
  Backend Libraries : Found
  Library Version   : OpenCL 2.0 Adreno(TM) 612
  Core Version      : Adreno(TM) 608
  Unit Test         : Passed
}

And I have used a mobilenet model from Qualcomm as quantization is not needed

Non-quantized models files use 32 bit floating point representations of network parameters. 
...
GPU - Choose a non-quantized model. Quantized models are currently incompatible with the GPU backend.

(https://huggingface.co/qualcomm/MobileNet-v2/tree/main)

and which has float32 in/output

---

Configuration is okay.
All nodes are placed well without failing validation.

But when I run on the GPU backend(libQnnGpu.so), it crashes with error code 6999.
(on CPU backend libQnnCpu.so, it works fine)

// Configuration
11:19:21.297  V   [V:onnxruntime:, session_state.cc:1148 VerifyEachNodeIsAssignedToAnEp] Node placements
11:19:21.297  V   [V:onnxruntime:, session_state.cc:1151 VerifyEachNodeIsAssignedToAnEp]  All nodes placed on [QNNExecutionProvider]. Number of nodes: 1
11:19:21.298  V   [V:onnxruntime:, session_state.cc:128 CreateGraphInfo] SaveMLValueNameIndexMapping
11:19:21.299  V   [V:onnxruntime:, session_state.cc:174 CreateGraphInfo] Done saving OrtValue mappings.
11:19:21.299  I   [I:onnxruntime:, allocation_planner.cc:2567 CreateGraphPartitioner] Use DeviceBasedPartition as default
11:19:21.302  I   [I:onnxruntime:, session_state_utils.cc:276 SaveInitializedTensors] Saving initialized tensors.
11:19:21.316  I   [I:onnxruntime:, session_state_utils.cc:427 SaveInitializedTensors] Done saving initialized tensors
11:19:21.316  V   [V:onnxruntime:, qnn_execution_provider.cc:807 operator()] compute_info.create_state_func context->node_name: QNNExecutionProvider_QNN_13609413732656985736_1_0
11:19:21.317  I   [I:onnxruntime:, inference_session.cc:2106 Initialize] Session successfully initialized.

// Inference
11:27:31.135  V   [V:VLOG0:, sequential_executor.cc:536 ExecuteThePlan] Number of streams: 1
11:27:31.135  V   [V:VLOG0:, sequential_executor.cc:184 SessionScope] Begin execution
11:27:31.136  V   [V:onnxruntime:, qnn_model.cc:189 ExecuteGraph] QnnModel::ExecuteGraphs
11:27:31.136  V   [V:onnxruntime:, qnn_model.cc:208 ExecuteGraph] model_input = image_tensor index = 0
11:27:31.136  V   [V:onnxruntime:, qnn_model.cc:212 ExecuteGraph] Qnn tensor size: 602112Ort tensor size: 602112
11:27:31.136  V   [V:onnxruntime:, qnn_model.cc:227 ExecuteGraph] model_output = class_logits index = 0
11:27:31.136  V   [V:onnxruntime:, qnn_model.cc:232 ExecuteGraph] Qnn tensor size: 4000Ort tensor size: 4000
11:27:31.136  V   [V:onnxruntime:, qnn_model.cc:242 ExecuteGraph] Start execute QNN graph:QNNExecutionProvider_QNN_13609413732656985736_1_0

11:27:31.136  E   [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running QNN_13609413732656985736_1 node. Name:'QNNExecutionProvider_QNN_13609413732656985736_1_0' Status Message: QNN graph execute error. Error code: 6999
11:27:31.136  E  terminating with uncaught exception of type Ort::Exception: Non-zero status code returned while running QNN_13609413732656985736_1 node. Name:'QNNExecutionProvider_QNN_13609413732656985736_1_0' Status Message: QNN graph execute error. Error code: 6999

I have set log severity 0 and level verbose, but the above is the only meaningful log when it crashes.

I have tested several models but the results were the same.

11:53:50.640  E   [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running QNN_3988460305798194574_24 node. Name:'QNNExecutionProvider_QNN_3988460305798194574_24_3' Status Message: QNN graph execute error. Error code: 6999
11:53:50.641  E  terminating due to uncaught exception of type Ort::Exception: Non-zero status code returned while running QNN_3988460305798194574_24 node. Name:'QNNExecutionProvider_QNN_3988460305798194574_24_3' Status Message: QNN graph execute error. Error code: 6999

The node's number doesn't matter, it always crashes from the first node placed on the QnnExecution(GPU) provider, not the CPU provider for the unsupported functions as a fallback.
(like CPU nodes: 1-23, GPU nodes: 24-)

I am currently doubting some reasons.

• Some vendor or system libraries need to be linked to my App explicitly for GPU usage.
  (But when I did the platform test, it automatically linked it 🤔 )
• Missing library from the onnxruntime or qairt for GPU usage.
• The version of either onnxruntime or qairt I am using doesn't support my device QCS610 which also supports hexagon-v66

Any Idea?
(a.k.a I am running it in C++ for a native Android application.)

To reproduce

Build Onnxruntime with QNN
Run with GPU execution provider backend on Qualcomm device

Urgency

No response

Platform

Android

OS Version

29

ONNX Runtime Installation

Built from Source

Compiler Version (if 'Built from Source')

clang 17.0.2

Package Name (if 'Released Package')

None

ONNX Runtime Version or Commit ID

Rel-1.20.1

ONNX Runtime API

C++/C

Architecture

X64

Execution Provider

Other / Unknown

Execution Provider Library Version

QNN (qairt 2.28.2.241116)