Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crashes when executing model quantification on Deeplabv3 #23985

Open
EzioQR opened this issue Mar 11, 2025 · 0 comments
Open

Crashes when executing model quantification on Deeplabv3 #23985

EzioQR opened this issue Mar 11, 2025 · 0 comments
Labels
quantization issues related to quantization

Comments

@EzioQR
Copy link

EzioQR commented Mar 11, 2025

Describe the issue

I want to perform INT8 precision inference on the Deeplabv3 model on the CPU. I first quantified the model, but during execution, I threw NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for ConvInteger(10) node with name '/resnet/conv1/Conv_quant' Error. The model file is shown in the attachment (https://drive.google.com/file/d/10QhV6_lqoD4nnGx3a4HJV-zA7bQJ86E5/view?usp=drive_link).

To reproduce

import onnxruntime as ort
from onnxruntime.quantization import quantize_dynamic, QuantType
import numpy as np

session = ort.InferenceSession("./deeplabv3.onnx",
providers=["CPUExecutionProvider"],graph_optimization_level=ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED)

quantized_model = quantize_dynamic("./deeplabv3.onnx", "./deeplabv3_quantized.onnx", weight_type=QuantType.QInt8)

input_data = np.random.rand(2, 3, 513, 513).astype(np.float32)
input_data_int8 = input_data.astype(np.int8)
inputs = {session.get_inputs()[0].name: input_data}
outputs = session.run(["infer_output"], inputs)

session_int8 = ort.InferenceSession("../deeplabv3_quantized.onnx", providers=["CPUExecutionProvider"])

outputs_int8 = session_int8.run(["infer_output"], quantized_model)

Expected behavior
Execute normally.

Urgency

No response

Platform

Linux

OS Version

(Linux Ubuntu 20.04)

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.17.0)

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

@github-actions github-actions bot added the quantization issues related to quantization label Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
quantization issues related to quantization
Projects
None yet
Development

No branches or pull requests

1 participant