Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracker] All the issue related with ONNX model zoo models #886

Open
pdhirajkumarprasad opened this issue Nov 20, 2024 · 2 comments
Open

Comments

@pdhirajkumarprasad
Copy link

pdhirajkumarprasad commented Nov 20, 2024

We have enabled total 2275 models from ONNX model zoo. This issue is to track all the issues seen in the models.

Passing Summary

TOTAL TESTS = 2275

Stage # Passing % of Total % of Attempted
Setup 2274 100.0% 100.0%
IREE Compilation 2175 95.6% 95.6%
Gold Inference 2175 95.6% 100.0%
IREE Inference Invocation 2170 95.4% 99.8%
Inference Comparison (PASS) 2130 93.6% 98.2%

Fail Summary

TOTAL TESTS = 2275

Stage # Failed at Stage % of Total
Setup 1 0.0%
IREE Compilation 99 4.4%
Gold Inference 0 0.0%
IREE Inference Invocation 5 0.2%
Inference Comparison 40 1.8%

Detail categorization is present here

CPU

# issue type issue no #model impacted list of model assignee status
1 failed to legalize operation 'torch.operator' -> QLinearConv 894 21 QLinearConv Torch MLIR (Review) - 3917
2 failed to legalize operation 'torch.constant.float 908 15 Roi Align Torch MLIR (In Progress)
3 Assertion `cast(expr.getRHS()).getValue() > 0 && "nonpositive multiplying coefficient 20110 10 Tile Issue IREE - Unassigned
4 failed to legalize operation 'onnx.Loop' 948 7 Loop Torch MLIR - In Progress
5 error: 'iree_linalg_ext.scan' op expected type of operand #1 ('tensor<1x128xi64>') to match type of corresponding result ('tensor<1x?xi64>') 17526 6 Scan IREE - In Progress
6 Assertion `result && ""Zero-sized vectors are not allowed""' failed.\n' 913 6 Zero Torch MLIR - Unassigned
7 expected offsets to be non-negative, but got -1 19935 6 Offsets IREE - Unassigned
8 Incorrect imported IR 905 6 Incorrect IR Torch MLIR - Unassigned
9 failed to legalize operation 'torch.operator' -> AvgPool 895 5 AvgPool Torch MLIR (Review) - 3923
10 'tensor.cast' op operand type 'tensor<80x15130xf32>' and result type 'tensor<15130xf32>' are cast incompatible 891 3 NMS Torch MLIR - Review 3999, IREE - Assigned19474
11 error: invalid size -9223372036854775808 for !torch.tensor type 3 Invalid
12 failed to legalize operation 'torch.operator' -> MatMulInteger 906 2 MatmulInteger Torch MLIR (Review) - 4036
13 error: expected type to be 'tensor<1x?x1xi64>' or a rank-reduced version. (size mismatch) 2 Rank Mismatch
14 failed to legalize operation "onnx.CategoryMapper" 910 1 CategoryMapper
15 failed to legalize operation 'torch.operator' DequantizeLinear 914 1 DequantizeLinear Torch MLIR - Unassigned
16 failed to legalize operation 'torch.operator' QLinearMatMul 916 1 QLinearMatMul Torch MLIR - Unassigned
17 'hal.tensor.barrier' op failed to verify that all of {sources, results} have same type 1 Barrier
@pravg-amd
Copy link

Latest categorization of the errors is updated here

@pravg-amd
Copy link

Models failing at setup stage are disabled as part of the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants