Repository navigation
onnxruntime optimizers fails #22512
Description
Describe the issue
The attached model succeeds when onnxruntime optimization are disabled, and fails when all of them are.
File "onnxruntime/capi/onnxruntime_inference_collection.py", line 491, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (MatMulBnFusion_Gemm) Op (Gemm) [ShapeInferenceError] First input does not have rank 2
model_levit_128-onnx_dynamo-ir-d0rt0.zip
To reproduce
import onnxruntime
onnxruntime.InferenceSession("<model.onnx>", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])Urgency
No response
Platform
Linux
OS Version
ubuntu-22.04
ONNX Runtime Installation
Built from Source
ONNX Runtime Version or Commit ID
ONNX Runtime API
Python
Architecture
X64
Execution Provider
CUDA
Execution Provider Library Version
CUDA 11.8
Activity
xadupre commentedon Nov 6, 2024
The issue is here: https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/optimizer/matmul_bn_fusion.cc#L207. A Gemm operator is introduced to fuse
Add(MatMul(X,Y), Z)intoGemm(X,Y,Z). In my case, it fails becauserank(X) == 3andrank(Y) == 2which is not allowed by this kernel.Fix MatMulBnFusion to exclude cases when tensors are not 2D tensors (#…
Fix MatMulBnFusion to exclude cases when tensors are not 2D tensors (#…
Fix MatMulBnFusion to exclude cases when tensors are not 2D tensors (m…
Fix MatMulBnFusion to exclude cases when tensors are not 2D tensors (m…
merge to 1b60209