Error Code 2: Internal Error (Assertion !mValueMapUndo failed. )（Trt10.5/10.7 trtexec convert onnx model failed)

[TigerSong]

https://forums.developer.nvidia.com/t/trt10-5-10-7-trtexec-convert-onnx-model-failed-error-code-2-internal-error-assertion-mvaluemapundo-failed/317205

[asfiyab-nvidia]

@TigerSong please provide the ONNX model and the trtexec command used so we can investigate. You can upload models less than 2GB in size to this issue

[TigerSong]

@TigerSong please provide the ONNX model and the trtexec command used so we can investigate. You can upload models less than 2GB in size to this issue

@asfiyab-nvidia
https://bucket-standard-chatdata.bj.bcebos.com/temp/model.zip
model.onnx in model.zip. Thanks.

[TigerSong]

This is a multi-task audio model(4 sub-models, all have sequence input). I do the following things to make partial model works.

1. I devide the model into pieces
2. fix the input_ids's 2nd shape(from None -> 80)
3. use builderOptimizationLevel=4/5(3 will failed)

After doing these things, 3/4 model convert(trtexec) successfully.

BUT, the inference outputs has different with the original tf model. That's why i try to downgrade builderOptimizationLevel to 3, but failed.

IF I SET builderOptimizationLevel=3, I get this

[12/30/2024-17:42:43] [E] Error[9]: Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [myelin_graph.h:attachExceptionMsgToGraph:835] MyelinCheckException: bb.cpp:138: CHECK(op->parent() == this || op->parent() == nullptr) failed. 
[12/30/2024-17:42:43] [V] [TRT] {ForeignNode[ReverseSequence__3352__3353...ReverseSequence__3076]} (Myelin[0x80000023]) profiling completed in 0.930368 seconds. Fastest Tactic: 0xd15ea5edd15ea5ed Time: inf
[12/30/2024-17:42:43] [E] Error[10]: IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[ReverseSequence__3352__3353...ReverseSequence__3076]}.)