Does the plug-in code implemented in cubin such as FlashAttention support the orin platform?

[lzymmm123]

I see that the x86 platform trtexec reads the fMHA plugin, but the trtexec running on orin does not have the plugin。

ON x86 Tensorrt 8.5.2.2

ON orin Tensorrt 8.5.2.2

[shiyu-ren]

I also want to use attn plugin in orin platform. But in trt 8.5.2.2, attn plugins include multiHeadCrossAttentionPlugin and multiHeadFlashAttentionPlugin don't support orin which is compute capability 8.7. I would like to ask if there is a way to modify it to use the plugins on Orin, or only add plugin manually to make flash attention work?

[lix19937]

@lzymmm123 Because orin gpu SM is 87, not in
{#if defined(ENABLE_SM75) || defined(ENABLE_SM80) || defined(ENABLE_SM86) || defined(ENABLE_SM89)}

@shiyu-ren This plugin (multiHeadCrossAttentionPlugin and multiHeadFlashAttentionPlugin trt 8.5) only supports GPUs with compute capability 8.0, 8.6 and 8.9. If you want to run on orin sm87, you should write the cuda kernel by yourself.
https://github.com/NVIDIA/TensorRT/tree/release/8.5/plugin/multiHeadFlashAttentionPlugin