Parallel Grid loop creation for gemms

[KavithaTipturMadhu]

No description provided.

[rengolin]

Awesome! I'm seeing 38% improvement on mlir-gen BF16 16-threads (smaller improvements with less threads), and 22% for our PyTorch models BF16 16-threads, and no reductions in single-threaded benchmarks. I'll look at the code soon, but at least now we can run some benchmarks in parallel.

[alheinecke]

BTW: I cannot build this PR:

/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/../../../../bin/ld: lib/libTPPPipeline.a(DefaultPipeline.cpp.o): in function (anonymous namespace)::DefaultPipeline::constructPipeline()': /../lib/TPP/DefaultPipeline.cpp:159: undefined reference to mlir::tpp::createSCFParallelLoopTiling(mlir::tpp::SCFParallelLoopTilingOptions const&)'
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

[alheinecke]

I could make it compile, after making it a default TPP pass
alheinecke@f97604c

[rengolin]

I could make it compile, after making it a default TPP pass alheinecke@f97604c

This looks like the right fix. Adding it to the transform dialect is the wrong direction.

I'm not sure why the CI can build it and you can't. Perhaps a clean build would works? Regardless, your patch is the right direction anyway.

[alheinecke]

I did a fully clean build... and following the instructions from our ReadME (have them in a script, where I just set the branch name and origin). It seems something went sideways that pass was in the LinAlgXTransform library?!

[rengolin]

I did a fully clean build... and following the instructions from our ReadME (have them in a script, where I just set the branch name and origin). It seems something went sideways that pass was in the LinAlgXTransform library?!

It was, wrong place. Not sure how the CI build it either. I get this error with the PR:

lib/TPP/Dialect/Transform/SCFParallelLoopTiling.cpp:266:9: error: ignoring return value of function declared with 'nodiscard' attribute [-Werror,-Wunused-result]
        tileParallelLoop(ploop, tileSizes, noMinMaxBounds);
        ^~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

But that's unrelated to your error / fix, I think.

[KavithaTipturMadhu]

I did a fully clean build... and following the instructions from our ReadME (have them in a script, where I just set the branch name and origin). It seems something went sideways that pass was in the LinAlgXTransform library?!

It was, wrong place. Not sure how the CI build it either. I get this error with the PR:

lib/TPP/Dialect/Transform/SCFParallelLoopTiling.cpp:266:9: error: ignoring return value of function declared with 'nodiscard' attribute [-Werror,-Wunused-result]
        tileParallelLoop(ploop, tileSizes, noMinMaxBounds);
        ^~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

But that's unrelated to your error / fix, I think.

Alex has fixed the return value of the pass to fix this error.

[rengolin]

The code itself is very confusing, with manual copying the basic block, #if 0 and dead variables, but as discussed, it should be fine until we gather more examples and understand all the behaviour we want form it, including the dynamic case.

See if you can add some comment to the confusing parts trying to explain what you were trying to do or didn't do because of another problem (for example, the basic block copy), so that we can more easily read it later and fix it.

[KavithaTipturMadhu]

The code itself is very confusing, with manual copying the basic block, #if 0 and dead variables, but as discussed, it should be fine until we gather more examples and understand all the behaviour we want form it, including the dynamic case.

See if you can add some comment to the confusing parts trying to explain what you were trying to do or didn't do because of another problem (for example, the basic block copy), so that we can more easily read it later and fix it.

I have added a couple of comments for now and gotten rid of #if 0 block and the dead code.