Hyperelastic RMT code with updated modular precision

[mrodrig6]

Type of change

• New feature (non-breaking change which adds functionality)
• Something else

Scope

• This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Tested the code on NCSA Delta and passed the full test suite and most example cases.

Provide instructions so we can reproduce.
Ran on 2 GPUs with one CPU each on an interactive node
./mfc.sh test -j 8 --max-attempt 3 -- -c delta

Test Configuration:

• What computers and compilers did you use to test this:

Checklist

• I have added comments for the new code
• I added Doxygen docstrings to the new code
• I have made corresponding changes to the documentation (docs/)
• I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
• I have added example cases in examples/ that demonstrate my new feature performing as expected.
  They run to completion and demonstrate "interesting physics"
• I ran ./mfc.sh format before committing my code
• New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
• This PR does not introduce any repeated code (it follows the DRY principle)
• I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)

To make sure the code is performing as expected on GPU devices, I have:

• Checked that the code compiles using NVHPC compilers
• Checked that the code compiles using CRAY compilers
• Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
• Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
• Enclosed the new feature via nvtx ranges so that they can be identified in profiles
• Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
• Ran an Omniperf profile using ./mfc.sh run XXXX --gpu -t simulation --omniperf, and have attached the output file and plain text results to this PR.
• Ran my code using various numbers of different GPUs (1, 2, and 8, for example) in parallel and made sure that the results scale similarly to what happens if you run without the new code/feature

[sbryngelson]

@ChrisZYJ I merged your PR if you could try to help get those features back into this one, it would be highly appreciated!

[ChrisZYJ]

I can first help merge that PR. @mrodrig6 Could you please give me the access to edit? Thanks!

[ChrisZYJ]

I’ve identified the issue. In the original code, hypoelastic calculations defaulted to 4th-order finite differencing regardless of input file parameters. However, the updated code adjusts the order based on the fd_order parameter.

Initially, fd_order was intended solely as a post-processing parameter for vorticity and numerical Schlieren calculations. It didn't affect the simulations, as stated in the documentation and case file comments. In the #771 case file, fd_order was set to 1. With the change in default behavior, the simulation now uses first-order finite differencing, which is less stable and led to the observed stability issues.

@sbryngelson, could you please revert my previous PR? The simulation results have always been correct, and the discrepancy came from the change in default behavior of hypoelastic calculations. Sorry for my oversight. I can update the documentation later to clarify the role of fd_order.

[sbryngelson]

I’ve identified the issue. In the original code, hypoelastic calculations defaulted to 4th-order finite differencing regardless of input file parameters. However, the updated code adjusts the order based on the fd_order parameter.

Initially, fd_order was intended solely as a post-processing parameter for vorticity and numerical Schlieren calculations. It didn't affect the simulations, as stated in the documentation and case file comments. In the #771 case file, fd_order was set to 1. With the change in default behavior, the simulation now uses first-order finite differencing, which is less stable and led to the observed stability issues.

@sbryngelson, could you please revert my previous PR? The simulation results have always been correct, and the discrepancy came from the change in default behavior of hypoelastic calculations. Sorry for my oversight. I can update the documentation later to clarify the role of fd_order.

I created a PR for this: #776

@ChrisZYJ can you look it over to be sure it's correct and then approve it and I will merge. Thanks!

[mrodrig6]

I can first help merge that PR. @mrodrig6 Could you please give me the access to edit? Thanks!

@ChrisZYJ Yes, and I added you now.

[mrodrig6]

@mrodrig6 Could you please take a look at issue #771? It would be very helpful if you could identify the error and implement a quick fix. If not, I'm happy to help revert your previous PR's changes to the hypoelasticity module - it seems that would solve the problem. (It appears your previous PR changed how finite difference coefficients are computed, but didn't add new functionality for hypoelasticity. Please correct me if I'm wrong.)

@ChrisZYJ Yes and done. I responded earlier this week to this comment. I agree with @sbryngelson that this is resolved

[mrodrig6]

I’ve identified the issue. In the original code, hypoelastic calculations defaulted to 4th-order finite differencing regardless of input file parameters. However, the updated code adjusts the order based on the fd_order parameter.
Initially, fd_order was intended solely as a post-processing parameter for vorticity and numerical Schlieren calculations. It didn't affect the simulations, as stated in the documentation and case file comments. In the #771 case file, fd_order was set to 1. With the change in default behavior, the simulation now uses first-order finite differencing, which is less stable and led to the observed stability issues.
@sbryngelson, could you please revert my previous PR? The simulation results have always been correct, and the discrepancy came from the change in default behavior of hypoelastic calculations. Sorry for my oversight. I can update the documentation later to clarify the role of fd_order.

I created a PR for this: #776

@ChrisZYJ can you look it over to be sure it's correct and then approve it and I will merge. Thanks!

I agree with this causing the issue and good to know.

[sbryngelson]

So if I understand correctly, this PR is waiting on tests and then it will be ready for code review? (of course I can review it earlier as well)

[sbryngelson]

@mrodrig6 please add tests and whatever else is needed to merge this PR. @ChrisZYJ can you comment on it?

[ChrisZYJ]

@sbryngelson Of course! Happy to help

[ChrisZYJ]

@mrodrig6 I can help make a couple of minor changes. Also, if you'd like to convert an example into a test suite, I can do that - please just let me know which examples would cover all the added code. Sorry that your invitation has expired; could you please invite me to your repository again? Thanks!

Also, I see you removed the checker that prohibits hypoelasticity with HLLC. Do both hypoelasticity and hyperelasticity now work with both HLL and HLLC?