ch4: use am_tag_{send,recv} in RMA get/put #7202
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
test:mpich/ch4/ucx |
Move the wrapper into mpidig.h so we can use it in other paths. Rename the interface to MPIDIG_can_do_tag(bool is_local).
Rearrange the branches for cleaner code and prepare for the next patch.
When target reply data to origin get, use am_tag_send if available.
Move the code that unflatens the datatype from MPIDIG_put_data_target_msg_cb to put_dt_target_cmpl_cb. This allows us to post tag_am_recv in put_dt_target_cmpl_cb.
In the MPIDIG_PUT_DT_REQ protocol, use am_tag_{send,recv} when
available.
Potentially I should squash this...
When the datatype is a struct with mixed elements, we set its basic_type to MPI_DATATYPE_NULL. This certainly is not ideal. But for now, lets avoid divide by 0 error in MPIR_Type_get_basic_type_elements and simply return 0. MPIR_Type_get_basic_type_elements is used in the OFI receive complete event to check whether we received partial elements. We'll skip the error checking for struct types for now.
We need call MPID_Type_commit_hook for unflattened datatypes, or UCX
won't able to send or receive an unflattened datatypes. Previously,
unflattened datatypes are only used in RMA AM and they are handled in MPIDIG.
Now with am_tag_{send,recv}, we are directly using UCX to send/recv such
unflattened types.
Without explicit 1ULL, C will default it to int, resulting overflows.
We need set is_local in a requests inside the vci critical section or race condition may happen. Only the send requests that may go into RNDV active messages need it set. This fixes the occasional failures in am-only threads/pt2pt/multisend2 test.
|
test:mpich/ch4/most |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
Use am_tag_{send,recv} in RMA get/put.
This potentially works around the issue #7118
[skip warnings]
TODO
Author Checklist
Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
Commits are self-contained and do not do two things at once.
Commit message is of the form:
module: short descriptionCommit message explains what's in the commit.
Whitespace checker. Warnings test. Additional tests via comments.
For non-Argonne authors, check contribution agreement.
If necessary, request an explicit comment from your companies PR approval manager.