romio: implement binding generation for ROMIO #7228
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hzhou
force-pushed
the
2308_romio_binding
branch
14 times, most recently
from
966f859 to
d00e9f7
Compare
|
test:mpich/ch3/tcp ✔️ test:mpich/custom |
hzhou
force-pushed
the
2308_romio_binding
branch
2 times, most recently
from
35655c4 to
3c43546
Compare
|
test:mpich/custom |
hzhou
force-pushed
the
2308_romio_binding
branch
from
3c43546 to
5ac180d
Compare
These looks like vendor-specific code. We can easily generate them if needed. Remove them for now to simplify refactor.
All romio error check macros takes a myname parameter (which can be replaced by __func__). Make MPIO_CHECK_INFO_ALL consistent.
Refactor to split interface and implementation. This prepares for binding generations. The binding layer, e.g. open.c, close.c, etc. will be replaced by python generation when ROMIO is building FROM_MPICH.
Add impl functions MPI_File_f2c and MPI_File_c2f for consistency and to avoid exposing MPIOI functions. They will be used when we generate IO bindings from MPICH and are not used if we build ROMIO outside MPICH.
Use "goto fn_fail" instead of "goto fn_exit". Skip MPIO_Err_return_file since we will do that in top-level routines.
We call MPI_File_delete in case there is a leftover testfile, which will interfere with later MPI_File_open. Not checking error return and call "delete" on a seemingly non-existent file is myterious. Add a comment and error checking for maintenance.
hzhou
force-pushed
the
2308_romio_binding
branch
from
5ac180d to
3c5dc12
Compare
Need double cast to suppress warnings when converting between pointers and integers. Also remove an unused macro, HAVE_INT_LT_POINTER.
Define the complete MPI-IO interface in mpi.h. In all_romio_symbols.c, we can use MPI_Request instead of MPIO_Reuqest since MPIO_Request is not longer defined in mpi.h (since it no longer includes mpio.h).
Add io_api.txt and generate io bindings
This function is now defined in src/binding/c/io_api.txt.
We don't have error checking for async_thing, amode, whence.
Wrap the code that generates c_bindings.c in a function. This prepares for later adding io binding generations. ROMIO bindings will be thinner than non-IO functions. So we will a separate function to generate IO bindings.
Refactor code to function dump_body_of_routine. This is a preparation for generating io bindings. Since ROMIO is designed on top of the rest of MPI, we will handle differently for parameter filtering and error checking, but the part that handles large count and calling the internal impl functions can be shared.
MPICH will generate the binding layer. Now that libromio just supply internal code, we no longer need worry about libromio vs libpromio. We don't separate man pages and include mpio.h in mpi.h either.
We no longer have any convenience libraries that need go into libmpi.so when libpmpi.so is also built. Thus, the legacy mpi_convenience_libs variable is no longer in use. However, we need differentiate libromio from the other convenience libraries for the ABI build. ROMIO depends on MPI, thus will need separate version for ABI and non-ABI build. Thus, we create two variables -- mpi_convenience_libs and abi_convenience_libs. Note: the former reuses the name but now with different meanings.
hzhou
force-pushed
the
2308_romio_binding
branch
from
3c5dc12 to
06fc378
Compare
Generate io bindings including prototypes in mpi_proto.h and mpir_impl.h. src/binding/c/io.c caintains all the IO binding functions that calls the corresponding _impl functions. But it is not included in the Makefile yet. Dump the prototypes for io impl functions separately to allow inserting necessary missing declarations such as MPIR_Ext_cs_enter/exit. IO functions need use MPIR_Ext_cs_enter/exit for global cs. Now it gets more complicated, refactor into dump_global_cs_enter/exit for better readability.
Because ROMIO is built on top of the rest of MPI, it is difficult to use the abi wrappers because we need either - a. convert from ABI into MPICH abi and then convert back to ABI before calling into ROMIO. This seems silly. Or - b. build ROMIO on MPICH abi. But in libmpi_abi.so, MPICH abi is not available. Thus we need - c. Switch ROMIO to call MPICH impl functions instead of `MPI/PMPI` functions. We'll need to build a layer in ROMIO switch between using `MPI/PMPI` or using `MPIR_Xxx_impl` depending on build options. This will result in ROMIO having full access to MPICH internals! But let's delay work until we have a convincing need. Thus, this commit implements io abi bindings without the abi wrappers. We implemented a separate path rather than making the existing code more complex.
hzhou
force-pushed
the
2308_romio_binding
branch
from
06fc378 to
a4c17e9
Compare
|
test:mpich/ch3/tcp test:mpich/custom |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
[note: split preparation PR in #7232]
Integrate ROMIO into MPICH's binding generation framework. This will unify and simplify the IO binding tasks such as large count support, fortran support, man page support.
We'll retain the current stand-alone nature of ROMIO as much as we can in this PR. There shouldn't be any noticeable difference for users who build ROMIO separate from MPICH.
MPI_File_open) into a binding layer and an implementation layer --MPI_File_openinopen.candMPI_File_open_implinio_impl.cFROM_MPICHdefined stays exactly the samesrc/binding/c/io.c, which calls into theimpllayer that is defined in ROMIO.mpl.#include "mpio.h"inmpi.hAuthor Checklist
Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
Commits are self-contained and do not do two things at once.
Commit message is of the form:
module: short descriptionCommit message explains what's in the commit.
Whitespace checker. Warnings test. Additional tests via comments.
For non-Argonne authors, check contribution agreement.
If necessary, request an explicit comment from your companies PR approval manager.