Idea: collaborate more tightly with Git for Windows to provide a working mingw-w64-git

[dscho]

The main obstacle when I migrated Git for Windows from MSys to MSYS2 was that the MSYS2 runtime is opinionated in a way that disagrees with Git's expectations (as documented in its test suite).

That is the reason why Git for Windows ships with a fork of the MSYS2 runtime, and that in turn is the obstacle why the Git for Windows cannot trivially maintain a mingw-w64-git definition in msys2/MINGW-packages: it would not pass the test suite.

So here is an idea: try to work out how the expectations of Git can be met, maybe by making minor adjustments to the MSYS2 runtime, and once that is done, modify Git for Windows' mingw-w64-git package in a way that the identical definition can be used in msys2/MINGW-packages (one thing that would obviously need to be changed is that mingw-w64-git must not put a git-bash.exe into the root directory 😁).

Are there people interested in working with me on this project?

EDIT: Here is an analysis of the deviations between MSYS2's and Git for Windows' flavor of the MSYS2 runtime:

• git-for-windows/msys2-runtime@589fac65ff: this is a real bug fix for running find on a FAT file system and should probably go into MSYS2 as-is
• git-for-windows/msys2-runtime@3e9c003101: this is a real bug fix for setting empty-valued environment variables (think about the difference between ${var:-hello} and ${var-hello} after setting export var=). The fix might even be welcome in the Cygwin project. -> Allow empty-valued environment variables to be passed to non-MSYS programs msys2-runtime#101
• git-for-windows/msys2-runtime@1e0136ee86: this might be a big controversial, but Git's test suite needs this (and thereby suggests that this is standard Linux behavior, something Cygwin always puts above POSIX)
• git-for-windows/msys2-runtime@0b2d287629: a real bug fix that should be contributed to msys2/msys2-runtime -> Stop assuming that there are no spaces in POSIX-style paths msys2-runtime#143 runtime: add tests for pathlists with spaces msys2-tests#33
• git-for-windows/msys2-runtime@a5012b9e83: a documentation improvement that should be contributed to the Cygwin project
• git-for-windows/msys2-runtime@d2456ea67e: improving debug facilities in the MSYS2 runtime that should go into msys2/msys2-runtime -> Add a helpful debug message for posix-to-windows conversion msys2-runtime#148
• git-for-windows/msys2-runtime@78fbdbbd5e: readibility (and slight performance) improvement that should go into msys2/msys2-runtime -> Avoid unnecessary recursion in find_path_start_and_type() msys2-runtime#149
• git-for-windows/msys2-runtime@707f9b218a, git-for-windows/msys2-runtime@c312f3d3c6, git-for-windows/msys2-runtime@99a84e0081, git-for-windows/msys2-runtime@6efd127eeb, git-for-windows/msys2-runtime@4076235378, git-for-windows/msys2-runtime@12461acc8a, git-for-windows/msys2-runtime@aeb2e423d9, git-for-windows/msys2-runtime@9f3e592333, and git-for-windows/msys2-runtime@92a0032d1a: adjust the implicit Windows <-> Unix-y path conversion to accommodate common Git invocations better, i.e. accommodating expectations documented by Git's test suite.
• git-for-windows/msys2-runtime@f874ac108a: in addition to the horrible-to-use, poorly-named MSYS2_ENV_CONV_EXCL and MSYS2_ARG_CONV_EXCL variables, this commit introduces a poorly-named but at least dirt-easy-to-use MSYS_NO_PATHCONV variable to stop the MSYS2 runtime from misinterpreting command-line arguments starting with a slash as referring to an absolute Unix-y path that needs to be converted to a valid Windows one (think cmd /?).
• git-for-windows/msys2-runtime@ad711d9aa5, git-for-windows/msys2-runtime@90d8aad19b and git-for-windows/msys2-runtime@4935495078: offer to use the standard Windows profile directories as home directories (as is expected by Git for Windows). I contributed two iterations of these patches to the Cygwin project already, and simply have to find the time to do the final touch-up. It's on my ever-growing TODO list.
• git-for-windows/msys2-runtime@07e3f3f107: this is a net improvement that should go into msys2/msys2-runtime as-is.
• git-for-windows/msys2-runtime@99c3c01445: this should probably go to the Cygwin project, making it harder to introduce Unicode conversions that inadvertently use the private code page intended to pretend that Windows has no forbidden filename characters
• git-for-windows/msys2-runtime@d6849e0baa: this fixes a real bug, and I am not quite certain whether this should go into Cygwin or just msys2/msys2-runtime
• git-for-windows/msys2-runtime@1fbb6ee0c1: needs analysis whether this should directly go into msys2/msys2-runtime, or into Cygwin
• git-for-windows/msys2-runtime@65ce633af3 (with git-for-windows/msys2-runtime@f5f594c2ff squashed in): this works around (as well as possible) issues calling an MSYS2 runtime from another MSYS2 runtime (or even the Cygwin runtime); It is not possible to reuse the pseudo terminals, but at least most other operations should work.
• git-for-windows/msys2-runtime@bec3d608e8: this is a net improvement: it lets uname report the commit from which the MSYS2 runtime was built. This should go into msys2/msys2-runtime; We may eventually even want to contribute this to the Cygwin project, they may like it, too.

There are a couple of challenges with this idea outside of the MSYS2 runtime:

• Git for Windows' mingw-w64-git installs a /git-bash.exe (and other files that are most inappropriate for MSYS2). We will need to split them into their own package, e.g. mingw-w64-git-for-windows-root, that only gets built and installed in the Git for Windows project.
• Several of MSYS2's defaults are not amenable to Git for Windows and have to be overridden in the latter, e.g. to use the Windows profile directory as home directory rather than /home/<user> (which would be unwritable in an installed Git for Windows). There is really no good way to override this other than editing files that are owned by MSYS2's packages (such as filesystem). Git for Windows does this via a hack: the post-install script of the mingw-w64-git-extra package performs those edits as needed. It would be nice if we found a non-hacky solution for these modifications.
• Git for Windows builds a libcurl that offers both Secure Channel and OpenSSL backends (but not GNU TLS) with a runtime switch (CURL_SSL_BACKEND for curl.exe, http.sslBackend for Git). The MSYS2 project insteads builds various flavors of libcurl; The user has to decide which flavor to install and then cannot switch between them at runtime. I am ready to bite the bullet and adjust Git for Windows' libcurl to align with MSYS2's paradigm (and I think I found a hacky way to still sort of support runtime-switching).

[DakshKK]

Interested, but I lack sufficient knowledge to do anything beneficial or fundamental.

[lazka]

Thanks @dscho. We had a small meetup just now and talked about working through your list.

Some small notes that came out of it (just from me and @elieux):

• we are generally fine with small breakages for MSYS2 users due to behavior differences, as we think this is more important for the project health long term
• upstreaming to cygwin will come later, to keep this doable

[Biswa96]

• Several of MSYS2's defaults are not amenable to Git for Windows and have to be overridden in the latter, e.g. to use the Windows profile directory as home directory rather than /home/<user>

Is it possible not to change that behavior please? 🥺 I prefer config files to be store in /home/$USER directory. Alternatively, add option or config to restore that default directory.

[lazka]

That's definitely not something we will change by default.

[dscho]

Right, I should have clarified that my plan was never to force Git for Windows' behavior onto MSYS2 here. Instead, I would like to find ways how we can keep the behaviors separate without requiring Git for Windows to edit filesystems' version of /etc/nsswitch.conf.

[lazka]

I've started with something easy: msys2/msys2-runtime#143

How do we want to keep track of the list above? Can I edit the original post, check those boxes and add links to PRs?

[rimrul]

How do we want to keep track of the list above? Can I edit the original post, check those boxes and add links to PRs?

That's probably the cleanest approach.

[rimrul]

git-for-windows/msys2-runtime@ad711d9aa5, git-for-windows/msys2-runtime@90d8aad19b and git-for-windows/msys2-runtime@4935495078: offer to use the standard Windows profile directories as home directories (as is expected by Git for Windows). I contributed two iterations of these patches to the Cygwin project already, and simply have to find the time to do the final touch-up. It's on my ever-growing TODO list.

The first of these three patches made it to cygwin as cygwin/cygwin@27376c6

IDK how to keep track of that though. Can't exactly partially check a checkbox.

[dscho]

The first of these three patches made it to cygwin as cygwin/cygwin@27376c6

IDK how to keep track of that though. Can't exactly partiallycheck a checkbox.

Right. I am working on the second and third patch for that, and then we can check the box.

[dscho]

So the second of these three patches made it to cygwin as cygwin/cygwin@cc84040. Unfortunately I had to rewrite the third patch completely, and add a bug fix for the first.

I saw that @lazka opened msys2/msys2-runtime#148 and msys2/msys2-runtime#149... Thank you so much!