Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase to 3.5.4 #71

Merged
merged 775 commits into from
Sep 25, 2024
Merged

Rebase to 3.5.4 #71

merged 775 commits into from
Sep 25, 2024

Conversation

dscho
Copy link
Member

@dscho dscho commented Aug 30, 2024

This finally makes the jump to Cygwin v3.5, leaving Windows 7 and Windows 8 support behind.

It also cleans up a lot of stuff, in particular dropping the upstreamed/backported patches. Basically, this is what is left in Git for Windows, on top of MSYS2:

*   27a56f38b5 Merge pull request #57 from rglidden/main
|\
| * b04c85093b msys2-runtime: restore fast path for current user primary group
|/
*   d348c78b68 Merge branch 'revert-pipe-blocking-regression'
|\
| * 0ee763fc22 Revert "Cygwin: pipe: Restore blocking mode for cygwin process at startup."
* |   0dc855f8d2 Merge pull request #37 from chirpnot/main
|\ \
| * | 81da75e85b Change the default base address for x86_64
| |/
* |   553fc9396c Merge branch 'wcstombs'
|\ \
| * | 060026beb8 Use MB_CUR_MAX == 6 by default
| * | f04a45615c Make paths' WCS->MBS conversion explicit
| |/
* |   ba3fefdab0 Merge pull request #10 from kblees/kb/symlinks
|\ \
| * | 7fcfaab3a9 Allow native symlinks to non-existing targets in 'nativestrict' mode
| |/
* |   61e5b3dda1 Merge pull request #1 from dscho/git-for-windows
|\ \
| |/
|/|
| * 2854bea098 Fixed path converting with non ascii char.
| * 7077e7e0f5 Mention the extremely useful small_printf() function
| * acf439b6dd Handle 8-bit characters under LOCALE=C
|/
* 52bdc892e9 Do not try to sync with Cygwin
* bfeec8172f dependabot: help keeping GitHub Actions versions up to date
* 386fe831b8 Bump actions/checkout from 2 to 4
* ce42b7f7fd fixup!!! CI: add a GHA for doing a basic build test
* a98eedda40 Merge branch 'msys2-3.5.4'

This addresses git-for-windows/git#5110

JeffreyALaw and others added 30 commits December 22, 2023 20:40
Same as we've seen elsewhere.  Add missing prototype and header include.
Just one missing header for this libgloss port.
Just one missing header in this case...
Needed a prototype for SYS_write and the prototype for write was incorrect
(returns an int, not a char *).
Missing headers and prototypes, much like other ports.
Make implicit its explicit, add missing prototypes and header includes.  This
does not fix m32r-stub.c which needs more than the trivial work that Jeff J has
pre-approved.
So in this case we mostly need to include an internal header
(epiphany-syscalls.h).  In a few cases an explicit prototype is added.  We've
also got a return with no value in a function with a non-void return type.
Finally the asm_syscall interface expects a pointer as its first argument.  In
a few cases we've actually got an int (file descriptor) which we just cast to a
void *.

It's slightly more than Jeff J's pre-approval, but I think still reasonable.
... so that all of 'exit', '_exit', '_Exit' work.  'exit' thus becomes the
standard 'newlib/libc/stdlib/exit.c'.  (Getting 'atexit' functional needs
further work elsewhere.)

See also commit 5841b2f
"nvptx: Implement '_exit' instead of 'exit'".
Without this, when building with recent gcc, we'll see errors when
compiling for --target mmix the first being:
  CC       libc/sys/mmixware/libc_a-chmod.o
In file included from /x/newlib/libc/sys/mmixware/chmod.c:17:
/x/newlib/libc/sys/mmixware/chmod.c: In function 'chmod':
/x/newlib/libc/sys/mmixware/sys/syscall.h:139:6: error: implicit declaration \
of function 'sprintf' [-Wimplicit-function-declaration]
  139 |      sprintf (buf, "UNIMPLEMENTED %s in %s\n", __FUNCTION__,
 __FILE__); \

Other warnings also quelled.

	* libc/sys/mmixware/sys/syscall.h: Include stdio.h, string.h
	and unistd.h.
	* libc/sys/mmixware/_exit.c: Call __unreachable after simulator exit.
	* libc/sys/mmixware/chown.c (chown): Match declaration in unistd.h.
	* libc/sys/mmixware/getpid.c (_getpid): Ditto.
	* libc/sys/mmixware/kill.c (_kill): Ditto.
	* libc/sys/mmixware/link.c (_link): Ditto.
	* libc/sys/mmixware/read.c (_read): Ditto.
	* libc/sys/mmixware/sbrk.c (_sbrk): Ditto.
	* libc/sys/mmixware/unlink.c (_unlink): Ditto.
	* libc/sys/mmixware/write.c (_write): Ditto.
Don't hardcode a custom rule for this -- reuse libobjs_a_SOURCES
which was designed for this exact scenario.
Commit 8d75828 ("libgloss: merge
sparc configure script up a level") missed including the sparc
acinclude.m4 file which meant none of the sparc-specific vars were
propagating to the sub-makefile.
I guess I must have missed this when working through the trivial port specific
changes.

This adds an include of stdlib.h to get a prototype for exit().

Pushed to the trunk.
Similar to what's been done in the ports, but this time in generic code.

Add missing #includes to pick up prototypes.  Add prototypes for various
internal functions where needed.  Fix signatures of various functions to match
what's provided by the headers (read, sbrk, write, unlink).

Nearly done with this effort ;-)

Pushed to the trunk,
Basically the startup code needs to clear memory from _edata to _end. In the
past it's been done with a fairly naive copy loop.  This changes the code to
just call memset and let memset figure out a sensible way to handle the
operation given the size and alignment requirements.

I don't have performance data on this.  I cobbled it together some time ago in
response to seeing some of the GCC tests with larger .bss sections taking an
insane amount of time to just get from _start to main.  With the fixes to the
H8 decoder in the simulator it may not matter nearly as much anymore.

This has been in my tester for months.  Naturally it does not cause any
regressions in the H8 port.
- bump up version to 4.4.0
Signed-off-by: Xiao Zeng <zengxiao@eswincomputing.com>
This is a .c file, not assembly, and it's a duplicate (same content)
as the existing sim-lseek.c.  Delete it since it's unused and doesn't
actually compile.
Avoid a recursive make to speed things up a bit.
A moxie-elf build shows installed objects & libs produce same code.
Avoid a recursive make to speed things up a bit.
A v850e-elf build shows installed objects & libs produce same code.
The i960 port has a lot of references to sim files that have never been
included in the tree afaict, and thus has never worked.  Delete all the
code to simplify.
Avoid a recursive make to speed things up a bit.
I can't test this as current versions of binutils doesn't support i960.
This enables automatic sparse file support for remote SSDs.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Set symbol '__jvt_base$' as weak. So if the symbol is not set in the
linker script, the address would be 0. We initialize jvt CSR only if
the address is not 0.

Also use csr number directly instead of using symbolic name to prevent the
backward incompatible issue.

psabi reference:
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/2d770815dc9a8b11e61ea1abd487cb25ee56ad5e/riscv-elf.adoc#table-jump-relaxation
From: Rich Felker <dalias@aerifal.cx>
Date: Mon, 7 Nov 2022 22:17:55 -0500

the rule that longest digit sequence not beginning with a zero is
greater only applies when both sequences being compared are
non-degenerate. this is spelled out explicitly in the man page, which
may be deemed authoritative for this nonstandard function: "If one or
both of these is empty, then return what strcmp(3) would have
returned..."

we were wrongly treating any sequence of digits not beginning with a
zero as greater than a non-digit in the other string.

Signed-off-by: Brian Inglis <Brian.Inglis@SystematicSW.ab.ca>
Currently init_semihosting() assumes the return value from _open()
call as the file descriptor handle and that is incorrect.

The semihost _open() call returns the fdtable index returned by the
__add_fdentry() for the file opened.
Signed-off-by: Jon Turney <jon.turney@dronecode.org.uk>
Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
Also add notes in terms of fallocate quirks.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
dscho and others added 15 commits September 18, 2024 14:39
This updates a GitHub Action's version to stay up to date.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Even when the character set is specified as ASCII, we should handle data
outside the 7-bit range gracefully by simply copying it, even if it is
technically no longer ASCII.

This fixes several of Git for Windows' tests, e.g. t7400.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
It came in real handy while debugging an issue that strace 'fixed'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When a non ascii char is at the beginning of a path the current conversion
destroys the path. This fix will prevent this with an extra check for
non-ascii UTF-8 characters.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: 마누엘 <nalla@hamal.uberspace.de>
This is a forked repository...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
	* dcrt0.cc (dll_crt0_1), dtable.cc (handle_to_fn), environ.cc
	(environ_init, getwinenveq, build_env), external.cc
	(fillout_pinfo), fhandler_disk_file.cc (__DIR_mounts::eval_ino,
	fhandler_disk_file::readdir_helper), fhandler_netdrive.cc
	(fhandler_netdrive::readdir), fhandler_process.cc
	(format_process_winexename, format_process_maps,
	format_process_stat, format_process_status), fhandler_procsys.cc
	(fill_filebuf, fhandler_procsys::readdir), mount.cc
	(fs_info::update, mount_info::create_root_entry,
	mount_info::conv_to_posix_path, mount_info::from_fstab_line),
	nlsfuncs.cc (internal_setlocale), path.cc (path_conv::check,
	sysmlink_info::check_shortcut, symlink_info::check_sysfile,
	symlink_info::check_reparse_point,
	symlink_info::check_nfs_symlink, cygwin_conv_path,
	cygwin_conv_path_list, cwdstuff::get_error_desc, cwdstuff::get),
	strfuncs.cc (sys_wcstombs_no_path, sys_wcstombs_alloc_no_path),
	uinfo.cc (ontherange, fetch_from_path, cygheap_pwdgrp::get_home,
	cygheap_pwdgrp::get_shell, cygheap_pwdgrp::get_gecos), wchar.h
	(sys_wcstombs_no_path, sys_wcstombs_alloc_no_path): Convert call
	sites of the sys_wcstombs*() family to specify explicitly when the
	parameter refers to a path or file name, to avoid future
	misconversions.

Detailed explanation:

The sys_wcstombs() function contains special handling for paths/file
names, to work around file name restriction on Windows that are
unexpected in the POSIX context of Cygwin.

We actually do not want that special handling for WCS strings that
do *not* refer to paths or file names.

Neither do we want to convert those special file names unless they come
from inside Cygwin: if the source of the string value is the Windows API,
we *know* it cannot be such a special file name because Windows itself
would not be able to handle it in the way Cygwin does.

So let's switch the previous sys_wcstombs()/sys_wcstombs_no_path() (and
the *_alloc* variant) around to sys_wcstombs_path()/sys_wcstombs(). We do
this for several reasons:

- whenever a call site wants to convert a WCS representation of a path or
  file name to an MBS one, it should be made very clear that we *want* the
  special file name conversion to happen.

- it is shorter to read and write.

- future calls to sys_wcstombs() will not incur unwanted conversion by
  accident (it is easy for unsuspecting programmers to assume that the
  function name "sys_wcstombs()" refers to a regular text conversion that
  has nothing to do with paths or filenames).

By keeping the name sys_wcstombs() (and not switching to
sys_wcstombs_path()), the following call sites are implicitly changed to
*exclude* the special path/file name conversion:

cygheap.h (get_drive):
	Cannot contain special characters

external.cc (cygwin_internal):
	Refers to user/domain names, not paths

fhandler_clipboard.cc (fhandler_dev_clipboard::read):
	Is not a path or file name but characters from the Windows
	clipboard

fhandler_console.cc: (dev_console::con_to_str):
	Is not a path or file name but characters from the console

fhandler_registry.cc (encode_regname):
	Is a registry key, not a path or filename

fhandler_registry.cc (multi_wcstombs):
	All call sites pass registry values, not paths or filenames

fhandler_registry.cc (fstat):
	Is a registry value, not a path or filename

fhandler_registry.cc (fill_filebuf):
	Is a registry value, not a path or filename

net.cc (get_ipv4fromreg):
	Is a registry value, not a path or filename

net.cc (get_friendlyname):
	Is a device name, not a path or filename

netdb.cc (open_system_file):
	Is from outside Cygwin

smallprint.cc (__small_vsprintf):
	Is a free text, not a path or filename

strfuncs.cc (strlwr):
	Should preserve the characters from the private page if there
	are any

strfuncs.cc (strupr):
	Should preserve the characters from the private page if there
	are any

uinfo.cc (cygheap_user::init):
	Refers to a user name, not a path or filename

uinfo.cc (pwdgrp::fetch_account_from_windows):
	Refers to value from outside Cygwin

By keeping the function name sys_wcstombs_alloc() (and not changing it to
sys_wcstombs_alloc_path()), the following call sites are implicitly
changed to *exclude* the special path/file name conversion:

ldap.cc (cyg_ldap::remap_uid):
	Refers to a user name, not a path or filename

ldap.cc (cyg_ldap::remap_gid):
	Refers to a group name, not a path or filename

pinfo.cc (_pinfo::cmdline):
	Refers to a command line from Windows, outside Cygwin

uinfo.cc (cygheap_user::env_logsrv):
	Is a server name, not a path or filename

uinfo.cc (cygheap_user::env_domain):
	Refers to the user/domain name, not a path or filename

uinfo.cc (cygheap_user::env_userprofile):
	Refers to Windows' idea of a path, outside Cygwin

uinfo.cc (cygheap_user::env_systemroot):
	Refers to Windows' idea of a path, outside Cygwin

uinfo.cc (fetch_from_description):
	Refers to values from outside of Cygwin

uinfo.cc (cygheap_pwdgrp::get_gecos):
	Refers to user/domain name and email address, not path nor filename

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows native symlinks must match the type of their target (file or
directory), otherwise native Windows tools will fail. Creating symlinks in
'nativestrict' mode currently requires the target to exist in order to
check its type.

However, the target of a symlink can change at any time after the symlink
has been created. Thus users of native symlinks must be prepared to deal
with type mismatches anyway. Checking the target type at symlink creation
time is not a good reason to violate the symlink() API specification.

In 'nativestrict' mode, always create native symlinks. Choose the symlink
type according to the target if it exists. Otherwise check the target path
for a trailing '/' as hint to create a directory symlink.

This allows callers to explicitly specify the expected target type, e.g.:

  $ ln -s test/ link-to-test
  $ mkdir test

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Internally, Cygwin already uses __utf8_mbtowc(), even if it still claims
to use the "ASCII" charset.

But the `MB_CUR_MAX` value (which is not actually a constant, but
dependent on the current locale) was still 1, which broke the initial
`globify()` call while parsing the the command-line in `build_argv()`
for non-ASCII arguments.

This fixes git-for-windows/git#2189

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This might break things, but it turns out several Windows libraries like
to be loaded at 0x180000000.

This causes a problem, because `msys-2.0.dll` loads at `0x180040000` and
expects `0x180000000-0x180040000` to be available. A problem arises when
Antiviruses (or other DLL hooking mechanisms) load a DLL whose preferred
load address is `0x180000000` and fits in size before `0x180010000`:

1. `msys-2.0.dll` loads and fills `0x180010000-0x180040000` assuming no
   shared console structure is going to be needed.

2. Another DLL loads and fills `0x180000000-0x18000xxxx`

3. `msys-2.0.dll` tries to load `0x180000000-0x180010000` but it's not
   available. It falls back to another address, but down the line
   something else fails.

This bug triggers when using subshells (e.g.: `git clone --recursive`).

The MSYS2 runtime should be able to work around the address conflict,
but the code is failing in some way or other...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Mikael Larsson <95430516+chirpnot@users.noreply.github.com>
Allow native symlinks to non-existing targets in 'nativestrict' mode
This topic branch fixes the problem where a UTF-16 command-line was
converted to UTF-8 in an incorrect way (because Cygwin treated it as if
it was a file name and applied some magic that is intended to allow for
otherwise invalid file names on Windows).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Workaround certain anti-malware programs

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Member Author

dscho commented Sep 18, 2024

I've force-pushed an update after merging #72 (and after backporting the latest iteration of that patch that made it onto the cygwin-3_5-branch).

tyan0 and others added 6 commits September 25, 2024 20:44
When win32-input-mode (which is supported by Windows Termainal) is
set by "\033[?9001h", cons_master_thread does not work properly and
consumes larger and larger memory space. This is because sending
event by WriteConsoleInput() is translated into the sequence that
is used by win32-input-mode. Due to this behaviour, write-back
of the INPUT_RECORDs does not work as expected. With this patch,
cons_master_thread is disabled on win32-input-mode where the signal
keys such as Ctrl-C, Ctrl-Z etc. never comes.

Backported-from: 68a14b6 (Cygwin: console: Disable cons_master_thread in win32-input-mode, 2024-08-31)
Addresses: https://cygwin.com/pipermail/cygwin/2024-August/256380.html
Fixes: ff4440f ("Cygwin: console: Introduce new thread which handles input signal.")
Reported-by: Adamyg Mob <adamyg.mob@gmail.com>
Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
(cherry picked from commit 84d77e5918e18170c393407d477140fcf5d3e432)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
After the commit 7f3c22532577, writing to pipe extremely slows down.
This is because cygwait(select_sem, 10, cw_cancel) is called even
when write operation is already completed. With this patch, the
cygwait() is called only if the write operation is not completed.

Backported-from: 37ab3e0 (Cygwin: pipe: Fix a regression that raw_write() slows down, 2024-09-01)
Addresses: https://cygwin.com/pipermail/cygwin/2024-August/256398.html
Fixes: 7f3c22532577 ("Cygwin: pipe: handle signals explicitely in raw_write")
Reported-by: Jim Reisert AD1C <jjreisert@alum.mit.edu>
Reviewed-by: Corinna Vinschen <corinna@vinschen.de>
Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
(cherry picked from commit f78009cb1ccf84cc343cf2441c76196461d87532)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Commit a5bcfe6 removed an optimization that fetches the
default group from the current user token, as it is sometimes
not accurate such as when groups like the builtin
Administrators group is the primary group.

However, removing this optimization causes extremely poor
performance when connected to some Active Directory
environments.

Restored this optimization as the default behaviour, and
added a `group: db-accurate` option to `nsswitch.conf` that
can be used to disable the optimization in cases where
accurate group information is required.

This fixes git-for-windows/git#4459

Signed-off-by: Richard Glidden <richard@glidden.org>
If a cygwin app is executed from a non-cygwin app and the cygwin
app exits, the read pipe remains in the non-blocking mode because
of the commit fc691d0246b9. Due to this behaviour, the non-cygwin
app cannot read the pipe correctly after that. Similarly, if a
non-cygwin app is executed from a cygwin app and the non-cygwin
app exits, the read pipe remains in the blocking mode. With this
patch, the blocking mode of the read pipe is stored into a variable
was_blocking_read_pipe on set_pipe_non_blocking() when the cygwin
app starts and restored on close(). In addition, the pipe mode is
set to non-blocking mode in raw_read() if the mode is blocking
mode by referring the variable is_blocking_read_pipe as well.
is_blocking_read_pipe is a member of fhandler_pipe class and is set
by set_pipe_non_blocking(), so if other process sets the pipe mode
to blocking mode, the current process cannot know the pipe is
blocking mode. Therefore, is_blocking_read_pipe is also set on the
signal __SIGNONCYGCHLD, which is sent to the process group when
non-cygwin app is started.

Backported-from: c7fe29f (Cygwin: pipe: Restore blocking mode of read pipe on close()/raw_read(), 2024-09-06)
Addresses: git-for-windows/git#5115
Fixes: fc691d0246b9 ("Cygwin: pipe: Make sure to set read pipe non-blocking for cygwin apps.");
Reported-by: isaacag, Johannes Schindelin <Johannes.Schindelin@gmx.de>
Reviewed-by: Corinna Vinschen <corinna@vinschen.de>, Ken Brown <kbrown@cornell.edu>
Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
msys2-runtime: restore fast path for current user primary group
This includes the pull request git-for-windows#72 from
dscho/restore-pipes-blocking-mode, and adds two more backported patches.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Member Author

dscho commented Sep 25, 2024

I just force-pushed an update, in preparation for integrating this in time for v2.47.0-rc0 (which is due tomorrow). Range-diff:

  • 1: d7d010e < -: ---------- Revert "Cygwin: pipe: Restore blocking mode for cygwin process at startup."
  • 2: ffb4029 = 1: f4fb564 msys2-runtime: restore fast path for current user primary group
  • -: ---------- > 2: 900a410 Cygwin: console: Disable cons_master_thread in win32-input-mode
  • -: ---------- > 3: 8854bb4 Cygwin: pipe: Fix a regression that raw_write() slows down
  • 3: 8095439 = 4: 9d0d6e1 Cygwin: pipe: Restore blocking mode of read pipe on close()/raw_read()

There are two new backports, fresh from the cygwin-3_5-branch, that I think I want to have in Git for Windows.

Also, this drops that Revert at long last that I had figured out to be an incorrect fix for the add -i hang: The correct fix was actually 19ed0dff8f3f19fc15e79d188d7bcaadf26f2f11 (which I came up with when trying to upstream Git for Windows' patches to msys2/msys2-runtime and was forced to have a much closer look than before). This fix has made it into v2.44.0 via v2.44.0-rc0~15, and I hope this is reasonably old enough by now that I can risk updating the minimal SDK (this has an impact on all of those topic branches at https://github.com/gitster/git which use an entire zoo of base commits that are too often well in the past: their CI builds will start to fail, and so will the CI builds of new topic branches based on old base commits).

@dscho dscho marked this pull request as ready for review September 25, 2024 19:27
@dscho
Copy link
Member Author

dscho commented Sep 25, 2024

I've verified in https://github.com/dscho/git-for-windows-automation/actions/runs/11039614276 that Git's test suite passes with this MSYS2 runtime version.

@dscho dscho merged commit dc59218 into git-for-windows:main Sep 25, 2024
2 checks passed
@dscho dscho deleted the rebase-to-3.5.4 branch September 25, 2024 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.