clang UBSan and other sanitizers #3020

silverqx · 2022-05-28T10:13:28Z

silverqx
May 28, 2022

Does MSYS2 support these sanitizers like UBSan? Eg. in the gentoo they are provided by the sys-libs/compiler-rt-sanitizers-14.0.3 package, which provides shared libs for the UBSan, asan, memproof, and a few other sanitizers.

Here is the content of sys-libs/compiler-rt-sanitizers-14.0.3 package:
sys-libs_compiler-rt-sanitizers-14.0.3.txt

When I try to compile eg. with the -O1 -fsanitize=undefined flags to enable UBSan I get this compiler error about it can not find these libraries during linkage:

...
compiling E:/c/qMedia/TinyOrm/TinyOrm/tom/src/tom/tomconstants_extern.cpp 
compiling E:/c/qMedia/TinyOrm/TinyOrm/tom/src/tom/tomutils.cpp 
linking debug/TinyOrm0.dll 
ld.lld: error: could not open 'liblibclang_rt.ubsan_standalone-x86_64.a': No such file or directory
ld.lld: error: could not open 'liblibclang_rt.ubsan_standalone_cxx-x86_64.a': No such file or directory
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
mingw32-make[2]: *** [Makefile.Debug:508: debug/TinyOrm0.dll] Error 1
mingw32-make[1]: *** [Makefile:45: debug] Error 2
mingw32-make[2]: Leaving directory 'E:/c/qMedia/TinyOrm/TinyOrm-builds-qmake/build-TinyOrm-Desktop_Qt_5_15_3_MSYS2_UCRT64_clang_64bit-Debug/src'
mingw32-make[1]: Leaving directory 'E:/c/qMedia/TinyOrm/TinyOrm-builds-qmake/build-TinyOrm-Desktop_Qt_5_15_3_MSYS2_UCRT64_clang_64bit-Debug/src'
mingw32-make: *** [Makefile:50: sub-src-make_first] Error 2
12:10:58: The process "C:\msys64\ucrt64\bin\mingw32-make.exe" exited with code 2.
Error while building/deploying project TinyOrm (kit: Desktop Qt 5.15.3 MSYS2 UCRT64 clang 64bit)
When executing step "Make"
12:10:58: Elapsed time: 00:49.

Answered by mati865

May 28, 2022

We do but they are available only in clang* environments: https://www.msys2.org/docs/environments/#gcc-vs-llvmclang since they require both Clang and LLD to build.

Package contents:

$ pacman -Ql mingw-w64-clang-x86_64-compiler-rt
mingw-w64-clang-x86_64-compiler-rt /clang64/
mingw-w64-clang-x86_64-compiler-rt /clang64/bin/
mingw-w64-clang-x86_64-compiler-rt /clang64/bin/libclang_rt.asan_dynamic-x86_64.dll
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/cla…

View full answer

mati865 · 2022-05-28T13:43:02Z

mati865
May 28, 2022
Collaborator

We do but they are available only in clang* environments: https://www.msys2.org/docs/environments/#gcc-vs-llvmclang since they require both Clang and LLD to build.

Package contents:

$ pacman -Ql mingw-w64-clang-x86_64-compiler-rt
mingw-w64-clang-x86_64-compiler-rt /clang64/
mingw-w64-clang-x86_64-compiler-rt /clang64/bin/
mingw-w64-clang-x86_64-compiler-rt /clang64/bin/libclang_rt.asan_dynamic-x86_64.dll
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/fuzzer/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/fuzzer/FuzzedDataProvider.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/profile/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/profile/InstrProfData.inc
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/allocator_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/asan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/common_interface_defs.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/coverage_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/dfsan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/hwasan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/linux_syscall_hooks.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/lsan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/msan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/netbsd_syscall_hooks.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/scudo_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/tsan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/tsan_interface_atomic.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/sanitizer/ubsan_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/xray/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/xray/xray_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/xray/xray_log_interface.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/include/xray/xray_records.h
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan-preinit-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan_cxx-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan_dll_thunk-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan_dynamic-x86_64.dll.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan_dynamic_runtime_thunk-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.asan_static-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.builtins-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.fuzzer-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.fuzzer_interceptors-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.fuzzer_no_main-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.profile-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.stats-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.stats_client-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.ubsan_standalone-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/lib/windows/libclang_rt.ubsan_standalone_cxx-x86_64.a
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/share/
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/share/asan_ignorelist.txt
mingw-w64-clang-x86_64-compiler-rt /clang64/lib/clang/14.0.3/share/cfi_ignorelist.txt

0 replies

silverqx · 2022-05-28T14:24:31Z

silverqx
May 28, 2022
Author

I don't know what is the situation now, but I have tried to compile simple programs with the clang's std c++ library in the past and I had many failures because of that I don't support clang's std c++ library.

Currently, I'm only supporting ucrt64 GCC and clang, they work amazing, practically the same way as on Linux, of course except performance 😁.

Do you have any plans to add these clang sanitizers to the ucrt64 env.? Of course, it's not critical.

They can be enabled by LLVM_USE_SANITIZER:STRING=Address;Memory;MemoryWithOrigins;Undefined;Thread;DataFlow as is described here.

4 replies

mati865 May 28, 2022
Collaborator

I don't know what is the situation now, but I have tried to compile simple programs with the clang's std c++ library in the past and I had many failures because of that I don't support clang's std c++ library.

Lib++ used by Clang still isn't drop-in replacement for libstdc++ but it's evolving fast so your issues might already solved.
One area where libc++ is currently lacking is locale support, most of other things work as you can see in our repo.

Do you have any plans to add these clang sanitizers to the ucrt64 env.? Of course, it's not critical.

They can be enabled by LLVM_USE_SANITIZER:STRING=Address;Memory;MemoryWithOrigins;Undefined;Thread;DataFlow as is described here.

As I said before it's not that easy because they don't build with GCC and ld.bfd.
However if GCC and ld.bfd ever gain features required to build these sanitizers we will surely enable them.

silverqx May 28, 2022
Author

As I said before it's not that easy because they don't build with GCC and ld.bfd. However if GCC and ld.bfd ever gain features required to build these sanitizers we will surely enable them.

So you are currently building whole ucrt64 with the GCC and ld.bfd as the linker and these sanitizers are failing? You can use the lld linker to build clang in ucrt64, can you? Or its bad idea?

mati865 May 28, 2022
Collaborator

So you are currently building whole ucrt64 with the GCC and ld.bfd as the linker and these sanitizers are failing?

Yes, they fail for mingw* and ucrt* environments so they were disabled.

You can use the lld linker to build clang in ucrt64, can you? Or its bad idea?

First of all: you need Clang since GCC doesn't have required features. In the past there were some differences that prevented mixing GCC and Clang static objects but probably shouldn't be problem now.

Second: LLD is also required because ld.bfd doesn't have necessary features either.
This is more problematic because have single PKGBUILD (and need it that way for Clang subsystems) that produces Clang+LLVM+compiler-rt. Why is it a problem?
We cannot simply switch whole PKGBUILD to use Clang+LLD because ld.bfd which is the main linker doesn't properly support import libraries produced by LLD (ld.bfd works correctly only with legacy format that LLD cannot produce).
As an alternative we could add more complexity to that PKGBUILD and build only compiler-rt with Clang+LLD. I won't block such PR but I'm not willing to do it myself. And there is another issue here, sanitizers are tested only in purely LLVM based environment at https://github.com/mstorsjo/llvm-mingw (our clang* environments are almost the same thing). Nobody can tell for sure if sanitizers will work properly in GCC based environment even when using Clang+LLD (Clang+LLD+libgcc+libstdc++).

silverqx May 28, 2022
Author

We cannot simply switch whole PKGBUILD to use Clang+LLD

This is understandable. 👍

build only compiler-rt with Clang+LLD

I think this would be the correct solution. 😁

I won't block such PR but I'm not willing to do it myself

Neither I do 😎, but it would be great to have it 🔥

Nobody can tell for sure if sanitizers will work properly in GCC based environment even when using Clang+LLD

I believe they would work if compiled on Linux but I can not tell how it would turn out with the msys2 ucrt64, can be assumed that with a lot of little work 😁 it would compile ok.

Gentoo is compiling it with the Clang and probably lld here is the ebuild and whole Gentoo is compiled with the GCC and linked with the ld.bfd, so it is absolutely the same situation.

silverqx · 2022-05-28T14:28:37Z

silverqx
May 28, 2022
Author

At least the UBsan and memory sanitizers would be very helpful.

0 replies

efa · 2024-11-14T13:47:57Z

efa
Nov 14, 2024

just tested with updated CLANG, but still got error:
CLANG64 ~/c
$ gcc -O1 -g -fsanitize=address sizeof.c -Wall -o sizeof_Clang64.exe
/usr/lib/gcc/x86_64-pc-msys/13.3.0/../../../../x86_64-pc-msys/bin/ld: cannot find -lasan: No such file or directory
collect2: error: ld returned 1 exit status

1 reply

efa Nov 14, 2024

sorry for the noise, changing gcc to clang work

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

clang UBSan and other sanitizers #3020

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

clang UBSan and other sanitizers #3020

silverqx May 28, 2022

Replies: 4 comments · 5 replies

mati865 May 28, 2022 Collaborator

silverqx May 28, 2022 Author

mati865 May 28, 2022 Collaborator

silverqx May 28, 2022 Author

mati865 May 28, 2022 Collaborator

silverqx May 28, 2022 Author

silverqx May 28, 2022 Author

efa Nov 14, 2024

efa Nov 14, 2024

silverqx
May 28, 2022

Replies: 4 comments 5 replies

mati865
May 28, 2022
Collaborator

silverqx
May 28, 2022
Author

mati865 May 28, 2022
Collaborator

silverqx May 28, 2022
Author

mati865 May 28, 2022
Collaborator

silverqx May 28, 2022
Author

silverqx
May 28, 2022
Author

efa
Nov 14, 2024