Building CMLFS with LTO #27

dslm4515 · 2021-07-09T18:03:27Z

dslm4515
Jul 9, 2021
Maintainer

LTO, Link-Time Optimization, promises faster execution time in exchange for longer compile time and possible smaller binaries/libraries. There is also ThinLTO that is like LTO but with short compile times than 'Full-LTO'

Binutils, LLVM, and GCC of llvmtools would need LTO feature turned on.

For GCC, add to configure command:

--enable-stage1-languages=c,lto 
--enable-languages=c,c++,lto
--with-build-config=bootstrap-lto
--enable-lto

For LLVM, add to cmake command:

-DLLVM_ENABLE_LTO=ON

Set flags to use LTO:

CFLAGS="-flto"
CXXFLAGS="-flto"
LDFLAGS="-flto -Wl,-O2 -Wl,--as-needed"
# When adding additional flags, -flto has to be first.

or for ThinLTO:

CFLAGS="-flto=thin"
CXXFLAGS="-flto=thin"
LDFLAGS="-flto -Wl,-O2 -Wl,--as-needed" 
# When adding additional flags, -flto has to be first.

Per owl4ce (for more optimization):

You may want to use -march=<arch> (prefer with native) as well. 
Also I recommend to use -pipe flag to C and CXX flags.

For LTO and more optimization:

export CFLAGS="-flto -pipe -Oz -fno-semantic-interposition "
export CXXFLAGS="-flto -pipe -Oz -fno-semantic-interposition "
export LDFLAGS="-flto -Wl,-O2 -Wl,--as-needed"
# Use -Oz only with clang, as GCC does not recognize it

Interesting projects:

https://github.com/InBetweenNames/gentooLTO

dslm4515 · 2021-07-09T18:09:19Z

dslm4515
Jul 9, 2021
Maintainer Author

For build-time comparison:

# Building LLVM-12 under chroot with 2 job threads (`make -j2`)
non-LTO: 171m27.003s
LTO-built: 218m57.358s

Host machine specs: AMD Phenom(tm) II X2 550 (2 x 3.1 GHz), 4GB DDR2 800, SATA II HDD

4 replies

dslm4515 Jul 14, 2021
Maintainer Author

LTO Status:

cgnutools:

binutils, builds fine with --enable-lto ... 7min27.310s
gcc-static - ~~builds fine (with above flags) ... 10m27.899s~~ Do not enable LTO
musl libc - will not compile with LTO (unresolved symbol) ... 2m10.369s
gcc-final - ~~builds fine (with above flags) ... 26m9.678s~~ Do not enable LTO
file - builds fine (with above flags) ... 8.001s
libexecinfo - builds fine (with above flags)
ncurses - have not tried LTO
LLVM - build fine with -DLLVM_ENABLE_LTO=ON ... 171m5958s

llvmtools:

Musl libc - will not compile with LTO (unresolved symbol)
libunwind - builds fine (with above flags) ... 7.86s
libcxxabi - builds fine (with above flags) ... 7.852s
libcxx - builds fine (with above flags) ... 54.155s
libexecinfo - builds fine (with above flags) ... 2.592s
ncurses - builds fine (with above flags) ... 24.203s
LLVM - pending, so far compiling for 24 hours and counting!

dslm4515 Jul 15, 2021
Maintainer Author

Update: Do not build GCC with LTO for cgnutools.

Seems to cause issues with file failing to configure unless --host is defined and cgnutool's GCC unable to find libcurses when building LLVM. Also file segfaults if GCC of cgnutools was built with LTO.

dslm4515 Jul 15, 2021
Maintainer Author

Update: Ncurses required as a shared library in cgnutools

Build fails to link with.. static ncurses?

[ 54%] Linking CXX shared library ../../lib/libLTO.so                                                                                           
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cg
nutools/lib/../lib/libcurses.a(lib_cur_term.o): relocation R_X86_64_32 against symbol `ttytype' can not be used when making a shared object; rec
ompile with -fPIC                                                                                                                               
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cg
nutools/lib/../lib/libcurses.a(lib_setup.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recomp
ile with -fPIC                                                                                                                                  
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(lib_tputs.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(read_entry.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(alloc_ttype.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(comp_error.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(comp_userdefs.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile
 with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(db_iterator.o): relocation R_X86_64_32S against symbol `_nc_globals' can not be used when making a shared object;
 recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(getenv_num.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(home_terminfo.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(lib_baudrate.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile
 with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cg
nutools/lib/../lib/libcurses.a(lib_options.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(comp_captab.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(init_keytry.o): relocation R_X86_64_32 against symbol `_nc_tinfo_fkeys' can not be used when making a shared object; recompile with -fPIC
/mnt/cmlto/cgnutools/bin/../lib/gcc/x86_64-cmlfs-linux-musl/10.3.1/../../../../x86_64-cmlfs-linux-musl/bin/ld: /mnt/cmlto/cgnutools/bin/../../cgnutools/lib/../lib/libcurses.a(lib_tputs.o): warning: relocation against `stdout' in read-only section `.text'
collect2: error: ld returned 1 exit status
make[2]: *** [tools/lto/CMakeFiles/LTO.dir/build.make:166: lib/libLTO.so.12] Error 1
make[2]: Leaving directory '/mnt/cmlto/sources/llvm/build'
make[1]: *** [CMakeFiles/Makefile2:15804: tools/lto/CMakeFiles/LTO.dir/all] Error 2
make[1]: Leaving directory '/mnt/cmlto/sources/llvm/build'
make: *** [Makefile:156: all] Error 2
make: Leaving directory '/mnt/cmlto/sources/llvm/build'

dslm4515 Jul 15, 2021
Maintainer Author

Issue fixed. Ncurses should not be static, must be built as a shared library

dslm4515 · 2021-07-09T20:29:02Z

dslm4515
Jul 9, 2021
Maintainer Author

I do wonder if stripping (--strip-unneeded and --strip-debug) is necessary or optional. Not sure if stripping is an indication if package was successfully compiled with LTO:

# when stripping binaries and libraries of bison:
 /llvmtools/bin/strip: /BUILD/usr/lib/liby.a(main.o): Unable to recognise the format of file: file format not recognized
/llvmtools/bin/strip: /BUILD/usr/lib/liby.a(yyerror.o): Unable to recognise the format of file: file format not recognized

Or is the above warning for static libraries built with LTO?

5 replies

dslm4515 Jul 10, 2021
Maintainer Author

Looks like if binaries & libraries are stripped with strip and the above error appears (file format *.o is unrecognized) then building that package with LTO failed.

dslm4515 Jul 10, 2021
Maintainer Author

List of 'LTO' status of packages:

Python 3.9.5 - no -flto (configure finds an error), no --enable-optimizations since system LLVM is missing libclang_rt.profile-x86_64.a
argp-standalone - no -flto (builds but libraries are useless)
zlib/zlib-ng - no -flto (builds but libraries are useless)
e2fsprogs - no -flto (builds but libraries are useless)
execline, skalibg, utmps, s6, s6-rc, s6 suite - no -flto (builds but libraries are useless)

owl4ce Jul 18, 2021

About missing libclang_rt.profile-x86_64.a

owl4ce Jul 18, 2021

Untested, I'm not sure ..

-DLLVM_ENABLE_PROJECTS
https://clang.llvm.org/get_started.html

owl4ce Jul 19, 2021

I confirm, need to enable profiler or something (disabled in this repo).

Gentoo has separated package to enable sanitizer.
https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-libs/compiler-rt-sanitizers/compiler-rt-sanitizers-12.0.1.ebuild

owl4ce · 2021-07-11T06:29:51Z

owl4ce
Jul 11, 2021

I suggest to use -Wno-dev while configure LLVM to suppress developer warnings.

0 replies

dslm4515 · 2021-08-21T20:26:18Z

dslm4515
Aug 21, 2021
Maintainer Author

I will likely open an new repo with optimization in mind and CMLFS as the base

4 replies

dslm4515 Aug 21, 2021
Maintainer Author

I may call it ... Optimux-C

ghost Aug 24, 2021

I was just going to try and build optimux with lld since I've heard that it's faster than gold.

I guess optimux-c will be a frankenstein of all of your repos.

dslm4515 Aug 25, 2021
Maintainer Author

With my 2nd build of optimux, I will try to use the gold linker... since its faster than ld.bfd.

But yes, with Optimux-C, i too will build it with LLD

dslm4515 Nov 9, 2021
Maintainer Author

Quick research shows GCC only has 1-3 % performance gain (when optimizing for execution speed primarily and size secondarily) over LLVM built libraries and binaries.

I suppose would not recommend Optimux-C for embedded systems due to a potentially larger rootfs compared to a GCC built rootfs.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building CMLFS with LTO #27

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 13 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Building CMLFS with LTO #27

dslm4515 Jul 9, 2021 Maintainer

For LTO and more optimization:

Replies: 4 comments · 13 replies

dslm4515 Jul 9, 2021 Maintainer Author

dslm4515 Jul 14, 2021 Maintainer Author

LTO Status:

cgnutools:

llvmtools:

dslm4515 Jul 15, 2021 Maintainer Author

Update: Do not build GCC with LTO for cgnutools.

dslm4515 Jul 15, 2021 Maintainer Author

Update: Ncurses required as a shared library in cgnutools

dslm4515 Jul 15, 2021 Maintainer Author

dslm4515 Jul 9, 2021 Maintainer Author

dslm4515 Jul 10, 2021 Maintainer Author

dslm4515 Jul 10, 2021 Maintainer Author

owl4ce Jul 18, 2021

owl4ce Jul 18, 2021

owl4ce Jul 19, 2021

owl4ce Jul 11, 2021

dslm4515 Aug 21, 2021 Maintainer Author

I will likely open an new repo with optimization in mind and CMLFS as the base

dslm4515 Aug 21, 2021 Maintainer Author

ghost Aug 24, 2021

dslm4515 Aug 25, 2021 Maintainer Author

dslm4515 Nov 9, 2021 Maintainer Author

dslm4515
Jul 9, 2021
Maintainer

Replies: 4 comments 13 replies

dslm4515
Jul 9, 2021
Maintainer Author

dslm4515 Jul 14, 2021
Maintainer Author

dslm4515 Jul 15, 2021
Maintainer Author

dslm4515 Jul 15, 2021
Maintainer Author

dslm4515 Jul 15, 2021
Maintainer Author

dslm4515
Jul 9, 2021
Maintainer Author

dslm4515 Jul 10, 2021
Maintainer Author

dslm4515 Jul 10, 2021
Maintainer Author

owl4ce
Jul 11, 2021

dslm4515
Aug 21, 2021
Maintainer Author

dslm4515 Aug 21, 2021
Maintainer Author

dslm4515 Aug 25, 2021
Maintainer Author

dslm4515 Nov 9, 2021
Maintainer Author