Debug musl build #263
Debug musl build #263
Conversation
|
Thank you. I wasn't able to find any significant differences in the config.mk, though there were a few that probably don't matter. I'll have to look more closely later. In the meantime, not sure if this is helpful but here's the final compile+link command when I run, that produces a binary about 2.5MB in size. Maybe it's related to gcc version? Mine is x86_64-linux-musl-gcc (GCC) 9.4.0. In any case, if we can't figure this out it's not a big deal for now. The musl build is useful for some cases, but will be very significantly limited by its inability to use dlopen which therefore will make extensions unavailable. |
|
@liquidaty: |
Regarding future directions and road map, do you think we should keep the musl build? |
|
@liquidaty: Ran benchmarks with GCC 9.3.0 (locally via Docker and CI) and 9.4.0 (locally via musl toolchain). Here are the select command results only (ran multiple times, pasting only one instance here for brevity): GCC 9.3.0 GCC 9.4.0 It's clearly slower than As for the size, the size of ## alpine:latest container
760782 /usr/lib/libncursesw.a
## built with musl toolchain
808316 ncurses-6.5/lib/libncursesw.aThe compiler must be adding the rest of the size. Apart from that, you might want to run benchmarks on your side on a Linux box. The sequence of commands will look like this (need to update env vars and paths in case of musl toolchain): PREFIX=amd64-linux-musl \
CC=musl-gcc \
MAKE=make \
ARTIFACT_DIR=artifacts \
STATIC_BUILD=1 \
CFLAGS="-I/path/to/ncurses-6.5/include" \
LDFLAGS="-L/path/to/ncurses-6.5/lib" \
RUN_TESTS=false \
SKIP_ZIP_ARCHIVE=true \
SKIP_TAR_ARCHIVE=true \
./scripts/ci-build.sh
tar -czvf "zsv-0.3.9-alpha-amd64-linux-musl.tar.gz" "amd64-linux-musl"
git clone https://github.com/liquidaty/zsv.git
cd zsv
mkdir -p .benchmarks
cp /path/to/zsv-0.3.9-alpha-amd64-linux-musl.tar.gz .benchmarks
ZSV_LINUX_BUILD_COMPILER=musl ./scripts/ci-run-benchmarks.sh |
|
@liquidaty: UPDATE Command: zsv select -W -n -- 2 1 3-7 < worldcitiespop_mil.csv >/dev/nullHere are the
|
|
@liquidaty: musl build has been used from this workflow run: Results from CI (benchmarks workflow used from #274): $ ZSV_LINUX_BUILD_COMPILER=musl ./scripts/ci-run-benchmarks.sh
[INF] Running ./scripts/ci-run-benchmarks.sh
[INF] OS: linux
[INF] RUNS: 6
[INF] SKIP_FIRST_RUN: true
[INF] BENCHMARKS_DIR: .benchmarks
[INF] ZSV_TAG: 0.3.9-alpha
[INF] WORKFLOW_RUN_ID: 11758172816
[INF] ZSV_LINUX_BUILD_COMPILER: musl
[INF] Downloading CSV file... [worldcitiespop_mil.csv] [DOWNLOADED]
[INF] Downloading... [zsv-0.3.9-alpha-amd64-linux-musl.tar.gz] [SKIPPED]
[INF] Extracting... [zsv-0.3.9-alpha-amd64-linux-musl.tar.gz] [DONE]
[INF] Downloading... [tsv-utils-v2.2.0_linux-x86_64_ldc2.tar.gz] [DONE]
[INF] Extracting... [tsv-utils-v2.2.0_linux-x86_64_ldc2.tar.gz] [DONE]
[INF] Downloading... [xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz] [DONE]
[INF] Extracting... [xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz] [DONE]
[INF] Running count benchmarks...
1 | zsv : real 0.04 user 0.04 sys 0.00
2 | zsv : real 0.04 user 0.04 sys 0.00
3 | zsv : real 0.04 user 0.04 sys 0.00
4 | zsv : real 0.04 user 0.04 sys 0.00
5 | zsv : real 0.04 user 0.04 sys 0.00
1 | xsv : real 0.10 user 0.09 sys 0.00
2 | xsv : real 0.10 user 0.10 sys 0.00
3 | xsv : real 0.10 user 0.09 sys 0.00
4 | xsv : real 0.10 user 0.10 sys 0.00
5 | xsv : real 0.10 user 0.09 sys 0.00
1 | tsv : real 0.02 user 0.02 sys 0.00
2 | tsv : real 0.03 user 0.01 sys 0.01
3 | tsv : real 0.02 user 0.02 sys 0.00
4 | tsv : real 0.03 user 0.01 sys 0.01
5 | tsv : real 0.03 user 0.02 sys 0.00
[INF] Running select benchmarks...
1 | zsv : real 0.15 user 0.14 sys 0.00
2 | zsv : real 0.15 user 0.14 sys 0.00
3 | zsv : real 0.17 user 0.17 sys 0.00
4 | zsv : real 0.15 user 0.15 sys 0.00
5 | zsv : real 0.15 user 0.14 sys 0.00
1 | xsv : real 0.32 user 0.31 sys 0.00
2 | xsv : real 0.31 user 0.31 sys 0.00
3 | xsv : real 0.33 user 0.31 sys 0.01
4 | xsv : real 0.32 user 0.31 sys 0.00
5 | xsv : real 0.31 user 0.30 sys 0.01
1 | tsv : real 0.15 user 0.14 sys 0.01
2 | tsv : real 0.15 user 0.14 sys 0.00
3 | tsv : real 0.15 user 0.15 sys 0.00
4 | tsv : real 0.15 user 0.14 sys 0.00
5 | tsv : real 0.15 user 0.14 sys 0.00
[INF] Generating Markdown output... [benchmarks.md]
[INF] Generated Markdown output successfully!
[INF] Generating step summary...
[INF] Generated step summary successfully!
[INF] --- [DONE] ---Results from local runs: $ ZSV_LINUX_BUILD_COMPILER=musl ./scripts/ci-run-benchmarks.sh
[INF] Running ./scripts/ci-run-benchmarks.sh
[INF] OS: linux
[INF] RUNS: 6
[INF] SKIP_FIRST_RUN: true
[INF] BENCHMARKS_DIR: .benchmarks
[INF] ZSV_TAG: 0.3.9-alpha
[INF] ZSV_LINUX_BUILD_COMPILER: musl
[INF] Downloading CSV file... [worldcitiespop_mil.csv] [SKIPPED]
[INF] Downloading... [zsv-0.3.9-alpha-amd64-linux-musl.tar.gz] [SKIPPED]
[INF] Extracting... [zsv-0.3.9-alpha-amd64-linux-musl.tar.gz] [DONE]
[INF] Downloading... [tsv-utils-v2.2.0_linux-x86_64_ldc2.tar.gz] [SKIPPED]
[INF] Extracting... [tsv-utils-v2.2.0_linux-x86_64_ldc2.tar.gz] [DONE]
[INF] Downloading... [xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz] [SKIPPED]
[INF] Extracting... [xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz] [DONE]
[INF] Running count benchmarks...
1 | zsv : real 0.04 user 0.03 sys 0.00
2 | zsv : real 0.05 user 0.05 sys 0.00
3 | zsv : real 0.04 user 0.04 sys 0.00
4 | zsv : real 0.05 user 0.04 sys 0.00
5 | zsv : real 0.05 user 0.04 sys 0.00
1 | xsv : real 0.08 user 0.07 sys 0.01
2 | xsv : real 0.08 user 0.08 sys 0.00
3 | xsv : real 0.08 user 0.06 sys 0.01
4 | xsv : real 0.08 user 0.08 sys 0.00
5 | xsv : real 0.08 user 0.08 sys 0.00
1 | tsv : real 0.08 user 0.09 sys 0.00
2 | tsv : real 0.09 user 0.08 sys 0.00
3 | tsv : real 0.08 user 0.07 sys 0.00
4 | tsv : real 0.08 user 0.08 sys 0.00
5 | tsv : real 0.08 user 0.07 sys 0.01
[INF] Running select benchmarks...
1 | zsv : real 0.11 user 0.11 sys 0.00
2 | zsv : real 0.13 user 0.13 sys 0.00
3 | zsv : real 0.11 user 0.09 sys 0.01
4 | zsv : real 0.11 user 0.10 sys 0.00
5 | zsv : real 0.11 user 0.10 sys 0.00
1 | xsv : real 0.37 user 0.37 sys 0.00
2 | xsv : real 0.37 user 0.36 sys 0.00
3 | xsv : real 0.36 user 0.35 sys 0.00
4 | xsv : real 0.36 user 0.35 sys 0.00
5 | xsv : real 0.36 user 0.34 sys 0.01
1 | tsv : real 0.11 user 0.10 sys 0.00
2 | tsv : real 0.11 user 0.11 sys 0.00
3 | tsv : real 0.11 user 0.11 sys 0.00
4 | tsv : real 0.11 user 0.10 sys 0.00
5 | tsv : real 0.10 user 0.09 sys 0.01
[INF] Generating Markdown output... [benchmarks.md]
[INF] Generated Markdown output successfully!
[INF] --- [DONE] ---So, as observed earlier, the musl libc implementation of |
|
@iamazeem that is very interesting thank you. How does the memcpy change impact the benchmarks on other OSes? Should this change be isolated to musl builds? If not, would it be advantageous to replace all instances of memcpy in the entire codebase? |
Ran some benchmarks with the same change for Linux and macOS builds.
Apparently, this change only affects the musl build significantly.
Yes, apparently, looks like it.
Without measuring, it's hard to say. |
iamazeem
force-pushed
the
debug-musl-build
branch
from
bb6d204 to
7d166c6
Compare
Signed-off-by: Azeem Sajid azeem.sajid@gmail.com