playdate-sys causes rust to always rebuild everything, even when nothing has changed

[jcornaz]

Hi,

Usually, working with rust cargo only needs to recompile the crates that have changed and their dependent crates.

But it looks like playdate-sys causes a full rebuild everything, even when nothing has changed at all.

I am not sure what's the exact cause of this. But I assume it's due to one of the rerun-if-changed in the playdate-sys build script?

[boozook]

If you're using cargo-playdate,
it causes because I have to fix inner cargo ctx configuration to do not touch cache of your real cargo.

If you're building just playdate-sys as dependency or standalone, I'm not sure why it happening and should investigate.

Little workaround to speed-up build time:

1. install playdate-bindgen binary (cargo install playdate-bindgen)
2. using --no-default-features disable all features that requires bindgen, that is just two:

• bindgen-runtime
• bindgen-static

4. (optional) also set env PD_BUILD_BINDINGS_ONCE=1 to reuse generated bindings (def)

So now build script will execute your installed pd-bindgen, and cargo will not compile it for dev-deps.

---

If you have any idea how to improve it or fix cargo-playdate's inner cargo configuration to don't touch cache - it could be great and highly welcome! ❤️

[boozook]

@jcornaz, could you please try to build with -vv and tell me list of prints that contains cargo:rerun-?

[jcornaz]

Thanks for the help.

Here are my lines containing cargo:rerun-:

[libm 0.2.8] cargo:rerun-if-changed=build.rs
[semver 1.0.20] cargo:rerun-if-changed=build.rs
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=PLAYDATE_SDK_PATH
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=PD_BUILD_PREBUILT
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=TARGET
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=PROFILE
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/VERSION.txt
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api.h
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=PLAYDATE_SDK_PATH
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=TARGET
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_x86_64-unknown-linux-gnu
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_x86_64_unknown_linux_gnu
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stdint.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/stdint.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/libc-header-start.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features-time64.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/timesize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/stdc-predef.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/sys/cdefs.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/long-double.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/gnu/stubs.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/gnu/stubs-64.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/timesize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/typesizes.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/time64.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wchar.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/stdint-intn.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/stdint-uintn.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/stdlib.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/libc-header-start.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stddef.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/waitflags.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/waitstatus.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/floatn.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/floatn-common.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/long-double.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/sys/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/clock_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/clockid_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/time_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/timer_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stddef.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/stdint-intn.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/endian.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/endian.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/endianness.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/byteswap.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/uintn-identity.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/sys/select.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/select.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/sigset_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/__sigset_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/time_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/struct_timeval.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/struct_timespec.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/endian.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/time_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/pthreadtypes.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/thread-shared-types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/pthreadtypes-arch.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/wordsize.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/atomic_wide_counter.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/struct_mutex.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/struct_rwlock.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/alloca.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stddef.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/stdlib-float.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/floatn.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_gfx.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_sys.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_lua.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_json.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/stdlib.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/string.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/libc-header-start.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stddef.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/locale_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/__locale_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/strings.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stddef.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types/locale_t.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_file.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_sprite.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_sound.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/math.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/libc-header-start.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/features.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/types.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/math-vector.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/libm-simd-decl-stubs.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/floatn.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/flt-eval-method.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/fp-logb.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/fp-fast.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/mathcalls-helper-functions.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/mathcalls.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/mathcalls-helper-functions.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/mathcalls.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/mathcalls-helper-functions.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/include/bits/mathcalls.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_display.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/home/jonathan/PlaydateSDK-2.1.0/C_API/pd_api/pd_api_scoreboards.h
[playdate-sys 0.2.10] cargo:rerun-if-changed=/usr/lib/clang/16/include/stdint.h

[boozook]

Hm... probably my mistake in this two lines:

[playdate-sys 0.2.10] cargo:rerun-if-env-changed=TARGET
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=PROFILE

I'll try without it, by absolutely not sure. 🤷🏻‍♂️
Also I don't remember why and for what I've added it.

[notpeter]

Bump. This unnecessarily adds a 10 second delay to cargo playdate run which is pretty bad. Even worse, it prevents using IDE tools like rust-analyzer because it too has rebuild everything for 10secs before doing anything. All this compiling means I can't code while on battery 🤣.

I removed [these two lines]: (5729acf)

[playdate-sys 0.2.10] cargo:rerun-if-env-changed=TARGET
[playdate-sys 0.2.10] cargo:rerun-if-env-changed=PROFILE

But that wasn't enough, something is still triggering a rebuild on each invocation.

Let me know if you have alternate suggestions or if I can help troubleshoot in anyway.

[boozook]

I almost sure there is need to dramatically tune cargo-playdate cache and fs-lock used in target dir.
In my plan it's probably next after issue 354 (I don't want to link the two issues, so just a number.).

Also if it's not cargo-playdate it could be:

• rerun-if-env-changed=PLAYDATE_SDK_PATH if it not set in rust-analyser invocation context,
• may be other env vars

It can be tested with .cargo/config.toml - add env table and check the time

[env]
PLAYDATE_SDK_PATH = "/path/to/PlaydateSDK-xxx"

requires nightly

[boozook]

If you want to try some, rename there .cargo-lock to something any other. But don't run tests in multiple threads with this change.
I'm not sure but prooooooobably can be helpful 🤷🏻‍♂️

[boozook]

@notpeter,
IMHO, main problem is because I creating cargo::core::compiler::build_context::BuildContext and it's invalidating your cargo cache. I need BuildContext to get actual dependency tree with playdate.assets metadata to properly build assets for the package. Previously I tried with only build-plan by cargo, but there is no so much info as needed. I have some ideas that I'll try in nearest future.

[boozook]

@notpeter I’m just clarifying. Is the problem reproduced without using cargo-playdate?

[notpeter]

I was using cargo-playdate to build/run, is there documentation or an example of how to build without it?

[boozook]

Nonono, I was just wondering if the cargo-playdate is involved in the process or if the problem is reproducible using only cargo check or cargo build or rust-analyzer.

how to build without it

The tool cargo playdate doing many tasks such as preparing assets and manifest, packaging... But minimally necessary is "compilation configuration and linking" that explained in the playdate-sys's readme, as I remember. But I don't recommend it - complex and horrible. But very soon I'll publish standalone rustc target and it will be simpler to create your own build pipeline without cargo-playdate.

[boozook]

One more workaround - set another target-directory, add argument --target-dir=./target-pd for cargo-playdate.

[boozook]

Found this - rust-analyzer #14722.

[boozook]

Finally I've got it. It's not a playdate-sys, but cargo-playdate corrupts cargo's cache when planning units compilation. Trying to fix it... 🥵

[boozook]

I mean that main problem is in the cargo-playdate, but playdate-sys has build-script that also need a "tuning".

[boozook]

Have been closed automatically by a mention.

@jcornaz, @notpeter,
This issue is partially fixed in #385. Now cargo-playdate doesn't corrupts cargo's cache, so incremental build works now normally. And so now we can tune playdate-sys' build-script and (bindgen probably) to make it less trigger rebuild.

Please try the latest beta 6.
❤️‍🔥

[boozook]

Well, now playdate-sys causes cargo to rebuild only when there is no pre-generated bindings and/or bindgen feature.
Also now cargo-playdate respects cargo's cache and so doesn't corrupts incremental builds.
Also in some near future I'll remove bindgen feature from defaults, probably.
So I closing this issue. Feel free to reopen if needed.