Julia 1.10 and higher wait for all running Tasks to finish before writing out the precompiled (cached) version of the package. One consequence is that a package that launches Tasks in its __init__ function may precompile successfully, but block precompilation of any packages that depend on it.
The symptom of this problem is a message
◐ MyPackage: Waiting for background task / IO / timer. Interrupt to inspect...
that may appear during precompilation, with that precompilation process "hanging" until you press Ctrl-C.
Aqua has checks to determine whether your package causes this problem. Conversely, if you're a victim of this problem, it also has tools to help you determine which of your dependencies is causing the problem.
Let's create a dummy package, PkgA, that launches a persistent Task:
module PkgA
+const t = Ref{Timer}() # used to prevent the Timer from being garbage-collected
+__init__() = t[] = Timer(0.1; interval=1) # create a persistent `Timer` `Task`
+end
PkgA will precompile successfully, because PkgA.__init__() does not run when PkgA is precompiled. However,
module PkgB
+using PkgA
+end
fails to precompile: using PkgA runs PkgA.__init__(), which leaves the Timer Task running, and that causes precompilation of PkgB to hang.
Without Aqua's tests, the developers of PkgA might not realize that their package is essentially unusable with any other package.
Running all of Aqua's tests will automatically check whether your package falls into this trap. In addition, there are ways to manually run (or tweak) this specific test.
Aqua.test_persistent_tasks(MyPackage) will check whether MyPackage blocks precompilation for any packages that depend on it.
By default, Aqua.test_persistent_tasks only checks whether a package's __init__ function leaves persistent tasks running. To check whether other package functions leave persistent tasks running, pass a quoted expression:
Aqua.test_persistent_tasks(MyPackage, quote
+ # Code to run after loading MyPackage
+ server = MyPackage.start_server()
+ MyPackage.stop_server!(server) # ideally, this this should cleanly shut everything down. Does it?
+end)
Here is an example test with a dummy expr which will obviously fail, because it's explicitly spawning a Task that never dies.
julia> using Aqua
julia> Aqua.test_persistent_tasks(Aqua, expr = quote
+ Threads.@spawn while true sleep(0.5) end
+ end)Test Failed at /home/runner/work/Aqua.jl/Aqua.jl/src/persistent_tasks.jl:38
+ Expression: !(has_persistent_tasks(package; kwargs...))
+
+ERROR: There was an error during testing
This test works by launching a Julia process that tries to precompile a dummy package similar to PkgB above, modified to signal back to Aqua when PkgA has finished loading. The test fails if the gap between loading PkgA and finishing precompilation exceeds time tmax.
Often, the easiest fix is to modify the __init__ function to check whether the Julia process is precompiling some other package; if so, don't launch the persistent Tasks.
function __init__()
+ # Other setup code here
+ if ccall(:jl_generating_output, Cint, ()) == 0 # if we're not precompiling...
+ # launch persistent tasks here
+ end
+end
In more complex cases, you may need to modify the task to support a clean shutdown. For example, if you have a Task that runs a never-terminating while loop, you could change
while true
+ ⋮
+ end
to
while task_should_run[]
+ ⋮
+ end
where
const task_should_run = Ref(true)
is a global constant in your module. Setting task_should_run[] = false from outside that while loop will cause it to terminate on its next iteration, allowing the Task to finish.
Julia's devdocs also discuss this issue.
Aqua.test_persistent_tasks(package)
Test whether loading package creates persistent Tasks which may block precompilation of dependent packages. See also Aqua.find_persistent_tasks_deps.
If you provide an optional expr, this tests whether loading package and running expr creates persistent Tasks. For example, you might start and shutdown a web server, and this will test that there aren't any persistent Tasks.
On Julia version 1.9 and before, this test always succeeds.
Arguments
package: a top-level Module or Base.PkgId.
Keyword Arguments
broken::Bool = false: If true, it uses @test_broken instead of @test.tmax::Real = 5: the maximum time (in seconds) to wait after loading the package before forcibly shutting down the precompilation process (triggering a test failure).expr::Expr = quote end: An expression to run in the precompile package.
Aqua.test_persistent_tasks(package) creates a package with package as a dependency and runs the precompilation process. This requires that package is instantiable with the information in the Project.toml file alone. In particular, this will not work if some of package's dependencies are deved packages or are given as a local path or a git repository in the Manifest.toml.
Aqua.find_persistent_tasks_deps(package; broken = Dict{String,Bool}(), kwargs...)
Test all the dependencies of package with Aqua.test_persistent_tasks. On Julia 1.10 and higher, it returns a list of all dependencies failing the test. These are likely the ones blocking precompilation of your package.
Any additional kwargs (e.g., tmax) are passed to Aqua.test_persistent_tasks.