Skip to content

Commit

Permalink
Parallelise building dynlink_compilerlibs
Browse files Browse the repository at this point in the history
This performs two weird tricks that together knock ~30 seconds off the build
time in dev mode and ~40 seconds outside dev mode:

- To build `dynlink_compilerlibs`, copy both `parser.ml` and `parser.mli` from
  the main compiler source. Copying just `parser.ml` means that the native build
  depends on the .cmi produced by the bytecode build.

- Explicitly add the .cma and .cmxa files for `dynlink` and
  `flambda2_from_lambda` to the `install` alias. It's not clear why this helps,
  but it appears that Dune is being excessively lazy in discovering installable
  targets, waiting until `ocamlcommon` is built before it even considers any
  targets in these libraries, including those that don't depend on `ocamlcommon`
  (and take quite a while to build). Redundantly adding targets to `install`
  seems to be the nudge Dune needs.
  • Loading branch information
lukemaurer committed Nov 11, 2024
1 parent d97816a commit a3fe78c
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 7 deletions.
17 changes: 17 additions & 0 deletions dune
Original file line number Diff line number Diff line change
Expand Up @@ -866,3 +866,20 @@
(package ocaml)
(files
(include compiler-libs-installation.sexp)))

; This is entirely redundant, but without it, each target in these libraries
; and their dependencies waits until ocamlcommon is built, including those
; targets that don't actually depend on it. This produces severe bottlenecking,
; costing about 15 seconds when building in dev mode. It's not immediately clear
; why Dune fails to parallelise here. Going by the progress indicator, my best
; guess is that targets in subdirectories aren't even discovered until
; ocamlcommon is built, unless something in the subdirectory is explicitly made
; a dependency.
(alias
(name install)
(package ocaml)
(deps
otherlibs/dynlink/dynlink.cma
otherlibs/dynlink/dynlink.cmxa
middle_end/flambda2/from_lambda/flambda2_from_lambda.cma
middle_end/flambda2/from_lambda/flambda2_from_lambda.cmxa))
19 changes: 12 additions & 7 deletions otherlibs/dynlink/dune
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,21 @@
(targets parser.ml)
(deps ../../parser.ml)
(action
(bash
"echo \"module MenhirLib = CamlinternalMenhirLib\" > parser.ml && cat ../../parser.ml >> parser.ml")))
(with-stdout-to %{targets}
(progn
(echo "module MenhirLib = CamlinternalMenhirLib")
(cat %{deps})))))

(rule
(targets camlinternalMenhirLib.ml)
(deps ../../parsing/camlinternalMenhirLib.ml)
(targets parser.mli)
(deps ../../parser.mli)
(action
(copy %{deps} %{targets})))
(with-stdout-to %{targets}
(progn
(echo "module MenhirLib = CamlinternalMenhirLib")
(cat %{deps})))))

(copy_files ../../parsing/camlinternalMenhirLib.ml{,i})

(library
(name dynlink_compilerlibs)
Expand All @@ -59,8 +66,6 @@
(:standard
-strict-sequence
-absname
-w
+67
-bin-annot
-safe-string
-strict-formats))
Expand Down

0 comments on commit a3fe78c

Please sign in to comment.