Trouble with Z3 on OCaml 4.10 on Linux and OCaml 4.x on macOS

@mgree wrote:

I’m having trouble getting a project to work using Z3. The project, KMT uses the Z3 SMT solver to generate Kleene algebras with tests (KATs) and decide equivalences between them.

Some configurations work fine, others don’t.

• On my own computer, macOS 10.13.6, my project works with OCaml 4.07 and 4.10 just fine.
• On Travis’s Linux build, my project works with OCaml 4.07, 4.08, and 4.09 just fine.
• On Travis’s Linux build, my project fails in 4.10 with an error related to caml_local_roots (see below).
• On Travis’s macOS build, my project fails in 4.07 through 4.10 with a missing libz3.dylib.

I suspect these are totally different issues. There’s also a third issue, perhaps related, perhaps not: with Z3 breaking on versions higher than 4.8.1. I’m only using 4.8.1 in my project, though.

Linux and OCaml 4.10

Here’s the build error, excerpted from my CI log.

$ opam config exec -- dune build src/main.exe
(cd _build/default && /home/travis/.opam/ocaml-base-compiler.4.10.0/bin/ocamlopt.opt -thread -cclib -lstdc++ -g -o src/main.exe /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml/nums.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/z3ml.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3 /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml/unix.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml/bigarray.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml/str.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml/threads/threads.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ocaml /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/batteries/batteries.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/batteries/batteriesThread.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ANSITerminal/ANSITerminal.cmxa -I /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ANSITerminal /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/stdlib-shims/stdlib_shims.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/fmt/fmt.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/cmdliner/cmdliner.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/fmt/fmt_cli.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/fmt/fmt_tty.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/astring/astring.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/uuidm/uuidm.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/re/re.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/alcotest/alcotest.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/logs/logs.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/logs/logs_fmt.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/logs/logs_cli.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/result/result.cmxa /home/travis/.opam/ocaml-base-compiler.4.10.0/lib/ppx_deriving/runtime/ppx_deriving_runtime.cmxa src/kmt.cmxa src/.main.eobjs/native/main.cmx)
/home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/libz3ml.a(z3native_stubs.o): In function `n_context_of_ast':
z3native_stubs.c:(.text+0x133a): undefined reference to `caml_local_roots'
/home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/libz3ml.a(z3native_stubs.o): In function `n_mk_null_ast':
z3native_stubs.c:(.text+0x142a): undefined reference to `caml_local_roots'
/home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/libz3ml.a(z3native_stubs.o): In function `n_context_of_symbol':
z3native_stubs.c:(.text+0x153a): undefined reference to `caml_local_roots'
/home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/libz3ml.a(z3native_stubs.o): In function `n_mk_null_symbol':
z3native_stubs.c:(.text+0x162a): undefined reference to `caml_local_roots'
/home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/libz3ml.a(z3native_stubs.o): In function `n_context_of_constructor':
z3native_stubs.c:(.text+0x173a): undefined reference to `caml_local_roots'
/home/travis/.opam/ocaml-base-compiler.4.10.0/lib/z3/libz3ml.a(z3native_stubs.o):z3native_stubs.c:(.text+0x182a): more undefined references to `caml_local_roots' follow
collect2: error: ld returned 1 exit status

File "caml_startup", line 1:
Error: Error during linking

My understanding is that OCaml 4.10 changed caml_local_roots to be a macro referring to Caml_state_field(local_roots). My guess at what’s going wrong is that Z3 is somehow using older headers (maybe from the system ocaml?). I’m not certain how to figure this out, though, since I’m installing Z3 from OPAM. (Which, you would think, would make sure that the right headers got found.) Thoughts?

macOS and libz3.dylib (resolved, but not to my satisfaction)

On the macOS builds, libz3.dylib isn’t found. Here’s the build error, excerpted from my CI log:

$ opam config exec -- dune exec src/test_equivalence.exe
dyld: Library not loaded: libz3.dylib
  Referenced from: /Users/travis/build/mgree/kmt/_build/default/src/test_equivalence.exe
  Reason: image not found
/Users/travis/.travis/functions: line 109: 48960 Abort trap: 6           opam config exec -- dune exec src/test_equivalence.exe
The command "opam config exec -- dune exec src/test_equivalence.exe" exited with 134.

I’ve found a temporary but unsatisfactory solution: setting DYLD_LIBRARY_PATH or, preferably, DYLD_FALLBACK_LIBRARY_PATH. (Apparently the latter plays better with correctly configured executables.)

An ideal solution here wouldn’t require environment variables for my executables to run. I can see two ways forward here:

1. Change the Z3 OPAM file to use -install_name correctly when building libz3.dylib.
2. Change my build to use install_name_tool when generating Z3-linked executables on macOS.

The first option seems better—it should fix things for more people—but the second has the advantage of being local.

Update 1: I’ve got the basics working, though not particularly nicely. I’m not certain of the right way to accomplish executable post-processing in dune. Currently I have:

(rule
 (target kmt)
 (enabled_if (= %{ocaml-config:system} macosx))
 (deps main.exe)
 (action (progn
           (copy main.exe %{target})
           (run install_name_tool -change libz3.dylib "%{lib:z3:libz3.dylib}" %{target}))))

(rule
 (target kmt)
 (enabled_if (<> %{ocaml-config:system} macosx))
 (deps main.exe)
 (action (copy main.exe %{target})))

Doing so allows me to drop the DYLD_FALLBACK_LIBRARY_PATH variable setting, but then I have to munge every executable—including tests! (Here’s a CI log showing success for munged and failure for unmunged executables.) Is there a better way to do this? How do you postprocess an executable in dune? While I need the copy command to generate the correctly named kmt executable, it’d be great to be able to skip the copying (and verbose rules).

Posts: 5

Participants: 3

Read full topic