Running rules against multiple input files for promotion testing

I currently have a directory full of a bunch of *.mltt files that I want to run promotion tests against. Currently I have to hard-code these:

(rule (with-stdout-to let.mltt.stdout.exp     (with-stdin-from let.mltt     (run mltt))))
(rule (with-stdout-to monoid.mltt.stdout.exp  (with-stdin-from monoid.mltt  (run mltt))))
(rule (with-stdout-to type.mltt.stdout.exp    (with-stdin-from type.mltt    (run mltt))))

(rule (alias runtest) (action (diff let.mltt.stdout     let.mltt.stdout.exp)))
(rule (alias runtest) (action (diff monoid.mltt.stdout  monoid.mltt.stdout.exp)))
(rule (alias runtest) (action (diff type.mltt.stdout    type.mltt.stdout.exp)))

This is pretty error prone, and frustrating to keep up to date. Is there a more systematic way of describing these rules based on globbing the test input files, or would I have to write a custom test harness for this?

I tried this before I really understood how dependencies and globbing worked:

; note: this does not work!

(rule
  (deps (:mltt (glob_files tests/*.mltt)))
  (action
    (with-stdout-to %{mltt}.stdout.exp
      (with-stdin-from %{mltt} (run ./mltt.exe)))))

(rule
  (alias runtest)
  (deps (:mltt (glob_files tests/*.mltt.stdout)))
  (action
    (diff %{mltt} %{mltt}.exp)))

By searching github I’ve seen that some people generate the dune file with OCaml, but this seems a bit like a bodge, and liable to break in the future?

3 posts - 2 participants

Read full topic