diff options
| author | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-13 16:54:02 -0400 |
|---|---|---|
| committer | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-13 16:54:02 -0400 |
| commit | 3f34ec6afde21801eff0b655949a6d219e47943c (patch) | |
| tree | 6e95a210663bcc7f41d0791de1769a2535988180 /pipeline | |
| parent | format 2.1: the pipeline reads with ocda-read (diff) | |
The writer's version follows the format's: 2.1.0 in dub.json,
gleam.toml and both flakes (it was 2.0.0), and manifest.toml locks the
library at 2.1.0 to match. A release is numbered major.minor.patch,
major.minor being the format and shared with the specification and
the library, the patch the writer's own.
flake.lock, in dlang/ and gleam/, pins the library at its 2.1.0
release and nixpkgs as that release locks it, so the writer builds
with what the library was built and checked with. The READMEs say
plain nix build, and how to build against a working copy of the
library instead.
The pipeline no longer builds or runs sisudoc-ssp-reader and
sisudoc-ocda-db-reader, which are frozen: ocda-read, the library's
reader, covers both, and read.sh already ran it from both forms and in
both languages. Gone from build.sh, config.sh, one.sh and read.sh, and
from pipeline/README.md, which with config.sh now names spine's
--show-abstraction as the README does. build.sh still gives the writer
the library beside it, so that the library run and the library built in
are the same one.
README, to what was decided for these writers: the outputs must be
identical from every form and the D and Gleam writers must agree byte
for byte, but they are not held to spine's bytes; each format has its
own judge, and a difference from spine is there to read better. The
introduction said "identical to each other and to spine's", and the
example compared --text with spine's text file, which fails by
design; the example now compares the .ssp and the .ocda.db. The
documents the checks run over are the markup document samples, named
with their repository and the spine flags that build the artefacts,
and the counts read "all" or "every document" rather than a number,
here and in dlang/, gleam/ and pipeline/README.md. The format is 2.1,
and the license is stated.
Checked: D and Gleam build from the lock; over the markup document
samples, D == Gleam in text, markdown, html, epub and typst for every
document-language, and the .ssp and the .ocda.db give the same output.
(assisted by Claude-Code)
Diffstat (limited to 'pipeline')
| -rw-r--r-- | pipeline/README.md | 16 | ||||
| -rwxr-xr-x | pipeline/build.sh | 13 | ||||
| -rw-r--r-- | pipeline/config.sh | 5 | ||||
| -rwxr-xr-x | pipeline/one.sh | 6 | ||||
| -rwxr-xr-x | pipeline/read.sh | 10 |
5 files changed, 16 insertions, 34 deletions
diff --git a/pipeline/README.md b/pipeline/README.md index c5920f7..b5adc84 100644 --- a/pipeline/README.md +++ b/pipeline/README.md @@ -1,13 +1,13 @@ -# The pipeline: how to run the readers, the library and the writers +# The pipeline: how to run the library, its reader and the writers Fixed scripts rather than a test suite. They produce output that is meant to be **looked at**, and they double as the answer to "what do I type?". ``` config.sh where everything is, and where the output goes -build.sh build every reader, library and writer, in both languages +build.sh build the library and the writer, in both languages one.sh one tool, one document - the answer to "how do I run this?" -read.sh every reader and both libraries, over every document +read.sh the library and its reader, over every document write.sh every writer, in both languages, over every document pdf.sh compile the .typ files that write.sh produced run.sh all of the above, in order @@ -20,7 +20,7 @@ document-language and a `.ocda.db` for each document, and, where there is one, a `.sisupod` beside it. That is what spine writes: ```sh -spine --abstraction --ocda-db --pod2 --output=<dir> <pod>... +spine --show-abstraction --ocda-db --pod2 --output=<dir> <pod>... ``` Everything downstream reads those files and nothing else. **No tool here @@ -44,8 +44,6 @@ at something. ## Where the output goes ``` -$OUT/read/ssp-d/ ssp-g/ the .ssp readers' summaries -$OUT/read/db-d/ db-g/ the .ocda.db readers' summaries $OUT/read/dump-d/ dump-g/ the library's dump, from each serialisation $OUT/write/text/{d,g}/ <doc>.txt $OUT/write/md/{d,g}/ <doc>.md @@ -70,8 +68,8 @@ sh pipeline/one.sh read-d <doc>.sisupod en --tree The tool names are `<format>-<language>`: `text`, `md`, `html`, `epub`, `typst` for the writers, `read` for the library's reader (ocda-read, which -reads every form and takes its modes after the file), and `ssp`, `db`, `dump`, -`agree` for the older readers and the library; `-d` for D and `-g` for Gleam. +reads every form and takes its modes after the file), and `dump`, `agree` for +the library; `-d` for D and `-g` for Gleam. Output goes to stdout, so a comparison is a pipeline. The epub and the typst name a file instead, because an epub is a zip and a `.typ` names its images by @@ -86,7 +84,7 @@ typst compile --root d d/x.typ d/x.pdf ## What the counts at the end mean Each stage prints a few numbers. They are not a pass, they are a way to see a -regression without reading 36 documents. +regression without reading every document. | `all four dumps identical` | the library's whole claim: two serialisations, two languages, one model | |---|---| diff --git a/pipeline/build.sh b/pipeline/build.sh index 72c9b11..6e1ef10 100755 --- a/pipeline/build.sh +++ b/pipeline/build.sh @@ -1,9 +1,10 @@ #!/usr/bin/env sh -# Build every reader, every library and every writer, in both languages. +# Build the library (with its reader, ocda-read) and the writer, in both languages. # -# Each project is its own flake and builds on its own. The library has no -# public url yet, so the three that depend on it are given the local copy with -# --override-input; that argument goes away when it is published. +# Each project is its own flake and builds on its own. The writer's flakes lock +# the published library; here they are given the copy at $OCDA_LIB with +# --override-input, so that the library run and the library built into the +# writer are the same one. Where that fails, the build falls back to the lock. set -e . "$(dirname "$0")/config.sh" @@ -24,10 +25,6 @@ failed=0 # first, and the dump this pipeline reads was not there. build "$OCDA_LIB/dlang" build "$OCDA_LIB/gleam" -build "$SSP_READER/dlang" -build "$SSP_READER/gleam" -build "$DB_READER/dlang" -build "$DB_READER/gleam" build "$WRITERS/dlang" build "$WRITERS/gleam" exit $failed diff --git a/pipeline/config.sh b/pipeline/config.sh index 04c4bfd..ef81ee9 100644 --- a/pipeline/config.sh +++ b/pipeline/config.sh @@ -18,12 +18,11 @@ WRITERS=${WRITERS:-$(cd "$(dirname "$0")/.." && pwd)} # short name; where they sit relative to each other is the only local fact. SIBLINGS=${SIBLINGS:-$(cd "$WRITERS/.." && pwd)} OCDA_LIB=${OCDA_LIB:-$SIBLINGS/sisudoc-ocda-lib} -SSP_READER=${SSP_READER:-$SIBLINGS/sisudoc-ssp-reader} -DB_READER=${DB_READER:-$SIBLINGS/sisudoc-ocda-db-reader} # the abstraction artefacts to read: a directory holding .ssp files, and for # each of them a .ocda.db and, where there is one, a .sisupod beside it. This -# is what spine writes with --abstraction --ocda-db --pod2; see pipeline/README. +# is what spine writes with --show-abstraction --ocda-db --pod2; see +# pipeline/README. ARTEFACTS=${ARTEFACTS:-} # where the output goes. One directory per stage, so a run can be looked at. diff --git a/pipeline/one.sh b/pipeline/one.sh index 55e9ca6..a018fec 100755 --- a/pipeline/one.sh +++ b/pipeline/one.sh @@ -7,8 +7,6 @@ # # read-d read-g ocda-read, the library's reader of every form; takes its # modes after the file: one.sh read-d <file> en --tree -# ssp-d ssp-g the .ssp readers, D and Gleam -# db-d db-g the .ocda.db readers # dump-d dump-g the library's canonical dump of the whole model # agree-d agree-g the library's two loaders against each other # text-d text-g the writers. Each takes a .ssp, a .ocda.db or a @@ -28,10 +26,6 @@ lang=${3:-} case $tool in read-d) shift; exec "$OCDA_LIB/dlang/result/bin/ocda-read" "$@" ;; read-g) shift; exec "$OCDA_LIB/gleam/result/bin/ocda-read" "$@" ;; - ssp-d) exec "$SSP_READER/dlang/result/bin/ssp-reader" "$file" ;; - ssp-g) exec "$SSP_READER/gleam/result/bin/ssp_reader" "$file" ;; - db-d) exec "$DB_READER/dlang/result/bin/ocda-db-reader" "$file" ;; - db-g) exec "$DB_READER/gleam/result/bin/ocda-db-reader" "$file" ;; dump-d) exec "$OCDA_LIB/dlang/result/bin/ocda-dump" "$file" $lang ;; dump-g) exec "$OCDA_LIB/gleam/result/bin/ocda-lib" dump "$file" $lang ;; agree-d) exec "$OCDA_LIB/dlang/result/bin/ocda-agree" "$file" $lang ;; diff --git a/pipeline/read.sh b/pipeline/read.sh index ddbebe4..451f725 100755 --- a/pipeline/read.sh +++ b/pipeline/read.sh @@ -1,10 +1,8 @@ #!/usr/bin/env sh -# Every reader and both libraries, over every document, into $OUT/read/. +# The library and its reader, over every document, into $OUT/read/. # # What each produces: # -# read/ssp-{d,g}/<doc>.txt the .ssp readers' summary -# read/db-{d,g}/<doc>.txt the .ocda.db readers' summary # read/dump-{d,g}/<doc>.ssp the library's dump, loaded from the .ssp # read/dump-{d,g}/<doc>.db the same, loaded from the .ocda.db # read/read-{d,g}/<doc>.ssp ocda-read --tree, reading the .ssp @@ -17,7 +15,7 @@ set -e . "$(dirname "$0")/config.sh" : "${ARTEFACTS:?set ARTEFACTS to a directory of .ssp and .ocda.db files}" mkdir -p "$OUT/read" -for d in ssp-d ssp-g db-d db-g dump-d dump-g read-d read-g; do mkdir -p "$OUT/read/$d"; done +for d in dump-d dump-g read-d read-g; do mkdir -p "$OUT/read/$d"; done list=$(find "$ARTEFACTS" -name '*.ssp' | sort) [ -n "$LIMIT" ] && list=$(echo "$list" | head -n "$LIMIT") @@ -27,13 +25,9 @@ for ssp in $list; do stem=$(basename "$ssp" .ssp); lang=${stem##*.}; uid=${stem%.*} db=$(find "$ARTEFACTS" -name "$uid.ocda.db" | head -1) n=$((n+1)) - sh "$(dirname "$0")/one.sh" ssp-d "$ssp" > "$OUT/read/ssp-d/$stem.txt" - sh "$(dirname "$0")/one.sh" ssp-g "$ssp" > "$OUT/read/ssp-g/$stem.txt" sh "$(dirname "$0")/one.sh" dump-d "$ssp" > "$OUT/read/dump-d/$stem.ssp" sh "$(dirname "$0")/one.sh" dump-g "$ssp" > "$OUT/read/dump-g/$stem.ssp" [ -n "$db" ] || continue - sh "$(dirname "$0")/one.sh" db-d "$db" > "$OUT/read/db-d/$stem.txt" - sh "$(dirname "$0")/one.sh" db-g "$db" > "$OUT/read/db-g/$stem.txt" sh "$(dirname "$0")/one.sh" dump-d "$db" "$lang" > "$OUT/read/dump-d/$stem.db" sh "$(dirname "$0")/one.sh" dump-g "$db" "$lang" > "$OUT/read/dump-g/$stem.db" # the four-way agreement, which is the library's whole claim |
