format 2.0: sample writers over the abstraction, in D and in Gleam

Writers over sisudoc-ocda-lib: functions from the document model to
bytes, in five formats, one program, ocda-write, in each language.
Begun 2026-09-25; this is the writer as it stood at format 2.0.

  ocda-write --text|--md|--html|--epub|--typst <file> [lang]

They are samples. Spine is the output pipeline and nothing here
replaces it. What they show is that the abstraction is sufficient:
given a .ssp, a .ocda.db or a .sisupod and nothing else, a few hundred
lines per format make a complete document. A writer is never told
which form it was given, and every writer is run against every form
of every document.

Each format is held to an oracle that can say no:

  --text   the words equal the html writer's objects, as a multiset;
           every object number exactly once
  --md     CommonMark; every (#x) resolves to an <a id=x>, no id twice
  --html   it parses; every internal link resolves; the objects' text
           equals spine's scroll html
  --epub   epubcheck
  --typst  typst itself: a link to a missing label does not compile

and in every format the same bytes from every input form, and the D
and Gleam writers byte for byte alike. Over the markup document
samples every check holds for every document-language. The two
languages' writers were not written from each other, and their
agreement is the strongest check: twice it caught what nothing else
did (an ampersand in an attribute mangled by Erlang's replacement
string; typst label names built from bytes in one language and
graphemes in the other).

Byte identity with spine is not the aim, by decision. The text writer
began byte identical to spine --text, and was rewritten: a note is
written once, markup comments and dummy headings are not written, an
image keeps its caption, a table is padded to display width. The html
writer is valid html5 rather than a copy of spine's (h1..h6 where
spine writes <h0>; one charset; no placeholder metadata; an id for
every anchor tag; self-contained). Each difference is a better
presentation, and the code says why at the point of choice.

The typst writer puts the object number in the margin as the label
every citation points at, a note as a real footnote with a link back,
and a table of contents that locates by object number rather than by
page, as the markdown writer's does: a page number is a fact about one
typesetting, the object number is not. The markdown writer is
CommonMark with yaml front matter; a note is at its reference, with
the document's own number, and --image-prefix gives the image path,
which the model rightly does not know.

Holding the writers to these checks found defects in spine, several
since fixed there, and spine's typst and markdown writers were ported
from these.

Use: nix build, in dlang/ or in gleam/. Each flake takes the library
from git.sisudoc.org, and flake.lock pins the commit of it these were
built and checked against. gleam/ builds with nix-gleam's builder,
kept in gleam/nix-gleam/ (MIT); in a working tree, gleam.toml reads
the library through the symlink ./ocda-lib.

See README.md, dlang/README.md and gleam/README.md, and pipeline/
for building everything and running it over a directory of artefacts.

(assisted by Claude-Code)

1 files changed, 113 insertions, 0 deletions

diff --git a/gleam/README.md b/gleam/README.md
new file mode 100644
index 0000000..8a2bea9
--- /dev/null
+++ b/gleam/README.md
@@ -0,0 +1,113 @@
+# ocda-write, in Gleam
+
+Sample writers over the abstraction:
+
+    ocda-write --text|--md|--html|--epub|--typst <file> [lang]
+    ocda-write --md <file> [lang] [--image-prefix <path>]
+
+writes a document out, from any serialisation of its abstraction.
+
+## Modules
+
+| | |
+|---|---|
+| `write/text` | the model -> spine's plain text, byte for byte |
+| `write/html` | the model -> one self-contained, valid html5 file |
+| `write/epub` | the model -> an epub3, one file per segment |
+| `write/markdown` | the model -> CommonMark, object numbers and all |
+| `write/typst` | the model -> typst markup, for a pdf |
+| `write/zip` | writing a zip, stored only, with no timestamps |
+| `write/rgx` | the internal markup, as compiled patterns |
+| `ocda_write` | the program: which format, which file, which language |
+
+`write.text.write` takes an `ocda/model.Document` and returns a `String`. It
+is given no file, no path and no option: a writer is a function from the
+model to bytes.
+
+## Build and run
+
+```sh
+nix build --override-input ocda-lib path:../../sisudoc-ocda-lib
+./result/bin/ocda-write --text <doc>.ssp
+./result/bin/ocda-write --text <doc>.ocda.db de
+./result/bin/ocda-write --md <doc>.ssp
+./result/bin/ocda-write --epub <doc>.ocda.db en --out <doc>.epub
+./result/bin/ocda-write --typst <doc>.ocda.db en --out out/<doc>.typ
+```
+
+The override applies to `nix develop` too, until the library is published:
+
+```sh
+nix develop --override-input ocda-lib path:../../sisudoc-ocda-lib -c gleam build
+```
+
+## The check
+
+```sh
+./result/bin/ocda-write --text <doc>.ssp | cmp - <the D writer's>
+```
+
+**36 of 36 document-languages, from both serialisations, byte identical to the
+D writer** - which is the check that matters now, since byte identity with
+`spine --text` is no longer the target.
+
+And for `--md`, `--html`, `--epub` and `--typst`, **byte identical to the D
+writer on all 36**, which is the agreement that says the model is enough. None
+of them is held to spine's bytes: the epub's judge is epubcheck, which read the
+specification so that the writer need not be trusted to have, and the pdf's is
+typst, where a link to a label that does not exist is a compile error. See the
+repository README for what they do differently and why.
+
+## The library, as a path dependency inside the package
+
+`gleam.toml` names it `./ocda-lib`, and that is a **symlink** to
+`../../sisudoc-ocda-lib/gleam`, kept out of git.
+
+It has to be inside the package: a path dependency reaching outside cannot
+be seen by a sandboxed build, and a store path in `gleam.toml` would change
+with every change to the library. So the path is stable, and what sits at
+the end of it is arranged by whoever is building: a symlink in a working
+tree, a copy from the store in the nix build, where nix-gleam puts the
+library in place as a local package.
+
+Create it after cloning:
+
+```sh
+ln -s ../../sisudoc-ocda-lib/gleam ocda-lib
+```
+
+## Places this is not idiomatic Gleam, on purpose
+
+- **A column's width is the widest cell in bytes, and the padding is counted
+  in codepoints.** Not a typo. Spine takes the width from `cell.length`,
+  which for a D string is a byte count, and pads with `format("%-*s")`, which
+  counts codepoints. So a cell holding `©` is padded one place further than
+  one holding `c`. Measured rather than guessed, and reproduced rather than
+  corrected, because the output is compared byte for byte.
+- **The escape in `write/typst` walks codepoints and folds a list**, where
+  everything else here is a regular expression. It has to: the escaping skips
+  protected runs, the protection nests, and the D writer walks codepoints, so
+  a grapheme walk would part company with it over a combining mark. `label_of`
+  walks codepoints for the same reason, and once did not: it used graphemes
+  while the D one used bytes, and five documents with typographic quotation
+  marks in a book index entry disagreed.
+- **`gleam/regexp` has no replace-with-function**, so `replace_all` here is
+  `scan` plus a walk over the subject. It is the one piece of scaffolding
+  this writer needs that the D one gets from Phobos.
+- **The claimed-id set is threaded across segments, not restarted in each.**
+  An id has to be unique in a *file* to be valid, and unique in the
+  *publication* to be useful: a link carries a file and a fragment, and an
+  anchor with two destinations has none. Restarting the set per segment gave
+  twelve documents that epubcheck passed and the D writer disagreed with.
+- **Every replacement goes through the function form, never the string
+  form.** Erlang's `re:replace` reads `&` in a replacement *string* as the
+  whole match, so replacing `&` with `"&amp;"` yields `"quot;` where the
+  match was a quotation mark. Byte identity with the D writer is what caught
+  it, on one document's copyright notice, and nothing else would have: the
+  file still parsed and still read correctly everywhere else.
+
+## Shipping
+
+An Erlang shipment rather than an escript, because the library reaches
+sqlite through a NIF and a NIF cannot be loaded from inside an escript
+archive.