1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
|
# The pipeline: how to run the library, its reader and the writers
Fixed scripts rather than a test suite. They produce output that is meant to
be **looked at**, and they double as the answer to "what do I type?".
```
config.sh where everything is, and where the output goes
build.sh build the library and the writer, in both languages
one.sh one tool, one document - the answer to "how do I run this?"
read.sh the library and its reader, over every document
write.sh every writer, in both languages, over every document
pdf.sh compile the .typ files that write.sh produced
run.sh all of the above, in order
```
## What you need first
An **abstraction** to read: a directory holding a `.ssp` for each
document-language and a `.ocda.db` for each document, and, where there is
one, a `.sisupod` beside it. That is what spine writes:
```sh
spine --show-abstraction --ocda-db --pod2 --output=<dir> <pod>...
```
Everything downstream reads those files and nothing else. **No tool here
parses markup**: parsing is spine's work, and the point of the exercise is
that the abstraction is enough without it.
## Running it
```sh
ARTEFACTS=<dir> OUT=<dir> sh pipeline/run.sh
```
`ARTEFACTS` is the directory above. `OUT` is where the output goes and
defaults to `./pipeline-out`. Both, and every path to a sibling repository,
are variables with defaults in `config.sh`; override them in the environment
rather than editing anything.
`LIMIT=3` takes the first three documents, which is what to use while looking
at something.
## Where the output goes
```
$OUT/read/dump-d/ dump-g/ the library's dump, from each serialisation
$OUT/write/text/{d,g}/ <doc>.txt
$OUT/write/md/{d,g}/ <doc>.md
$OUT/write/html/{d,g}/ <doc>.html
$OUT/write/epub/{d,g}/ <doc>.epub
$OUT/write/typst/{d,g}/<doc>/ <doc>.typ, and image/ beside it
$OUT/write/pdf/ <doc>.pdf
```
`d` is the D implementation and `g` the Gleam one, throughout.
## One tool, one document
```sh
sh pipeline/one.sh text-d <doc>.ssp
sh pipeline/one.sh md-g <doc>.ocda.db en
sh pipeline/one.sh html-d <doc>.sisupod
sh pipeline/one.sh dump-d <doc>.ocda.db en
sh pipeline/one.sh agree-g <doc>.ssp <doc>.ocda.db
sh pipeline/one.sh read-d <doc>.sisupod en --tree
```
The tool names are `<format>-<language>`: `text`, `md`, `html`, `epub`,
`typst` for the writers, `read` for the library's reader (ocda-read, which
reads every form and takes its modes after the file), and `dump`, `agree` for
the library; `-d` for D and `-g` for Gleam.
Output goes to stdout, so a comparison is a pipeline. The epub and the typst
name a file instead, because an epub is a zip and a `.typ` names its images by
path:
```sh
"$WRITERS/dlang/result/bin/ocda-write" --epub <doc>.ocda.db en --out x.epub
"$WRITERS/dlang/result/bin/ocda-write" --typst <doc>.ocda.db en --out d/x.typ
typst compile --root d d/x.typ d/x.pdf
```
## What the counts at the end mean
Each stage prints a few numbers. They are not a pass, they are a way to see a
regression without reading every document.
| `all four dumps identical` | the library's whole claim: two serialisations, two languages, one model |
|---|---|
| `ocda-read, all four identical` | the reader's tree of a document, the same from its .ssp and its .ocda.db, in D and in Gleam |
| `D == Gleam, <format>` | the writers agree, in unrelated languages, neither written from the other. This is what says the model is sufficient |
| `.ssp and .ocda.db give the same .typ` | a writer is not told which serialisation it was handed |
| `typst compiled silently` | a link to a label that does not exist is a typst compile error, so this says every internal link in every document resolves |
## What this is not
It is not the project's checking. The sweeps that hold each writer against an
oracle - `spine --text`, spine's scroll html, epubcheck, typst - are separate
and stricter. This is the pipeline you run to *look* at all of it at once.
|
