1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
|
# ocda-lib, in Gleam
The abstraction's model and both its loaders, as a Gleam library, with its
reader and the two programs that put it to the question.
| | |
|------------------|-----------------------------------------------------------------------------------------|
| `ocda/model` | the document, the object, the image. Depends on nothing but the standard library |
| `ocda/load_ssp` | text -> model |
| `ocda/load_db` | sqlite -> model |
| `ocda/load` | what a path is, and loading it |
| `ocda/inventory` | what a file holds beyond the one document: its languages, and what a `.ocda.db` carries |
| `ocda/dump` | the model written out canonically, as an oracle |
| `read/main` | `ocda-read`, the reader |
| `ocda_ffi.erl` | a sha256, from OTP's crypto, which the standard library lacks |
## Use
```gleam
import ocda/load
import ocda/model
case load.load(path, "en", False) {
Error(why) -> io.println_error(why)
Ok(doc) -> {
// and from here a writer neither knows nor cares which of the two
// files it was handed
list.each(model.section(doc, "body"), fn(obj) { ... })
}
}
```
## The reader
```sh
nix build
./result/bin/ocda-read [--tree|--outline|--languages|--files|--report-unknown]
<file>.ssp|<file>.ocda.db|<file>.zip [lang]
```
`ocda-lib read`, installed also as `ocda-read`. The modes the Gleam readers
had, over the library, so each reads every form this library reads; the
others are the D reader's. **Its output is the D `ocda-read`'s, byte for
byte**, in every mode both have. In `src/read/`, because Gleam compiles
nothing outside `src/`; a program depending on the library compiles it too,
and does not use it.
A `.sisupod` it refuses, saying why: the zstd frame, below.
## The two programs
```sh
nix build
./result/bin/ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
./result/bin/ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
```
`agree` asks whether the two loaders give the same model. `dump` writes
the whole model out canonically, and is the stronger check of the two,
because it is **byte for byte what the D library's `ocda-dump` writes**.
So the four questions can all be asked with `diff`:
| | |
|-----------------------------|-------------------------|
| D: `.ssp` == `.ocda.db` | every document-language |
| Gleam: `.ssp` == `.ocda.db` | every document-language |
| `.ssp`: D == Gleam | every document-language |
| `.ocda.db`: D == Gleam | every document-language |
## Deliberately a mirror of the D library
Field for field, name for name, including where that is not the most
Gleamish choice:
- `text` is one `String` with the lines joined by newline, not a
`List(String)`. So an object whose only text line is blank is
indistinguishable from an object with no text, *in both languages, in the
same place*. A difference that existed in one and not the other would be
worse than a loss both share, because it would show up in every dump and
mean nothing.
- the properties are typed fields rather than a list of pairs. A writer
wants `obj.parent_ocn`, not a lookup that might miss.
The point is that the two agree. Making them agree is much cheaper when
they are shaped alike, and the shape is the specification's, not either
language's.
## What the agreement found
Four disagreements, none of which a comparison of section counts or of a
handful of fields would have shown. All four are recorded in the D
library's history; the short version:
1. `.identifier` and `.anchor` are omitted by the text form when they can
be worked out (the identifier is the ocn, the anchor is the segment).
Both loaders now derive them. Over one document that is 1,314
identifiers.
2. `make` and `doc_has` are one metadata table in the database,
distinguished by a key prefix. Both loaders unprefix them.
3. The JSON columns must be *parsed*, not scanned: the encoder writes `/`
as `\/` and every heading with a slash in it came back wrong.
4. `translation.ocn_aligned` is in the database and cannot be in a `.ssp`,
which is one language. It is a fact about the file, so neither loader
puts it in the document.
## Build
```sh
nix build # sandboxed, no network
nix develop -c gleam test
```
Shipped as an Erlang shipment rather than an escript, because sqlite on the
BEAM is a NIF and a NIF cannot be loaded from inside an escript archive.
The build is nix-gleam's, kept in `nix-gleam/` (see its README): each Hex
dependency is fetched on its own, with the checksum gleam wrote into
`manifest.toml`, so there is no hash in `flake.nix` to keep up to date.
## Format
Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model,
`ssp-format-specification.md` and `ocda-db-format-specification.md` for the
two serialisations. At format **2.1**.
|