summaryrefslogtreecommitdiffhomepage
path: root/gleam/README.md
blob: d586bd38bc4ff28c5557f595f79ff16b4b088d92 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
# ocda-lib, in Gleam

The abstraction's model and both its loaders, as a Gleam library, and the
two programs that put it to the question.

|                 |                                                                                  |
|-----------------|----------------------------------------------------------------------------------|
| `ocda/model`    | the document, the object, the image. Depends on nothing but the standard library |
| `ocda/load_ssp` | text -> model                                                                    |
| `ocda/load_db`  | sqlite -> model                                                                  |
| `ocda/load`     | what a path is, and loading it                                                   |
| `ocda/dump`     | the model written out canonically, as an oracle                                  |

## Use

```gleam
import ocda/load
import ocda/model

case load.load(path, "en", False) {
  Error(why) -> io.println_error(why)
  Ok(doc) -> {
    // and from here a writer neither knows nor cares which of the two
    // files it was handed
    list.each(model.section(doc, "body"), fn(obj) { ... })
  }
}
```

## The two programs

```sh
nix build
./result/bin/ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
./result/bin/ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
```

`agree` asks whether the two loaders give the same model. `dump` writes
the whole model out canonically, and is the stronger check of the two,
because it is **byte for byte what the D library's `ocda-dump` writes**.
So the four questions can all be asked with `diff`:

|                             |                             |
|-----------------------------|-----------------------------|
| D: `.ssp` == `.ocda.db`     | 36 of 36 document-languages |
| Gleam: `.ssp` == `.ocda.db` | 36 of 36                    |
| `.ssp`: D == Gleam          | 36 of 36                    |
| `.ocda.db`: D == Gleam      | 36 of 36                    |

## Deliberately a mirror of the D library

Field for field, name for name, including where that is not the most
Gleamish choice:

- `text` is one `String` with the lines joined by newline, not a
  `List(String)`. So an object whose only text line is blank is
  indistinguishable from an object with no text, *in both languages, in the
  same place*. A difference that existed in one and not the other would be
  worse than a loss both share, because it would show up in every dump and
  mean nothing.
- the properties are typed fields rather than a list of pairs. A writer
  wants `obj.parent_ocn`, not a lookup that might miss.

The point is that the two agree. Making them agree is much cheaper when
they are shaped alike, and the shape is the specification's, not either
language's.

## What the agreement found

Four disagreements, none of which a comparison of section counts or of a
handful of fields would have shown. All four are recorded in the D
library's history; the short version:

1. `.identifier` and `.anchor` are omitted by the text form when they can
   be worked out (the identifier is the ocn, the anchor is the segment).
   Both loaders now derive them. Over one document that is 1,314
   identifiers.
2. `make` and `doc_has` are one metadata table in the database,
   distinguished by a key prefix. Both loaders unprefix them.
3. The JSON columns must be *parsed*, not scanned: the encoder writes `/`
   as `\/` and every heading with a slash in it came back wrong.
4. `translation.ocn_aligned` is in the database and cannot be in a `.ssp`,
   which is one language. It is a fact about the file, so neither loader
   puts it in the document.

## Build

```sh
nix build            # sandboxed, no network
nix develop -c gleam test
```

Shipped as an Erlang shipment rather than an escript, because sqlite on the
BEAM is a NIF and a NIF cannot be loaded from inside an escript archive.
The dependency hash in `flake.nix` pins what `manifest.toml` pins; change
one and the other must change with it.

## Format

Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model,
`ssp-format-specification.md` and `ocda-db-format-specification.md` for the
two serialisations. At format **2.x**.