summaryrefslogtreecommitdiffhomepage
path: root/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'README.md')
-rw-r--r--README.md77
1 files changed, 58 insertions, 19 deletions
diff --git a/README.md b/README.md
index 39da9f9..efa9e70 100644
--- a/README.md
+++ b/README.md
@@ -8,7 +8,7 @@ loaders for both of the forms it is written in.
| | |
|---|---|
| [`dlang/`](dlang/) | D |
-| `gleam/` | to come |
+| [`gleam/`](gleam/) | Gleam |
| `ruby/` | to come |
## What it is for
@@ -29,38 +29,77 @@ cannot, since spine 0.25.0.
That is the whole of the idea. The readers become thin programs over this,
and the writers are functions from the model to bytes.
-## The check that matters
+## The checks that matter
-The claim above is only true if the two loaders agree, so the library asks:
+The claim above is only true if the loaders agree, so the library asks. Two
+programs, in each language:
```sh
-nix build && ./result/bin/ocda-agree <doc>.ssp <doc>.ocda.db [lang]
+ocda-agree <doc>.ssp <doc>.ocda.db [lang] # do the two loaders agree?
+ocda-dump <file>.ssp|<file>.ocda.db [lang] # the whole model, canonically
```
-It compares the two models object by object: citation number, kind, type,
-part, parent, heading level, digest, text, ancestors and image count.
+`agree` compares two models within one language. **`dump` is the stronger
+of the two**, and the reason it exists: it writes the entire loaded model
+out in a fully determined form, and the D and the Gleam library write it
+*byte for byte alike*. So four questions can be asked, all of them with
+`diff`:
-**Over the reference collection the two loaders agree on 36 of 36
-document-languages.** They did not at first: see below.
+| | |
+|---|---|
+| D: `.ssp` == `.ocda.db` | **36 of 36** document-languages |
+| Gleam: `.ssp` == `.ocda.db` | **36 of 36** |
+| `.ssp`: D == Gleam | **36 of 36** |
+| `.ocda.db`: D == Gleam | **36 of 36** |
-## What the check found, first time it was run
+None of those held at first.
-Eight of thirty-six disagreed, and the cause was a defect in the `.ssp`
-loader rather than a difference between the formats.
+## What the checks found
+
+### the ten-field check: a blank first text line
An object whose **first text line is blank** lost it. The loader decided
whether to start a new line by asking whether the accumulated text was
empty, which is not the same question as whether a line has been seen. Of
the sample collection **seven documents have verse opening on a blank
-line**, and in every one of them the blank was dropped.
+line**, and in every one of them the blank was dropped. Eight of
+thirty-six documents disagreed.
-Nothing else had caught it. The two `.ssp` readers were compared on
-*section counts*, which this does not change; spine's own output was never
-diffed against the reader's. It took loading the same document two ways and
-comparing the objects.
+Nothing else had caught it. The two `.ssp` readers had been compared on
+*section counts*, which this does not change. It took loading the same
+document two ways and comparing the objects.
Fixed here and in `sisudoc-ssp-reader/dlang`, which carried the same code.
+### the whole-model dump: four more
+
+The ten-field check was enough for that and not enough for anything
+subtler. Dumping the whole model found four further disagreements between
+the two loaders:
+
+1. **`.identifier` and `.anchor`.** The `.ssp` emits a property only when it
+ carries information, so the identifier is absent when it is simply the
+ ocn and the anchor is absent when it is the segment name; the database
+ stores both unconditionally. Over one document that is **1,314
+ identifiers and 69 anchors**. The loaders now derive them, which is what
+ the model means.
+2. **`make` and `doc_has`.** The database has one metadata table where the
+ text form has three blocks, and tells them apart by a key prefix. The
+ loaders unprefix them, so a writer asks `make["footer"]` whichever
+ serialisation it was handed.
+3. **The JSON columns were scanned rather than parsed.** Phobos writes `/`
+ as `\/`, so every heading containing a slash came back with a backslash
+ in it: **all 36 document-languages** differed on
+ `heading_ancestors_text` and nothing else had shown it.
+4. **`translation.ocn_aligned`** is in the database and cannot be in a
+ `.ssp`, which is one language and cannot know whether the languages
+ share a numbering. It is a fact about the *file*, so the loaders leave
+ it out rather than letting a writer tell which serialisation it was
+ given.
+
+The first three are defects. The fourth is a real asymmetry between the two
+serialisations, and naming it is the useful part.
+
## What it reads, and what it does not
| | |
@@ -71,14 +110,14 @@ Fixed here and in `sisudoc-ssp-reader/dlang`, which carried the same code.
| `.sst`, `.ssm` markup | **no, and not planned**: that is a parser, which is spine's work |
| a url | no |
-`ocda/load.d` is the one place that decides what a path is, so a new input
-type is added once and every reader and writer gains it.
+`ocda/load` is the one place in each language that decides what a path is,
+so a new input form is added once and every reader and writer gains it.
## The format
Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model
this library implements, and the two format specifications for the loaders.
Section 4 of `ocda-db-format-specification.md` gives the column by column
-correspondence that `load_db.d` follows.
+correspondence the `load_db` of each language follows.
At format **2.x**.