diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 77 |
1 files changed, 58 insertions, 19 deletions
@@ -8,7 +8,7 @@ loaders for both of the forms it is written in. | | | |---|---| | [`dlang/`](dlang/) | D | -| `gleam/` | to come | +| [`gleam/`](gleam/) | Gleam | | `ruby/` | to come | ## What it is for @@ -29,38 +29,77 @@ cannot, since spine 0.25.0. That is the whole of the idea. The readers become thin programs over this, and the writers are functions from the model to bytes. -## The check that matters +## The checks that matter -The claim above is only true if the two loaders agree, so the library asks: +The claim above is only true if the loaders agree, so the library asks. Two +programs, in each language: ```sh -nix build && ./result/bin/ocda-agree <doc>.ssp <doc>.ocda.db [lang] +ocda-agree <doc>.ssp <doc>.ocda.db [lang] # do the two loaders agree? +ocda-dump <file>.ssp|<file>.ocda.db [lang] # the whole model, canonically ``` -It compares the two models object by object: citation number, kind, type, -part, parent, heading level, digest, text, ancestors and image count. +`agree` compares two models within one language. **`dump` is the stronger +of the two**, and the reason it exists: it writes the entire loaded model +out in a fully determined form, and the D and the Gleam library write it +*byte for byte alike*. So four questions can be asked, all of them with +`diff`: -**Over the reference collection the two loaders agree on 36 of 36 -document-languages.** They did not at first: see below. +| | | +|---|---| +| D: `.ssp` == `.ocda.db` | **36 of 36** document-languages | +| Gleam: `.ssp` == `.ocda.db` | **36 of 36** | +| `.ssp`: D == Gleam | **36 of 36** | +| `.ocda.db`: D == Gleam | **36 of 36** | -## What the check found, first time it was run +None of those held at first. -Eight of thirty-six disagreed, and the cause was a defect in the `.ssp` -loader rather than a difference between the formats. +## What the checks found + +### the ten-field check: a blank first text line An object whose **first text line is blank** lost it. The loader decided whether to start a new line by asking whether the accumulated text was empty, which is not the same question as whether a line has been seen. Of the sample collection **seven documents have verse opening on a blank -line**, and in every one of them the blank was dropped. +line**, and in every one of them the blank was dropped. Eight of +thirty-six documents disagreed. -Nothing else had caught it. The two `.ssp` readers were compared on -*section counts*, which this does not change; spine's own output was never -diffed against the reader's. It took loading the same document two ways and -comparing the objects. +Nothing else had caught it. The two `.ssp` readers had been compared on +*section counts*, which this does not change. It took loading the same +document two ways and comparing the objects. Fixed here and in `sisudoc-ssp-reader/dlang`, which carried the same code. +### the whole-model dump: four more + +The ten-field check was enough for that and not enough for anything +subtler. Dumping the whole model found four further disagreements between +the two loaders: + +1. **`.identifier` and `.anchor`.** The `.ssp` emits a property only when it + carries information, so the identifier is absent when it is simply the + ocn and the anchor is absent when it is the segment name; the database + stores both unconditionally. Over one document that is **1,314 + identifiers and 69 anchors**. The loaders now derive them, which is what + the model means. +2. **`make` and `doc_has`.** The database has one metadata table where the + text form has three blocks, and tells them apart by a key prefix. The + loaders unprefix them, so a writer asks `make["footer"]` whichever + serialisation it was handed. +3. **The JSON columns were scanned rather than parsed.** Phobos writes `/` + as `\/`, so every heading containing a slash came back with a backslash + in it: **all 36 document-languages** differed on + `heading_ancestors_text` and nothing else had shown it. +4. **`translation.ocn_aligned`** is in the database and cannot be in a + `.ssp`, which is one language and cannot know whether the languages + share a numbering. It is a fact about the *file*, so the loaders leave + it out rather than letting a writer tell which serialisation it was + given. + +The first three are defects. The fourth is a real asymmetry between the two +serialisations, and naming it is the useful part. + ## What it reads, and what it does not | | | @@ -71,14 +110,14 @@ Fixed here and in `sisudoc-ssp-reader/dlang`, which carried the same code. | `.sst`, `.ssm` markup | **no, and not planned**: that is a parser, which is spine's work | | a url | no | -`ocda/load.d` is the one place that decides what a path is, so a new input -type is added once and every reader and writer gains it. +`ocda/load` is the one place in each language that decides what a path is, +so a new input form is added once and every reader and writer gains it. ## The format Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model this library implements, and the two format specifications for the loaders. Section 4 of `ocda-db-format-specification.md` gives the column by column -correspondence that `load_db.d` follows. +correspondence the `load_db` of each language follows. At format **2.x**. |
