diff options
| author | Ralph Amissah <ralph.amissah@gmail.com> | 2026-09-25 22:28:50 -0400 |
|---|---|---|
| committer | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-03 21:50:59 -0400 |
| commit | 5a5036e53aa18afa61d6e475336cd96c115dac00 (patch) | |
| tree | 669a9190703b6833591311c73f4646a123c11dc3 /ocda-db-format-specification.md | |
| parent | .ocda.db specified & the two serialisations agree (diff) | |
derived values that the model left unstated
The model says nothing about an object's identifier or its anchor, and
both are derived: the identifier is the ocn where there is one, the
anchor is the segment name. The .ssp omits each where it can be worked
out; the .ocda.db states both on every row.
This is existing behaviour, not a new rule. Spine's own .ssp reader has
always filled both in, which is why the database carries them: the
database is written from the .ssp read straight back. It should have
been written down first. Two readers written independently from these
documents disagreed for want of it, on nearly every object, and nothing
short of comparing whole models showed it.
ocda-model.md gains 3.1 the identifier, 7.1 the anchor and 9 derived
values, with the loading rule in a table; "what the model does not hold"
becomes 10.
The two format specifications gain what a reader needs and did not have:
- the .ssp's property table now says what the absent value is, not only
when it is written;
- the .ocda.db gains 3.2.1, the metadata key prefix that carries what
the .ssp puts in four blocks, which a reader must strip to give one
model from either file, and the note that a value here may carry
whitespace the text form cannot represent;
- 3.3 gains the element types of each JSON column and a warning to
parse rather than scan: the encoder writes "/" as "\/" and every
document in the reference collection has a heading containing one;
- 4.1 names the three places the two files are not symmetric,
including translation.ocn_aligned, which a .ssp cannot hold because
it is one language, and which a reader should keep out of the
document model so that nothing built on the model can tell which
file it was given.
(assisted by Claude-Code)
Diffstat (limited to 'ocda-db-format-specification.md')
| -rw-r--r-- | ocda-db-format-specification.md | 83 |
1 files changed, 83 insertions, 0 deletions
diff --git a/ocda-db-format-specification.md b/ocda-db-format-specification.md index ebeeaa6..eacc618 100644 --- a/ocda-db-format-specification.md +++ b/ocda-db-format-specification.md @@ -101,6 +101,33 @@ written with `doc_id` null: `schema.name`, `schema.version` and `UNIQUE(doc_id, key)` does not constrain them, and a partial unique index over `key` alone `WHERE doc_id IS NULL` is what makes them singular. +#### 3.2.1 The key prefix carries what the `.ssp` puts in blocks + +The `.ssp` has four preamble blocks, `@source`, `@meta`, `@make` and +`@doc_has`. The database has **one** `metadata` table, and tells them apart +by a **prefix on the key**: + +| `.ssp` block | database key | +|--------------|--------------| +| `@meta` | the key as it stands: `title.main`, `creator.author`, ... | +| `@make` | `make.` + the key: `make.footer`, `make.doc_type` | +| `@doc_has` | `doc_has.` + the key: `doc_has.images`, `doc_has.tables` | +| `@source` | `source.` + the key: `source.language`, `source.digest` | + +**A reader must strip the prefix and restore the grouping**, or a writer +over it has to ask `metadata["make.footer"]` from a database and +`make["footer"]` from a `.ssp`, which is exactly the difference the two +serialisations exist not to make. + +`source.` also carries two keys with no `@source` counterpart, +`source.filename` and `source.ssp_digest`; the `.ssp` puts the first in its +`% Source:` comment and cannot hold the second, which is a digest *of* it. + +Values may carry leading or trailing whitespace here where the `.ssp` +cannot represent it, its `key: value` line being unable to distinguish a +value that begins with a space from the separator. **Strip on load**, as a +reader of the text form necessarily does. + ### 3.3 Fixed-width arrays are JSON `children`, `ancestors`, `ancestors_collapsed`, `dom_status`, @@ -108,6 +135,33 @@ over `key` alone `WHERE doc_id IS NULL` is what makes them singular. `table_aligns` are stored as **JSON arrays**, with validity checks, so that `json_extract`, `json_each` and `json_array_length` reach inside them. +> **Parse them as JSON.** A scan that strips `[`, `]`, `,` and `"` looks +> adequate and is wrong on the first escape: the encoder writes `/` as +> `\/`, so `FREE/OPEN-SOURCE SOFTWARE` comes back as +> `FREE\/OPEN-SOURCE SOFTWARE`. All 36 document-languages of the reference +> collection contain at least one, and the defect is invisible to any check +> that does not compare the field itself. +> +> Note also that `table_widths` is an array of **integers except where the +> columns divide evenly**, when it is `[33.3333,33.3333,33.3333]`. A decoder +> that accepts strings and integers and then falls back to a default turns +> a value it could not read into one that looks read. + +The **element types**, so that a decoder can be written to accept exactly +what occurs: + +| column | elements | +|--------|----------| +| `children`, `ancestors`, `ancestors_collapsed`, `dom_status`, `dom_status_collapsed` | integers | +| `table_widths` | integers, or floats where the columns divide evenly | +| `table_aligns` | strings, one of `l` `c` `r` | +| `heading_ancestors_text` | strings, eight of them, empty where that level is not an ancestor | + +The `.ssp` writes the first three groups **space separated** and +`heading_ancestors_text` **`|` separated**. The empty slots of the last are +significant: they say which levels a heading sits under, so the separators +carry as much as the text does. + The open-ended lists are **rows in the four child tables** instead of columns: images, links, anchor tags and sub-table-of-contents entries. Each child table carries `seq`, which preserves the order they appeared in. @@ -159,6 +213,35 @@ object's position within its section, and `(doc_id, section, seq)` is what identifies an object when `ocn` will not (see [`ocda-model.md`](ocda-model.md) section 3). +### 4.1 Where the two are not symmetric + +"Nothing is lost in either direction" is true of the model, and there are +three places where the *files* nonetheless differ. A reader must know all +three, because each of them made two independently written readers disagree. + +**1. The database states what the `.ssp` derives.** `identifier` and +`anchor` are written on every row. The `.ssp` omits each where it can be +worked out, the identifier being the ocn and the anchor the segment name +(sections 6.1 and 6.3 of +[`ssp-format-specification.md`](ssp-format-specification.md), section 9 of +[`ocda-model.md`](ocda-model.md)). In `free_culture.lawrence_lessig` that is +**1,314 identifiers and 69 anchors** stated here and absent there. Neither +file is the poorer; a reader that does not derive is. + +**2. `translation.ocn_aligned` is here and cannot be in a `.ssp`.** It says +whether the languages of a document share one numbering, and a `.ssp` is one +language. It is therefore a fact about **the file**, not about the document +in it, and it is the one such fact either serialisation carries. + +A reader should **keep it out of the document model**, for the same reason +the `schema.*` and `build.*` rows are kept out: a writer over the model must +not be able to tell which serialisation it was handed, and a field that +exists only when the input was a database is exactly that tell. The model +has no place for collection-wide facts yet; when it grows one, this belongs +there. + +**3. Whitespace and grouping**, as section 3.2.1 gives them. + ## 5. The `files` table: what it carries, and how | role | named by | `compression` | `data` holds | |
