<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sisudoc-ocda-lib, branch claude-compact</title>
<subtitle>SiSU Spine: tools</subtitle>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-ocda-lib/'/>
<entry>
<title>copyright</title>
<updated>2026-10-13T00:24:11+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-10-13T00:24:11+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-ocda-lib/commit/?id=5ee4372851486da97d64a2a2d1d45df74e9ab919'/>
<id>5ee4372851486da97d64a2a2d1d45df74e9ab919</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>the abstraction as a library, and a defect it found at once</title>
<updated>2026-10-05T04:53:04+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-25T20:34:44+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-ocda-lib/commit/?id=f7da955e7ddcdd0da83c3b318418ec86822ac731'/>
<id>f7da955e7ddcdd0da83c3b318418ec86822ac731</id>
<content type='text'>
The readers each read one serialisation and print.  A writer needs
neither: it needs the document.  So the model and both loaders live
here, and a writer cannot tell which serialisation it was handed,
exactly as spine own writers cannot.

ocda.model imports nothing, which is the property that makes writers
possible: they depend on the model and on neither loader.  The model
and the .ssp loader are the ssp-reader code, moved and renamed to the
level they belong to: the model is neither the .ssp nor the .ocda.db.
ocda.load_db is new, and returns the same value from sqlite that
load_ssp returns from text.  ocda.load is the one place that decides
what a path is.

The claim is only true if the two loaders agree, so test/agree.d asks:
load a document both ways and compare the models object by object.

It found a real defect on its first run.  Eight of thirty-six
disagreed, and the cause was the .ssp loader, not the formats: an
object whose first text line is blank lost it, the loader having
decided whether to start a new line by asking whether the accumulated
text was empty, which is a different question from whether a line has
been seen.  Seven documents of the sample set have verse opening on a
blank line and every one of them was wrong.

Nothing else had caught it: the two .ssp readers were compared on
section counts, which the bug does not disturb.  It took loading one
document two ways and comparing the objects.  Fixed here, and in
ssp-reader/dlang which carried the same code.  36 of 36 now agree.

docs: the repository is sisudoc-ocda-lib

Prefix the repository name, as every other repository of this
project's own is prefixed. This one needed it most: "ocda-lib" is a
three letter acronym and the word "lib", and in a clone, away from
the parent directory that supplied the context, it reads as a generic
library belonging to no-one.

The dub path-dependency example and the note on where the blank-line
fix also went follow the new sibling name.

Binary, package and module names are unchanged: ocda-agree and the
ocda.* modules are read with the repository already in hand.

style: restore the comment arrows, double-encoded by an edit

The /+ ↓ +/ arrows had each of their three UTF-8 bytes re-encoded as
UTF-8 in turn, by an in-place edit that read the file as latin-1. 13
comments in load_ssp.d. Comment text only; the parser is unchanged and
the agreement check still passes.

ocda-dump, and the four disagreements it found

The agreement check compared ten fields of each object. That was enough
to find the blank-first-line defect and not enough for anything
subtler, so this adds ocda-dump: the whole loaded model written out
canonically, every field, sorted dictionaries, sections in format
order, defaults omitted. It is an oracle, not an output format. Two
implementations of this model can now be compared with diff.

Run against the reference collection it found four disagreements
between the two loaders, none of which the ten-field check could see.

1. .identifier and .anchor. The .ssp emits a property only when it
   carries information, so the identifier is absent when it is simply
   the ocn and the anchor is absent when it is the segment name; the
   database stores both unconditionally. Over one document that is
   1,314 identifiers and 69 anchors. The loader now derives them, which
   is what the model means: an object's identifier is its ocn unless it
   says otherwise.

2. make and doc_has. The database has one metadata table where the
   text form has three blocks, and tells them apart by a key prefix.
   The loader now unprefixes them, so a writer asks doc.make["footer"]
   whichever serialisation it was handed and never
   doc.metadata["make.footer"].

3. The JSON columns were read by a character scan that stripped
   brackets and quotes, which is wrong the moment a value holds an
   escape: JSONValue.toString escapes "/" as "\/", so every heading
   containing a slash came back with a backslash in it. All 36
   document-languages differed on heading_ancestors_text. Now parsed as
   JSON.

4. translation.ocn_aligned is in the database and not in the .ssp,
   legitimately: it says whether the languages share one numbering, and
   a .ssp is one language and cannot know. It is a fact about the file
   rather than the document, so the loader leaves it out rather than
   letting a writer tell which serialisation it was given.

Metadata values are also stripped now, as the text loader strips them:
one topic_register value carried surrounding spaces in the database
that the text form cannot represent.

After all four: 36 of 36 document-languages give byte identical dumps
from their .ssp and from their .ocda.db.

ocda-lib in Gleam, and four agreements of 36 of 36

The model, both loaders, the dispatch and the canonical dump, in Gleam.
A deliberate field-for-field mirror of the D library, including where
that is not the most idiomatic Gleam: text is one string with the lines
joined by newline rather than a list of lines, and the properties are
typed fields rather than a list of pairs. A difference that existed in
one language and not the other would show in every dump and mean
nothing, so the two lose the same distinctions in the same places.

The dump is what makes the agreement testable. Both languages write it
byte for byte alike, so four questions are diff, and over the reference
collection all four hold:

  D:       .ssp == .ocda.db      36 of 36
  Gleam:   .ssp == .ocda.db      36 of 36
  .ssp:      D == Gleam          36 of 36
  .ocda.db:  D == Gleam          36 of 36

and a fifth, that the two report exactly the same ignored properties
with the same counts.

The Gleam .ssp loader agreed with the D one at the first attempt. The
Gleam .ocda.db loader had one defect of its own, found by the sweep and
worth recording because of its shape: a JSON element decoder handled
strings and integers and fell through to a fallback that yielded "".
table_widths is an integer array except where the columns divide
evenly, when it is [33.3333,...], and one table in the collection does
that. Its widths came back as three empty strings, so a value that
could not be read turned into one that looked read. That is the failure
the unknown counts exist to prevent, and a decoder fallback quietly
reintroduces it.

Shipped as an Erlang shipment rather than an escript, as the .ocda.db
reader is and for the same reason: sqlite on the BEAM is a NIF and a
NIF cannot load from inside an escript archive.

gleam.toml and the model came in with the previous commit, which was
about the dump; they belong here.

test: the fixture indented its property lines, so none were read

A `.ssp` property line is `.name: value` at the start of the line. The
Gleam test fixture indented them by two spaces, which makes each a line
of no known shape, so every property in the fixture was being counted as
malformed and ignored. The tests around it passed for the wrong reason.

Corrected, and a test added that reads a property, a digest and a
parent, and asserts that the fixture now holds no malformed line. Only
the fixture was wrong: the loaders agree with each other and with the D
library over the whole reference collection, which is real files.

a third input form: the abstraction out of a pod archive

A pod built with --pod2 carries the abstraction inside it, at
media/abstraction/&lt;uid&gt;.&lt;lang&gt;.ssp. So ocda.load gains .sisupod and .zip,
and because the dispatch is in one place a writer over the library gained
a third input form without being touched.

Over the reference collection a .sisupod gives the same model as the .ssp
beside it: 36 of 36 document-languages, byte identical dumps. And the
text writer, unchanged, renders a document from its .sisupod byte
identically to spine --text, 36 of 36.

Two small readers, neither of them a library: ocda.zstd declares the four
libzstd entry points needed to unwrap a frame, as ocda.sqlite declares
sqlite3; ocda.zip reads a central directory and one stored entry. Deflate
is refused rather than mishandled, since spine writes stored entries and
a library that says it read what it skipped is worse than one that says
it cannot. A frame that declares an expansion over 512 MiB is refused
before anything is allocated: a reader of an artefact it did not make
declines to be told how much memory to take.

The Gleam library reads the zip and not the frame. The only zstd on the
BEAM is a NIF that clones zstd from github in its own build, so it cannot
be built sandboxed without patching it. A .zip pod is read in both
languages, a .sisupod in D only, and the Gleam refusal says why and says
"zstd -d" rather than claiming the file is malformed. It is a packaging
limit, not anything about the format, and the only place the two
libraries differ in what they accept.

A pod without an abstraction is refused with a message naming the flag
that puts one in, because that is a fact about the pod.

two fields the model was missing, found by writing an html writer

anchor_tags and heading_lev_anchor. Both are in the specification's
property table and neither was read, so they went to the unknown counts
and a writer over the library had no way to emit a link target.

The cost was visible the moment an html writer existed: 154 of its
internal links pointed at nothing, because a note reference points at
the endnote's anchor and the endnote's anchor is an anchor_tag.
heading_lev_anchor is the name a document's own cross references use,
1,854 of them in the reference collection.

The .ssp carries them as repeated .anchor_tag and as
.heading_lev_anchor; the database as the object_anchors child table and
the heading_lev_anchor column. Both loaders read both now, the dump
reports them, and the four-way agreement still holds: 36 of 36 each way.

One thing to watch when adding a column to the database query: the read
is by position, so a column inserted in the middle shifts every one
after it. Adding heading_lev_anchor after segment_html_is broke all 36
documents until it moved to the end of the select.

images from the source, and the epub segment fields

Two additions the epub writer needed and could not have without.

ocda.images: an image's bytes, from whichever input form the document came
from. The model names images, with digest, byte size and pixel size, and
does not hold them - which is right, an abstraction is not a bundle of
assets - so something has to know where they are, and the answer depends
on the input form rather than on the output. Hence the library.

  .ocda.db   the files table, role 'image', by name
  .sisupod   the archive entry whose path ends in /image/&lt;name&gt;
  .zip       the same
  .ssp       beside it: a pod puts the abstraction in media/abstraction/
             and its images in media/image/, so the path is documented
             rather than guessed

canGiveImages says whether a source can answer at all, because a writer
has to know before it emits an &lt;img&gt;: a manifest item whose file is absent
and an &lt;img&gt; whose file is not in the manifest are both epubcheck errors.
A .ssp outside a pod has nowhere to look, and that is absence rather than
error - the document is still readable.

Reading a blob needed sqlite3_column_blob rather than _column_text: an
image read as text stops at the first zero byte, and a png's ninth byte
is zero.

And two more model fields: segment_epub and segment_epub_is. The epub
segments at heading levels 1 to 4 where the html segments at level 4
alone, so the two are different files and the abstraction records both.
Nine of the 36 documents have images; the four-way agreement still holds
at 36 of 36 each way.

model 2.1: ocn_range, the verses a poem holds

The format now states, for a container, the first and last ocn of
what it holds: .ocn_range in the .ssp, ocn_range_first and
ocn_range_last in the .ocda.db. A poem is the one container, and since
2.1 it carries ocn 0 and comes before its verses.

Both fields in the model, read by both loaders, in both languages, and
written by the dump. A 2.0 database has neither column, so each
loader asks sqlite whether it is there and reads a 2.0 file as having
no ranges, which is what 2.0 meant; checked on a published 2.0
database, D and Gleam giving the same dump.

Over the 36 document-languages written by spine at 2.1, the four
dumps (D and Gleam, .ssp and .ocda.db) are byte identical, 36 of 36.

D: what a source holds, and what a database holds unread

ocda.inventory answers two questions about the file rather than the
document. languages(path), for every form alike: the languages in the
manifest's order (which every .ssp records in @source languages), each
with the markup it was made from and the sha256 of its .ssp - stored
by the database, taken from the bytes for a .ssp or a pod, and the
same. carried(path), for a .ocda.db: what it holds besides the
abstraction, by role and storage.

And the database loader now counts what it holds and the model does
not. The text loader meets every property line and counts those it
does not keep; this one asks for columns by name and so never saw the
others, and reported that it had understood everything, which is the
silence the count exists to prevent. It counts them on the conditions
under which the .ssp writes the same property, and the report is now
the same from a .ssp, a .ocda.db and a .sisupod: 36 of 36.

The pod loader's archive and language-name helpers become package
visible for the inventory.

D: ocda-read, the reader, over the library, of every form

One reader where there were two. sisudoc-ssp-reader read a .ssp and
sisudoc-ocda-db-reader a .ocda.db, each with its own parser; this one
is built on the library, so every mode reads a .ssp, a .ocda.db and a
.sisupod, and says the same thing of each. Its output modules are the
.ssp reader's, copied (the reader repositories are unchanged) and
moved onto the library's model, which was the reader's types renamed
and added to. From the database reader, --languages and --files.

  ocda-read [--tree|--outline|--html|--llm-text|--llm-jsonl|--images|
             --verify --image-dir=&lt;d&gt;|--languages|--files|
             --report-unknown|--validate] &lt;file&gt; [lang]

Two modes are about the file and say so: --files answers for a
.ocda.db, and --validate checks a .ssp as text and refuses anything
else. Where a file holds several languages and none is named, every
form reads the first and says so, where the database used to take one
and the pod refuse.

In src/read/ beside the library in src/ocda/; dub.json keeps it out of
the library (configurations "library" and "ocda-read"), and the flake
builds it as a package. The flake's default is now all three programs
in one bin/: built separately into the same result link, each replaced
the one before, and the pipeline's dump looked for ocda-dump where
only ocda-agree was left.

Checked over the 36 document-languages:
- against the .ssp reader, on a .ssp, byte for byte after the first
  line (Parsed: is Read:): --tree, --html, --llm-text, --llm-jsonl,
  --images, --verify, --validate, 36 of 36; the summary differs only
  in the closing hint, which names the two new modes;
- against the database reader, on a .ocda.db: --files 27 of 27, and
  --languages 27 of 27 on language, source and digest (its id column,
  the database's own key, is left out);
- the same output from the .ssp, the .ocda.db and the .sisupod: the
  summary and --tree, --html, --llm-text, --llm-jsonl, --images and
  --report-unknown, 36 of 36; --languages, .ocda.db == .sisupod 27 of
  27, and a single-language .ssp == its database, 26 of 26.

Not reproduced, deliberately: the database reader's --outline, which
ordered the headings by their position within their own section, so
that the toc and the "(skip)" headings of other sections fell among
the body's. --outline here is --tree.

Gleam: what a source holds, and what a database holds unread

The mirror of the D commit before it. ocda/inventory gives the
languages of any form, in the manifest's order, each with its markup
file and the sha256 of its .ssp (stored by the database, taken from
the bytes for a .ssp or a pod), and what a .ocda.db carries. The
sha256 comes from OTP's crypto through a four line Erlang FFI file,
since the standard library has none and a Hex package would be one
more fixed output hash; the whitelist admits src/*.erl for it.

The database loader counts the columns it does not read, on the
conditions the .ssp writer uses, so that it can say what it did not
understand; it could not before. The pod loader's archive, language
and text helpers become public for the inventory.

Checked: the four dumps, D and Gleam of .ssp and .ocda.db, identical
36 of 36, and with --unknown also 36 of 36; --languages of a database
== of its pod, 27 of 27.

Gleam: ocda-read, byte for byte the D reader's

ocda-lib read, installed also as ocda-read: the modes the Gleam readers
had (the summary, --tree/--outline, --languages, --files,
--report-unknown), over the library, in src/read/, which is where
Gleam must put it. A D-only mode asked of it says so.

Its output is the D ocda-read's. For that the D reader's closing hint
no longer lists its modes, which this one has fewer of: both say that
--help lists them.

Checked over the 36 document-languages, on the .ssp and the .ocda.db:
D == Gleam byte for byte in all five modes, 36 of 36. And the same
from the .ssp, the .ocda.db and the .zip (a .sisupod unwrapped, the
Gleam library having no zstd): the summary, --tree and
--report-unknown 36 of 36; --languages of a database == of its pod
27 of 27, and of a single-language .ssp == its database; --files
answers for a database and says why not for the others, in the same
words in both languages.

README: the reader, in each language, and what writing it found

The library's reader, ocda-read: where it lives and why (src/read/,
kept out of the library in D by dub configuration, in Gleam because
nothing outside src/ is compiled), its modes in each language, and
the two properties it holds, the same output from every form and D ==
Gleam byte for byte. And the one defect it found in the library, the
database loader unable to say what it had not read. The module tables
gain ocda.inventory and the reader, the build notes the three
programs in one result.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The readers each read one serialisation and print.  A writer needs
neither: it needs the document.  So the model and both loaders live
here, and a writer cannot tell which serialisation it was handed,
exactly as spine own writers cannot.

ocda.model imports nothing, which is the property that makes writers
possible: they depend on the model and on neither loader.  The model
and the .ssp loader are the ssp-reader code, moved and renamed to the
level they belong to: the model is neither the .ssp nor the .ocda.db.
ocda.load_db is new, and returns the same value from sqlite that
load_ssp returns from text.  ocda.load is the one place that decides
what a path is.

The claim is only true if the two loaders agree, so test/agree.d asks:
load a document both ways and compare the models object by object.

It found a real defect on its first run.  Eight of thirty-six
disagreed, and the cause was the .ssp loader, not the formats: an
object whose first text line is blank lost it, the loader having
decided whether to start a new line by asking whether the accumulated
text was empty, which is a different question from whether a line has
been seen.  Seven documents of the sample set have verse opening on a
blank line and every one of them was wrong.

Nothing else had caught it: the two .ssp readers were compared on
section counts, which the bug does not disturb.  It took loading one
document two ways and comparing the objects.  Fixed here, and in
ssp-reader/dlang which carried the same code.  36 of 36 now agree.

docs: the repository is sisudoc-ocda-lib

Prefix the repository name, as every other repository of this
project's own is prefixed. This one needed it most: "ocda-lib" is a
three letter acronym and the word "lib", and in a clone, away from
the parent directory that supplied the context, it reads as a generic
library belonging to no-one.

The dub path-dependency example and the note on where the blank-line
fix also went follow the new sibling name.

Binary, package and module names are unchanged: ocda-agree and the
ocda.* modules are read with the repository already in hand.

style: restore the comment arrows, double-encoded by an edit

The /+ ↓ +/ arrows had each of their three UTF-8 bytes re-encoded as
UTF-8 in turn, by an in-place edit that read the file as latin-1. 13
comments in load_ssp.d. Comment text only; the parser is unchanged and
the agreement check still passes.

ocda-dump, and the four disagreements it found

The agreement check compared ten fields of each object. That was enough
to find the blank-first-line defect and not enough for anything
subtler, so this adds ocda-dump: the whole loaded model written out
canonically, every field, sorted dictionaries, sections in format
order, defaults omitted. It is an oracle, not an output format. Two
implementations of this model can now be compared with diff.

Run against the reference collection it found four disagreements
between the two loaders, none of which the ten-field check could see.

1. .identifier and .anchor. The .ssp emits a property only when it
   carries information, so the identifier is absent when it is simply
   the ocn and the anchor is absent when it is the segment name; the
   database stores both unconditionally. Over one document that is
   1,314 identifiers and 69 anchors. The loader now derives them, which
   is what the model means: an object's identifier is its ocn unless it
   says otherwise.

2. make and doc_has. The database has one metadata table where the
   text form has three blocks, and tells them apart by a key prefix.
   The loader now unprefixes them, so a writer asks doc.make["footer"]
   whichever serialisation it was handed and never
   doc.metadata["make.footer"].

3. The JSON columns were read by a character scan that stripped
   brackets and quotes, which is wrong the moment a value holds an
   escape: JSONValue.toString escapes "/" as "\/", so every heading
   containing a slash came back with a backslash in it. All 36
   document-languages differed on heading_ancestors_text. Now parsed as
   JSON.

4. translation.ocn_aligned is in the database and not in the .ssp,
   legitimately: it says whether the languages share one numbering, and
   a .ssp is one language and cannot know. It is a fact about the file
   rather than the document, so the loader leaves it out rather than
   letting a writer tell which serialisation it was given.

Metadata values are also stripped now, as the text loader strips them:
one topic_register value carried surrounding spaces in the database
that the text form cannot represent.

After all four: 36 of 36 document-languages give byte identical dumps
from their .ssp and from their .ocda.db.

ocda-lib in Gleam, and four agreements of 36 of 36

The model, both loaders, the dispatch and the canonical dump, in Gleam.
A deliberate field-for-field mirror of the D library, including where
that is not the most idiomatic Gleam: text is one string with the lines
joined by newline rather than a list of lines, and the properties are
typed fields rather than a list of pairs. A difference that existed in
one language and not the other would show in every dump and mean
nothing, so the two lose the same distinctions in the same places.

The dump is what makes the agreement testable. Both languages write it
byte for byte alike, so four questions are diff, and over the reference
collection all four hold:

  D:       .ssp == .ocda.db      36 of 36
  Gleam:   .ssp == .ocda.db      36 of 36
  .ssp:      D == Gleam          36 of 36
  .ocda.db:  D == Gleam          36 of 36

and a fifth, that the two report exactly the same ignored properties
with the same counts.

The Gleam .ssp loader agreed with the D one at the first attempt. The
Gleam .ocda.db loader had one defect of its own, found by the sweep and
worth recording because of its shape: a JSON element decoder handled
strings and integers and fell through to a fallback that yielded "".
table_widths is an integer array except where the columns divide
evenly, when it is [33.3333,...], and one table in the collection does
that. Its widths came back as three empty strings, so a value that
could not be read turned into one that looked read. That is the failure
the unknown counts exist to prevent, and a decoder fallback quietly
reintroduces it.

Shipped as an Erlang shipment rather than an escript, as the .ocda.db
reader is and for the same reason: sqlite on the BEAM is a NIF and a
NIF cannot load from inside an escript archive.

gleam.toml and the model came in with the previous commit, which was
about the dump; they belong here.

test: the fixture indented its property lines, so none were read

A `.ssp` property line is `.name: value` at the start of the line. The
Gleam test fixture indented them by two spaces, which makes each a line
of no known shape, so every property in the fixture was being counted as
malformed and ignored. The tests around it passed for the wrong reason.

Corrected, and a test added that reads a property, a digest and a
parent, and asserts that the fixture now holds no malformed line. Only
the fixture was wrong: the loaders agree with each other and with the D
library over the whole reference collection, which is real files.

a third input form: the abstraction out of a pod archive

A pod built with --pod2 carries the abstraction inside it, at
media/abstraction/&lt;uid&gt;.&lt;lang&gt;.ssp. So ocda.load gains .sisupod and .zip,
and because the dispatch is in one place a writer over the library gained
a third input form without being touched.

Over the reference collection a .sisupod gives the same model as the .ssp
beside it: 36 of 36 document-languages, byte identical dumps. And the
text writer, unchanged, renders a document from its .sisupod byte
identically to spine --text, 36 of 36.

Two small readers, neither of them a library: ocda.zstd declares the four
libzstd entry points needed to unwrap a frame, as ocda.sqlite declares
sqlite3; ocda.zip reads a central directory and one stored entry. Deflate
is refused rather than mishandled, since spine writes stored entries and
a library that says it read what it skipped is worse than one that says
it cannot. A frame that declares an expansion over 512 MiB is refused
before anything is allocated: a reader of an artefact it did not make
declines to be told how much memory to take.

The Gleam library reads the zip and not the frame. The only zstd on the
BEAM is a NIF that clones zstd from github in its own build, so it cannot
be built sandboxed without patching it. A .zip pod is read in both
languages, a .sisupod in D only, and the Gleam refusal says why and says
"zstd -d" rather than claiming the file is malformed. It is a packaging
limit, not anything about the format, and the only place the two
libraries differ in what they accept.

A pod without an abstraction is refused with a message naming the flag
that puts one in, because that is a fact about the pod.

two fields the model was missing, found by writing an html writer

anchor_tags and heading_lev_anchor. Both are in the specification's
property table and neither was read, so they went to the unknown counts
and a writer over the library had no way to emit a link target.

The cost was visible the moment an html writer existed: 154 of its
internal links pointed at nothing, because a note reference points at
the endnote's anchor and the endnote's anchor is an anchor_tag.
heading_lev_anchor is the name a document's own cross references use,
1,854 of them in the reference collection.

The .ssp carries them as repeated .anchor_tag and as
.heading_lev_anchor; the database as the object_anchors child table and
the heading_lev_anchor column. Both loaders read both now, the dump
reports them, and the four-way agreement still holds: 36 of 36 each way.

One thing to watch when adding a column to the database query: the read
is by position, so a column inserted in the middle shifts every one
after it. Adding heading_lev_anchor after segment_html_is broke all 36
documents until it moved to the end of the select.

images from the source, and the epub segment fields

Two additions the epub writer needed and could not have without.

ocda.images: an image's bytes, from whichever input form the document came
from. The model names images, with digest, byte size and pixel size, and
does not hold them - which is right, an abstraction is not a bundle of
assets - so something has to know where they are, and the answer depends
on the input form rather than on the output. Hence the library.

  .ocda.db   the files table, role 'image', by name
  .sisupod   the archive entry whose path ends in /image/&lt;name&gt;
  .zip       the same
  .ssp       beside it: a pod puts the abstraction in media/abstraction/
             and its images in media/image/, so the path is documented
             rather than guessed

canGiveImages says whether a source can answer at all, because a writer
has to know before it emits an &lt;img&gt;: a manifest item whose file is absent
and an &lt;img&gt; whose file is not in the manifest are both epubcheck errors.
A .ssp outside a pod has nowhere to look, and that is absence rather than
error - the document is still readable.

Reading a blob needed sqlite3_column_blob rather than _column_text: an
image read as text stops at the first zero byte, and a png's ninth byte
is zero.

And two more model fields: segment_epub and segment_epub_is. The epub
segments at heading levels 1 to 4 where the html segments at level 4
alone, so the two are different files and the abstraction records both.
Nine of the 36 documents have images; the four-way agreement still holds
at 36 of 36 each way.

model 2.1: ocn_range, the verses a poem holds

The format now states, for a container, the first and last ocn of
what it holds: .ocn_range in the .ssp, ocn_range_first and
ocn_range_last in the .ocda.db. A poem is the one container, and since
2.1 it carries ocn 0 and comes before its verses.

Both fields in the model, read by both loaders, in both languages, and
written by the dump. A 2.0 database has neither column, so each
loader asks sqlite whether it is there and reads a 2.0 file as having
no ranges, which is what 2.0 meant; checked on a published 2.0
database, D and Gleam giving the same dump.

Over the 36 document-languages written by spine at 2.1, the four
dumps (D and Gleam, .ssp and .ocda.db) are byte identical, 36 of 36.

D: what a source holds, and what a database holds unread

ocda.inventory answers two questions about the file rather than the
document. languages(path), for every form alike: the languages in the
manifest's order (which every .ssp records in @source languages), each
with the markup it was made from and the sha256 of its .ssp - stored
by the database, taken from the bytes for a .ssp or a pod, and the
same. carried(path), for a .ocda.db: what it holds besides the
abstraction, by role and storage.

And the database loader now counts what it holds and the model does
not. The text loader meets every property line and counts those it
does not keep; this one asks for columns by name and so never saw the
others, and reported that it had understood everything, which is the
silence the count exists to prevent. It counts them on the conditions
under which the .ssp writes the same property, and the report is now
the same from a .ssp, a .ocda.db and a .sisupod: 36 of 36.

The pod loader's archive and language-name helpers become package
visible for the inventory.

D: ocda-read, the reader, over the library, of every form

One reader where there were two. sisudoc-ssp-reader read a .ssp and
sisudoc-ocda-db-reader a .ocda.db, each with its own parser; this one
is built on the library, so every mode reads a .ssp, a .ocda.db and a
.sisupod, and says the same thing of each. Its output modules are the
.ssp reader's, copied (the reader repositories are unchanged) and
moved onto the library's model, which was the reader's types renamed
and added to. From the database reader, --languages and --files.

  ocda-read [--tree|--outline|--html|--llm-text|--llm-jsonl|--images|
             --verify --image-dir=&lt;d&gt;|--languages|--files|
             --report-unknown|--validate] &lt;file&gt; [lang]

Two modes are about the file and say so: --files answers for a
.ocda.db, and --validate checks a .ssp as text and refuses anything
else. Where a file holds several languages and none is named, every
form reads the first and says so, where the database used to take one
and the pod refuse.

In src/read/ beside the library in src/ocda/; dub.json keeps it out of
the library (configurations "library" and "ocda-read"), and the flake
builds it as a package. The flake's default is now all three programs
in one bin/: built separately into the same result link, each replaced
the one before, and the pipeline's dump looked for ocda-dump where
only ocda-agree was left.

Checked over the 36 document-languages:
- against the .ssp reader, on a .ssp, byte for byte after the first
  line (Parsed: is Read:): --tree, --html, --llm-text, --llm-jsonl,
  --images, --verify, --validate, 36 of 36; the summary differs only
  in the closing hint, which names the two new modes;
- against the database reader, on a .ocda.db: --files 27 of 27, and
  --languages 27 of 27 on language, source and digest (its id column,
  the database's own key, is left out);
- the same output from the .ssp, the .ocda.db and the .sisupod: the
  summary and --tree, --html, --llm-text, --llm-jsonl, --images and
  --report-unknown, 36 of 36; --languages, .ocda.db == .sisupod 27 of
  27, and a single-language .ssp == its database, 26 of 26.

Not reproduced, deliberately: the database reader's --outline, which
ordered the headings by their position within their own section, so
that the toc and the "(skip)" headings of other sections fell among
the body's. --outline here is --tree.

Gleam: what a source holds, and what a database holds unread

The mirror of the D commit before it. ocda/inventory gives the
languages of any form, in the manifest's order, each with its markup
file and the sha256 of its .ssp (stored by the database, taken from
the bytes for a .ssp or a pod), and what a .ocda.db carries. The
sha256 comes from OTP's crypto through a four line Erlang FFI file,
since the standard library has none and a Hex package would be one
more fixed output hash; the whitelist admits src/*.erl for it.

The database loader counts the columns it does not read, on the
conditions the .ssp writer uses, so that it can say what it did not
understand; it could not before. The pod loader's archive, language
and text helpers become public for the inventory.

Checked: the four dumps, D and Gleam of .ssp and .ocda.db, identical
36 of 36, and with --unknown also 36 of 36; --languages of a database
== of its pod, 27 of 27.

Gleam: ocda-read, byte for byte the D reader's

ocda-lib read, installed also as ocda-read: the modes the Gleam readers
had (the summary, --tree/--outline, --languages, --files,
--report-unknown), over the library, in src/read/, which is where
Gleam must put it. A D-only mode asked of it says so.

Its output is the D ocda-read's. For that the D reader's closing hint
no longer lists its modes, which this one has fewer of: both say that
--help lists them.

Checked over the 36 document-languages, on the .ssp and the .ocda.db:
D == Gleam byte for byte in all five modes, 36 of 36. And the same
from the .ssp, the .ocda.db and the .zip (a .sisupod unwrapped, the
Gleam library having no zstd): the summary, --tree and
--report-unknown 36 of 36; --languages of a database == of its pod
27 of 27, and of a single-language .ssp == its database; --files
answers for a database and says why not for the others, in the same
words in both languages.

README: the reader, in each language, and what writing it found

The library's reader, ocda-read: where it lives and why (src/read/,
kept out of the library in D by dub configuration, in Gleam because
nothing outside src/ is compiled), its modes in each language, and
the two properties it holds, the same output from every form and D ==
Gleam byte for byte. And the one defect it found in the library, the
database loader unable to say what it had not read. The module tables
gain ocda.inventory and the reader, the build notes the three
programs in one result.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
</feed>
