<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sisudoc-ocda-lib/gleam/src/ocda/load_db.gleam, branch orgify</title>
<subtitle>SiSU Spine: tools</subtitle>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/'/>
<entry>
<title>track</title>
<updated>2026-10-03T01:39:17+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-10-02T17:30:25+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=c458d2112ce91ccd55bcb2e5b6e7c6fc4ac083a3'/>
<id>c458d2112ce91ccd55bcb2e5b6e7c6fc4ac083a3</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>gleam sisudoc-ocda-lib</title>
<updated>2026-10-03T01:39:08+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-10-01T21:23:00+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=919942b1b5222cf00a5f68605bc383462770fa22'/>
<id>919942b1b5222cf00a5f68605bc383462770fa22</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Gleam: what a source holds, and what a database holds unread</title>
<updated>2026-10-03T01:38:54+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-30T00:00:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=8faa5db8e78973122c46b0ba03ad8064f678f150'/>
<id>8faa5db8e78973122c46b0ba03ad8064f678f150</id>
<content type='text'>
The mirror of the D commit before it. ocda/inventory gives the
languages of any form, in the manifest's order, each with its markup
file and the sha256 of its .ssp (stored by the database, taken from
the bytes for a .ssp or a pod), and what a .ocda.db carries. The
sha256 comes from OTP's crypto through a four line Erlang FFI file,
since the standard library has none and a Hex package would be one
more fixed output hash; the whitelist admits src/*.erl for it.

The database loader counts the columns it does not read, on the
conditions the .ssp writer uses, so that it can say what it did not
understand; it could not before. The pod loader's archive, language
and text helpers become public for the inventory.

Checked: the four dumps, D and Gleam of .ssp and .ocda.db, identical
36 of 36, and with --unknown also 36 of 36; --languages of a database
== of its pod, 27 of 27.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The mirror of the D commit before it. ocda/inventory gives the
languages of any form, in the manifest's order, each with its markup
file and the sha256 of its .ssp (stored by the database, taken from
the bytes for a .ssp or a pod), and what a .ocda.db carries. The
sha256 comes from OTP's crypto through a four line Erlang FFI file,
since the standard library has none and a Hex package would be one
more fixed output hash; the whitelist admits src/*.erl for it.

The database loader counts the columns it does not read, on the
conditions the .ssp writer uses, so that it can say what it did not
understand; it could not before. The pod loader's archive, language
and text helpers become public for the inventory.

Checked: the four dumps, D and Gleam of .ssp and .ocda.db, identical
36 of 36, and with --unknown also 36 of 36; --languages of a database
== of its pod, 27 of 27.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>model 2.1: ocn_range, the verses a poem holds</title>
<updated>2026-10-03T01:38:40+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-29T18:07:32+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=597d73db60679b8ed13524f9dd3832bb7476038f'/>
<id>597d73db60679b8ed13524f9dd3832bb7476038f</id>
<content type='text'>
The format now states, for a container, the first and last ocn of
what it holds: .ocn_range in the .ssp, ocn_range_first and
ocn_range_last in the .ocda.db. A poem is the one container, and since
2.1 it carries ocn 0 and comes before its verses.

Both fields in the model, read by both loaders, in both languages, and
written by the dump. A 2.0 database has neither column, so each
loader asks sqlite whether it is there and reads a 2.0 file as having
no ranges, which is what 2.0 meant; checked on a published 2.0
database, D and Gleam giving the same dump.

Over the 36 document-languages written by spine at 2.1, the four
dumps (D and Gleam, .ssp and .ocda.db) are byte identical, 36 of 36.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The format now states, for a container, the first and last ocn of
what it holds: .ocn_range in the .ssp, ocn_range_first and
ocn_range_last in the .ocda.db. A poem is the one container, and since
2.1 it carries ocn 0 and comes before its verses.

Both fields in the model, read by both loaders, in both languages, and
written by the dump. A 2.0 database has neither column, so each
loader asks sqlite whether it is there and reads a 2.0 file as having
no ranges, which is what 2.0 meant; checked on a published 2.0
database, D and Gleam giving the same dump.

Over the 36 document-languages written by spine at 2.1, the four
dumps (D and Gleam, .ssp and .ocda.db) are byte identical, 36 of 36.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>images from the source, and the epub segment fields</title>
<updated>2026-10-03T01:38:33+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-26T13:47:07+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=6c14c162bdec65934fbda3b14295efcfc12d28d2'/>
<id>6c14c162bdec65934fbda3b14295efcfc12d28d2</id>
<content type='text'>
Two additions the epub writer needed and could not have without.

ocda.images: an image's bytes, from whichever input form the document came
from. The model names images, with digest, byte size and pixel size, and
does not hold them - which is right, an abstraction is not a bundle of
assets - so something has to know where they are, and the answer depends
on the input form rather than on the output. Hence the library.

  .ocda.db   the files table, role 'image', by name
  .sisupod   the archive entry whose path ends in /image/&lt;name&gt;
  .zip       the same
  .ssp       beside it: a pod puts the abstraction in media/abstraction/
             and its images in media/image/, so the path is documented
             rather than guessed

canGiveImages says whether a source can answer at all, because a writer
has to know before it emits an &lt;img&gt;: a manifest item whose file is absent
and an &lt;img&gt; whose file is not in the manifest are both epubcheck errors.
A .ssp outside a pod has nowhere to look, and that is absence rather than
error - the document is still readable.

Reading a blob needed sqlite3_column_blob rather than _column_text: an
image read as text stops at the first zero byte, and a png's ninth byte
is zero.

And two more model fields: segment_epub and segment_epub_is. The epub
segments at heading levels 1 to 4 where the html segments at level 4
alone, so the two are different files and the abstraction records both.
Nine of the 36 documents have images; the four-way agreement still holds
at 36 of 36 each way.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Two additions the epub writer needed and could not have without.

ocda.images: an image's bytes, from whichever input form the document came
from. The model names images, with digest, byte size and pixel size, and
does not hold them - which is right, an abstraction is not a bundle of
assets - so something has to know where they are, and the answer depends
on the input form rather than on the output. Hence the library.

  .ocda.db   the files table, role 'image', by name
  .sisupod   the archive entry whose path ends in /image/&lt;name&gt;
  .zip       the same
  .ssp       beside it: a pod puts the abstraction in media/abstraction/
             and its images in media/image/, so the path is documented
             rather than guessed

canGiveImages says whether a source can answer at all, because a writer
has to know before it emits an &lt;img&gt;: a manifest item whose file is absent
and an &lt;img&gt; whose file is not in the manifest are both epubcheck errors.
A .ssp outside a pod has nowhere to look, and that is absence rather than
error - the document is still readable.

Reading a blob needed sqlite3_column_blob rather than _column_text: an
image read as text stops at the first zero byte, and a png's ninth byte
is zero.

And two more model fields: segment_epub and segment_epub_is. The epub
segments at heading levels 1 to 4 where the html segments at level 4
alone, so the two are different files and the abstraction records both.
Nine of the 36 documents have images; the four-way agreement still holds
at 36 of 36 each way.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>two fields the model was missing, found by writing an html writer</title>
<updated>2026-10-03T01:38:29+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-26T04:50:16+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=8fd585f659c5dfc7c822cb8b7d66d2c5f9e901a1'/>
<id>8fd585f659c5dfc7c822cb8b7d66d2c5f9e901a1</id>
<content type='text'>
anchor_tags and heading_lev_anchor. Both are in the specification's
property table and neither was read, so they went to the unknown counts
and a writer over the library had no way to emit a link target.

The cost was visible the moment an html writer existed: 154 of its
internal links pointed at nothing, because a note reference points at
the endnote's anchor and the endnote's anchor is an anchor_tag.
heading_lev_anchor is the name a document's own cross references use,
1,854 of them in the reference collection.

The .ssp carries them as repeated .anchor_tag and as
.heading_lev_anchor; the database as the object_anchors child table and
the heading_lev_anchor column. Both loaders read both now, the dump
reports them, and the four-way agreement still holds: 36 of 36 each way.

One thing to watch when adding a column to the database query: the read
is by position, so a column inserted in the middle shifts every one
after it. Adding heading_lev_anchor after segment_html_is broke all 36
documents until it moved to the end of the select.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
anchor_tags and heading_lev_anchor. Both are in the specification's
property table and neither was read, so they went to the unknown counts
and a writer over the library had no way to emit a link target.

The cost was visible the moment an html writer existed: 154 of its
internal links pointed at nothing, because a note reference points at
the endnote's anchor and the endnote's anchor is an anchor_tag.
heading_lev_anchor is the name a document's own cross references use,
1,854 of them in the reference collection.

The .ssp carries them as repeated .anchor_tag and as
.heading_lev_anchor; the database as the object_anchors child table and
the heading_lev_anchor column. Both loaders read both now, the dump
reports them, and the four-way agreement still holds: 36 of 36 each way.

One thing to watch when adding a column to the database query: the read
is by position, so a column inserted in the middle shifts every one
after it. Adding heading_lev_anchor after segment_html_is broke all 36
documents until it moved to the end of the select.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>ocda-lib in Gleam, and four agreements of 36 of 36</title>
<updated>2026-10-03T01:38:14+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-26T01:18:06+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/projects/sisudoc-ocda-lib/commit/?id=e1994c4e01788fe46f534dddb104a1c808f47f48'/>
<id>e1994c4e01788fe46f534dddb104a1c808f47f48</id>
<content type='text'>
The model, both loaders, the dispatch and the canonical dump, in Gleam.
A deliberate field-for-field mirror of the D library, including where
that is not the most idiomatic Gleam: text is one string with the lines
joined by newline rather than a list of lines, and the properties are
typed fields rather than a list of pairs. A difference that existed in
one language and not the other would show in every dump and mean
nothing, so the two lose the same distinctions in the same places.

The dump is what makes the agreement testable. Both languages write it
byte for byte alike, so four questions are diff, and over the reference
collection all four hold:

  D:       .ssp == .ocda.db      36 of 36
  Gleam:   .ssp == .ocda.db      36 of 36
  .ssp:      D == Gleam          36 of 36
  .ocda.db:  D == Gleam          36 of 36

and a fifth, that the two report exactly the same ignored properties
with the same counts.

The Gleam .ssp loader agreed with the D one at the first attempt. The
Gleam .ocda.db loader had one defect of its own, found by the sweep and
worth recording because of its shape: a JSON element decoder handled
strings and integers and fell through to a fallback that yielded "".
table_widths is an integer array except where the columns divide
evenly, when it is [33.3333,...], and one table in the collection does
that. Its widths came back as three empty strings, so a value that
could not be read turned into one that looked read. That is the failure
the unknown counts exist to prevent, and a decoder fallback quietly
reintroduces it.

Shipped as an Erlang shipment rather than an escript, as the .ocda.db
reader is and for the same reason: sqlite on the BEAM is a NIF and a
NIF cannot load from inside an escript archive.

gleam.toml and the model came in with the previous commit, which was
about the dump; they belong here.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The model, both loaders, the dispatch and the canonical dump, in Gleam.
A deliberate field-for-field mirror of the D library, including where
that is not the most idiomatic Gleam: text is one string with the lines
joined by newline rather than a list of lines, and the properties are
typed fields rather than a list of pairs. A difference that existed in
one language and not the other would show in every dump and mean
nothing, so the two lose the same distinctions in the same places.

The dump is what makes the agreement testable. Both languages write it
byte for byte alike, so four questions are diff, and over the reference
collection all four hold:

  D:       .ssp == .ocda.db      36 of 36
  Gleam:   .ssp == .ocda.db      36 of 36
  .ssp:      D == Gleam          36 of 36
  .ocda.db:  D == Gleam          36 of 36

and a fifth, that the two report exactly the same ignored properties
with the same counts.

The Gleam .ssp loader agreed with the D one at the first attempt. The
Gleam .ocda.db loader had one defect of its own, found by the sweep and
worth recording because of its shape: a JSON element decoder handled
strings and integers and fell through to a fallback that yielded "".
table_widths is an integer array except where the columns divide
evenly, when it is [33.3333,...], and one table in the collection does
that. Its widths came back as three empty strings, so a value that
could not be read turned into one that looked read. That is the failure
the unknown counts exist to prevent, and a decoder fallback quietly
reintroduces it.

Shipped as an Erlang shipment rather than an escript, as the .ocda.db
reader is and for the same reason: sqlite on the BEAM is a NIF and a
NIF cannot load from inside an escript archive.

gleam.toml and the model came in with the previous commit, which was
about the dump; they belong here.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
</feed>
