<feed xmlns='http://www.w3.org/2005/Atom'>
<title>sisudoc-spine/org/tests_for_document_abstraction_shell_scripts.org, branch main</title>
<subtitle>SiSU Spine: document publishing and search (in D) 2015</subtitle>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/'/>
<entry>
<title>org minor tidy</title>
<updated>2026-10-24T03:19:44+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-10-24T03:19:44+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=aa770ddd1748fc838fa71ed34dcf224a6914a165'/>
<id>aa770ddd1748fc838fa71ed34dcf224a6914a165</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>flags: --ocda-db and matching --ocda-db-round-trip</title>
<updated>2026-09-28T15:49:59+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-28T00:09:53+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=029d7af58d499a8dafd6438e62499ec05df93178'/>
<id>029d7af58d499a8dafd6438e62499ec05df93178</id>
<content type='text'>
flags names tidied so that each names the artefact it acts on.

--ocda-db and --abstraction-db do the same thing. --ocda-db is the
official flag (--abstraction-db remains but undocumented)

--db-round-trip becomes --ocda-db-round-trip, so that it agrees with
--ssp-round-trip: each names the artefact it reads rather than one
naming a container and the other a format.

--abstraction-source is untouched named after he stage as it takes .sst,
a pod, .ssp or .ocda.db, rather than any one artefact.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
flags names tidied so that each names the artefact it acts on.

--ocda-db and --abstraction-db do the same thing. --ocda-db is the
official flag (--abstraction-db remains but undocumented)

--db-round-trip becomes --ocda-db-round-trip, so that it agrees with
--ssp-round-trip: each names the artefact it reads rather than one
naming a container and the other a format.

--abstraction-source is untouched named after he stage as it takes .sst,
a pod, .ssp or .ocda.db, rather than any one artefact.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>ocda: warn if heading claims reserved segment name</title>
<updated>2026-09-23T12:22:15+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T20:12:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=8b81e9a11c7af642c4d0d8584a9cb348cc887d6e'/>
<id>8b81e9a11c7af642c4d0d8584a9cb348cc887d6e</id>
<content type='text'>
spine automatically builds some segments, including: (toc, endnotes,
glossary, bibliography, bookindex, blurb, _the_title) these are now
identified as reserved names and a user is now warned if any of these
names have been manually assigned to a heading by markup. A document
still builds but to disambiguate the ocn of the heading is attached to
the markup (reserved) name and this is seeded before a document not
after. It is read off the finished abstraction rather than reported by
the parser, and beside the ocn alignment check for the same reason: it
is a statement about a document rather than a step in building one.

  WARNING reserved segment name: the_autonomous_contract... [en]
    heading 1 at ocn 135 asks for "endnotes", which spine gives its
    own generated section
      it is named "endnotes-135" instead; ...

A warning: the document is correct and complete and the name it ends
up with works.

--strict makes it a failure for the run, as it does for ocn alignment,
and by the same reasoning: the outputs are written and can be looked at,
and the exit status is taken at the end.

Two of the thirty-six sample documents have reserved segment names,
"1~endnotes" heading. Output is unchanged: nothing here touches the
abstraction.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
spine automatically builds some segments, including: (toc, endnotes,
glossary, bibliography, bookindex, blurb, _the_title) these are now
identified as reserved names and a user is now warned if any of these
names have been manually assigned to a heading by markup. A document
still builds but to disambiguate the ocn of the heading is attached to
the markup (reserved) name and this is seeded before a document not
after. It is read off the finished abstraction rather than reported by
the parser, and beside the ocn alignment check for the same reason: it
is a statement about a document rather than a step in building one.

  WARNING reserved segment name: the_autonomous_contract... [en]
    heading 1 at ocn 135 asks for "endnotes", which spine gives its
    own generated section
      it is named "endnotes-135" instead; ...

A warning: the document is correct and complete and the name it ends
up with works.

--strict makes it a failure for the run, as it does for ocn alignment,
and by the same reasoning: the outputs are written and can be looked at,
and the exit status is taken at the end.

Two of the thirty-six sample documents have reserved segment names,
"1~endnotes" heading. Output is unchanged: nothing here touches the
abstraction.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>test: language alignment, (live-manual)</title>
<updated>2026-09-22T21:24:15+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T19:54:37+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=21979975f0323b299615b01aa58dac9b4d7aad03'/>
<id>21979975f0323b299615b01aa58dac9b4d7aad03</id>
<content type='text'>
The ja and pl translations were mended in the sample set, so the two
divergences this script asserted by name are gone. It failed and named
every stale expectation, which is what it was for.

The reference set is regenerated with two files affected:
live-manual.ja.ssp and live-manual.pl.ssp.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The ja and pl translations were mended in the sample set, so the two
divergences this script asserted by name are gone. It failed and named
every stale expectation, which is what it was for.

The reference set is regenerated with two files affected:
live-manual.ja.ssp and live-manual.pl.ssp.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>check required tools; skipped test is incomplete</title>
<updated>2026-09-22T21:07:31+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T19:46:00+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=b1f8c8a980295a8d16b2eb689f8137d712f2cce4'/>
<id>b1f8c8a980295a8d16b2eb689f8137d712f2cce4</id>
<content type='text'>
- check that tools required for a test are present
  (check for sqlite3, epubcheck; add tools as
  needed)
- report a a skipped test as incomplete

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- check that tools required for a test are present
  (check for sqlite3, epubcheck; add tools as
  needed)
- report a a skipped test as incomplete

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>test: database to pod to artefacts</title>
<updated>2026-09-22T21:07:29+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T18:21:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=4449ac39de863dc65521aea48b123adba7785fa4'/>
<id>4449ac39de863dc65521aea48b123adba7785fa4</id>
<content type='text'>
The acceptance test for the document source claim. Two runs of the
same spine with the same flags over the same document:
- the pod to artefacts and outputs, including database, and
- that run's database back to a pod and on to artefacts and outputs.
diff -r between the two trees must be empty.

One comparison covers the written pod tree with its markup, conf, images
and catalogues, the .ssp files, the database itself, the digests, and
the text, html and epub built from them. 690 files for live-manual, ten
languages and a catalogue tree and no images, and 58 for the wealth of
networks, ten images and one language: the two cover each other's gaps,
and SpineAcceptanceDocs widens the set.

--ocda-verify is asserted separately, before the build. The build would
check the same thing and stop, but asking on its own means a failure
says which claim broke: that the pod comes back, or that the abstraction
it makes is the one recorded with it.

The pod for the first run is copied out of the sample set rather than
read where it lies. Read in place it has a site configuration above it
and a materialised pod has none, and the site url reaches the generated
index pages: five files differing in a url, and nothing else, which
would read as corruption.

Both runs ask for the same flags, the dom structure tags the .ssp
records being computed only when an action needs them.

The archive is not compared. Neither run writes one: it is a container
around bytes compared here already, and its own bytes are not part of
what a database claims to carry.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The acceptance test for the document source claim. Two runs of the
same spine with the same flags over the same document:
- the pod to artefacts and outputs, including database, and
- that run's database back to a pod and on to artefacts and outputs.
diff -r between the two trees must be empty.

One comparison covers the written pod tree with its markup, conf, images
and catalogues, the .ssp files, the database itself, the digests, and
the text, html and epub built from them. 690 files for live-manual, ten
languages and a catalogue tree and no images, and 58 for the wealth of
networks, ten images and one language: the two cover each other's gaps,
and SpineAcceptanceDocs widens the set.

--ocda-verify is asserted separately, before the build. The build would
check the same thing and stop, but asking on its own means a failure
says which claim broke: that the pod comes back, or that the abstraction
it makes is the one recorded with it.

The pod for the first run is copied out of the sample set rather than
read where it lies. Read in place it has a site configuration above it
and a materialised pod has none, and the site url reaches the generated
index pages: five files differing in a url, and nothing else, which
would read as corruption.

Both runs ask for the same flags, the dom structure tags the .ssp
records being computed only when an action needs them.

The archive is not compared. Neither run writes one: it is a container
around bytes compared here already, and its own bytes are not part of
what a database claims to carry.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>test: the carried markup, by rebuilding its digest</title>
<updated>2026-09-22T19:11:02+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T17:03:03+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=ea1162e5cf6182ad296b4775c7acfb51ac435842'/>
<id>ea1162e5cf6182ad296b4775c7acfb51ac435842</id>
<content type='text'>
Check that they are the right files: source.digest is the sha256 over
one line per markup file, "&lt;SHA256&gt; &lt;basename&gt;", sorted and newline
joined, so the same value can be computed from the stored rows alone and
held against what the .ssp recorded. A file stored under the right name
with the wrong contents passes a count and fails this.

Two details the shell gets wrong by default: the sort has to be
LC_ALL=C, because the writer sorts by byte and a locale does not, and
the lines are joined with no trailing newline, so printf and not echo.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Check that they are the right files: source.digest is the sha256 over
one line per markup file, "&lt;SHA256&gt; &lt;basename&gt;", sorted and newline
joined, so the same value can be computed from the stored rows alone and
held against what the .ssp recorded. A file stored under the right name
with the wrong contents passes a count and fails this.

Two details the shell gets wrong by default: the sort has to be
LC_ALL=C, because the writer sorts by byte and a locale does not, and
the lines are joined with no trailing newline, so printf and not echo.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>test: ocn alignment check</title>
<updated>2026-09-22T19:11:02+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T16:49:54+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=eeabf6216fde5f387d6173da93d0df882ed8a34e'/>
<id>eeabf6216fde5f387d6173da93d0df882ed8a34e</id>
<content type='text'>
Asserts the two known divergences in live-manual's translations by name
and value: ja two objects short of en, and pl's ocn 313 a paragraph in
English and a heading in Polish.

It asserts that nothing else is reported: exactly one document warned
about, exactly two languages of it.

Also the exit codes: 0 without --strict, non-zero with it on the
diverging document, and 0 with it on a single language document.

It will fail when those two translations are mended, which is the right
moment to hear about it, and the message names the expectation that went
stale.

test-abstraction-db.sh excludes translation.ocn_aligned from its header
block comparison, with the other two keys that are the database's notes
about the artefact rather than properties of the document.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Asserts the two known divergences in live-manual's translations by name
and value: ja two objects short of en, and pl's ocn 313 a paragraph in
English and a heading in Polish.

It asserts that nothing else is reported: exactly one document warned
about, exactly two languages of it.

Also the exit codes: 0 without --strict, non-zero with it on the
diverging document, and 0 with it on a single language document.

It will fail when those two translations are mended, which is the right
moment to hear about it, and the message names the expectation that went
stale.

test-abstraction-db.sh excludes translation.ocn_aligned from its header
block comparison, with the other two keys that are the database's notes
about the artefact rather than properties of the document.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>test db: multilingual database test</title>
<updated>2026-09-22T18:54:35+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-21T16:00:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=774302487c8595ec4383558e9d8387a6219ef2ed'/>
<id>774302487c8595ec4383558e9d8387a6219ef2ed</id>
<content type='text'>
one file holding several languages.

Both assumed &lt;uid&gt;.&lt;lang&gt;.ocda.db beside &lt;uid&gt;.&lt;lang&gt;.ssp, one for
one.

- test-abstraction-db.sh takes the language from the .ssp name, the
  database from the stem, and scopes every query by the doc_id that
  language resolves to: objects by doc_id, the four child tables
  through their objects, metadata by doc_id, schema.version by doc_id
  is null. It also checks what a per language loop cannot see, that the
  languages a database holds are exactly the languages there are .ssp
  files for.

- test-abstraction-db-roundtrip.sh asks for each language by name and
  holds it against its own .ssp, and requires a database of more than
  one language to refuse a round trip that names none: a reader quietly
  emitting the first language would otherwise pass against the first
  .ssp and say nothing about the other nine. Its language list comes
  from the .ssp filenames, so it still needs no sqlite3.

databases (re-)emitting all .ssp files exactly.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
one file holding several languages.

Both assumed &lt;uid&gt;.&lt;lang&gt;.ocda.db beside &lt;uid&gt;.&lt;lang&gt;.ssp, one for
one.

- test-abstraction-db.sh takes the language from the .ssp name, the
  database from the stem, and scopes every query by the doc_id that
  language resolves to: objects by doc_id, the four child tables
  through their objects, metadata by doc_id, schema.version by doc_id
  is null. It also checks what a per language loop cannot see, that the
  languages a database holds are exactly the languages there are .ssp
  files for.

- test-abstraction-db-roundtrip.sh asks for each language by name and
  holds it against its own .ssp, and requires a database of more than
  one language to refuse a round trip that names none: a reader quietly
  emitting the first language would otherwise pass against the first
  .ssp and say nothing about the other nine. Its language list comes
  from the .ssp filenames, so it still needs no sqlite3.

databases (re-)emitting all .ssp files exactly.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
<entry>
<title>images alt text, and epub says what it is</title>
<updated>2026-09-14T16:18:51+00:00</updated>
<author>
<name>Ralph Amissah</name>
<email>ralph.amissah@gmail.com</email>
</author>
<published>2026-09-14T01:56:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.sisudoc.org/sisudoc-spine/commit/?id=eb5a9f3dab1d1848a1aa90c534a125c222327635'/>
<id>eb5a9f3dab1d1848a1aa90c534a125c222327635</id>
<content type='text'>
Four things, not visible in an error count:

- Images had no alt text, even where markup carried the text, and the
  field alt text is present, incorrectly used in code; fixed as the alt.

  { sm_tux.png 64x80 "Gnu/Linux - a better way" }image

  an image with nothing to say for itself gets alt=""

- Images with no dimensions were incorrectly written width="0"
  height="0", fixed

- Accessibility metadata, which EPUB Accessibility 1.1 requires:
  schema:accessMode, accessibilityFeature and accessibilityHazard, plus
  accessModeSufficient and accessibilitySummary, which epubcheck does
  not test for; used by
  - readers looking for a book it can use,
  - anyone distributing into the EU.
  Every value is derived from the document, not asserted: a document
  with no images says so, one whose images all carry alt text claims
  alternativeText and textual sufficiency, one where an image says
  nothing claims neither.
  Nothing claims WCAG conformance, which would be a claim about an
  evaluation that has not happened.

- DPUB-ARIA roles beside the structure spine already knew about.
  doc-toc, doc-endnotes, doc-bibliography, doc-glossary and doc-index on
  the sections, doc-noteref on a note reference and doc-footnote on the
  note it points at. "doc_endnotes" as a class name means something to a
  stylesheet and nothing to a reading system. Not doc-biblioentry: DPUB-
  ARIA 1.1 deprecated it. These are plain ARIA, so the html gets them
   too.

Also dealt with here:
- Spine makes certain sections such as endnotes, bookindex, ... and a
  heading that has a name for itself cannot share the same name, if it
  does the name is now disambiguated, disambiguated. (disambiguation
  similar to that used for a repeated anchor tag)

- Endnote anchors fixed to use "id" (being one unique tag in the
  document that is target of every reference to it)instead of obsolete
  form (they were &lt;a name="note_1"&gt;. "name" on an &lt;a&gt;).
  Book index markers keep "name": the same marker sits on every object
  an entry names, 296 times in one document, and those are markers of
  where a term occurs rather than distinct targets.

(assisted by Claude-Code)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Four things, not visible in an error count:

- Images had no alt text, even where markup carried the text, and the
  field alt text is present, incorrectly used in code; fixed as the alt.

  { sm_tux.png 64x80 "Gnu/Linux - a better way" }image

  an image with nothing to say for itself gets alt=""

- Images with no dimensions were incorrectly written width="0"
  height="0", fixed

- Accessibility metadata, which EPUB Accessibility 1.1 requires:
  schema:accessMode, accessibilityFeature and accessibilityHazard, plus
  accessModeSufficient and accessibilitySummary, which epubcheck does
  not test for; used by
  - readers looking for a book it can use,
  - anyone distributing into the EU.
  Every value is derived from the document, not asserted: a document
  with no images says so, one whose images all carry alt text claims
  alternativeText and textual sufficiency, one where an image says
  nothing claims neither.
  Nothing claims WCAG conformance, which would be a claim about an
  evaluation that has not happened.

- DPUB-ARIA roles beside the structure spine already knew about.
  doc-toc, doc-endnotes, doc-bibliography, doc-glossary and doc-index on
  the sections, doc-noteref on a note reference and doc-footnote on the
  note it points at. "doc_endnotes" as a class name means something to a
  stylesheet and nothing to a reading system. Not doc-biblioentry: DPUB-
  ARIA 1.1 deprecated it. These are plain ARIA, so the html gets them
   too.

Also dealt with here:
- Spine makes certain sections such as endnotes, bookindex, ... and a
  heading that has a name for itself cannot share the same name, if it
  does the name is now disambiguated, disambiguated. (disambiguation
  similar to that used for a repeated anchor tag)

- Endnote anchors fixed to use "id" (being one unique tag in the
  document that is target of every reference to it)instead of obsolete
  form (they were &lt;a name="note_1"&gt;. "name" on an &lt;a&gt;).
  Book index markers keep "name": the same marker sits on every object
  an entry names, 296 times in one document, and those are markers of
  where a term occurs rather than distinct targets.

(assisted by Claude-Code)
</pre>
</div>
</content>
</entry>
</feed>
