diff options
| author | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-01 19:06:40 -0400 |
|---|---|---|
| committer | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-01 19:06:40 -0400 |
| commit | 9b9391327424febd5bd774d2ce35ecc137c79a58 (patch) | |
| tree | a4b4467693d64dd3b55ff3a1b893cba2ffd59c0a | |
| parent | the specification in org: links to the section, not "see section 3" (diff) | |
the specification uses org, rather than being markdown in orgclaude-convert-md-to-org
The first conversion carried markdown's forms across. This one uses the
structures org has for what the specification holds:
- each .ssp property is a heading of its own under section 6.3, its
order, value, condition and count in a property drawer, with an id
(prop-parent) that the database specification's column tables link
to. The table at the head of 6.3 is a column view of those headings,
regenerated from them, so it cannot drift from them;
- the @meta keys are a checklist, checked where the reference
collection uses them, and the counts (28 of 43, and each group's) are
org's own count of the checks rather than a number in the prose;
- tables whose cells were prose are definition lists: the version
history, with org dates, the note marks, the @source and @make keys,
the model's three terms and two serialisations;
- what is formal is a named source block: the grammar (ssp-grammar,
tangled to ssp.peg), the database queries (sqlite, tangled to
ocda-db-queries.sql, runnable in place given :db), and .ssp lines as
ssp blocks. Tangled files are generated and not tracked;
- inline code is ~code~ rather than =verbatim=: org reads a table cell
that begins with "=" as a formula once the table is edited, and two
tables had been rewritten to #ERROR that way;
- README: a section on reading it in org, and the pandoc filter that
makes a markdown copy for a forge, an export rather than a second
source, with every link between the documents and to a section kept.
Fixed in passing, both from the first conversion: the @meta table's
"exercised" marks had been read by pandoc as emphasis and lost, and a
code span after ".." had not rendered.
Checked: each document exports with org's broken-link check on; every
link into another document names an id it has; every table row has its
table's number of cells; every word of the markdown is in the org bar
table headings, titles and the rewording named above; the tangled
queries run against a format 2.1 database; every link in the markdown
copy resolves.
(assisted by Claude-Code)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| -rw-r--r-- | README.org | 109 | ||||
| -rw-r--r-- | ocda-db-format-specification.org | 283 | ||||
| -rw-r--r-- | ocda-model.org | 158 | ||||
| -rw-r--r-- | ssp-format-specification.org | 877 |
4 files changed, 971 insertions, 456 deletions
@@ -7,8 +7,8 @@ Specification of the *object-centric document abstraction* that [[https://git.sisudoc.org/projects/sisudoc-spine][sisudoc-spine]] builds from -SiSU markup, and of the two forms it is written in: *=.ssp=*, text and -PEG-parsable, and *=.ocda.db=*, sqlite. +SiSU markup, and of the two forms it is written in: *~.ssp~*, text and +PEG-parsable, and *~.ocda.db~*, sqlite. The two carry one version number between them, because neither may express what the other cannot. @@ -19,19 +19,19 @@ what the other cannot. :END: A specification, not a program. The repository ships: -- [[file:ocda-model.org][=ocda-model.org=]] - *the model*: what an object is, +- [[file:ocda-model.org][~ocda-model.org~]] - *the model*: what an object is, the citation number, structure, sections, segments, digests. The part both serialisations share, and the place the three terms /ocda/, - /abstraction/ and =.ssp= are told apart. -- [[file:ssp-format-specification.org][=ssp-format-specification.org=]] - *the - =.ssp=*: the model written as text. -- [[file:ocda-db-format-specification.org][=ocda-db-format-specification.org=]] - - *the =.ocda.db=*: the model written as sqlite. -- [[file:example-autonomous-contract.ssp][=example-autonomous-contract.ssp=]] - + /abstraction/ and ~.ssp~ are told apart. +- [[file:ssp-format-specification.org][~ssp-format-specification.org~]] - *the + ~.ssp~*: the model written as text. +- [[file:ocda-db-format-specification.org][~ocda-db-format-specification.org~]] - + *the ~.ocda.db~*: the model written as sqlite. +- [[file:example-autonomous-contract.ssp][~example-autonomous-contract.ssp~]] - spine's output for that sample document, complete and unedited. -- [[file:example-wealth-of-networks.ssp][=example-wealth-of-networks.ssp=]] - +- [[file:example-wealth-of-networks.ssp][~example-wealth-of-networks.ssp~]] - an excerpt of spine's output for that sample document: verbatim - lines, marked elisions, keeping the preamble, the =@head= section + lines, marked elisions, keeping the preamble, the ~@head~ section and a run of body objects that includes an object embedding an image. The whole file runs to 27,769 lines. @@ -47,16 +47,16 @@ poem, so from 2.0 to 2.1 each changed by its version line alone). The format is at *2.1*, and says so on the first line of every file: -#+begin_example +#+begin_src ssp % SiSU Document Abstraction v2.1 -#+end_example +#+end_src The minor part is raised when a property is added, the major part when one changes meaning or goes away, and a reader checks the major part and refuses what it does not know. The history is in the specification. -* What is =.ssp=? +* What is ~.ssp~? :PROPERTIES: :CUSTOM_ID: what-is-ssp :END: @@ -64,22 +64,22 @@ Spine's processing pipeline goes *markup -> abstraction -> output*. The abstraction stage builds an in-memory object model: every paragraph, heading, table, footnote etc. is a numbered object with parent / sibling / type metadata -("OCN" - Object Citation Numbering). =.ssp= is a faithful text +("OCN" - Object Citation Numbering). ~.ssp~ is a faithful text serialisation of that object model. It is designed for: - Inspecting and debugging spine's abstraction stage. -- Diff-friendly version control of document structure (a =.ssp= +- Diff-friendly version control of document structure (a ~.ssp~ diff between two markup edits shows exactly which objects changed). -- Round-trip experiments: markup -> spine -> =.ssp= -> alternative +- Round-trip experiments: markup -> spine -> ~.ssp~ -> alternative renderers (HTML, LLM-ready chunks, ...) without rerunning spine. - A stable contract between spine and downstream consumers that do not want to link against spine's internal D types. Line types (full grammar in -[[file:ssp-format-specification.org][=ssp-format-specification.org=]]): +[[file:ssp-format-specification.org][~ssp-format-specification.org~]]): #+begin_example @name { section / block open @@ -104,7 +104,7 @@ Line types (full grammar in document data downstream of spine. End-user document authors do not need to read this; they write -=.sst= / =.ssm= markup, which spine then turns into =.ssp= along +~.sst~ / ~.ssm~ markup, which spine then turns into ~.ssp~ along the way to the final output formats. * Quickstart @@ -123,7 +123,7 @@ Look at a real example: $EDITOR example-wealth-of-networks.ssp #+end_src -Generate your own =.ssp= from a sample document via spine: +Generate your own ~.ssp~ from a sample document via spine: #+begin_src sh # inside sisudoc-spine/ @@ -131,9 +131,66 @@ result/bin/spine --source --pod2 --output=/tmp/ssp path/to/pod ls /tmp/ssp/*/abstraction/*.ssp #+end_src -=--pod2= writes the abstraction as =.ssp= and bundles it into the +~--pod2~ writes the abstraction as ~.ssp~ and bundles it into the source pod. (See sisudoc-spine's CLI help for the current flags.) +* Reading it in org +:PROPERTIES: +:CUSTOM_ID: reading-it-in-org +:END: +The specification is written in org, and uses it for more than markup: + +- *Each numbered section has an id* (~sec-6-3~), and every "section 6.3" in + the text is a link to it, in its own document or another. +- *Each ~.ssp~ property is a heading* of its own under + [[file:ssp-format-specification.org::#sec-6-3][section 6.3]] of the ~.ssp~ + specification, with its order, value, condition and count in a property + drawer and an id (~prop-parent~) that the database specification links to. + The table at the head of the section is a /column view/ of those headings: + ~C-c C-c~ on its ~#+BEGIN~ line regenerates it, so the table and the + properties cannot disagree for longer than it takes to press it. +- *The ~@meta~ keys are a checklist*, a key checked where the reference + collection uses it, and the counts on the heading and on each group are + org's own count of the checks: ~C-c #~ on a heading updates them. +- *What is formal is a named source block*: the grammar (~ssp-grammar~, + tangled to ~ssp.peg~), the database queries (~ocda-db-query-1~ to ~5~, as + ~sqlite~ blocks tangled to ~ocda-db-queries.sql~), and ~.ssp~ lines as + ~ssp~ blocks. ~C-c C-v t~ tangles a document's blocks to their files, which + are generated and not kept in the repository. A query block runs in place + with ~C-c C-c~ given a database, by adding ~:db <file>.ocda.db~ to its + header. + +A markdown copy, for a forge that renders markdown and not org, is an export +rather than a second source. The filter below keeps the links between the +documents and to their sections working in the copy: + +#+begin_src sh +for f in README ocda-model ssp-format-specification ocda-db-format-specification; do + pandoc -f org -t gfm -s --wrap=preserve --lua-filter org-to-md.lua "$f.org" -o "$f.md" +done +#+end_src + +#+NAME: org-to-md +#+begin_src lua :tangle org-to-md.lua +-- links between the documents go to the .md, and every section keeps the +-- id the org gave it, so that a link to a section still lands on it +function Link(el) + el.target = el.target:gsub("^([%w%-]+)%.org::#", "%1.md#"):gsub("^([%w%-]+)%.org$", "%1.md") + return el +end +function Header(el) + if el.identifier ~= "" then + table.insert(el.content, 1, pandoc.RawInline("html", '<a id="' .. el.identifier .. '"></a>')) + end + return el +end +function Code(el) + el.text = el.text:gsub("^(README)%.org$", "%1.md"):gsub("^(ocda%-model)%.org$", "%1.md") + :gsub("^([%w%-]+%-format%-specification)%.org$", "%1.md") + return el +end +#+end_src + * License :PROPERTIES: :CUSTOM_ID: license @@ -147,17 +204,17 @@ project. :CUSTOM_ID: repository :END: - *Canonical URL:* [[https://git.sisudoc.org/projects/sisudoc-abstraction-format]] -- *Clone:* =git clone --depth=1 git://git.sisudoc.org/specs/sisudoc-abstraction-format= +- *Clone:* ~git clone --depth=1 git://git.sisudoc.org/specs/sisudoc-abstraction-format~ * Related projects :PROPERTIES: :CUSTOM_ID: related-projects :END: - [[https://git.sisudoc.org/projects/sisudoc-spine][sisudoc-spine]] - - the document engine that produces =.ssp= files. + the document engine that produces ~.ssp~ files. - [[https://git.sisudoc.org/projects/ssp-reader][ssp-reader]] - - reference reader; renders =.ssp= to tree outline, HTML, plain + reference reader; renders ~.ssp~ to tree outline, HTML, plain text with breadcrumbs, or LLM-ready JSONL. - [[https://git.sisudoc.org/projects/tree-sitter-sisu][tree-sitter-sisu]] - - tree-sitter grammar for the upstream =.sst= / =.ssm= markup that - spine consumes before producing =.ssp=. + tree-sitter grammar for the upstream ~.sst~ / ~.ssm~ markup that + spine consumes before producing ~.ssp~. diff --git a/ocda-db-format-specification.org b/ocda-db-format-specification.org index 324babd..7655ad2 100644 --- a/ocda-db-format-specification.org +++ b/ocda-db-format-specification.org @@ -9,7 +9,7 @@ version 2.1*, as spine emits and reads it after 0.25.1. This document specifies /one serialisation/. The model it serialises is -specified in [[file:ocda-model.org][=ocda-model.org=]], which the text serialisation +specified in [[file:ocda-model.org][~ocda-model.org~]], which the text serialisation shares; read that first if the question is what an object /is/ rather than how it is stored. @@ -17,7 +17,7 @@ how it is stored. :PROPERTIES: :CUSTOM_ID: status :END: -Derived from the writer, =sisudoc/outputs/io_out/sqlite_ocda_db.d=, and +Derived from the writer, ~sisudoc/outputs/io_out/sqlite_ocda_db.d~, and checked against the 27 published databases of the project's reference collection. Where the two could differ the writer is normative and this document is wrong. @@ -27,26 +27,26 @@ document is wrong. :CUSTOM_ID: sec-1 :END: One file per *document*, holding *every language of it*: a ten language -document is one database with ten rows in =documents=, written one language +document is one database with ten rows in ~documents~, written one language at a time into the one file. -Two things distinguish it from the =.ssp=: +Two things distinguish it from the ~.ssp~: -- *It carries its own source.* The markup, the images, =conf/document_make=, - =pod.manifest= and the =tools/po4a= catalogues are all inside it, in the - =files= table. A pod can be written back out of it, and spine does exactly - that when asked for an artefact that needs markup. The =.ssp= names its +- *It carries its own source.* The markup, the images, ~conf/document_make~, + ~pod.manifest~ and the ~tools/po4a~ catalogues are all inside it, in the + ~files~ table. A pod can be written back out of it, and spine does exactly + that when asked for an artefact that needs markup. The ~.ssp~ names its images and carries none of them. - *It is queryable without being parsed.* A question about the document is a SELECT, answered from the file with no spine and no reader. -** 1.1 It is built from the =.ssp= +** 1.1 It is built from the ~.ssp~ :PROPERTIES: :CUSTOM_ID: sec-1-1 :END: -The document is emitted as =.ssp=, *read straight back*, and what comes +The document is emitted as ~.ssp~, *read straight back*, and what comes back is what is written. So *the database cannot carry an object property -the =.ssp= cannot express*, and the two cannot drift. That is also why they +the ~.ssp~ cannot express*, and the two cannot drift. That is also why they share one version number: a bump moves both. The carried /files/ are the deliberate exception. They come from the pod @@ -56,19 +56,20 @@ rather than from the abstraction, each with its digest. :PROPERTIES: :CUSTOM_ID: sec-2 :END: -Declared in band, in the =metadata= table: +Declared in band, in the ~metadata~ table: -#+begin_src sql +#+NAME: ocda-db-query-1 +#+begin_src sqlite :tangle ocda-db-queries.sql SELECT key, value FROM metadata WHERE doc_id IS NULL; -- build.spine_version | 0.25.0 -- schema.name | sisu-abstraction-db -- schema.version | 2.1 #+end_src -The rule is the model's, and identical to the =.ssp=: the minor part is +The rule is the model's, and identical to the ~.ssp~: the minor part is raised when a property is added, the major when one changes meaning or goes away, and *a reader checks the major part and refuses what it does not -know*. See [[file:ssp-format-specification.org][=ssp-format-specification.org=]] +know*. See [[file:ssp-format-specification.org][~ssp-format-specification.org~]] [[file:ssp-format-specification.org::#sec-2][section 2]] for the history, which is shared. * 3. The schema @@ -96,56 +97,56 @@ Eight tables, one virtual table, three views. objects_fts fts5 over objects.text, external content over objects.id #+end_example -** 3.1 =doc_id= is the language key +** 3.1 ~doc_id~ is the language key :PROPERTIES: :CUSTOM_ID: sec-3-1 :END: -=documents.id=, referred to as =doc_id= everywhere else, is *what tells one +~documents.id~, referred to as ~doc_id~ everywhere else, is *what tells one language's rows from another's*. It is the key to reach for in every query. -=objects.lang= and =objects.meta_lang= are *properties of the object* and +~objects.lang~ and ~objects.meta_lang~ are *properties of the object* and are not it. They are almost always empty ([[#sec-6][section 6]]). ** 3.2 The file-level metadata rows :PROPERTIES: :CUSTOM_ID: sec-3-2 :END: -=metadata= is keyed =UNIQUE(doc_id, key)= and not on key alone, because +~metadata~ is keyed ~UNIQUE(doc_id, key)~ and not on key alone, because every language has a title and a creator and a key-only key would refuse the second. Three rows describe *the file* rather than a document in it, and are -written with =doc_id= null: =schema.name=, =schema.version= and -=build.spine_version=. In SQL no null equals another null, so -=UNIQUE(doc_id, key)= does not constrain them, and a partial unique index -over =key= alone =WHERE doc_id IS NULL= is what makes them singular. +written with ~doc_id~ null: ~schema.name~, ~schema.version~ and +~build.spine_version~. In SQL no null equals another null, so +~UNIQUE(doc_id, key)~ does not constrain them, and a partial unique index +over ~key~ alone ~WHERE doc_id IS NULL~ is what makes them singular. -*** 3.2.1 The key prefix carries what the =.ssp= puts in blocks +*** 3.2.1 The key prefix carries what the ~.ssp~ puts in blocks :PROPERTIES: :CUSTOM_ID: sec-3-2-1 :END: -The =.ssp= has four preamble blocks, =@source=, =@meta=, =@make= and -=@doc_has=. The database has *one* =metadata= table, and tells them apart +The ~.ssp~ has four preamble blocks, ~@source~, ~@meta~, ~@make~ and +~@doc_has~. The database has *one* ~metadata~ table, and tells them apart by a *prefix on the key*: -| =.ssp= block | database key | +| ~.ssp~ block | database key | |--------------+-----------------------------------------------------------| -| =@meta= | the key as it stands: =title.main=, =creator.author=, ... | -| =@make= | =make.= + the key: =make.footer=, =make.doc_type= | -| =@doc_has= | =doc_has.= + the key: =doc_has.images=, =doc_has.tables= | -| =@source= | =source.= + the key: =source.language=, =source.digest= | +| ~@meta~ | the key as it stands: ~title.main~, ~creator.author~, ... | +| ~@make~ | ~make.~ + the key: ~make.footer~, ~make.doc_type~ | +| ~@doc_has~ | ~doc_has.~ + the key: ~doc_has.images~, ~doc_has.tables~ | +| ~@source~ | ~source.~ + the key: ~source.language~, ~source.digest~ | *A reader must strip the prefix and restore the grouping*, or a writer -over it has to ask =metadata["make.footer"]= from a database and -=make["footer"]= from a =.ssp=, which is exactly the difference the two +over it has to ask ~metadata["make.footer"]~ from a database and +~make["footer"]~ from a ~.ssp~, which is exactly the difference the two serialisations exist not to make. -=source.= also carries two keys with no =@source= counterpart, -=source.filename= and =source.ssp_digest=; the =.ssp= puts the first in its -=% Source:= comment and cannot hold the second, which is a digest /of/ it. +~source.~ also carries two keys with no ~@source~ counterpart, +~source.filename~ and ~source.ssp_digest~; the ~.ssp~ puts the first in its +~% Source:~ comment and cannot hold the second, which is a digest /of/ it. -Values may carry leading or trailing whitespace here where the =.ssp= -cannot represent it, its =key: value= line being unable to distinguish a +Values may carry leading or trailing whitespace here where the ~.ssp~ +cannot represent it, its ~key: value~ line being unable to distinguish a value that begins with a space from the separator. *Strip on load*, as a reader of the text form necessarily does. @@ -153,21 +154,21 @@ reader of the text form necessarily does. :PROPERTIES: :CUSTOM_ID: sec-3-3 :END: -=children=, =ancestors=, =ancestors_collapsed=, =dom_status=, -=dom_status_collapsed=, =heading_ancestors_text=, =table_widths= and -=table_aligns= are stored as *JSON arrays*, with validity checks, so that -=json_extract=, =json_each= and =json_array_length= reach inside them. +~children~, ~ancestors~, ~ancestors_collapsed~, ~dom_status~, +~dom_status_collapsed~, ~heading_ancestors_text~, ~table_widths~ and +~table_aligns~ are stored as *JSON arrays*, with validity checks, so that +~json_extract~, ~json_each~ and ~json_array_length~ reach inside them. #+begin_quote -*Parse them as JSON.* A scan that strips =[=, =]=, =,= and ="= looks -adequate and is wrong on the first escape: the encoder writes =/= as -=\/=, so =FREE/OPEN-SOURCE SOFTWARE= comes back as -=FREE\/OPEN-SOURCE SOFTWARE=. All 36 document-languages of the reference +*Parse them as JSON.* A scan that strips ~[~, ~]~, ~,~ and ~"~ looks +adequate and is wrong on the first escape: the encoder writes ~/~ as +~\/~, so ~FREE/OPEN-SOURCE SOFTWARE~ comes back as +~FREE\/OPEN-SOURCE SOFTWARE~. All 36 document-languages of the reference collection contain at least one, and the defect is invisible to any check that does not compare the field itself. -Note also that =table_widths= is an array of *integers except where the -columns divide evenly*, when it is =[33.3333,33.3333,33.3333]=. A decoder +Note also that ~table_widths~ is an array of *integers except where the +columns divide evenly*, when it is ~[33.3333,33.3333,33.3333]~. A decoder that accepts strings and integers and then falls back to a default turns a value it could not read into one that looks read. #+end_quote @@ -177,71 +178,71 @@ what occurs: | column | elements | |--------------------------------------------------------------------------------------+-------------------------------------------------------------------| -| =children=, =ancestors=, =ancestors_collapsed=, =dom_status=, =dom_status_collapsed= | integers | -| =table_widths= | integers, or floats where the columns divide evenly | -| =table_aligns= | strings, one of =l= =c= =r= | -| =heading_ancestors_text= | strings, eight of them, empty where that level is not an ancestor | +| ~children~, ~ancestors~, ~ancestors_collapsed~, ~dom_status~, ~dom_status_collapsed~ | integers | +| ~table_widths~ | integers, or floats where the columns divide evenly | +| ~table_aligns~ | strings, one of ~l~ ~c~ ~r~ | +| ~heading_ancestors_text~ | strings, eight of them, empty where that level is not an ancestor | -The =.ssp= writes the first three groups *space separated* and -=heading_ancestors_text= *=|= separated*. The empty slots of the last are +The ~.ssp~ writes the first three groups *space separated* and +~heading_ancestors_text~ *~|~ separated*. The empty slots of the last are significant: they say which levels a heading sits under, so the separators carry as much as the text does. The open-ended lists are *rows in the four child tables* instead of columns: images, links, anchor tags and sub-table-of-contents entries. Each -child table carries =seq=, which preserves the order they appeared in. +child table carries ~seq~, which preserves the order they appeared in. ** 3.4 Indexes :PROPERTIES: :CUSTOM_ID: sec-3-4 :END: -=objects(doc_id, section)=, =objects(ocn)=, =objects(parent_ocn)=, -=objects(is_a)=, =objects(heading_level)= where it is a heading, and -=files(role)=, =object_images(name)=. +~objects(doc_id, section)~, ~objects(ocn)~, ~objects(parent_ocn)~, +~objects(is_a)~, ~objects(heading_level)~ where it is a heading, and +~files(role)~, ~object_images(name)~. -* 4. The relationship to the =.ssp=, column by column +* 4. The relationship to the ~.ssp~, column by column :PROPERTIES: :CUSTOM_ID: sec-4 :END: -*Nothing is lost in either direction.* Every =.ssp= property is a column +*Nothing is lost in either direction.* Every ~.ssp~ property is a column or a child table; every column is a property, a value from the object declaration line, or a database key. -Where the names differ they differ for a reason, the =.ssp= preferring the +Where the names differ they differ for a reason, the ~.ssp~ preferring the short form and the database the explicit one: -| =.ssp= property | database | +| ~.ssp~ property | database | |--------------------+-----------------------| -| =.parent= | =parent_ocn= | -| =.last_descendant= | =last_descendant_ocn= | -| =.dummy= | =dummy_heading= | -| =.ocn_off= | =object_number_off= | -| =.part= | =is_of_part= | -| =.section= | =is_of_section= | +| [[file:ssp-format-specification.org::#prop-parent][~.parent~]] | ~parent_ocn~ | +| [[file:ssp-format-specification.org::#prop-last-descendant][~.last_descendant~]] | ~last_descendant_ocn~ | +| [[file:ssp-format-specification.org::#prop-dummy][~.dummy~]] | ~dummy_heading~ | +| [[file:ssp-format-specification.org::#prop-ocn-off][~.ocn_off~]] | ~object_number_off~ | +| [[file:ssp-format-specification.org::#prop-part][~.part~]] | ~is_of_part~ | +| [[file:ssp-format-specification.org::#prop-section][~.section~]] | ~is_of_section~ | -Three are *expanded*, the =.ssp= folding what the database spreads: +Three are *expanded*, the ~.ssp~ folding what the database spreads: -| =.ssp= property | database | +| ~.ssp~ property | database | |---------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------| -| =.indent: <base> <hang>= | =indent_base=, =indent_hang= | -| =.ocn_range: <first> <last>= | =ocn_range_first=, =ocn_range_last= (2.1: =0= where the object is not a container; a 2.0 database has neither column, and reads as no range) | -| =.has: links notes_reg notes_star images images_no_dim= | =has_links=, =has_notes_reg=, =has_notes_star=, =has_images=, =has_images_no_dim= | +| [[file:ssp-format-specification.org::#prop-indent][~.indent: <base> <hang>~]] | ~indent_base~, ~indent_hang~ | +| [[file:ssp-format-specification.org::#prop-ocn-range][~.ocn_range: <first> <last>~]] | ~ocn_range_first~, ~ocn_range_last~ (2.1: ~0~ where the object is not a container; a 2.0 database has neither column, and reads as no range) | +| [[file:ssp-format-specification.org::#prop-has][~.has: links notes_reg notes_star images images_no_dim~]] | ~has_links~, ~has_notes_reg~, ~has_notes_star~, ~has_images~, ~has_images_no_dim~ | The four *repeatable* properties become the four child tables: -| =.ssp= property | database | +| ~.ssp~ property | database | |-----------------+------------------| -| =.image= | =object_images= | -| =.stow_link= | =object_links= | -| =.anchor_tag= | =object_anchors= | -| =.lev4_subtoc= | =object_subtoc= | - -And four columns come from outside the property list: =ocn=, =is_a= and -=heading_level= from the object declaration line, and =text= from its =|= -lines. =id=, =doc_id= and =seq= are the database's own keys; =seq= is the -object's position within its section, and =(doc_id, section, seq)= is what -identifies an object when =ocn= will not: =0=, or, before 2.1, a poem -sharing its last verse's number (see [[file:ocda-model.org][=ocda-model.org=]] +| [[file:ssp-format-specification.org::#prop-image][~.image~]] | ~object_images~ | +| [[file:ssp-format-specification.org::#prop-stow-link][~.stow_link~]] | ~object_links~ | +| [[file:ssp-format-specification.org::#prop-anchor-tag][~.anchor_tag~]] | ~object_anchors~ | +| [[file:ssp-format-specification.org::#prop-lev4-subtoc][~.lev4_subtoc~]] | ~object_subtoc~ | + +And four columns come from outside the property list: ~ocn~, ~is_a~ and +~heading_level~ from the object declaration line, and ~text~ from its ~|~ +lines. ~id~, ~doc_id~ and ~seq~ are the database's own keys; ~seq~ is the +object's position within its section, and ~(doc_id, section, seq)~ is what +identifies an object when ~ocn~ will not: ~0~, or, before 2.1, a poem +sharing its last verse's number (see [[file:ocda-model.org][~ocda-model.org~]] [[file:ocda-model.org::#sec-3][section 3]]). ** 4.1 Where the two are not symmetric @@ -252,22 +253,22 @@ sharing its last verse's number (see [[file:ocda-model.org][=ocda-model.org=]] three places where the /files/ nonetheless differ. A reader must know all three, because each of them made two independently written readers disagree. -*1. The database states what the =.ssp= derives.* =identifier= and -=anchor= are written on every row. The =.ssp= omits each where it can be +*1. The database states what the ~.ssp~ derives.* ~identifier~ and +~anchor~ are written on every row. The ~.ssp~ omits each where it can be worked out, the identifier being the ocn and the anchor the segment name ([[file:ssp-format-specification.org::#sec-6-1][sections 6.1]] and [[file:ssp-format-specification.org::#sec-6-3][6.3]] of -[[file:ssp-format-specification.org][=ssp-format-specification.org=]], [[file:ocda-model.org::#sec-9][section 9]] of -[[file:ocda-model.org][=ocda-model.org=]]). In =free_culture.lawrence_lessig= that is +[[file:ssp-format-specification.org][~ssp-format-specification.org~]], [[file:ocda-model.org::#sec-9][section 9]] of +[[file:ocda-model.org][~ocda-model.org~]]). In ~free_culture.lawrence_lessig~ that is *1,314 identifiers and 69 anchors* stated here and absent there. Neither file is the poorer; a reader that does not derive is. -*2. =translation.ocn_aligned= is here and cannot be in a =.ssp=.* It says -whether the languages of a document share one numbering, and a =.ssp= is one +*2. ~translation.ocn_aligned~ is here and cannot be in a ~.ssp~.* It says +whether the languages of a document share one numbering, and a ~.ssp~ is one language. It is therefore a fact about *the file*, not about the document in it, and it is the one such fact either serialisation carries. A reader should *keep it out of the document model*, for the same reason -the =schema.*= and =build.*= rows are kept out: a writer over the model must +the ~schema.*~ and ~build.*~ rows are kept out: a writer over the model must not be able to tell which serialisation it was handed, and a field that exists only when the input was a database is exactly that tell. The model has no place for collection-wide facts yet; when it grows one, this belongs @@ -275,19 +276,19 @@ there. *3. Whitespace and grouping*, as [[#sec-3-2-1][section 3.2.1]] gives them. -* 5. The =files= table: what it carries, and how +* 5. The ~files~ table: what it carries, and how :PROPERTIES: :CUSTOM_ID: sec-5 :END: -| role | named by | =compression= | =data= holds | +| role | named by | ~compression~ | ~data~ holds | |------------+----------------------------+---------------+------------------| -| =image= | its bare filename | =none= | the file as read | -| =source= | =media/text/<lang>/<file>= | =zstd= | a zstd frame | -| =conf= | =conf/document_make= | =zstd= | a zstd frame | -| =manifest= | =pod.manifest= | =zstd= | a zstd frame | -| =tools= | =tools/po4a/...= | =zstd= | a zstd frame | +| ~image~ | its bare filename | ~none~ | the file as read | +| ~source~ | ~media/text/<lang>/<file>~ | ~zstd~ | a zstd frame | +| ~conf~ | ~conf/document_make~ | ~zstd~ | a zstd frame | +| ~manifest~ | ~pod.manifest~ | ~zstd~ | a zstd frame | +| ~tools~ | ~tools/po4a/...~ | ~zstd~ | a zstd frame | -The =compression= column carries the literal =none= or =zstd=, so a reader +The ~compression~ column carries the literal ~none~ or ~zstd~, so a reader need not infer it from the role. Three rules make the file predictable: @@ -300,27 +301,28 @@ Three rules make the file predictable: differently for no reason a reader could see. 2. *Images are not compressed.* png and jpeg are compressed already; zstd gains nothing and costs a decompression on every read. -3. *=objects.text= is never compressed.* The full text index reads that +3. *~objects.text~ is never compressed.* The full text index reads that column directly, and compressing it would stop search working completely rather than subtly. -*=bytes= and =sha256= are always those of the original file*, whatever -=compression= says. So a digest check, a =digests.txt= line and a rebuilt -=source.digest= all keep working, and a reader that does not decompress can +*~bytes~ and ~sha256~ are always those of the original file*, whatever +~compression~ says. So a digest check, a ~digests.txt~ line and a rebuilt +~source.digest~ all keep working, and a reader that does not decompress can still say what it is looking at. * 6. Querying it :PROPERTIES: :CUSTOM_ID: sec-6 :END: -=sqlite3= is all that is needed. The recipes below were run against the +~sqlite3~ is all that is needed. The recipes below were run against the published databases. ** 6.1 What is this file :PROPERTIES: :CUSTOM_ID: sec-6-1 :END: -#+begin_src sql +#+NAME: ocda-db-query-2 +#+begin_src sqlite :tangle ocda-db-queries.sql SELECT key, value FROM metadata WHERE doc_id IS NULL; SELECT id, lang, source_filename FROM documents ORDER BY id; SELECT role, count(*), sum(bytes) FROM files GROUP BY role; @@ -332,27 +334,28 @@ SELECT role, count(*), sum(bytes) FROM files GROUP BY role; :END: | view | gives | |------------------+---------------------------------------------------------------------------------------------| -| =outline= | the headings: lang, section, seq, ocn, both levels, =last_descendant_ocn=, identifier, text | -| =citable= | every object with an ocn, ordered by ocn, language named | -| =document_files= | what the file carries: role, name, bytes, sha256, pixel size | +| ~outline~ | the headings: lang, section, seq, ocn, both levels, ~last_descendant_ocn~, identifier, text | +| ~citable~ | every object with an ocn, ordered by ocn, language named | +| ~document_files~ | what the file carries: role, name, bytes, sha256, pixel size | -*=outline= and =citable= name the language and order by it first.* A view +*~outline~ and ~citable~ name the language and order by it first.* A view over a file that may hold ten languages and does not say which would read as one document and be several. -*A trap in =citable=:* it is =SELECT d.lang AS lang, o.*=, and =objects= -has a =lang= column of its own, so sqlite renames the second one. The view -has *=lang=, the document's language, and =lang:1=, the object's own*. -Quote the second if you want it: =SELECT "lang:1" FROM citable=. +*A trap in ~citable~:* it is ~SELECT d.lang AS lang, o.*~, and ~objects~ +has a ~lang~ column of its own, so sqlite renames the second one. The view +has *~lang~, the document's language, and ~lang:1~, the object's own*. +Quote the second if you want it: ~SELECT "lang:1" FROM citable~. ** 6.3 Structure :PROPERTIES: :CUSTOM_ID: sec-6-3 :END: A heading's subtree is a *range*, not a traversal, because every heading -carries =last_descendant_ocn=: +carries ~last_descendant_ocn~: -#+begin_src sql +#+NAME: ocda-db-query-3 +#+begin_src sqlite :tangle ocda-db-queries.sql SELECT count(*) FROM objects o, (SELECT doc_id, ocn, last_descendant_ocn FROM objects WHERE is_a='heading' AND ocn=66) h @@ -360,9 +363,10 @@ SELECT count(*) FROM objects o, AND o.ocn BETWEEN h.ocn AND h.last_descendant_ocn; #+end_src -And the path back up, on =parent_ocn=: +And the path back up, on ~parent_ocn~: -#+begin_src sql +#+NAME: ocda-db-query-4 +#+begin_src sqlite :tangle ocda-db-queries.sql WITH RECURSIVE up(ocn, parent_ocn, text) AS ( SELECT ocn, parent_ocn, text FROM objects WHERE doc_id=1 AND ocn=345 UNION ALL @@ -372,7 +376,7 @@ WITH RECURSIVE up(ocn, parent_ocn, text) AS ( SELECT ocn, substr(text,1,44) FROM up; #+end_src -*Keep the =parent_ocn > 0= guard.* A parent of 0 is a real, +*Keep the ~parent_ocn > 0~ guard.* A parent of 0 is a real, non-substantive parent and not a missing value, and without the guard the recursion does not terminate. @@ -380,7 +384,8 @@ recursion does not terminate. :PROPERTIES: :CUSTOM_ID: sec-6-4 :END: -#+begin_src sql +#+NAME: ocda-db-query-5 +#+begin_src sqlite :tangle ocda-db-queries.sql SELECT o.ocn, substr(o.text,1,44) FROM objects_fts f JOIN objects o ON o.id = f.rowid JOIN documents d ON d.id = o.doc_id @@ -388,26 +393,26 @@ SELECT o.ocn, substr(o.text,1,44) FROM objects_fts f ORDER BY rank LIMIT 3; #+end_src -fts5 syntax is available in full: =MATCH 'peer AND production'=, -=MATCH '"peer production"'=, =MATCH 'peer NEAR production'=. +fts5 syntax is available in full: ~MATCH 'peer AND production'~, +~MATCH '"peer production"'~, ~MATCH 'peer NEAR production'~. -The index is *external content* over =objects.id=: the text is indexed, +The index is *external content* over ~objects.id~: the text is indexed, not stored twice. ** 6.5 Three things to know before trusting a query :PROPERTIES: :CUSTOM_ID: sec-6-5 :END: -1. *An ocn is not unique within a document*, and =0= is shared by many - objects. Key on =objects.id=, or on =(doc_id, section, seq)=. -2. *=objects.text= is internal markup*, neither the author's markup nor +1. *An ocn is not unique within a document*, and ~0~ is shared by many + objects. Key on ~objects.id~, or on ~(doc_id, section, seq)~. +2. *~objects.text~ is internal markup*, neither the author's markup nor any output. Fine for search and for reading structure, not for display without decoding. The markers are in - [[file:ssp-format-specification.org][=ssp-format-specification.org=]] [[file:ssp-format-specification.org::#sec-3-2][section 3.2]]. + [[file:ssp-format-specification.org][~ssp-format-specification.org~]] [[file:ssp-format-specification.org::#sec-3-2][section 3.2]]. 3. *A multi-language file answers an unfiltered query with every language - at once.* A single-language document has one =documents= row, so plain - queries read naturally there; anything else wants =WHERE lang='..'= or a - join to =documents=. + at once.* A single-language document has one ~documents~ row, so plain + queries read naturally there; anything else wants ~WHERE lang~'..'= or a + join to ~documents~. * 7. Digests, and verifying a file :PROPERTIES: @@ -422,8 +427,8 @@ each carried file──> files.sha256 over the original bytes the spine used ──> metadata build.spine_version #+end_example -So =.sst -> .ssp -> .ocda.db= is checkable end to end, and -=spine --ocda-verify=<file>= closes the loop by writing the carried markup +So ~.sst -> .ssp -> .ocda.db~ is checkable end to end, and +~spine --ocda-verify=<file>~ closes the loop by writing the carried markup back out and re-parsing it: *does this spine still make this abstraction from the markup this file carries?* It writes nothing and the exit status is the answer. @@ -432,7 +437,7 @@ is the answer. strongest sense in which it is the one to work with. The database's own digest is taken once, when the last language has been -written, and appears in =digests.txt=. The metadata page links the database +written, and appears in ~digests.txt~. The metadata page links the database and claims no hash for it, deliberately: the page is written per language while the database is added to per language, so a digest taken there would be of a file that no longer exists by the end of the run. @@ -441,11 +446,11 @@ be of a file that no longer exists by the end of the run. :PROPERTIES: :CUSTOM_ID: sec-8 :END: -A reader *must* check =schema.version='s major part and refuse a major it -does not know; *must* treat =ocn= as not an identity; *must* stop at -=parent_ocn = 0= when walking upward; and *should* say what it did not +A reader *must* check ~schema.version~'s major part and refuse a major it +does not know; *must* treat ~ocn~ as not an identity; *must* stop at +~parent_ocn = 0~ when walking upward; and *should* say what it did not understand, for the reason given in -[[file:ssp-format-specification.org][=ssp-format-specification.org=]] [[file:ssp-format-specification.org::#sec-8-2][section 8.2]]. +[[file:ssp-format-specification.org][~ssp-format-specification.org~]] [[file:ssp-format-specification.org::#sec-8-2][section 8.2]]. -A writer *must* keep =bytes= and =sha256= those of the original file -whatever =compression= says, and *must not* compress =objects.text=. +A writer *must* keep ~bytes~ and ~sha256~ those of the original file +whatever ~compression~ says, and *must not* compress ~objects.text~. diff --git a/ocda-model.org b/ocda-model.org index 17e8132..ad869fc 100644 --- a/ocda-model.org +++ b/ocda-model.org @@ -11,8 +11,8 @@ of any file it is written to. Two serialisations carry this model, and they carry *one version number between them*, because neither may express what the other cannot: -| [[file:ssp-format-specification.org][=ssp-format-specification.org=]] | the OCDA as text, =.ssp= | -| [[file:ocda-db-format-specification.org][=ocda-db-format-specification.org=]] | the OCDA as sqlite, =.ocda.db= | +- [[file:ssp-format-specification.org][~ssp-format-specification.org~]] :: the OCDA as text, ~.ssp~ +- [[file:ocda-db-format-specification.org][~ocda-db-format-specification.org~]] :: the OCDA as sqlite, ~.ocda.db~ This document is what they share. It describes objects; they describe writing objects down. @@ -24,11 +24,13 @@ writing objects down. The terms are used precisely in this repository and in spine, and the distinction is worth holding: -| level | term | names | -|--------------------+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------| -| the model | *OCDA* | what a document is once parsed: numbered objects with structure, identity and digests. Independent of any file, any stage, any tool | -| the stage | *abstraction* | the middle of spine's *markup -> abstraction -> output*: the act of building the model, and the point at which a document may now enter the pipeline | -| the serialisations | *=.ssp=*, *=.ocda.db=* | the model written down, as text and as sqlite | +- the model, *OCDA* :: what a document is once parsed: numbered objects with + structure, identity and digests. Independent of any file, any stage, any tool +- the stage, *abstraction* :: the middle of spine's *markup -> abstraction -> + output*: the act of building the model, and the point at which a document may + now enter the pipeline +- the serialisations, *~.ssp~*, *~.ocda.db~* :: the model written down, as text + and as sqlite A thing is named for the level it belongs to. Something true of the model whatever file it is in takes /ocda/; something about the pipeline stage @@ -61,39 +63,39 @@ Every object carries: :END: An object is classified twice, and the two answer different questions. -*Kind* (=is_a=) says /what the object is/, one of sixteen. Each kind -belongs to one section, except =heading=, which appears in all nine: +*Kind* (~is_a~) says /what the object is/, one of sixteen. Each kind +belongs to one section, except ~heading~, which appears in all nine: -| kind | section | what it is | count | +| kind | section | what it is | count | |----------------+--------------+--------------------------------------------------------------------------------+--------| -| =para= | body | a paragraph, the ordinary unit of a document | 41,885 | -| =heading= | /all nine/ | a heading at one of the eight levels | 3,503 | -| =toc= | toc | an entry in the generated table of contents | 3,215 | -| =bookindex= | bookindex | an entry in the generated book index | 3,034 | -| =endnote= | endnotes | a note, gathered from where it was called | 2,294 | -| =code= | body | a code block | 1,623 | -| =bibliography= | bibliography | a bibliography entry | 585 | -| =comment= | body | markup comment, carried and not rendered | 450 | -| =verse= | body | one verse of a poem | 151 | -| =glossary= | glossary | a glossary entry | 56 | -| =quote= | body | a block quotation | 53 | -| =poem= | body | a poem: the container of its verses, OCN =0=, cited as their range ([[#sec-5][section 5]]) | 48 | -| =block= | body | a block of grouped text | 45 | -| =table= | body | a table, with its columns described | 32 | -| =blurb= | blurb | a blurb entry | 28 | -| =group= | body | a group block | 16 | - -*Type* (=is_of_type=) says /how the object's text behaves/, and takes +| ~para~ | body | a paragraph, the ordinary unit of a document | 41,885 | +| ~heading~ | /all nine/ | a heading at one of the eight levels | 3,503 | +| ~toc~ | toc | an entry in the generated table of contents | 3,215 | +| ~bookindex~ | bookindex | an entry in the generated book index | 3,034 | +| ~endnote~ | endnotes | a note, gathered from where it was called | 2,294 | +| ~code~ | body | a code block | 1,623 | +| ~bibliography~ | bibliography | a bibliography entry | 585 | +| ~comment~ | body | markup comment, carried and not rendered | 450 | +| ~verse~ | body | one verse of a poem | 151 | +| ~glossary~ | glossary | a glossary entry | 56 | +| ~quote~ | body | a block quotation | 53 | +| ~poem~ | body | a poem: the container of its verses, OCN ~0~, cited as their range ([[#sec-5][section 5]]) | 48 | +| ~block~ | body | a block of grouped text | 45 | +| ~table~ | body | a table, with its columns described | 32 | +| ~blurb~ | blurb | a blurb entry | 28 | +| ~group~ | body | a group block | 16 | + +*Type* (~is_of_type~) says /how the object's text behaves/, and takes only three values: | type | means | kinds | |-----------+---------------------------------------------------------+---------------------------------------------------------------------------------------| -| =para= | flowing text, wrapped as the output sees fit | =para=, =heading=, =toc=, =bookindex=, =endnote=, =bibliography=, =glossary=, =blurb= | -| =block= | text whose line breaks are significant and must be kept | =code=, =verse=, =quote=, =poem=, =block=, =table=, =group= | -| =comment= | not rendered | =comment= | +| ~para~ | flowing text, wrapped as the output sees fit | ~para~, ~heading~, ~toc~, ~bookindex~, ~endnote~, ~bibliography~, ~glossary~, ~blurb~ | +| ~block~ | text whose line breaks are significant and must be kept | ~code~, ~verse~, ~quote~, ~poem~, ~block~, ~table~, ~group~ | +| ~comment~ | not rendered | ~comment~ | -The mapping from kind to type is fixed: every =code= object is of type -=block=, every =para= object of type =para=. *A renderer that has the type +The mapping from kind to type is fixed: every ~code~ object is of type +~block~, every ~para~ object of type ~para~. *A renderer that has the type does not need to know all sixteen kinds* to decide whether it may reflow the text, which is what the second axis is for. @@ -114,15 +116,15 @@ Three properties follow, and each is a trap for a reader that assumes otherwise: - *Not every object has one.* Generated furniture, and objects the markup - has suppressed, carry =0=. There are many of them: 7,543 across the 36 + has suppressed, carry ~0~. There are many of them: 7,543 across the 36 document reference collection. -- *=0= is therefore not an identity.* Objects sharing =0= are not the - same object, and =0= must never be used as a key. +- *~0~ is therefore not an identity.* Objects sharing ~0~ are not the + same object, and ~0~ must never be used as a key. - *A non-zero OCN is unique within a document*, from format 2.1: checked over the reference collection, no non-zero OCN occurs twice in any of the 36 document-languages. *Before 2.1 it was not*: a poem carried the OCN of its last verse, so the same number named two objects (48 such pairs in the - collection, 21 in =through_the_looking_glass=). A reader of an older + collection, 21 in ~through_the_looking_glass~). A reader of an older artefact keys on position within a section when one row is wanted. ** 3.1 The identifier @@ -130,15 +132,15 @@ otherwise: :CUSTOM_ID: sec-3-1 :END: Every object also carries an *identifier*: the name by which an output -refers to it. An HTML anchor, a LaTeX =\hypertarget=, a link from a book +refers to it. An HTML anchor, a LaTeX ~\hypertarget~, a link from a book index entry to the object it indexes, are all written from the identifier. *The identifier of an object with a non-zero OCN is its OCN*, written as text. That is the ordinary case and it is the reason a citation to object 42 can be turned into a link without consulting anything. -An object with OCN =0= has no OCN to take an identifier from, so one is -assigned: =a1=, =a2=, =a3=, ... in document order over the objects that +An object with OCN ~0~ has no OCN to take an identifier from, so one is +assigned: ~a1~, ~a2~, ~a3~, ... in document order over the objects that need one. There are few of them, eleven in a book, and they are the generated furniture of [[#sec-3][section 3]]. @@ -153,22 +155,22 @@ A document has *eight heading levels*, in two groups of four: | level | index | what it is | |-------+-------+----------------------------------------------| -| =A= | 0 | the document title. Exactly one per document | -| =B= | 1 | | -| =C= | 2 | the title matter below it | -| =D= | 3 | | -| =1= | 4 | the top level of the document body | -| =2= | 5 | | -| =3= | 6 | below that | -| =4= | 7 | | - -=A= to =D= are the title levels, =1= to =4= the body levels, and the index +| ~A~ | 0 | the document title. Exactly one per document | +| ~B~ | 1 | | +| ~C~ | 2 | the title matter below it | +| ~D~ | 3 | | +| ~1~ | 4 | the top level of the document body | +| ~2~ | 5 | | +| ~3~ | 6 | below that | +| ~4~ | 7 | | + +~A~ to ~D~ are the title levels, ~1~ to ~4~ the body levels, and the index is what the eight-slot arrays of [[#sec-5][section 5]] are indexed by. A heading that claims no level has 9, which is the "not a heading level" value and is why -=.heading_lev_collapsed= is emitted only when it is below 9. +~.heading_lev_collapsed~ is emitted only when it is below 9. -*Collapsed levels.* A document that uses =A= and then =1=, skipping =B=, -=C= and =D=, has a gap in its levels that is an artefact of the markup +*Collapsed levels.* A document that uses ~A~ and then ~1~, skipping ~B~, +~C~ and ~D~, has a gap in its levels that is an artefact of the markup rather than of the document. Every level-indexed array therefore comes in two forms: as marked up, and *collapsed*, with the unused levels removed so that the remaining ones are contiguous. Outputs that care about nesting @@ -183,31 +185,31 @@ Structure is carried by number, not by nesting: an object names its parent rather than containing its children. This is what lets the serialisations be flat and the model be walked in either direction. -- *=parent=* - the OCN of the object this one hangs from. - *A parent of =0= is not "no parent"*: it is a parent heading the author +- *~parent~* - the OCN of the object this one hangs from. + *A parent of ~0~ is not "no parent"*: it is a parent heading the author has made non-substantive, its OCN suppressed or a dummy heading introduced to segment an output. The distinction is preserved - deliberately, and a reader walking upward must stop at =0= rather than + deliberately, and a reader walking upward must stop at ~0~ rather than treat it as missing, or the walk will not terminate. -- *=ancestors=* - *eight slots, indexed by level* ([[#sec-4][section 4]]): the OCN - of the heading at each level above this object, =0= where there is none, +- *~ancestors~* - *eight slots, indexed by level* ([[#sec-4][section 4]]): the OCN + of the heading at each level above this object, ~0~ where there is none, so the path to the root is available without walking it. *A heading's own OCN sits in its own level's slot*: checked over the reference collection, 3,432 headings of 3,432. So for a heading the array is its path including itself, and for any other object it is the path to the heading above it. -- *=ancestors_collapsed=* - the same, in collapsed levels. -- *=children=* - a heading's child headings. -- *=last_descendant=* - the last OCN under a heading, which makes a +- *~ancestors_collapsed~* - the same, in collapsed levels. +- *~children~* - a heading's child headings. +- *~last_descendant~* - the last OCN under a heading, which makes a subtree a *range* rather than a traversal: everything from the heading's OCN to its last descendant is inside it. -- *=ocn_range=* - for a *container*, the first and last OCN of what it +- *~ocn_range~* - for a *container*, the first and last OCN of what it holds (2.1). A poem is the one container: its verses are the citable - units, one object each, and the poem is not; it carries OCN =0= and no + units, one object each, and the poem is not; it carries OCN ~0~ and no identifier, *comes immediately before its first verse*, and is cited as - the range of them (=173-180=). So a writer that meets a poem knows what + the range of them (~173-180~). So a writer that meets a poem knows what follows and where it ends, and can wrap the verses in one pass. The poem - has no text of its own. =0 0=, or absent, where the object is not a + has no text of its own. ~0 0~, or absent, where the object is not a container. *Before 2.1* the poem came /after/ its last verse, carried that verse's OCN and identifier, and stated no range. @@ -222,8 +224,8 @@ Objects are grouped into nine sections, in this order: head toc body endnotes glossary bibliography bookindex blurb tail #+end_example -=head= is the document's title matter, =body= the document proper, and the -rest are generated or gathered. =tail= is the end of document some output +~head~ is the document's title matter, ~body~ the document proper, and the +rest are generated or gathered. ~tail~ is the end of document some output writers use. A section with no objects is not carried. * 7. Segments @@ -285,10 +287,10 @@ Two of the values above are *derived*: the identifier from the OCN ([[#sec-3-1][section 3.1]]), and the anchor from the segment name ([[#sec-7-1][section 7.1]]). The model holds them; a serialisation need not, and the text form does not. -*A serialisation may omit a value it can compute.* The =.ssp= writes a -property only when it carries information, so it omits =.identifier= where -the identifier is the OCN and =.anchor= where the anchor is the segment -name. The =.ocda.db= is columns rather than lines and writes both +*A serialisation may omit a value it can compute.* The ~.ssp~ writes a +property only when it carries information, so it omits ~.identifier~ where +the identifier is the OCN and ~.anchor~ where the anchor is the segment +name. The ~.ocda.db~ is columns rather than lines and writes both unconditionally. Both are conforming. What is *not* conforming is a reader that takes the @@ -297,8 +299,8 @@ that model is not the same model the database gives, and a writer over it will produce different output from the two serialisations of one document. The size of the difference is easy to underestimate. In -=free_culture.lawrence_lessig= the =.ssp= omits *1,314 identifiers and 69 -anchors* that the =.ocda.db= states, which is nearly every object in the +~free_culture.lawrence_lessig~ the ~.ssp~ omits *1,314 identifiers and 69 +anchors* that the ~.ocda.db~ states, which is nearly every object in the document. So, on loading: @@ -306,18 +308,18 @@ So, on loading: | field | when absent | take | |------------+--------------------------+------------------------------------------------------------------| | identifier | OCN is non-zero | the OCN as text | -| identifier | OCN is =0= | whatever the serialisation gave; there is nothing to derive from | +| identifier | OCN is ~0~ | whatever the serialisation gave; there is nothing to derive from | | anchor | the object has a segment | the segment name | *This is a property of the model, not of either format*, which is why it is stated here and not only in the two format specifications. It is stated here /because/ two readers written independently from these documents -disagreed for want of it: the rule was implied by the =.ssp='s emission +disagreed for want of it: the rule was implied by the ~.ssp~'s emission conditions and nowhere asserted. It is also what *spine's own reader* does, and always did, which is why -the =.ocda.db= carries the derived values: the database is written from the -=.ssp= read straight back, and the reading fills both in. So this section +the ~.ocda.db~ carries the derived values: the database is written from the +~.ssp~ read straight back, and the reading fills both in. So this section documents existing behaviour rather than prescribing new. It should have been written down first. @@ -326,7 +328,7 @@ been written down first. :CUSTOM_ID: sec-10 :END: - *The markup.* The model is what the markup produced, not the markup - itself. An artefact that carries both is the =.ocda.db=, which carries + itself. An artefact that carries both is the ~.ocda.db~, which carries the markup alongside the model rather than within it. - *The images.* The model /names/ them, with digest, byte size and pixel size, and they must be found beside the file for a document to be diff --git a/ssp-format-specification.org b/ssp-format-specification.org index 7258862..d0846de 100644 --- a/ssp-format-specification.org +++ b/ssp-format-specification.org @@ -9,7 +9,7 @@ 2.1*, as spine emits and reads it after 0.25.1. This document specifies /one serialisation/. The model it serialises is -specified in [[file:ocda-model.org][=ocda-model.org=]], which the sqlite +specified in [[file:ocda-model.org][~ocda-model.org~]], which the sqlite serialisation shares; read that first if the question is what an object /is/ rather than how it is written down. @@ -17,14 +17,14 @@ serialisation shares; read that first if the question is what an object :PROPERTIES: :CUSTOM_ID: the-name :END: -=.ssp= belongs to the family of SiSU markup extensions, =.sst= (sisu text), -=.ssm= (sisu master) and =.ssi= (sisu insert), and a short extension was +~.ssp~ belongs to the family of SiSU markup extensions, ~.sst~ (sisu text), +~.ssm~ (sisu master) and ~.ssi~ (sisu insert), and a short extension was wanted that sat with them. It stands loosely for *sisu peg*, or *sisu -parsable*. =.ssa=, for sisu abstraction, was thought to be taken by other +parsable*. ~.ssa~, for sisu abstraction, was thought to be taken by other projects. So the extension names the /file family and its parsability/, where -=.ocda.db= names the /model and its container/. The two are not parallel in +~.ocda.db~ names the /model and its container/. The two are not parallel in construction and are not meant to be. * Status @@ -32,21 +32,21 @@ construction and are not meant to be. :CUSTOM_ID: status :END: This specification is derived from the emitter -(=sisudoc/ocda/abstraction/ssp.d= in sisudoc-spine, whose =sspObjectRecord= -is "the single definition of what an object looks like in a =.ssp= file"), +(~sisudoc/ocda/abstraction/ssp.d~ in sisudoc-spine, whose ~sspObjectRecord~ +is "the single definition of what an object looks like in a ~.ssp~ file"), and every claim in it is checked against the project's 36 document reference set. Where the two could differ, the emitter is normative and this document is wrong. -* 1. What a =.ssp= is +* 1. What a ~.ssp~ is :PROPERTIES: :CUSTOM_ID: sec-1 :END: -A =.ssp= file is the serialisation, as text, of one language of one +A ~.ssp~ file is the serialisation, as text, of one language of one document after spine has parsed it: the object model, not the markup and not any output format. -Spine's pipeline is *markup -> abstraction -> output*. A =.ssp= is the +Spine's pipeline is *markup -> abstraction -> output*. A ~.ssp~ is the middle stage written down. Since spine 0.25.0 it is also read back, so a document may enter the pipeline at the abstraction stage instead of being parsed, and the output writers cannot tell which happened. @@ -57,7 +57,7 @@ The format is: - *order significant.* Sections appear in a fixed order, objects in document order, and an object's properties in a fixed order ([[#sec-6-3][section 6.3]]). - *UTF-8*, without a byte order mark. -- *lossless for the abstraction*, not for the markup: a =.ssp= read back +- *lossless for the abstraction*, not for the markup: a ~.ssp~ read back and re-emitted is byte identical, which the project's test suite asserts over the whole reference set. @@ -67,9 +67,9 @@ The format is: :END: It does not carry the markup it was made from, and it does not carry its images: it /names/ them, with their digests and pixel sizes, and they must -be found beside it (=../image/= within a pod) for the document to be -complete. An artefact that carries both is the =.ocda.db=, which is built -by reading a =.ssp= back, so it can hold nothing this format cannot express. +be found beside it (~../image/~ within a pod) for the document to be +complete. An artefact that carries both is the ~.ocda.db~, which is built +by reading a ~.ssp~ back, so it can hold nothing this format cannot express. * 2. Version and compatibility :PROPERTIES: @@ -77,11 +77,11 @@ by reading a =.ssp= back, so it can hold nothing this format cannot express. :END: The version is declared *in band*, on the first line of every file: -#+begin_example +#+begin_src ssp % SiSU Document Abstraction v2.1 -#+end_example +#+end_src -and by the sibling sqlite serialisation in its =schema.version= metadata +and by the sibling sqlite serialisation in its ~schema.version~ metadata row, so that one bump moves both. ** 2.1 The rule @@ -99,12 +99,21 @@ row, so that one bump moves both. :PROPERTIES: :CUSTOM_ID: sec-2-2 :END: -| version | date | what moved | -|---------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 2.1 | 2026-09-29 | =.ocn_range=, the first and last ocn of what a container holds. A poem is a container of verses and is cited as their range: it now comes before its first verse, carries ocn =0= and no identifier, and states the range, where it had come after its last verse carrying that verse's ocn and identifier, so that one number named two objects. A minor bump: the property is an addition, the poem was never rendered and has no text, the verses are unchanged, and a 2.0 reader reads a 2.1 file. The same run gathers a note in a poem's last verse into the endnotes, which it had not been: one more endnote in the reference collection. And the editor's notes, =~[* ]~= and =~[+ ]~=, which spine had not recognised and passed through as text, are notes: a third form of note mark, a symbol and a number (=*1=, =+1=), each series numbered through the document; 135 more endnotes, 2,294 in all ([[#sec-3-2][section 3.2]], "A note begins with its mark"). | -| 2.0 | 2026-09-18 | =source.digest= changes meaning: it covers every markup file the document is made of, where it had been the digest of the master file alone. A change of meaning, so the major part. | -| 1.1 | 2026-09-10 | The =@source= block; the document metadata the output writers read, carried in full (14 =@meta= and 3 =@make= properties added); =.segment_lv4_is=. | -| 0.1 | | The first form. | +- 2.1 [2026-09-29 Tue] :: ~.ocn_range~, the first and last ocn of what a + container holds. A poem is a container of verses and is cited as their range: + it now comes before its first verse, carries ocn + ~0~ and no identifier, and states the range, where it had come after its last verse carrying that verse's ocn and identifier, so that one number named two objects. A minor bump: the property is an addition, the poem was never rendered and has no text, the verses are unchanged, and a 2.0 reader reads a 2.1 file. The same run gathers a note in a poem's last verse into the endnotes, which it had not been: one more endnote in the reference collection. And the editor's notes, =~[* + ]~= and =~[+ + ]~=, which spine had not recognised and passed through as text, are notes: a third form of note mark, a symbol and a number (~*1~, ~+1~), + each series numbered through the document; 135 more endnotes, 2,294 in all + ([[#sec-3-2][section 3.2]], "A note begins with its mark"). +- 2.0 [2026-09-18 Fri] :: ~source.digest~ changes meaning: it covers every + markup file the document is made of, where it had been the digest of the + master file alone. A change of meaning, so the major part. +- 1.1 [2026-09-10 Thu] :: The ~@source~ block; the document metadata the output + writers read, carried in full (14 ~@meta~ and 3 ~@make~ properties added); + ~.segment_lv4_is~. +- 0.1 :: The first form. * 3. Lexical structure :PROPERTIES: @@ -113,29 +122,30 @@ row, so that one bump moves both. Six line types, distinguished by their first character. Nothing else is a valid line. -| line | form | meaning | -|--------------------+-----------------+-----------------------------------------------| -| comment | =% ...= | file header only ([[#sec-4][section 4]]) | -| block open | =@name {= | opens a preamble block or a section | -| block close | =}= | closes it | -| key | =key: value=, indented two spaces | inside a preamble block | -| object declaration | =[ocn] kind...= | opens an object record ([[#sec-6-1][section 6.1]]) | -| property | =.name: value= | a property of the object just declared | -| text | \vert{} then the text | a line of the object's text | -| blank | | separates records; carries no meaning | +| line | form | meaning | +|--------------------+-----------------------------------+----------------------------------------| +| comment | ~% ...~ | file header only ([[#sec-4][section 4]]) | +| block open | ~@name {~ | opens a preamble block or a section | +| block close | ~}~ | closes it | +| key | ~key: value~, indented two spaces | inside a preamble block | +| object declaration | ~[ocn] kind...~ | opens an object record ([[#sec-6-1][section 6.1]]) | +| property | ~.name: value~ | a property of the object just declared | +| text | \vert{} then the text | a line of the object's text | +| blank | | separates records; carries no meaning | A value runs to the *end of the line*. It is not quoted, and it is not trimmed beyond the single space after the colon. A value may therefore -contain =:=, spaces, and any UTF-8 character except a line break. +contain ~:~, spaces, and any UTF-8 character except a line break. ** 3.1 Escaping :PROPERTIES: :CUSTOM_ID: sec-3-1 :END: Only one value is escaped, because only one value is a *joined list*: -=.heading_ancestors_text= joins eight slots with =|=. A slot that itself -contains =|=, a backslash, or a line break would otherwise be unreadable. +~.heading_ancestors_text~ joins eight slots with ~|~. A slot that itself +contains ~|~, a backslash, or a line break would otherwise be unreadable. +#+NAME: ssp-escapes #+begin_example in the value written as \ \\ @@ -145,81 +155,82 @@ carriage return \r #+end_example Unescaping reads left to right: a backslash escapes the next character, -=\n= and =\r= become the control characters, any other escaped character -stands for itself, and an unescaped =|= ends the slot. +~\n~ and ~\r~ become the control characters, any other escaped character +stands for itself, and an unescaped ~|~ ends the slot. *No other value is escaped*, and text lines are not escaped at all: a text line is the object's text verbatim after its two-character prefix, -=|= and a space. +~|~ and a space. ** 3.2 Object text is internal markup :PROPERTIES: :CUSTOM_ID: sec-3-2 :END: -The text on =|= lines is spine's *internal* representation, neither the +The text on ~|~ lines is spine's *internal* representation, neither the author's markup nor any output. It carries the markers spine uses to hold inline structure: -The full set is defined in spine's =InlineMarkup= (=ocda/meta/defaults.d=). +The full set is defined in spine's ~InlineMarkup~ (~ocda/meta/defaults.d~). Counts are occurrences in object text across the 36 document reference collection, which distinguishes what a reader will certainly meet from what it may not. -| marker | means | count | where | +| marker | means | count | where | |-------------+---------------------------------------------+--------+----------------------| -| =░= | non-breaking space | 23,048 | | -| =┥= =┝= | link text, open and close | 20,290 | | -| =┤= =├= | link target, open and close | 20,290 | | -| =⑆= | a page number of the source edition follows | 19,099 | | -| =┨= =┣= | that page number, open and close | 19,099 | | -| =■= | monospace | 10,688 | | -| =【= =】= | an inline note, its mark first (below) | 2,294 | | -| =¤= | an internal site link | 2,157 | | -| =┊= | table cell separator | 695 | =table= objects only | -| =┙= | line break, inline | 184 | | -| =☼= | an image, in a link's text slot | 63 | | -| =┚= | line break, spaced | 50 | =group= objects only | -| =〖= =〗= | a starred note | 0 | | -| =‖= | a citation | 0 | | -| =┘= | line break, plain | 0 | | -| =┼= =┿= =╂= | page break: line, page, new page | 0 | | -| =┏= =┆= | table open, table paragraph | 0 | | -| =␣= | separator | 0 | | +| ~░~ | non-breaking space | 23,048 | | +| ~┥~ ~┝~ | link text, open and close | 20,290 | | +| ~┤~ ~├~ | link target, open and close | 20,290 | | +| ~⑆~ | a page number of the source edition follows | 19,099 | | +| ~┨~ ~┣~ | that page number, open and close | 19,099 | | +| ~■~ | monospace | 10,688 | | +| ~【~ ~】~ | an inline note, its mark first (below) | 2,294 | | +| ~¤~ | an internal site link | 2,157 | | +| ~┊~ | table cell separator | 695 | ~table~ objects only | +| ~┙~ | line break, inline | 184 | | +| ~☼~ | an image, in a link's text slot | 63 | | +| ~┚~ | line break, spaced | 50 | ~group~ objects only | +| ~〖~ ~〗~ | a starred note | 0 | | +| ~‖~ | a citation | 0 | | +| ~┘~ | line break, plain | 0 | | +| ~┼~ ~┿~ ~╂~ | page break: line, page, new page | 0 | | +| ~┏~ ~┆~ | table open, table paragraph | 0 | | +| ~␣~ | separator | 0 | | *A count of 0 means the reference collection does not exercise it, not that it cannot appear.* All of these are defined by the emitter and a reader must be prepared for them. -*A note begins with its mark*, then whitespace, then the note: =【12 text】=. +*A note begins with its mark*, then whitespace, then the note: ~【12 text】~. The mark is one of three forms, and a reader must take all three: -| mark | what | example | -|-------------------------+-------------------------------------------------------------------------------------------------------------------------------+----------------------------| -| digits | the author's notes, numbered through the document | =【12 ...】= | -| =*= or =+=, repeated | a special note, marked by the symbols alone | =【** ...】= | -| =*= or =+=, then digits | an editor's note (2.1): the markup's =~[* ]~= and =~[+ ]~=, each series numbered through the document apart from the author's | =【*3 ...】=, =【+1 ...】= | +- digits :: the author's notes, numbered through the document: ~【12 ...】~ +- ~*~ or ~+~, repeated :: a special note, marked by the symbols alone: + ~【** ...】~ +- ~*~ or ~+~, then digits :: an editor's note (2.1): the markup's =~[* ]~= and + =~[+ ]~=, each series numbered through the document apart from the author's: + ~【*3 ...】~, ~【+1 ...】~ -So the pattern is =【(\d+|[*+]+[0-9]*)\s=. Before 2.1 the third form did not +So the pattern is ~【(\d+|[*+]+[0-9]*)\s~. Before 2.1 the third form did not occur: spine did not recognise the editor's-note markup, and it came through as text, =~[* ...]~= and all. Each note of every form is gathered into the -=endnotes= section, where its =.anchor_tag= is =note_= and the mark -(=note_12=, =note_*3=). +~endnotes~ section, where its ~.anchor_tag~ is ~note_~ and the mark +(~note_12~, ~note_*3~). Two traps in this table, both found by counting rather than by reading: -*=┚= is defined twice*, as the spaced line break and as the table close. -In object text it occurs only in =group= objects, never in a =table=, so +*~┚~ is defined twice*, as the spaced line break and as the table close. +In object text it occurs only in ~group~ objects, never in a ~table~, so the object's kind resolves it. A reader should decide by kind rather than by character. -*=「= and =」= are not markers here.* They are defined in =InlineMarkup= +*~「~ and ~」~ are not markers here.* They are defined in ~InlineMarkup~ as the wrapper spine's ODT writer puts around an object number, but that happens at output, after the abstraction. Their 67 occurrences in the reference collection are *Japanese corner brackets in the document's own -text*, being quotation marks in Japanese. In a =.ssp= they are always +text*, being quotation marks in Japanese. In a ~.ssp~ they are always content. -Some markers are the ASCII characters =*=, =!=, =/=, =_=, =^= and =,= for +Some markers are the ASCII characters ~*~, ~!~, ~/~, ~_~, ~^~ and ~,~ for emphasis, bold, italic, underscore, superscript and subscript. These are *not distinguishable from ordinary text by inspection*, and a reader that needs them must follow spine's rules for where they apply rather than @@ -236,7 +247,8 @@ are part of the format. :END: In order, and this order is fixed: -#+begin_example +#+NAME: ssp-file-shape +#+begin_src ssp % SiSU Document Abstraction v2.1 % Source: <markup filename> @@ -257,9 +269,9 @@ In order, and this order is fixed: @bookindex { ... } @blurb { ... } @tail { ... } -#+end_example +#+end_src -The two =%= lines are the only comments in the file and appear only here. +The two ~%~ lines are the only comments in the file and appear only here. The four preamble blocks are always present and always in that order. The nine section blocks appear in that order. *A section is omitted entirely when the document has no objects for it*, though in practice that is rare: @@ -270,84 +282,129 @@ them holding a single generated object. :PROPERTIES: :CUSTOM_ID: sec-5 :END: -Inside a block, every line is =key: value= with exactly two spaces of +Inside a block, every line is ~key: value~ with exactly two spaces of indent. Keys are unique within a block. *A key whose value is empty is omitted*, so the set of keys present varies by document; the exceptions are noted below. -** 5.1 =@source= - what this abstraction was made from +** 5.1 ~@source~ - what this abstraction was made from :PROPERTIES: :CUSTOM_ID: sec-5-1 :END: -| key | always? | meaning | -|-------------+----------------------------+---------------------------------------------------------------------------| -| =language= | yes | the language of /this/ file | -| =languages= | when the pod declares them | every language of the document, space separated | -| =digest= | yes | sha256 over *every markup file the document is made of*, in a fixed order | +- ~language~, always :: the language of /this/ file +- ~languages~, when the pod declares them :: every language of the document, + space separated +- ~digest~, always :: sha256 over *every markup file the document is made of*, + in a fixed order -The source /filename/ is not repeated here; it is on the =% Source:= line. +The source /filename/ is not repeated here; it is on the ~% Source:~ line. -=digest= is what changed in 2.0. It had been the digest of the master file +~digest~ is what changed in 2.0. It had been the digest of the master file alone, which for a single file document was right and for a multi-file one was close to meaningless: any chapter could be edited without the digest moving. -** 5.2 =@meta= - the document's own metadata +** 5.2 ~@meta~ - the document's own metadata [28/43] :PROPERTIES: :CUSTOM_ID: sec-5-2 +:COOKIE_DATA: checkbox recursive :END: -Dublin-Core shaped, =group.name=. *43 keys are defined*; a given document +Dublin-Core shaped, ~group.name~. *43 keys are defined*; a given document carries only those it has. The 28 the reference collection exercises are -marked; the remainder are defined and simply unused by these documents. - -| group | keys | -|---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| =title.= | =main=/, =subtitle=/, =full=/, =language=/, =edition=* | -| =creator.= | =author=/, =author_surname=/, =author_surname_fn=/, =author_email=/, =illustrator=/, =translator=/ | -| =date.= | =published=/, =created=/, =issued=/, =available=/, =modified=/, =valid=/, =added_to_site=* | -| =rights.= | =copyright=/, =license=/, =copyright_text=, =copyright_translation=, =copyright_illustrations=, =copyright_photographs=, =copyright_cover=, =copyright_audio=, =copyright_video= | -| =classify.= | =topic_register=/, =subject=, =keywords=, =loc=/, =dewey=* | -| =identifier.= | =isbn=, =oclc= | -| =notes.= | =summary=/, =abstract=/, =description=* | -| =original.= | =source=*, =title=, =language=, =language_char= | -| =language.= | =document_char=*, =document= | - -=classify.topic_register= is structured within its value: topics separated -by =;=, levels within a topic by =:=, and alternatives at a level by =|=. - -** 5.3 =@make= - how the document is to be built +checked; the remainder are defined and simply unused by these documents. +The counts, on the heading and on each group, are of the checks. + +- ~title.~ [5/5] + - [X] ~main~ + - [X] ~subtitle~ + - [X] ~full~ + - [X] ~language~ + - [X] ~edition~ +- ~creator.~ [6/6] + - [X] ~author~ + - [X] ~author_surname~ + - [X] ~author_surname_fn~ + - [X] ~author_email~ + - [X] ~illustrator~ + - [X] ~translator~ +- ~date.~ [7/7] + - [X] ~published~ + - [X] ~created~ + - [X] ~issued~ + - [X] ~available~ + - [X] ~modified~ + - [X] ~valid~ + - [X] ~added_to_site~ +- ~rights.~ [2/9] + - [X] ~copyright~ + - [X] ~license~ + - [ ] ~copyright_text~ + - [ ] ~copyright_translation~ + - [ ] ~copyright_illustrations~ + - [ ] ~copyright_photographs~ + - [ ] ~copyright_cover~ + - [ ] ~copyright_audio~ + - [ ] ~copyright_video~ +- ~classify.~ [3/5] + - [X] ~topic_register~ + - [ ] ~subject~ + - [ ] ~keywords~ + - [X] ~loc~ + - [X] ~dewey~ +- ~identifier.~ [0/2] + - [ ] ~isbn~ + - [ ] ~oclc~ +- ~notes.~ [3/3] + - [X] ~summary~ + - [X] ~abstract~ + - [X] ~description~ +- ~original.~ [1/4] + - [X] ~source~ + - [ ] ~title~ + - [ ] ~language~ + - [ ] ~language_char~ +- ~language.~ [1/2] + - [X] ~document_char~ + - [ ] ~document~ + +~classify.topic_register~ is structured within its value: topics separated +by ~;~, levels within a topic by ~:~, and alternatives at a level by ~|~. + +** 5.3 ~@make~ - how the document is to be built :PROPERTIES: :CUSTOM_ID: sec-5-3 :END: -| key | value | meaning | -|-------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------| -| =doc_type= | =article=, =book= | what kind of document this is | -| =auto_num_top_at_level= | a level, e.g. =1= | the heading level automatic numbering starts at, *as written in the markup* | -| =auto_num_top_lv= | that level's index, 0..8 | the same level as a number ([[file:ocda-model.org::#sec-4][section 4]] of [[file:ocda-model.org][=ocda-model.org=]]); =9= where numbering is off | -| =auto_num_depth= | e.g. =2= | how many levels down numbering continues | -| =breaks= | see below | where the output starts a new page, and where it breaks one | -| =footer= | slots joined with \vert{} | the footer, each slot carrying link markup | -| =home_button_text= | link markup | the home link, as internal markup | - -=auto_num_top_at_level= and =auto_num_top_lv= are *the same value twice*, -as the markup wrote it and as an index: =1= and =4=, since the body levels +- ~doc_type~ (~article~, ~book~) :: what kind of document this is +- ~auto_num_top_at_level~ (a level, e.g. ~1~) :: the heading level automatic + numbering starts at, *as written in the markup* +- ~auto_num_top_lv~ (that level's index, 0..8) :: the same level as a number + ([[file:ocda-model.org::#sec-4][section 4]] of + [[file:ocda-model.org][~ocda-model.org~]]); ~9~ where numbering is off +- ~auto_num_depth~ (e.g. ~2~) :: how many levels down numbering continues +- ~breaks~ (see below) :: where the output starts a new page, and where it + breaks one +- ~footer~ (slots joined with ~|~) :: the footer, each slot carrying link markup +- ~home_button_text~ (link markup) :: the home link, as internal markup + +~auto_num_top_at_level~ and ~auto_num_top_lv~ are *the same value twice*, +as the markup wrote it and as an index: ~1~ and ~4~, since the body levels begin at index 4. -=breaks= is a small language of its own, =new== for a new page and -=break== for a page break, each taking a comma separated list of levels, -the two separated by =;=: +~breaks~ is a small language of its own, ~new=~ for a new page and +~break=~ for a page break, each taking a comma separated list of levels, +the two separated by ~;~: -#+begin_example +#+begin_src ssp breaks: new=:A,:B; break=:C,1 breaks: new=:B; break=1 breaks: break=1 -#+end_example +#+end_src The level names here are written as they are in the markup, so with or without the leading colon as the document wrote them; a reader should not assume either form. -** 5.4 =@doc_has= - what the document contains, by count +** 5.4 ~@doc_has~ - what the document contains, by count :PROPERTIES: :CUSTOM_ID: sec-5-4 :END: @@ -371,18 +428,18 @@ lines, then a blank line. :PROPERTIES: :CUSTOM_ID: sec-6-1 :END: -#+begin_example +#+begin_src ssp [<ocn>] <kind> [<ocn>] heading :<level> [<ocn>] heading :<level> <identifier> -#+end_example +#+end_src -=<ocn>= is the *object citation number* (see -[[file:ocda-model.org][=ocda-model.org=]] [[file:ocda-model.org::#sec-3][section 3]]). =[0]= means the object has +~<ocn>~ is the *object citation number* (see +[[file:ocda-model.org][~ocda-model.org~]] [[file:ocda-model.org::#sec-3][section 3]]). ~[0]~ means the object has none, is shared by many objects, and must never be used as a key; key on position within a section instead. -=<kind>= is one of sixteen: +~<kind>~ is one of sixteen: #+begin_example para heading toc bookindex endnote code bibliography comment @@ -390,18 +447,18 @@ verse glossary quote poem block table blurb group #+end_example A *heading* is the one kind whose declaration line carries more: its -markup level after a colon (=:A= =:B= =:C= =:D= for the title levels, -=:1=..=:4= for the body levels; =:D= is defined and unexercised by the +markup level after a colon (~:A~ ~:B~ ~:C~ ~:D~ for the title levels, +~:1~..~:4~ for the body levels; ~:D~ is defined and unexercised by the reference collection), and then its identifier, when it has one that is not simply the ocn. Every other kind carries an identifier, on the -same condition, as the =.identifier= property. +same condition, as the ~.identifier~ property. #+begin_quote *A reader must derive the identifier where it is absent, and the anchor likewise.* "when it has one that is not simply the ocn" means the identifier /is/ the ocn in every other case, and a reader that leaves the -field empty gives a different model from the one the =.ocda.db= gives, on -nearly every object. [[file:ocda-model.org::#sec-9][Section 9]] of [[file:ocda-model.org][=ocda-model.org=]] states +field empty gives a different model from the one the ~.ocda.db~ gives, on +nearly every object. [[file:ocda-model.org::#sec-9][Section 9]] of [[file:ocda-model.org][~ocda-model.org~]] states both derivations and why they belong to the model rather than to either format. Two readers written independently from these documents disagreed for want of it. @@ -411,17 +468,18 @@ for want of it. :PROPERTIES: :CUSTOM_ID: sec-6-2 :END: -#+begin_example +#+begin_src ssp | <text> -#+end_example +#+end_src One line per line of the object's text, in order, each prefixed with -=|= and a space. An object with no text has no text lines. The text is +~|~ and a space. An object with no text has no text lines. The text is internal markup ([[#sec-3-2][section 3.2]]) and is not escaped. ** 6.3 Properties :PROPERTIES: :CUSTOM_ID: sec-6-3 +:COLUMNS: %ORDER(#) %ITEM(property) %VALUE(value) %EMITTED(emitted when) %COUNT(count) :END: *Properties appear in a fixed order*, the order of the table below, and a reader may rely on it: checked over the reference set, no object among @@ -429,8 +487,8 @@ reader may rely on it: checked over the reference set, no object among *A property is omitted when it holds its default*, so absence is meaningful and is defined per property in the table. Four properties -repeat, appearing once per value rather than carrying a list: =.image=, -=.stow_link=, =.lev4_subtoc= and =.anchor_tag=. +repeat, appearing once per value rather than carrying a list: ~.image~, +~.stow_link~, ~.lev4_subtoc~ and ~.anchor_tag~. Two consequences of the order, which a validator needs stated and which hold over the whole reference collection: @@ -439,94 +497,486 @@ hold over the whole reference collection: position in the order, and nothing is written between them. Checked: no repeat in the collection is separated by another property. - *no property follows the text.* The record is declaration, then - properties, then text lines, and once a =|= line has been written no + properties, then text lines, and once a ~|~ line has been written no further property appears. Checked: 0 occurrences in the collection. In the table, "emitted when" is the condition under which the property appears at all. Counts are occurrences across the 36 document reference set, as a guide to which parts of the format are load bearing. -| # | property | value | emitted when | count | -|----+---------------------------+------------------------------+---------------------------------------------------------------------------------------------------+--------| -| 1 | =.identifier= | string | a non-heading has an identifier that is not its ocn (*otherwise it is the ocn*: [[file:ocda-model.org::#sec-9][model section 9]]) | 353 | -| 2 | =.part= | string | the object belongs to a part (=frontmatter=, =body=, ...) | 57,018 | -| 3 | =.section= | string | the object's own section differs from the block it sits in | 605 | -| 4 | =.parent= | int | *every object except* the document root and =toc= entries | 53,767 | -| 5 | =.last_descendant= | int | non-zero: the last ocn under this heading | 3,202 | -| 6 | =.ocn_range= | 2 ints | the object is a container: the first and last ocn of what it holds (2.1; a poem, [[file:ocda-model.org::#sec-5][model section 5]]) | 48 | -| 7 | =.children= | ints | the heading has child headings | 827 | -| 8 | =.ancestors= | 8 ints | any is non-zero | 53,731 | -| 9 | =.ancestors_collapsed= | 8 ints | any is non-zero | 53,731 | -| 10 | =.dom_status= | 8 ints | any is non-zero | 3,319 | -| 11 | =.dom_status_collapsed= | 8 ints | any is non-zero | 3,319 | -| 12 | =.heading_lev_collapsed= | int | it is below 9 | 3,503 | -| 13 | =.parent_lev= | int | non-zero | 52,984 | -| 14 | =.dummy= | =true= | the heading was introduced to segment an output | 183 | -| 15 | =.ocn_off= | =true= | the object has no citation number | 7,543 | -| 16 | =.is_of_type= | string | always, in practice | 57,018 | -| 17 | =.attrib= | string | set | 1 | -| 18 | =.meta_lang= | string | the object carries its own language | 0 | -| 19 | =.meta_syntax= | string | the object carries its own markup syntax | 0 | -| 20 | =.sha256= | hex | the object's text digest is non-zero | 51,253 | -| 21 | =.image= | record | once *per image*, in order of appearance | 63 | -| 22 | =.indent= | 2 ints | either indent is non-zero: =base hang= | 7,583 | -| 23 | =.bullet= | =true= | the object is a bullet item | 1,903 | -| 24 | =.lang= | string | the text attributes carry a language | 0 | -| 25 | =.has= | flags | any is set | 11,384 | -| 26 | =.table_cols= | int | the object is a table with columns | 32 | -| 27 | =.table_widths= | ints | the table has column widths | 32 | -| 28 | =.table_aligns= | letters | the table has alignments (=l= =c= =r=) | 32 | -| 29 | =.table_header= | =true= | the table has a header row | 29 | -| 30 | =.code_linenumbers= | =true= | the code block is numbered | 1 | -| 31 | =.stow_link= | url | once *per extracted link* | 1,014 | -| 32 | =.segment= | string | the heading opens a segment | 3,355 | -| 33 | =.anchor= | string | the anchor differs from the segment name (*otherwise it is the segment name*: [[file:ocda-model.org::#sec-9][model section 9]]) | 68 | -| 34 | =.segment_prev= | string | there is a previous segment | 1,022 | -| 35 | =.segment_next= | string | there is a next segment | 1,022 | -| 36 | =.heading_lev_anchor= | string | the heading has a level anchor tag | 1,854 | -| 37 | =.segment_epub= | string | the heading opens an epub segment | 3,355 | -| 38 | =.segment_html_is= | string | the object is inside an html segment | 47,072 | -| 39 | =.segment_epub_is= | string | the object is inside an epub segment | 47,072 | -| 40 | =.segment_lv4_is= | string | *and* it differs from =.segment_html_is= | 342 | -| 41 | =.heading_ancestors_text= | 8 slots, \vert{} joined, escaped | any slot is non-empty | 3,319 | -| 42 | =.lev4_subtoc= | string | once *per sub-table-of-contents entry* | 1,887 | -| 43 | =.anchor_tag= | string | once *per anchor tag* | 8,019 | - -Three of these need a word here, the first two because the model behind -them is easy to misread (see [[file:ocda-model.org][=ocda-model.org=]] [[file:ocda-model.org::#sec-5][sections 5]] -and [[file:ocda-model.org::#sec-7][7]]), the third because it is a property of this table rather than of the -format. - -*=.parent: 0= is emitted, not omitted.* A parent of 0 is a real, +# A column view of the headings below, one per property: C-c C-c on +# the #+BEGIN line regenerates it from their drawers. +#+BEGIN: columnview :id local :skip-empty-rows t :maxlevel 3 +| # | property | value | emitted when | count | +|----+---------------------------+----------------------------------+---------------------------------------------------------------------------------------------------+--------| +| 1 | ~.identifier~ | string | a non-heading has an identifier that is not its ocn (*otherwise it is the ocn*: [[file:ocda-model.org::#sec-9][model section 9]]) | 353 | +| 2 | ~.part~ | string | the object belongs to a part (~frontmatter~, ~body~, ...) | 57,018 | +| 3 | ~.section~ | string | the object's own section differs from the block it sits in | 605 | +| 4 | ~.parent~ | int | *every object except* the document root and ~toc~ entries | 53,767 | +| 5 | ~.last_descendant~ | int | non-zero: the last ocn under this heading | 3,202 | +| 6 | ~.ocn_range~ | 2 ints | the object is a container: the first and last ocn of what it holds (2.1; a poem, [[file:ocda-model.org::#sec-5][model section 5]]) | 48 | +| 7 | ~.children~ | ints | the heading has child headings | 827 | +| 8 | ~.ancestors~ | 8 ints | any is non-zero | 53,731 | +| 9 | ~.ancestors_collapsed~ | 8 ints | any is non-zero | 53,731 | +| 10 | ~.dom_status~ | 8 ints | any is non-zero | 3,319 | +| 11 | ~.dom_status_collapsed~ | 8 ints | any is non-zero | 3,319 | +| 12 | ~.heading_lev_collapsed~ | int | it is below 9 | 3,503 | +| 13 | ~.parent_lev~ | int | non-zero | 52,984 | +| 14 | ~.dummy~ | ~true~ | the heading was introduced to segment an output | 183 | +| 15 | ~.ocn_off~ | ~true~ | the object has no citation number | 7,543 | +| 16 | ~.is_of_type~ | string | always, in practice | 57,018 | +| 17 | ~.attrib~ | string | set | 1 | +| 18 | ~.meta_lang~ | string | the object carries its own language | 0 | +| 19 | ~.meta_syntax~ | string | the object carries its own markup syntax | 0 | +| 20 | ~.sha256~ | hex | the object's text digest is non-zero | 51,253 | +| 21 | ~.image~ | record | once *per image*, in order of appearance | 63 | +| 22 | ~.indent~ | 2 ints | either indent is non-zero: ~base hang~ | 7,583 | +| 23 | ~.bullet~ | ~true~ | the object is a bullet item | 1,903 | +| 24 | ~.lang~ | string | the text attributes carry a language | 0 | +| 25 | ~.has~ | flags | any is set | 11,384 | +| 26 | ~.table_cols~ | int | the object is a table with columns | 32 | +| 27 | ~.table_widths~ | ints | the table has column widths | 32 | +| 28 | ~.table_aligns~ | letters | the table has alignments (~l~ ~c~ ~r~) | 32 | +| 29 | ~.table_header~ | ~true~ | the table has a header row | 29 | +| 30 | ~.code_linenumbers~ | ~true~ | the code block is numbered | 1 | +| 31 | ~.stow_link~ | url | once *per extracted link* | 1,014 | +| 32 | ~.segment~ | string | the heading opens a segment | 3,355 | +| 33 | ~.anchor~ | string | the anchor differs from the segment name (*otherwise it is the segment name*: [[file:ocda-model.org::#sec-9][model section 9]]) | 68 | +| 34 | ~.segment_prev~ | string | there is a previous segment | 1,022 | +| 35 | ~.segment_next~ | string | there is a next segment | 1,022 | +| 36 | ~.heading_lev_anchor~ | string | the heading has a level anchor tag | 1,854 | +| 37 | ~.segment_epub~ | string | the heading opens an epub segment | 3,355 | +| 38 | ~.segment_html_is~ | string | the object is inside an html segment | 47,072 | +| 39 | ~.segment_epub_is~ | string | the object is inside an epub segment | 47,072 | +| 40 | ~.segment_lv4_is~ | string | *and* it differs from ~.segment_html_is~ | 342 | +| 41 | ~.heading_ancestors_text~ | 8 slots, \vert{} joined, escaped | any slot is non-empty | 3,319 | +| 42 | ~.lev4_subtoc~ | string | once *per sub-table-of-contents entry* | 1,887 | +| 43 | ~.anchor_tag~ | string | once *per anchor tag* | 8,019 | +#+END: + +Two of these carry a note under their own heading, ~.parent~ and ~.segment~, +because the model behind them is easy to misread (see +[[file:ocda-model.org][~ocda-model.org~]] [[file:ocda-model.org::#sec-5][sections 5]] and [[file:ocda-model.org::#sec-7][7]]). +A third thing needs a word here, because it is a property of this table +rather than of the format: + +*Count 0 does not mean "not in the format".* ~.meta_lang~, ~.meta_syntax~ +and ~.lang~ are emitted by spine under the conditions given, and no +document in the reference collection meets them. A reader must handle them; +a test corpus does not exercise them. + +*** ~.identifier~ +:PROPERTIES: +:CUSTOM_ID: prop-identifier +:ORDER: 1 +:VALUE: string +:EMITTED: a non-heading has an identifier that is not its ocn (*otherwise it is the ocn*: [[file:ocda-model.org::#sec-9][model section 9]]) +:COUNT: 353 +:END: + +*** ~.part~ +:PROPERTIES: +:CUSTOM_ID: prop-part +:ORDER: 2 +:VALUE: string +:EMITTED: the object belongs to a part (~frontmatter~, ~body~, ...) +:COUNT: 57,018 +:END: + +*** ~.section~ +:PROPERTIES: +:CUSTOM_ID: prop-section +:ORDER: 3 +:VALUE: string +:EMITTED: the object's own section differs from the block it sits in +:COUNT: 605 +:END: + +*** ~.parent~ +:PROPERTIES: +:CUSTOM_ID: prop-parent +:ORDER: 4 +:VALUE: int +:EMITTED: *every object except* the document root and ~toc~ entries +:COUNT: 53,767 +:END: +*~.parent: 0~ is emitted, not omitted.* A parent of 0 is a real, non-substantive parent and not a missing value, so the property is written out rather than defaulted away: omitting it would make the two cases indistinguishable in the file. A reader walking upward stops at 0. -*=.segment= and =.segment_html_is= are different questions*, and both are +*** ~.last_descendant~ +:PROPERTIES: +:CUSTOM_ID: prop-last-descendant +:ORDER: 5 +:VALUE: int +:EMITTED: non-zero: the last ocn under this heading +:COUNT: 3,202 +:END: + +*** ~.ocn_range~ +:PROPERTIES: +:CUSTOM_ID: prop-ocn-range +:ORDER: 6 +:VALUE: 2 ints +:EMITTED: the object is a container: the first and last ocn of what it holds (2.1; a poem, [[file:ocda-model.org::#sec-5][model section 5]]) +:COUNT: 48 +:END: + +*** ~.children~ +:PROPERTIES: +:CUSTOM_ID: prop-children +:ORDER: 7 +:VALUE: ints +:EMITTED: the heading has child headings +:COUNT: 827 +:END: + +*** ~.ancestors~ +:PROPERTIES: +:CUSTOM_ID: prop-ancestors +:ORDER: 8 +:VALUE: 8 ints +:EMITTED: any is non-zero +:COUNT: 53,731 +:END: + +*** ~.ancestors_collapsed~ +:PROPERTIES: +:CUSTOM_ID: prop-ancestors-collapsed +:ORDER: 9 +:VALUE: 8 ints +:EMITTED: any is non-zero +:COUNT: 53,731 +:END: + +*** ~.dom_status~ +:PROPERTIES: +:CUSTOM_ID: prop-dom-status +:ORDER: 10 +:VALUE: 8 ints +:EMITTED: any is non-zero +:COUNT: 3,319 +:END: + +*** ~.dom_status_collapsed~ +:PROPERTIES: +:CUSTOM_ID: prop-dom-status-collapsed +:ORDER: 11 +:VALUE: 8 ints +:EMITTED: any is non-zero +:COUNT: 3,319 +:END: + +*** ~.heading_lev_collapsed~ +:PROPERTIES: +:CUSTOM_ID: prop-heading-lev-collapsed +:ORDER: 12 +:VALUE: int +:EMITTED: it is below 9 +:COUNT: 3,503 +:END: + +*** ~.parent_lev~ +:PROPERTIES: +:CUSTOM_ID: prop-parent-lev +:ORDER: 13 +:VALUE: int +:EMITTED: non-zero +:COUNT: 52,984 +:END: + +*** ~.dummy~ +:PROPERTIES: +:CUSTOM_ID: prop-dummy +:ORDER: 14 +:VALUE: ~true~ +:EMITTED: the heading was introduced to segment an output +:COUNT: 183 +:END: + +*** ~.ocn_off~ +:PROPERTIES: +:CUSTOM_ID: prop-ocn-off +:ORDER: 15 +:VALUE: ~true~ +:EMITTED: the object has no citation number +:COUNT: 7,543 +:END: + +*** ~.is_of_type~ +:PROPERTIES: +:CUSTOM_ID: prop-is-of-type +:ORDER: 16 +:VALUE: string +:EMITTED: always, in practice +:COUNT: 57,018 +:END: + +*** ~.attrib~ +:PROPERTIES: +:CUSTOM_ID: prop-attrib +:ORDER: 17 +:VALUE: string +:EMITTED: set +:COUNT: 1 +:END: + +*** ~.meta_lang~ +:PROPERTIES: +:CUSTOM_ID: prop-meta-lang +:ORDER: 18 +:VALUE: string +:EMITTED: the object carries its own language +:COUNT: 0 +:END: + +*** ~.meta_syntax~ +:PROPERTIES: +:CUSTOM_ID: prop-meta-syntax +:ORDER: 19 +:VALUE: string +:EMITTED: the object carries its own markup syntax +:COUNT: 0 +:END: + +*** ~.sha256~ +:PROPERTIES: +:CUSTOM_ID: prop-sha256 +:ORDER: 20 +:VALUE: hex +:EMITTED: the object's text digest is non-zero +:COUNT: 51,253 +:END: + +*** ~.image~ +:PROPERTIES: +:CUSTOM_ID: prop-image +:ORDER: 21 +:VALUE: record +:EMITTED: once *per image*, in order of appearance +:COUNT: 63 +:END: +Its inner form is [[#sec-6-4][section 6.4]]. + +*** ~.indent~ +:PROPERTIES: +:CUSTOM_ID: prop-indent +:ORDER: 22 +:VALUE: 2 ints +:EMITTED: either indent is non-zero: ~base hang~ +:COUNT: 7,583 +:END: + +*** ~.bullet~ +:PROPERTIES: +:CUSTOM_ID: prop-bullet +:ORDER: 23 +:VALUE: ~true~ +:EMITTED: the object is a bullet item +:COUNT: 1,903 +:END: + +*** ~.lang~ +:PROPERTIES: +:CUSTOM_ID: prop-lang +:ORDER: 24 +:VALUE: string +:EMITTED: the text attributes carry a language +:COUNT: 0 +:END: + +*** ~.has~ +:PROPERTIES: +:CUSTOM_ID: prop-has +:ORDER: 25 +:VALUE: flags +:EMITTED: any is set +:COUNT: 11,384 +:END: +The flags are [[#sec-6-5][section 6.5]]. + +*** ~.table_cols~ +:PROPERTIES: +:CUSTOM_ID: prop-table-cols +:ORDER: 26 +:VALUE: int +:EMITTED: the object is a table with columns +:COUNT: 32 +:END: + +*** ~.table_widths~ +:PROPERTIES: +:CUSTOM_ID: prop-table-widths +:ORDER: 27 +:VALUE: ints +:EMITTED: the table has column widths +:COUNT: 32 +:END: + +*** ~.table_aligns~ +:PROPERTIES: +:CUSTOM_ID: prop-table-aligns +:ORDER: 28 +:VALUE: letters +:EMITTED: the table has alignments (~l~ ~c~ ~r~) +:COUNT: 32 +:END: + +*** ~.table_header~ +:PROPERTIES: +:CUSTOM_ID: prop-table-header +:ORDER: 29 +:VALUE: ~true~ +:EMITTED: the table has a header row +:COUNT: 29 +:END: + +*** ~.code_linenumbers~ +:PROPERTIES: +:CUSTOM_ID: prop-code-linenumbers +:ORDER: 30 +:VALUE: ~true~ +:EMITTED: the code block is numbered +:COUNT: 1 +:END: + +*** ~.stow_link~ +:PROPERTIES: +:CUSTOM_ID: prop-stow-link +:ORDER: 31 +:VALUE: url +:EMITTED: once *per extracted link* +:COUNT: 1,014 +:END: + +*** ~.segment~ +:PROPERTIES: +:CUSTOM_ID: prop-segment +:ORDER: 32 +:VALUE: string +:EMITTED: the heading opens a segment +:COUNT: 3,355 +:END: +*~.segment~ and ~.segment_html_is~ are different questions*, and both are carried: what a heading opens, and what an object sits in. A reader wanting to know where a paragraph belongs needs the second. -*Count 0 does not mean "not in the format".* =.meta_lang=, =.meta_syntax= -and =.lang= are emitted by spine under the conditions given, and no -document in the reference collection meets them. A reader must handle them; -a test corpus does not exercise them. +*** ~.anchor~ +:PROPERTIES: +:CUSTOM_ID: prop-anchor +:ORDER: 33 +:VALUE: string +:EMITTED: the anchor differs from the segment name (*otherwise it is the segment name*: [[file:ocda-model.org::#sec-9][model section 9]]) +:COUNT: 68 +:END: + +*** ~.segment_prev~ +:PROPERTIES: +:CUSTOM_ID: prop-segment-prev +:ORDER: 34 +:VALUE: string +:EMITTED: there is a previous segment +:COUNT: 1,022 +:END: + +*** ~.segment_next~ +:PROPERTIES: +:CUSTOM_ID: prop-segment-next +:ORDER: 35 +:VALUE: string +:EMITTED: there is a next segment +:COUNT: 1,022 +:END: + +*** ~.heading_lev_anchor~ +:PROPERTIES: +:CUSTOM_ID: prop-heading-lev-anchor +:ORDER: 36 +:VALUE: string +:EMITTED: the heading has a level anchor tag +:COUNT: 1,854 +:END: + +*** ~.segment_epub~ +:PROPERTIES: +:CUSTOM_ID: prop-segment-epub +:ORDER: 37 +:VALUE: string +:EMITTED: the heading opens an epub segment +:COUNT: 3,355 +:END: + +*** ~.segment_html_is~ +:PROPERTIES: +:CUSTOM_ID: prop-segment-html-is +:ORDER: 38 +:VALUE: string +:EMITTED: the object is inside an html segment +:COUNT: 47,072 +:END: + +*** ~.segment_epub_is~ +:PROPERTIES: +:CUSTOM_ID: prop-segment-epub-is +:ORDER: 39 +:VALUE: string +:EMITTED: the object is inside an epub segment +:COUNT: 47,072 +:END: + +*** ~.segment_lv4_is~ +:PROPERTIES: +:CUSTOM_ID: prop-segment-lv4-is +:ORDER: 40 +:VALUE: string +:EMITTED: *and* it differs from ~.segment_html_is~ +:COUNT: 342 +:END: + +*** ~.heading_ancestors_text~ +:PROPERTIES: +:CUSTOM_ID: prop-heading-ancestors-text +:ORDER: 41 +:VALUE: 8 slots, \vert{} joined, escaped +:EMITTED: any slot is non-empty +:COUNT: 3,319 +:END: -** 6.4 The =.image= record +*** ~.lev4_subtoc~ +:PROPERTIES: +:CUSTOM_ID: prop-lev4-subtoc +:ORDER: 42 +:VALUE: string +:EMITTED: once *per sub-table-of-contents entry* +:COUNT: 1,887 +:END: + +*** ~.anchor_tag~ +:PROPERTIES: +:CUSTOM_ID: prop-anchor-tag +:ORDER: 43 +:VALUE: string +:EMITTED: once *per anchor tag* +:COUNT: 8,019 +:END: + +** 6.4 The ~.image~ record :PROPERTIES: :CUSTOM_ID: sec-6-4 :END: -#+begin_example +#+begin_src ssp .image: <filename> sha256:<HEX> bytes:<n> px:<width>x<height> .image: <filename> missing:true -#+end_example +#+end_src Filename first, as the object's handle on the image. The remaining fields -are self describing =key:value= tokens, *so that further fields may be -added without invalidating existing readers*; =px:= is omitted when the -pixel size is unknown. =px:= is the file's own size, not the display size, +are self describing ~key:value~ tokens, *so that further fields may be +added without invalidating existing readers*; ~px:~ is omitted when the +pixel size is unknown. ~px:~ is the file's own size, not the display size, which the object's text already carries. -** 6.5 The =.has= flags +** 6.5 The ~.has~ flags :PROPERTIES: :CUSTOM_ID: sec-6-5 :END: @@ -555,7 +1005,8 @@ the table is the contract. *Checked*: every line of the 36 document reference collection matches one of the eight line shapes below, 721,363 lines with none left over. -#+begin_src peg +#+NAME: ssp-grammar +#+begin_src peg :tangle ssp.peg Document <- Header Source Meta Make DocHas Section* EOF Header <- "% SiSU Document Abstraction v" Version EOL "% Source: " Value EOL Blank @@ -608,9 +1059,9 @@ Digit <- [0-9] - *Tolerate unknown properties* within a known major version, since the minor part is raised precisely by adding them. Ignoring them is conforming behaviour. -- *Treat =[0]= as not an identity.* Objects with no citation number are +- *Treat ~[0]~ as not an identity.* Objects with no citation number are numerous and share the number. -- *Stop at =.parent: 0=* when walking upward. +- *Stop at ~.parent: 0~* when walking upward. ** 8.2 A reader should :PROPERTIES: @@ -635,12 +1086,12 @@ Digit <- [0-9] :CUSTOM_ID: sec-9 :END: The model this format writes down is specified in -[[file:ocda-model.org][=ocda-model.org=]]: objects, citation numbers, structure, +[[file:ocda-model.org][~ocda-model.org~]]: objects, citation numbers, structure, sections, segments and digests. This document adds nothing to the model and subtracts nothing from it; it says how the model is written as text. -The =.ocda.db= is the same abstraction as sqlite, and is *built by reading -a =.ssp= back*: the document is emitted as =.ssp=, read straight in, and +The ~.ocda.db~ is the same abstraction as sqlite, and is *built by reading +a ~.ssp~ back*: the document is emitted as ~.ssp~, read straight in, and what comes back is written. So the database cannot carry an object property this format cannot express, and the two cannot drift. They share the version, which is why a bump moves both. @@ -653,9 +1104,9 @@ well, which this format does not. :PROPERTIES: :CUSTOM_ID: sec-10 :END: -- =example-autonomous-contract.ssp= - one document, complete and unedited. -- =example-wealth-of-networks.ssp= - an excerpt of a large one: verbatim - lines with marked elisions, keeping the preamble, the =@head= section, +- ~example-autonomous-contract.ssp~ - one document, complete and unedited. +- ~example-wealth-of-networks.ssp~ - an excerpt of a large one: verbatim + lines with marked elisions, keeping the preamble, the ~@head~ section, and a run of body objects including one that embeds an image. Each carries, in its opening comments, the command that regenerates it. |
