# The object-centric document abstraction (OCDA) **The model.** What a document *is* once spine has parsed it, independent of any file it is written to. Two serialisations carry this model, and they carry **one version number between them**, because neither may express what the other cannot: | | | |----------------------------------------------------------------------|--------------------------------| | [`ssp-format-specification.md`](ssp-format-specification.md) | the OCDA as text, `.ssp` | | [`ocda-db-format-specification.md`](ocda-db-format-specification.md) | the OCDA as sqlite, `.ocda.db` | This document is what they share. It describes objects; they describe writing objects down. ## 1. Three words, and what each names The terms are used precisely in this repository and in spine, and the distinction is worth holding: | level | term | names | |--------------------|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------| | the model | **OCDA** | what a document is once parsed: numbered objects with structure, identity and digests. Independent of any file, any stage, any tool | | the stage | **abstraction** | the middle of spine's **markup -> abstraction -> output**: the act of building the model, and the point at which a document may now enter the pipeline | | the serialisations | **`.ssp`**, **`.ocda.db`** | the model written down, as text and as sqlite | A thing is named for the level it belongs to. Something true of the model whatever file it is in takes *ocda*; something about the pipeline stage takes *abstraction*; something that reads or writes one file form takes that form's name. ## 2. The object A document is a sequence of **objects**, grouped into sections. An object is a paragraph, a heading, a table, an endnote, a line of verse: the unit the document is made of and the unit it is cited by. Every object carries: - an **object citation number** (OCN), section 3; - an **identifier**, section 3.1; - a **kind** and a **type**, section 2.1; - its **place in the structure**: parent, ancestors, and for a heading its children and its last descendant, section 4; - the **segment** it is in, and for a heading an **anchor**, sections 7 and 7.1; - its **text**, in spine's internal representation; - a **digest** of that text. ### 2.1 Kind and type: two axes, not one An object is classified twice, and the two answer different questions. **Kind** (`is_a`) says *what the object is*, one of sixteen. Each kind belongs to one section, except `heading`, which appears in all nine: | kind | section | what it is | count | |----------------|--------------|--------------------------------------------------------------------------------|--------| | `para` | body | a paragraph, the ordinary unit of a document | 41,885 | | `heading` | *all nine* | a heading at one of the eight levels | 3,503 | | `toc` | toc | an entry in the generated table of contents | 3,215 | | `bookindex` | bookindex | an entry in the generated book index | 3,034 | | `endnote` | endnotes | a note, gathered from where it was called | 2,294 | | `code` | body | a code block | 1,623 | | `bibliography` | bibliography | a bibliography entry | 585 | | `comment` | body | markup comment, carried and not rendered | 450 | | `verse` | body | one verse of a poem | 151 | | `glossary` | glossary | a glossary entry | 56 | | `quote` | body | a block quotation | 53 | | `poem` | body | a poem: the container of its verses, OCN `0`, cited as their range (section 5) | 48 | | `block` | body | a block of grouped text | 45 | | `table` | body | a table, with its columns described | 32 | | `blurb` | blurb | a blurb entry | 28 | | `group` | body | a group block | 16 | **Type** (`is_of_type`) says *how the object's text behaves*, and takes only three values: | type | means | kinds | |-----------|---------------------------------------------------------|---------------------------------------------------------------------------------------| | `para` | flowing text, wrapped as the output sees fit | `para`, `heading`, `toc`, `bookindex`, `endnote`, `bibliography`, `glossary`, `blurb` | | `block` | text whose line breaks are significant and must be kept | `code`, `verse`, `quote`, `poem`, `block`, `table`, `group` | | `comment` | not rendered | `comment` | The mapping from kind to type is fixed: every `code` object is of type `block`, every `para` object of type `para`. **A renderer that has the type does not need to know all sixteen kinds** to decide whether it may reflow the text, which is what the second axis is for. ## 3. The object citation number The OCN is the heart of the model and the reason for it. **An object's OCN is the same in every format spine writes.** A paragraph that is object 42 in the HTML is object 42 in the EPUB, in the PDF, in the plain text, in the search database and in both serialisations of the abstraction. A citation to object 42 of a document is therefore a citation to a fixed piece of text, not to a page in one edition or a scroll position in one rendering. Three properties follow, and each is a trap for a reader that assumes otherwise: - **Not every object has one.** Generated furniture, and objects the markup has suppressed, carry `0`. There are many of them: 7,543 across the 36 document reference collection. - **`0` is therefore not an identity.** Objects sharing `0` are not the same object, and `0` must never be used as a key. - **A non-zero OCN is unique within a document**, from format 2.1: checked over the reference collection, no non-zero OCN occurs twice in any of the 36 document-languages. **Before 2.1 it was not**: a poem carried the OCN of its last verse, so the same number named two objects (48 such pairs in the collection, 21 in `through_the_looking_glass`). A reader of an older artefact keys on position within a section when one row is wanted. ### 3.1 The identifier Every object also carries an **identifier**: the name by which an output refers to it. An HTML anchor, a LaTeX `\hypertarget`, a link from a book index entry to the object it indexes, are all written from the identifier. **The identifier of an object with a non-zero OCN is its OCN**, written as text. That is the ordinary case and it is the reason a citation to object 42 can be turned into a link without consulting anything. An object with OCN `0` has no OCN to take an identifier from, so one is assigned: `a1`, `a2`, `a3`, ... in document order over the objects that need one. There are few of them, eleven in a book, and they are the generated furniture of section 3. **A reader must arrive at the identifier whether or not the serialisation wrote it down**; see section 9. ## 4. Levels A document has **eight heading levels**, in two groups of four: | level | index | what it is | |-------|-------|----------------------------------------------| | `A` | 0 | the document title. Exactly one per document | | `B` | 1 | | | `C` | 2 | the title matter below it | | `D` | 3 | | | `1` | 4 | the top level of the document body | | `2` | 5 | | | `3` | 6 | below that | | `4` | 7 | | `A` to `D` are the title levels, `1` to `4` the body levels, and the index is what the eight-slot arrays of section 5 are indexed by. A heading that claims no level has 9, which is the "not a heading level" value and is why `.heading_lev_collapsed` is emitted only when it is below 9. **Collapsed levels.** A document that uses `A` and then `1`, skipping `B`, `C` and `D`, has a gap in its levels that is an artefact of the markup rather than of the document. Every level-indexed array therefore comes in two forms: as marked up, and **collapsed**, with the unused levels removed so that the remaining ones are contiguous. Outputs that care about nesting depth use the collapsed form; anything tracing back to the markup uses the other. ## 5. Structure Structure is carried by number, not by nesting: an object names its parent rather than containing its children. This is what lets the serialisations be flat and the model be walked in either direction. - **`parent`** - the OCN of the object this one hangs from. **A parent of `0` is not "no parent"**: it is a parent heading the author has made non-substantive, its OCN suppressed or a dummy heading introduced to segment an output. The distinction is preserved deliberately, and a reader walking upward must stop at `0` rather than treat it as missing, or the walk will not terminate. - **`ancestors`** - **eight slots, indexed by level** (section 4): the OCN of the heading at each level above this object, `0` where there is none, so the path to the root is available without walking it. **A heading's own OCN sits in its own level's slot**: checked over the reference collection, 3,432 headings of 3,432. So for a heading the array is its path including itself, and for any other object it is the path to the heading above it. - **`ancestors_collapsed`** - the same, in collapsed levels. - **`children`** - a heading's child headings. - **`last_descendant`** - the last OCN under a heading, which makes a subtree a **range** rather than a traversal: everything from the heading's OCN to its last descendant is inside it. - **`ocn_range`** - for a **container**, the first and last OCN of what it holds (2.1). A poem is the one container: its verses are the citable units, one object each, and the poem is not; it carries OCN `0` and no identifier, **comes immediately before its first verse**, and is cited as the range of them (`173-180`). So a writer that meets a poem knows what follows and where it ends, and can wrap the verses in one pass. The poem has no text of its own. `0 0`, or absent, where the object is not a container. **Before 2.1** the poem came *after* its last verse, carried that verse's OCN and identifier, and stated no range. ## 6. Sections Objects are grouped into nine sections, in this order: ``` head toc body endnotes glossary bibliography bookindex blurb tail ``` `head` is the document's title matter, `body` the document proper, and the rest are generated or gathered. `tail` is the end of document some output writers use. A section with no objects is not carried. ## 7. Segments An output may be one file or many. A **segment** is one of the many: a chapter-sized piece of HTML or EPUB with its own name and its own navigation. Two different questions are asked about segments, and the model answers both separately because the answers differ: - **which segment does this heading open?** Set on headings only. - **which segment is this object in?** Set on nearly every object. Without the second, a reader cannot say where a paragraph belongs. The search database reads the second. ### 7.1 The anchor A heading carries an **anchor**: the name an output links to in order to arrive at that heading. It is what a table of contents entry and a cross reference point at. **An object's anchor is its segment name**, in the ordinary case, because a heading that opens a segment is arrived at by arriving at the segment. It differs only where a heading is not the opening of the segment it sits in: 69 of the 72 anchors in one reference document equal their segment and 3 do not. **A reader must arrive at the anchor whether or not the serialisation wrote it down**; see section 9. ## 8. Digests The model is checkable at four levels, and the chain is what lets an artefact be verified rather than trusted: ``` the markup files ──> a digest over every markup file of that language the serialisation ──> a digest of the .ssp a database was built from each object ──> a digest of the object's text each image ──> a digest of the image file, with its size in bytes and pixels ``` An object's digest is over its text alone, so a change to one paragraph moves one digest and no other. ## 9. Derived values, and why a reader must derive them Two of the values above are **derived**: the identifier from the OCN (section 3.1), and the anchor from the segment name (section 7.1). The model holds them; a serialisation need not, and the text form does not. **A serialisation may omit a value it can compute.** The `.ssp` writes a property only when it carries information, so it omits `.identifier` where the identifier is the OCN and `.anchor` where the anchor is the segment name. The `.ocda.db` is columns rather than lines and writes both unconditionally. Both are conforming. What is **not** conforming is a reader that takes the text form literally and hands on a model with those fields empty, because that model is not the same model the database gives, and a writer over it will produce different output from the two serialisations of one document. The size of the difference is easy to underestimate. In `free_culture.lawrence_lessig` the `.ssp` omits **1,314 identifiers and 69 anchors** that the `.ocda.db` states, which is nearly every object in the document. So, on loading: | field | when absent | take | |------------|--------------------------|------------------------------------------------------------------| | identifier | OCN is non-zero | the OCN as text | | identifier | OCN is `0` | whatever the serialisation gave; there is nothing to derive from | | anchor | the object has a segment | the segment name | **This is a property of the model, not of either format**, which is why it is stated here and not only in the two format specifications. It is stated here *because* two readers written independently from these documents disagreed for want of it: the rule was implied by the `.ssp`'s emission conditions and nowhere asserted. It is also what **spine's own reader** does, and always did, which is why the `.ocda.db` carries the derived values: the database is written from the `.ssp` read straight back, and the reading fills both in. So this section documents existing behaviour rather than prescribing new. It should have been written down first. ## 10. What the model does not hold - **The markup.** The model is what the markup produced, not the markup itself. An artefact that carries both is the `.ocda.db`, which carries the markup alongside the model rather than within it. - **The images.** The model *names* them, with digest, byte size and pixel size, and they must be found beside the file for a document to be complete. - **Any output format.** No HTML, no page numbers, no typesetting.