summaryrefslogtreecommitdiffhomepage
path: root/README.md
diff options
context:
space:
mode:
authorRalph Amissah <ralph.amissah@gmail.com>2026-09-24 11:12:28 -0400
committerRalph Amissah <ralph.amissah@gmail.com>2026-09-24 11:12:28 -0400
commitda833f8b157ae501c9f6eec827dd4a965bfa7265 (patch)
tree9368bbfe07dc130867dc8241d20bc3d125f3e510 /README.md
parentcommencement: a reader for the .ssp (diff)
format 2.0: the version gate
The reader as it stood when the format reached 2.0, in D: a parser of the .ssp with no dependency beyond phobos, and outputs from it - a heading tree, an html scroll, breadcrumbed text, chunked jsonl for embedding pipelines, an image manifest, and a check of image records against the images on disk. Brought to abstraction format 2.0, and made able to say what it does not understand, so a reader can check that its format specification is complete. Previously the reader ignored what it did not recognise and reported nothing, which made it seem it had understood a 2.0 file (and drop things in silence). Now: - the banner is read, and the major part checked. A major this reader does not know is refused, exit 3, because an unknown major means a property may have changed meaning or gone away and loading it half populated in silence is the failure the check exists to prevent. --any-version reads it regardless, for looking at a file rather than trusting one. A file with no banner warns and is read. - @source is a preamble block beside @meta, @make and @doc_has, and the summary line shows the language and the markup digest. - tail is a section like the rest. - --report-unknown lists every property, block and line shape met and not understood, with counts. Ignoring them remains the conforming behaviour within a major version; the counting is so that it is not silent. Over the markup document samples that report names 18 properties, from .is_of_type at 56,882 occurrences down to .attrib at 1. Several are legitimately of no interest to a renderer and several are not. Deciding which is which is the next pass, and it now has a measured work list rather than a guess. (assisted by Claude-Code)
Diffstat (limited to 'README.md')
-rw-r--r--README.md138
1 files changed, 131 insertions, 7 deletions
diff --git a/README.md b/README.md
index 802aed2..1daa461 100644
--- a/README.md
+++ b/README.md
@@ -1,9 +1,133 @@
-# sisudoc-ssp-reader
+# ssp-reader
-A reader for the `.ssp`, the text form of the object-centric document
-abstraction that [sisudoc-spine](https://git.sisudoc.org/sisudoc-spine)
-writes, to show that the abstraction carries what is needed to present a
-document without the markup it was made from.
+Reader for the **`.ssp` document-abstraction text format** emitted
+by [sisudoc-spine](https://git.sisudoc.org/projects/sisudoc-spine).
-Work on the reader begins here. The first version of it to be published
-reads format 2.0, which follows.
+Reads a `.ssp` file (spine's PEG-parsable serialisation of its
+internal document abstraction) and renders it into a chosen output:
+an outline tree, scroll HTML, plain text with heading breadcrumbs,
+JSONL chunks suitable for LLM / embedding pipelines, or an image
+manifest for verifying the images a document embeds.
+
+## What this is for
+
+- Inspecting and debugging the abstraction stage of spine without
+ rebuilding spine itself.
+- Round-trip experiments: markup -> spine -> `.ssp` -> ssp-reader
+ -> downstream artefact.
+- Re-using spine's parsed document structure outside the spine
+ binary: feeding it to an indexer, an LLM pipeline, or a custom
+ renderer.
+- A worked example of consuming `.ssp` for anyone implementing
+ another reader in another language.
+
+## Audience
+
+Developers building tools that consume spine's output: search
+indexers, LLM ingestion pipelines, custom renderers, alternative
+output formats, and anyone studying the abstraction stage of
+spine itself.
+
+End-user document authors do **not** need this tool to publish a
+document. Run spine directly for that.
+
+## Relationship to spine
+
+Spine's processing pipeline has three stages:
+
+1. **Input:** parse `.sst` / `.ssm` markup.
+2. **Abstraction:** build a numbered object model (the "spine
+ abstraction"; one of spine's named outputs is `.ssp`, a
+ PEG-parsable text representation of this model).
+3. **Output:** render to HTML / EPUB / PDF / SQLite / etc.
+
+`ssp-reader` picks up the abstraction at the end of stage 2 (the
+`.ssp` file) and runs alternative stage-3 renderings on top of it.
+The `.ssp` format itself is documented in
+[`doc-abstraction-format`](https://git.sisudoc.org/tools/doc-abstraction-format).
+
+## Quickstart
+
+### Build
+
+Inside the project's nix dev environment (the same one spine uses):
+
+```sh
+nix develop ".#dsh-nixpkgs-ldc" --print-build-logs -c $SHELL
+dub build --compiler=ldmd2
+```
+
+The binary lands at `./bin/ssp-reader`.
+
+### Run
+
+```sh
+# generate a .ssp from a sample document via spine, then:
+ssp-reader --tree doc.ssp # outline view
+ssp-reader --html doc.ssp # HTML scroll output
+ssp-reader --llm-text doc.ssp # plain text with breadcrumbs
+ssp-reader --llm-jsonl doc.ssp # JSONL chunks for embeddings
+ssp-reader --images doc.ssp # image manifest (digest, size, pixels)
+ssp-reader --verify --image-dir=pod/media/image doc.ssp # check images against the records
+ssp-reader --report-unknown doc.ssp # what it read and did not understand
+ssp-reader --tree --output=out.txt doc.ssp
+ssp-reader --help
+```
+
+## Format version
+
+The reader knows abstraction format **2.x**. Every `.ssp` declares its
+version on its first line, and the reader checks the **major** part and
+refuses what it does not know: an unknown major means a property may have
+changed meaning or gone away, and loading such a file half populated in
+silence is the failure the check exists to prevent. `--any-version` reads
+it regardless, for looking at a file rather than trusting one.
+
+## What it does not understand
+
+Within a major version a reader **should** ignore properties it does not
+know, since the minor part is raised precisely by adding them. This reader
+does, and `--report-unknown` says what it ignored and how often:
+
+```
+read and NOT understood:
+ properties ignored (15 distinct):
+ .is_of_type 328
+ .segment_html_is 138
+ ...
+```
+
+That is deliberate and is what makes the reader usable as a check on the
+[format specification](https://git.sisudoc.org/specs/doc-abstraction-format):
+a reader that drops what it does not know is conforming, but one that
+cannot say what it dropped cannot tell you whether the specification it was
+written from was complete. Silence and understanding look alike otherwise.
+
+## Documentation
+
+- The `.ssp` format itself: see
+ [doc-abstraction-format](https://git.sisudoc.org/tools/doc-abstraction-format).
+- Source: [`src/`](src/) - one module per output format
+ (`output_tree.d`, `output_html.d`, `output_llm_breadcrumbs.d`,
+ `output_llm_jsonl.d`, `output_images.d`, `verify_images.d`), plus
+ `ssp_parser.d` and `ssp_types.d`.
+
+## License
+
+AGPL-3.0-or-later. See [`dub.json`](dub.json) and the parent
+[sisudoc-spine](https://git.sisudoc.org/projects/sisudoc-spine)
+project for full licence text.
+
+## Repository
+
+- **Canonical URL:** https://git.sisudoc.org/tools/ssp-reader
+- **Clone:** `git clone --depth=1 git://git.sisudoc.org/tools/ssp-reader`
+
+## Related projects
+
+- [sisudoc-spine](https://git.sisudoc.org/projects/sisudoc-spine) -
+ produces the `.ssp` files this tool consumes.
+- [doc-abstraction-format](https://git.sisudoc.org/tools/doc-abstraction-format) -
+ specification of the `.ssp` text format.
+- [tree-sitter-sisu](https://git.sisudoc.org/tools/tree-sitter-sisu) -
+ tree-sitter grammar for the upstream `.sst` / `.ssm` markup.