# ssp-reader, in D Reader for the **`.ssp` document-abstraction text format** emitted by [sisudoc-spine](https://git.sisudoc.org/sisudoc-spine). Reads a `.ssp` file (spine's PEG-parsable serialisation of its internal document abstraction) and renders it into a chosen output: an outline tree, scroll HTML, plain text with heading breadcrumbs, JSONL chunks suitable for LLM / embedding pipelines, or an image manifest for verifying the images a document embeds. ## What this is for - Inspecting and debugging the abstraction stage of spine without rebuilding spine itself. - Round-trip experiments: markup -> spine -> `.ssp` -> ssp-reader -> downstream artefact. - Re-using spine's parsed document structure outside the spine binary: feeding it to an indexer, an LLM pipeline, or a custom renderer. - A worked example of consuming `.ssp` for anyone implementing another reader in another language. ## Audience Developers building tools that consume spine's output: search indexers, LLM ingestion pipelines, custom renderers, alternative output formats, and anyone studying the abstraction stage of spine itself. End-user document authors do **not** need this tool to publish a document. Run spine directly for that. ## Relationship to spine Spine's processing pipeline has three stages: 1. **Input:** parse `.sst` / `.ssm` markup. 2. **Abstraction:** build a numbered object model (the "spine abstraction"; one of spine's named outputs is `.ssp`, a PEG-parsable text representation of this model). 3. **Output:** render to HTML / EPUB / PDF / SQLite / etc. `ssp-reader` picks up the abstraction at the end of stage 2 (the `.ssp` file) and runs alternative stage-3 renderings on top of it. The `.ssp` format itself is documented in [`sisudoc-abstraction-format`](https://git.sisudoc.org/sisudoc-abstraction-format). ## Quickstart ### Build ```sh nix build # the reader, with ldc ./result/bin/ssp-reader ``` The reader depends on nothing beyond phobos, so `nix build` needs no network and works in a sandbox. `nix build .#ssp-reader-dmd` builds it with dmd instead, which is given gcc14 for the same reason spine gives it: GCC 15 introduced `nullptr` in its headers and dmd's ImportC parser does not yet handle it. For development, or to build without nix: ```sh nix develop # ldc, dub, dtools dub build --compiler=ldmd2 ``` The binary then lands at `./bin/ssp-reader`. ### Run ```sh # generate a .ssp from a sample document via spine, then: ssp-reader --tree doc.ssp # outline view ssp-reader --html doc.ssp # HTML scroll output ssp-reader --llm-text doc.ssp # plain text with breadcrumbs ssp-reader --llm-jsonl doc.ssp # JSONL chunks for embeddings ssp-reader --images doc.ssp # image manifest (digest, size, pixels) ssp-reader --verify --image-dir=pod/media/image doc.ssp # check images against the records ssp-reader --report-unknown doc.ssp # what it read and did not understand ssp-reader --tree --output=out.txt doc.ssp ssp-reader --help ``` ## Format version The reader knows abstraction format **2.x**. Every `.ssp` declares its version on its first line, and the reader checks the **major** part and refuses what it does not know: an unknown major means a property may have changed meaning or gone away, and loading such a file half populated in silence is the failure the check exists to prevent. `--any-version` reads it regardless, for looking at a file rather than trusting one. ## What it does not understand Within a major version a reader **should** ignore properties it does not know, since the minor part is raised precisely by adding them. This reader does, and `--report-unknown` says what it ignored and how often: ``` read and NOT understood: properties ignored (15 distinct): .is_of_type 328 .segment_html_is 138 ... ``` ### What it ignores, and why Twelve properties are ignored across the reference collection, and each is ignored for a reason rather than by oversight: | ignored | why | |--------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------| | `.ancestors_collapsed`, `.parent_lev`, `.heading_lev_collapsed`, `.dom_status`, `.dom_status_collapsed` | level bookkeeping; this reader works from `.ancestors` and the heading level | | `.segment_epub`, `.segment_epub_is`, `.segment_lv4_is`, `.heading_lev_anchor`, `.anchor_tag`, `.lev4_subtoc` | naming for segmented HTML and EPUB output, which this reader does not produce | | `.attrib` | one occurrence in the whole collection, and no output uses it | The list was longer. `--report-unknown` found that `.is_of_type` was ignored, and **that was a defect**: objects of type `comment` are markup commentary, carried by the abstraction and not put into output, and three of the four outputs were rendering them. A markup comment was appearing in the HTML, in the plain text, and as an embedding chunk in the JSONL. Fixed, and the reason the report exists. That is deliberate and is what makes the reader usable as a check on the [format specification](https://git.sisudoc.org/sisudoc-abstraction-format): a reader that drops what it does not know is conforming, but one that cannot say what it dropped cannot tell you whether the specification it was written from was complete. Silence and understanding look alike otherwise. ## Documentation - The `.ssp` format itself: see [sisudoc-abstraction-format](https://git.sisudoc.org/sisudoc-abstraction-format). - Source: [`src/`](src/) - one module per output format (`output_tree.d`, `output_html.d`, `output_llm_breadcrumbs.d`, `output_llm_jsonl.d`, `output_images.d`, `verify_images.d`), plus `ssp_parser.d` and `ssp_types.d`. ## License AGPL-3.0-or-later. See [`dub.json`](dub.json) and the parent [sisudoc-spine](https://git.sisudoc.org/sisudoc-spine) project for full licence text. ## Repository - **Canonical URL:** https://git.sisudoc.org/sisudoc-ssp-reader - **Clone:** `git clone --depth=1 git://git.sisudoc.org/tools/sisudoc-ssp-reader` ## Related projects - [sisudoc-spine](https://git.sisudoc.org/sisudoc-spine) - produces the `.ssp` files this tool consumes. - [sisudoc-abstraction-format](https://git.sisudoc.org/sisudoc-abstraction-format) - specification of the `.ssp` text format. - [tree-sitter-sisu](https://git.sisudoc.org/tree-sitter-sisu) - tree-sitter grammar for the upstream `.sst` / `.ssm` markup.