| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
ocda: fixed inverted object_number_off on body paragraphs
ssp: make use of curly braces (code-style)
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
--+# clears dummy_heading_multiple_objects, fixed
until fix, following any ---# ... --+# block every ~# object in
the document was marked dummy_heading, (and the output generators
that honour that flag suppressed its text: so ~# behaved as -#.
One line added to correct the --+# branch.
~# suppresses the object's number only, content is still shown.
-# is for a heading introduced solely to segment an output:
object number suppressed and the heading text omitted from the
body where the output format allows, the toc entry kept. Both were
already correct in isolation; only the leaked block flag broke ~#.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
| |
sisu-spine-markup.sst needs review and update, it was taken from
the much older sisu (ruby) writing and was itself a part of a more
comprehensive document (sisu-manual). Some updates.
(assisted by Claude-Code)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
fix to --abstraction/.ssp: fix document structure, representation
parent, ancestors, children. Make parent_ocn, the marked-up and
collapsed ancestor arrays, and the children correctly represent
the document structure. fixed
(assisted by Claude-Code)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01836BxBb8LvVKJ8sBg7Dx6U
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
for document abstraction and its .ssp output, removed the
requirement of including --abstraction & --serial flags to produce
correct output (for: .dom_status, .dom_status_collapsed &
.last_descendant)
- meta_processing_xml_dom() includes show_abstraction, so --pod2
and --show-abstraction run the dom pass; last_descendant is
derived from that pass via after_doc_get_descendants()
The accumulators are now verified as eight wide locals of
docAbstraction(), so each document starts clean and no two threads
share one.
- bug: the four dom accumulators were template scope (shared) and
nine wide, while their end of document reset was eight wide, so
the first document of a run differed from the rest and parallel
runs raced on one buffer, (which also mis-nested epub toc_nav)
test/ reference .ssp regenerated: accelerando only, trailing zero
dropped.
test-abstraction-ssp.sh now runs parallel and diffs output against
a serial run.
(assisted by Claude-Code)
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
- dmd 2.112.1
- ldc 1.43.0 beta1
- dub 1.41.0
- dtools 2.112.1
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
test-abstraction-ssp.sh should fail if $SpinePOD is unset.
SAMPLES_RAW="$SPINE_DIR/$SpinePOD" resolves to the spine directory
itself when the variable is unset. That directory exists, so the
"sample documents not found" check did not fire and spine was
handed every file in the spine tree.
Fail early instead, naming the variable and giving the path the
samples are usually at.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Assumed that headings do not include images, obj_digest() looked
for images only in the non-heading branch, however, the markup
does not forbid one, and an object type left out of the scan
carries neither digest nor .ssp record, silently.
Open Issue: image_list, which drives pod bundling and the @doc_has
images count, is gathered only from para objects (and only the
first image of each), so an image in a heading or in a block body
is recorded in the abstraction but not bundled.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Count them over the assembled body objects, (the body alone).
Previously dochas["inline_links"], ["inline_notes"] and
["inline_notes_star"] were initialised and read but never
incremented, so @doc_has reported 0 inline links and 0 notes for
every document while the block counters (codeblock, poem, group,
block, quote, table) were counted. create_abstraction_db.d records
the same values as document metadata.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
| |
every object with an .image record should now carry the images
flag.
Previously DocObj_Has_.images was read but never written anywhere
in the tree, so no object carried it: .ssp emitted has_images = 0
for every object of every document.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For image digests cache file read & hash, one read per image
file
Fix issue: images referenced from several objects were read and
hashed once per reference.
[as an example in the sisu-manual sm_tux.png accounts for 6 of
the document's image records (12 across the sample set), each
of them a fresh read of the same bytes, and source_pod.d hashes
the file again for digests.txt.]
Memoise on the full path within safeComputeHashAndSize. static is
thread-local in D, so documents processed in parallel each get
their own cache and no synchronisation is involved. Files are not
expected to change during a run.
Output is unchanged: identical .ssp output. (An image that is
missing or unreadable now warns once to stdout rather than once
per reference).
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Complete image file digest identity record: add
px:<width>x<height> to the .ssp image record, read from the
image file header (imageformats.read_image_info, no decode):
.image: won_benkler_2_1.png sha256:5BA4...D128 bytes:93861 px:420x342
(The w##h## dimensions in the markup are display dimensions, not the
file's: they are author set, or where given as w0h0 are derived by
_image_dimensions and capped to a maximum display width, and they
become the width/height attributes of the output format image display).
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
image digest: record missing or unreadable image, do not abort
safeComputeHashAndSize() open image with guard, so a document
naming an image absent from media/image/ throws
ErrnoException out of the abstraction stage.
as with _image_dimensions() (which warns and continues where issue
with image) warn to stdout and record the fact in the abstraction:
.image: ffa.png missing:true
the abstraction would now state that the object embeds an image it
could not read, rather than the .ssp resemble one produced from a
complete pod. The sha256 summary line marks the same case
MISSING OR UNREADABLE.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
| |
a repeatable record, one line per image, filename first:
.image: won_benkler_2_1.png sha256:5BA492C2...D128 bytes:93861
an object embedding more than one image emits more than one .image
line.
(assisted by Claude-Code)
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
corrects an omission this hould have been included to start with
reference tests texts regenerated
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Three small follow-ups to the ocda/outputs split:
1. Add src/sisudoc/ocda/package.d (module sisudoc.ocda) as a 2-line
public re-export of sisudoc.ocda.abstraction. Provides downstream
consumers with a canonical "import sisudoc.ocda;" entry point and
a stable handle for eventual peer-repo packaging of the
abstraction library.
2. Fix the D import-path root in dub.json so it matches the declared
module names:
- spine:abstraction sub-package
"importPaths": [ "./src/sisudoc" ] -> [ "./src" ]
- main package buildTypes (dmd, ldc2, ldmd2, gdc, gdmd)
"-I=src/sisudoc" -> "-I=src"
The modules are named sisudoc.ocda.* / sisudoc.outputs.* /
sisudoc.* so the filesystem-based resolver needs to see
./src as the root (so <root>/sisudoc/ocda/X.d resolves).
3. Replace dyaml sub-package's destructive preGenerateCommands
("rm -rf ./src/ext_depends/D-YAML/{examples,testsuite}") with
declarative excludedSourceFiles globs. The two directories do
not exist in the vendored D-YAML tree, so the rm was a no-op
in practice; the glob form is defensive (would silently skip
them if they were ever re-introduced) and removes the
destructive side-effect from every build.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
| |
Modules and imports rewritten to sisudoc.ocda.* and
sisudoc.outputs.*; dub.json excludedSourceFiles and the
spine:abstraction sub-package sourcePaths collapsed to
./src/sisudoc/ocda.
Verified: nix build .#spine-overlay-ldc clean.
(assisted by Claude-Code)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Declare sisudoc.abstraction as a dub sub-package (spine:abstraction)
within the existing dub.json. The sub-package's sourcePaths cover
exactly the abstraction-side clusters that were freed from io_out
edges in phase 0 and phase 1:
- src/sisudoc/abstraction (package surface + ssp serialiser)
- src/sisudoc/meta (per-document abstraction)
- src/sisudoc/io_in (file/pod ingress)
Its declared dependencies are spine:dyaml and spine:imageformats
only - no d2sqlite3, no io_out. The main "spine" package adds
spine:abstraction to its dependencies and lists those three
directories in excludedSourceFiles so dub does not compile the
abstraction-side sources twice when the executable is built with
--combined (as the nix derivation does).
(assisted by Claude-Code)
|
| | |
|
| |
|
|
| |
(also cgi_sqlite_search_form.d did not belong here)
|
| |
|
|
|
|
|
|
|
|
|
|
| |
css: align body-flow <ul>/<li> & <details>/<summary> with <p>
Not used by sisudoc-spine but for hand-authored body-flow markup such as
the current homepage / body-flow, added block to each of the four html
CSS string heredocs in src/sisudoc/io_out/xmls_css.d
Existing tags are left in place and untouched.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
phase1 step2: move SSP serialiser into sisudoc.abstraction package
git mv src/sisudoc/io_out/create_abstraction_txt.d to
src/sisudoc/abstraction/ssp.d
Module rename: sisudoc.io_out.create_abstraction_txt
-> sisudoc.abstraction.ssp
Completes phase1: after this commit the sisudoc.abstraction package has
zero outgoing edges into sisudoc.io_out. The library produces both the
in-memory document object model AND the .ssp text serialisation without
referencing any output-side module.
The serialiser previously imported sisudoc.io_out.paths_output for the
single purpose of constructing the .ssp output path. That import is
dropped; the path construction is inlined as three lines of std.path
(chainPath / asNormalizedPath / array) producing
<output_path>/<language>/abstraction/<doc_uid_out>.ssp
- byte-for-byte the same path the previous spineOutPaths!() call
produced.
Updated:
- src/sisudoc/abstraction/ssp.d - module decl + inline path
- src/sisudoc/abstraction/package.d - public import .ssp
- src/sisudoc/spine.d - import sisudoc.abstraction.ssp (x2)
Completes decouple abstraction phase1
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
phase1 step1: introduce sisudoc.abstraction package re-export surface
Create src/sisudoc/abstraction/package.d as a library-facing re-export
module for the document-abstraction stage.
The surface currently re-exports:
- sisudoc.meta.metadoc (spineAbstraction, A-layer entry)
- sisudoc.meta.metadoc_from_src (docAbstraction, B-layer entry)
No code moves; no behaviour change. The package exists so external
consumers can `import sisudoc.abstraction;` and reach the entry points
without depending on spine's internal directory layout.
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
phase0 step2: move curation modules from meta/ to io_out/curate/
Curation modules moved to src/sisudoc/io_out/curate/, module
declarations renamed sisudoc.io_out.curate.metadoc_curate* from
sisudoc.meta.metadoc_curate* and updated spine.d imports. File contents
are otherwise unchanged.
Completes phase0: meta/ now has zero io_out imports - the abstraction
core's outgoing deps are now only:
meta/ internals + io_in/ + ext_depends/D-YAML
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
phase0: drop vestigial io_out.hub coupling from meta/metadoc.d
phase0 step1: abstraction-library extraction/decoupling: meta/ should
not import io_out/.
Removed unused call to `import sisudoc.io_out.hub;` `mixin outputHub;`
from `template spineAbstraction()`. (the load-bearing UFCS site is
spine.d:92 which has its own `mixin outputHub).
(assisted by Claude-Code)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- include all (doc abstraction) .ssp in pod zip and in digests
- fixed: for multi-language pods built with --pod2, only the last
language's .ssp file was being written into pod.zip and listed in
.digests.txt each languages' .ssp files were on disk in the pod
directory (copied during their own per-language passes) but were not
in final zip as it was being built once for each language and
writing over previous, (only the last one remaining). The solution
is to follow the pattern already used to avoid this by .sstm and
.ssi, namely wait for the last language and iterate the
manifest_list_of_languages internaly.
(assisted by Claude-Code)
|
| |
|
|
| |
(assisted by Claude-Code)
|
| |
|
|
|
|
| |
- fatal error on missing/unwritable --sqlite-db-path
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- emacs syntax: add tree-sitter major mode for SiSU spine markup
New file sisu-spine-ts-mode.el is a sibling of the existing regex
mode, backed by Emacs 29+'s built-in treesit.el and the
tree-sitter-sisu grammar. It replaces the long font-lock keyword list
with treesit-font-lock-rules grouped into eight features (comment,
header, heading, block, inline, note, link, index, misc) so users
can dial verbosity via treesit-font-lock-level. It also wires up:
- treesit-simple-imenu-settings for a heading outline,
- treesit-thing-settings for sentence / paragraph motions,
- treesit-defun-type-regexp so C-M-a / C-M-e jump heading-to-heading,
- a sisu-spine-ts-install-grammar command that registers
treesit-language-source-alist and runs
treesit-install-language-grammar so users can install the parser
from inside Emacs without leaving the editor.
- the original sisu-spine-mode.el is unchanged and supported for
Emacs < 29 and for users who prefer regex highlighting.
- nvim drop-in: point parser fetch at tools/tree-sitter-sisu
sundry/editor-syntax-etc/nvim/
The nvim-treesitter install_info now fetches the parser from
https://git.sisudoc.org/projects/tree-sitter-sisu which can be cloned
via git://git.sisudoc.org/tools/tree-sitter-sisu The fetched paths:
files = { "src/parser.c", "src/scanner.c" }
submission of the sisu parser to nvim-treesitter's parsers.lua should
be a near-trivial one-liner.
- the original vim regex highlighter remains as before
- sundry/editor-syntax-etc/vim/syntax/sisu-spine.vim
- sundry/editor-syntax-etc/vim/templates/{sst,ssm,ssi}.tpl
new skeleton templates for the three SiSU markup file types
sets up the YAML header (title, creator, date, rights, classify,
identfier)
- .gitignore - whitelist the new files
(assisted by Claude-Code)
|
| |
|
|
|
|
|
|
|
| |
- test/test-abstraction-ssp.sh script minor improvement
- result change reflects content corrections: name spelling & a markup
error, document abstracton, ssp verified ok (may be considered a test)
(assisted by Claude-Code)
|
| | |
|
| |
|
|
|
| |
- odd hilighting issue ... must result from my org config, but "fix"
makes things easier for me.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
test/test-abstraction-ssp.sh: shell script that generates
.ssp files for all sample documents and diffs against
committed reference files in test/reference/abstraction/.
Usage:
./test/test-abstraction-ssp.sh --generate ./bin/spine-ldc
./test/test-abstraction-ssp.sh ./bin/spine-ldc
Exits 0 if all .ssp files match, 1 if differences found.
Reports changed, missing, and new files.
test/reference/abstraction/: 35 reference .ssp files
generated from the current abstraction, covering all
sample documents including 9-language live-manual.
Co-Authored-By: Anthropic Claude Opus 4.6 (1M context)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Finer-grained control over when .ssp files are produced:
--show-abstraction writes .ssp to OUTPUT/lang/abstraction/
independently of any pod flag
--pod builds pod without .ssp bundled
--pod2 builds pod with .ssp in media/abstraction/
Changes to spine.d:
- show_abstraction() now only responds to its own flag and
pod2, no longer triggered by source_or_pod
- Add pod2 to opts init, getopt, OptActions
- pod() returns true for both --pod and --pod2
- source_or_pod() includes pod2
Changes to source_pod.d:
- Remove per-document pod directory (rmdirRecurse) before
regeneration, ensuring clean slate on every run. This
prevents stale content from previous runs (e.g. a --pod2
run followed by --pod would otherwise leave an outdated
media/abstraction/ directory)
- Gate abstraction directory creation and .ssp bundling on
pod2 flag specifically
Tested: --pod (no .ssp), --pod2 (.ssp in pod + zip),
--show-abstraction (standalone .ssp), --pod after --pod2
(stale abstraction cleaned up). All 35 sample documents pass.
Co-Authored-By: Anthropic Claude Opus 4.6 (1M context)
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Add empty-string guards to array property loops
(.stow_link, .lev4_subtoc, .anchor_tag) so entries with
zero-length values are not emitted. Empty properties have
no value for PEG parsing - absent lines are faster to skip
than matching a property name to find an empty value.
Removes 1488 empty .anchor_tag: lines from Wealth of
Networks .ssp alone.
Co-Authored-By: Anthropic Claude Opus 4.6 (1M context)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Add explicit child heading OCN lists to heading objects,
pre-computed in a single O(n) pass over the body section
before serialization. This makes the document tree directly
navigable without scanning - each heading lists its direct
sub-heading OCNs.
- Example output for a chapter heading:
[10] heading :1
.last_descendant: 65
.children: 14 24 42 57
- Implementation: builds an int[][int] map (parent_ocn ->
child heading OCNs) from one pass over the body objects,
then emits .children: during serialization for headings
that have entries in the map.
- The tree was already reconstructable from parent_ocn +
last_descendant_ocn, but .children makes it immediate -
no scanning required to find a heading's sub-structure.
- Tested against all 35 sample documents - zero failures.
Co-Authored-By: Anthropic Claude Opus 4.6 (1M context)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Make the .ssp format a complete representation of the
document abstraction by serializing all remaining fields
from ObjGenericComposite (only omitting ptr.* runtime
indices which are meaningless outside the in-memory context).
- New fields added:
.ancestors_collapsed: - collapsed level ancestor chain
.dom_status: - DOM structure markedup tags status[8]
.dom_status_collapsed: - DOM structure collapsed status[8]
.heading_lev_collapsed: - collapsed heading level
.parent_lev: - parent heading level (markup)
.o_n_type: - object numbering type (0=ocn, 1=non, 2=bkidx)
.is_of_type: - para/block type classification
.attrib: - general attributes string
.meta_lang: - block language (group/block/quote)
.meta_syntax: - codeblock syntax from metainfo
.sha256: - hex-encoded SHA-256 digest of object content
.has: images_no_dim - image without dimensions flag
.table_aligns: - column alignment array
.table_walls: - table walls/borders flag
.stow_link: - extracted URLs (one per line)
.heading_lev_anchor: - heading level anchor tag
.segment_epub: - EPUB segment anchor tag
.heading_ancestors_text: - pipe-separated ancestor headings
.lev4_subtoc: - sub-table-of-contents entries (one per line)
.anchor_tag: - additional anchor tags (one per line)
- Tested against all 35 sample documents - zero failures.
Co-Authored-By: Anthropic Claude Opus 4.6 (1M context)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- For heading objects, the identifier was always emitted on the
declaration line (e.g. "[10] heading :1 10") even when it was
just the OCN repeated. Now only emits the identifier when it
differs from the OCN (i.e. when there is a named segment like
"acknowledgments" or "a1"), reducing redundancy.
Before: [10] heading :1 10
After: [10] heading :1
Named segments still appear: [0] heading :1 a1
Co-Authored-By: Anthropic Claude Opus 4.6 (1M context)
|
