aboutsummaryrefslogtreecommitdiffhomepage
path: root/test
diff options
context:
space:
mode:
authorRalph Amissah <ralph.amissah@gmail.com>2026-09-21 16:12:56 -0400
committerRalph Amissah <ralph.amissah@gmail.com>2026-09-23 08:22:15 -0400
commit8b81e9a11c7af642c4d0d8584a9cb348cc887d6e (patch)
tree8ba9c356ff71bf18f5623c22d30a8b2513321cfe /test
parentuid and paths separator "~" in place of ":" (diff)
ocda: warn if heading claims reserved segment name
spine automatically builds some segments, including: (toc, endnotes, glossary, bibliography, bookindex, blurb, _the_title) these are now identified as reserved names and a user is now warned if any of these names have been manually assigned to a heading by markup. A document still builds but to disambiguate the ocn of the heading is attached to the markup (reserved) name and this is seeded before a document not after. It is read off the finished abstraction rather than reported by the parser, and beside the ocn alignment check for the same reason: it is a statement about a document rather than a step in building one. WARNING reserved segment name: the_autonomous_contract... [en] heading 1 at ocn 135 asks for "endnotes", which spine gives its own generated section it is named "endnotes-135" instead; ... A warning: the document is correct and complete and the name it ends up with works. --strict makes it a failure for the run, as it does for ocn alignment, and by the same reasoning: the outputs are written and can be looked at, and the exit status is taken at the end. Two of the thirty-six sample documents have reserved segment names, "1~endnotes" heading. Output is unchanged: nothing here touches the abstraction. (assisted by Claude-Code)
Diffstat (limited to 'test')
-rwxr-xr-xtest/run-tests.sh29
-rwxr-xr-xtest/test-run-order.sh142
2 files changed, 160 insertions, 11 deletions
diff --git a/test/run-tests.sh b/test/run-tests.sh
index d8822d4..f970194 100755
--- a/test/run-tests.sh
+++ b/test/run-tests.sh
@@ -23,12 +23,13 @@
# Order, and what depends on what:
#
# 1 test-abstraction-ssp.sh the abstraction has not changed
-# 2 test-abstraction-ssp-roundtrip.sh the .ssp can be read back whole
-# 3 test-abstraction-db.sh the two serialisations agree
-# 4 test-abstraction-db-roundtrip.sh the .db can be read back whole
-# 5 test-ocn-alignment.sh the languages agree on their ocns
-# 6 test-acceptance-db-to-pod.sh a database gives its document back
-# 7 test-epub-validity.sh the epubs a reader can actually open
+# 2 test-run-order.sh nor does it depend on the run
+# 3 test-abstraction-ssp-roundtrip.sh the .ssp can be read back whole
+# 4 test-abstraction-db.sh the two serialisations agree
+# 5 test-abstraction-db-roundtrip.sh the .db can be read back whole
+# 6 test-ocn-alignment.sh the languages agree on their ocns
+# 7 test-acceptance-db-to-pod.sh a database gives its document back
+# 8 test-epub-validity.sh the epubs a reader can actually open
#
# None of them calls another, and each can be run on its own. The order
# matters only for reading the result:
@@ -36,21 +37,26 @@
# - 1 is first because it is the one that says whether the abstraction
# itself moved. If it fails, the others are answering a different
# question than you think, and its failure is the one to look at.
-# - 2 reads the committed reference set, so it is only a statement about
+# - 2 says that a document abstracted alone is the document abstracted in
+# company. 1 only ever runs the whole sample set, so an abstraction
+# that depended on what else was in the run passed it; this is the
+# check that would have caught that, and the one that says an artefact
+# can be verified on its own.
+# - 3 reads the committed reference set, so it is only a statement about
# the current binary if 1 passes. Run together, they say: the abstraction
# is unchanged, and this binary can read that abstraction back.
-# - 3 and 4 generate both artefacts themselves and depend on nothing
+# - 4 and 5 generate both artefacts themselves and depend on nothing
# committed. Since the database is built from the .ssp, they are checks
# on the reader rather than on two independent writers.
-# - 5 is about the sample documents rather than about a serialisation: it
+# - 6 is about the sample documents rather than about a serialisation: it
# asserts that spine reports the two known divergences in live-manual's
# translations and reports nothing about the other documents. It will
# fail if those translations are mended, which is the right moment to
# hear about it.
-# - 6 is the acceptance test for the document source claim, and the widest
+# - 7 is the acceptance test for the document source claim, and the widest
# of them: it builds a document from its pod and again from the database
# that build produced, and requires the two output trees to be identical.
-# Where 3 and 4 check one serialisation against another, this checks a
+# Where 4 and 5 check one serialisation against another, this checks a
# whole document through the whole pipeline twice. If it fails and the
# others pass, what broke is the carrying or the materialising rather
# than the abstraction.
@@ -91,6 +97,7 @@ if [ -n "$_missing" ]; then
fi
TESTS="test-abstraction-ssp.sh
+test-run-order.sh
test-abstraction-ssp-roundtrip.sh
test-abstraction-db.sh
test-abstraction-db-roundtrip.sh
diff --git a/test/test-run-order.sh b/test/test-run-order.sh
new file mode 100755
index 0000000..29c1b7b
--- /dev/null
+++ b/test/test-run-order.sh
@@ -0,0 +1,142 @@
+#!/usr/bin/env sh
+# test-run-order.sh
+#
+# A document's abstraction must not depend on what else is in the run.
+#
+# It did. The set of heading anchor names spine has seen is seeded with the
+# names it keeps for its own generated sections (toc, endnotes, glossary,
+# bibliography, bookindex, blurb, _the_title), and the seeding was written
+# only in scope(exit) and at the tail of docAbstraction(): both run *after* a
+# document, so they seeded the next one. The first document of a run began
+# with the set empty and was the one document not held to the reserved names,
+# and a heading of its own claiming "endnotes" kept the name of the generated
+# section rather than being renamed out of its way.
+#
+# So the same document abstracted alone and abstracted after another differed,
+# and an artefact written in a collection run could not be verified on its
+# own: --ocda-verify re-parses one document, where it is necessarily first,
+# and reported a correct artefact as one whose markup no longer makes it.
+#
+# The check: abstract two documents in one run, abstract each of them alone,
+# and require the .ssp to be identical either way. The second document is the
+# one that matters (it is seeded in the pair and not when alone), so the pair
+# is ordered with the affected document last.
+#
+# $SpineRunOrderDocs overrides the pair, space separated, unqualified
+# directory names within $SpinePOD. The default pair is chosen so that the
+# test would fail before the fix: the_autonomous_contract has a "1~endnotes"
+# heading, which is what the seeding renames.
+#
+# Usage:
+# SpinePOD=../../markup/sisudoc-spine-samples/markup/pod-samples/pod \
+# ./test/test-run-order.sh ./bin/spine-ldc
+#
+# Exit codes:
+# 0 a document abstracts the same alone as in company
+# 1 it does not (the differing files are printed)
+# 2 set-up problem (no $SpinePOD, no binary, no samples)
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+SPINE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
+
+if [ -z "$SpinePOD" ]; then
+ echo "ERROR: \$SpinePOD is not set." >&2
+ echo " e.g. SpinePOD=../../markup/sisudoc-spine-samples/markup/pod-samples/pod" >&2
+ exit 2
+fi
+SAMPLES_RAW="$SPINE_DIR/$SpinePOD"
+if [ -d "$SAMPLES_RAW" ]; then
+ SAMPLES_DIR="$(cd "$SAMPLES_RAW" && pwd)"
+else
+ SAMPLES_DIR="$SAMPLES_RAW"
+fi
+if [ ! -d "$SAMPLES_DIR" ]; then
+ echo "ERROR: sample documents not found at $SAMPLES_DIR" >&2
+ exit 2
+fi
+
+SPINE_BIN="${1:-}"
+if [ -z "$SPINE_BIN" ]; then
+ if [ -x "$SPINE_DIR/result/bin/spine" ]; then SPINE_BIN="$SPINE_DIR/result/bin/spine"
+ elif [ -x "$SPINE_DIR/bin/spine-ldc" ]; then SPINE_BIN="$SPINE_DIR/bin/spine-ldc"
+ elif [ -x "$SPINE_DIR/bin/spine" ]; then SPINE_BIN="$SPINE_DIR/bin/spine"
+ else
+ echo "ERROR: spine binary not found. Specify path as argument." >&2
+ exit 2
+ fi
+fi
+
+DOCS="${SpineRunOrderDocs:-gpl2.fsf the_autonomous_contract.ralph_amissah}"
+
+for d in $DOCS; do
+ if [ ! -d "$SAMPLES_DIR/$d" ]; then
+ echo "ERROR: sample document not found: $SAMPLES_DIR/$d" >&2
+ echo " set \$SpineRunOrderDocs to a pair that is present" >&2
+ exit 2
+ fi
+done
+
+OUT_DIR="$SCRIPT_DIR/current-run-order"
+rm -rf "$OUT_DIR"
+mkdir -p "$OUT_DIR"
+
+echo "spine binary: $SPINE_BIN"
+echo "samples: $SAMPLES_DIR"
+echo "documents: $DOCS"
+
+FAILURES=0
+fail() { echo " FAIL: $*"; FAILURES=$((FAILURES + 1)); }
+
+# the pair, in one run
+_pair_args=""
+for d in $DOCS; do
+ _pair_args="$_pair_args $SAMPLES_DIR/$d"
+done
+# shellcheck disable=SC2086
+$SPINE_BIN --show-abstraction --skip-output \
+ --output="$OUT_DIR/together" $_pair_args > "$OUT_DIR/together.out" 2>&1
+
+# and each of them on its own
+for d in $DOCS; do
+ $SPINE_BIN --show-abstraction --skip-output \
+ --output="$OUT_DIR/alone-$d" "$SAMPLES_DIR/$d" \
+ > "$OUT_DIR/alone-$d.out" 2>&1
+done
+
+# every .ssp the pair run produced has to have a twin, identical, in the run
+# that produced it alone
+_compared=0
+for f in $(find "$OUT_DIR/together" -name "*.ssp" | sort); do
+ _rel="${f#"$OUT_DIR/together/"}"
+ _alone=""
+ for d in $DOCS; do
+ if [ -f "$OUT_DIR/alone-$d/$_rel" ]; then _alone="$OUT_DIR/alone-$d/$_rel"; fi
+ done
+ if [ -z "$_alone" ]; then
+ fail "$_rel was produced in the pair run and by neither document alone"
+ continue
+ fi
+ _compared=$((_compared + 1))
+ if ! cmp -s "$f" "$_alone"; then
+ fail "$_rel differs between the pair run and the run of its own document"
+ diff --unified=2 "$_alone" "$f" | head -20 | sed 's/^/ /'
+ fi
+done
+
+if [ "$_compared" -eq 0 ]; then
+ echo "ERROR: no .ssp files were produced to compare" >&2
+ rm -rf "$OUT_DIR"
+ exit 2
+fi
+
+rm -rf "$OUT_DIR"
+
+if [ "$FAILURES" -eq 0 ]; then
+ echo "PASS: $_compared .ssp identical whether the document ran alone or in company"
+ exit 0
+else
+ echo "FAIL: $FAILURES check(s)"
+ exit 1
+fi