summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
authorRalph Amissah <ralph.amissah@gmail.com>2026-10-01 17:23:00 -0400
committerRalph Amissah <ralph.amissah@gmail.com>2026-10-01 17:26:10 -0400
commit987828d4fd07cb390d6c61e91cd29bec79d444eb (patch)
tree67b008c99d20a43e59798c4923f4659c8aa6d57c
parentREADME: the reader, in each language, and what writing it found (diff)
gleam sisudoc-ocda-libclaude-org
-rw-r--r--README.md38
-rw-r--r--dlang/flake.nix10
-rw-r--r--gleam/README.md30
-rw-r--r--gleam/flake.nix11
-rw-r--r--gleam/src/ocda/dump.gleam49
-rw-r--r--gleam/src/ocda/images.gleam49
-rw-r--r--gleam/src/ocda/inventory.gleam49
-rw-r--r--gleam/src/ocda/load.gleam49
-rw-r--r--gleam/src/ocda/load_db.gleam49
-rw-r--r--gleam/src/ocda/load_pod.gleam49
-rw-r--r--gleam/src/ocda/load_ssp.gleam49
-rw-r--r--gleam/src/ocda/model.gleam49
-rw-r--r--gleam/src/ocda/zip.gleam49
-rw-r--r--gleam/src/ocda_ffi.erl49
-rw-r--r--gleam/src/ocda_lib.gleam49
-rw-r--r--gleam/src/read/main.gleam49
-rw-r--r--gleam/test/ocda_lib_test.gleam53
-rw-r--r--org/config_git.org101
-rw-r--r--org/config_nix.org287
-rw-r--r--org/gleam-ocda-load.org1484
-rw-r--r--org/gleam-ocda-misc.org649
-rw-r--r--org/gleam-ocda-model_and_inventory.org494
-rw-r--r--org/gleam-ocda_lib.org381
-rw-r--r--org/gleam-read-main.org520
-rw-r--r--org/gleam-test.org209
-rw-r--r--org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org181
26 files changed, 4984 insertions, 52 deletions
diff --git a/README.md b/README.md
index ecda5fd..53494cc 100644
--- a/README.md
+++ b/README.md
@@ -5,11 +5,11 @@ loaders for both of the forms it is written in.
**One reader per language, in a directory named for the language.**
-| | |
-|---|---|
-| [`dlang/`](dlang/) | D |
-| [`gleam/`](gleam/) | Gleam |
-| `ruby/` | to come |
+| | |
+|--------------------|---------|
+| [`dlang/`](dlang/) | D |
+| [`gleam/`](gleam/) | Gleam |
+| `ruby/` | to come |
## What it is for
@@ -73,12 +73,12 @@ out in a fully determined form, and the D and the Gleam library write it
*byte for byte alike*. So four questions can be asked, all of them with
`diff`:
-| | |
-|---|---|
-| D: `.ssp` == `.ocda.db` | **36 of 36** document-languages |
-| Gleam: `.ssp` == `.ocda.db` | **36 of 36** |
-| `.ssp`: D == Gleam | **36 of 36** |
-| `.ocda.db`: D == Gleam | **36 of 36** |
+| | |
+|-----------------------------|---------------------------------|
+| D: `.ssp` == `.ocda.db` | **36 of 36** document-languages |
+| Gleam: `.ssp` == `.ocda.db` | **36 of 36** |
+| `.ssp`: D == Gleam | **36 of 36** |
+| `.ocda.db`: D == Gleam | **36 of 36** |
None of those held at first.
@@ -130,14 +130,14 @@ serialisations, and naming it is the useful part.
## What it reads, and what it does not
-| | | |
-|---|---|---|
-| `.ssp` | yes | |
-| `.ocda.db` | yes, any language of it | |
-| `.sisupod` | yes, any language it carries | D only, see below |
-| `.zip` pod | yes, any language it carries | |
-| `.sst`, `.ssm` markup | **no, and not planned**: that is a parser, which is spine's work | |
-| a url | no | |
+| | | |
+|-----------------------|------------------------------------------------------------------|-------------------|
+| `.ssp` | yes | |
+| `.ocda.db` | yes, any language of it | |
+| `.sisupod` | yes, any language it carries | D only, see below |
+| `.zip` pod | yes, any language it carries | |
+| `.sst`, `.ssm` markup | **no, and not planned**: that is a parser, which is spine's work | |
+| a url | no | |
A pod carries the abstraction only when spine was asked for it: `--pod2`
puts it in, `--source` does not. A pod without one is refused with a message
diff --git a/dlang/flake.nix b/dlang/flake.nix
index 09dce2e..1596888 100644
--- a/dlang/flake.nix
+++ b/dlang/flake.nix
@@ -1,13 +1,10 @@
{
description = "ocda-lib: the object-centric document abstraction as a D library";
-
- inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
-
+ inputs.nixpkgs.url = "https://channels.nixos.org/nixpkgs-unstable/nixexprs.tar.zst";
outputs = { self, nixpkgs }:
let
system = "x86_64-linux"; # adjust for aarch64-linux / darwin
pkgs = import nixpkgs { inherit system; };
-
/* A library, so the library itself is not a build product: a program
that uses it compiles its modules alongside its own, as the writer's
flake does, or takes it as a dub dependency. What is built here are
@@ -16,7 +13,7 @@
and its .ocda.db and compares the two models, and the dump, which
writes a loaded model out canonically so that an implementation in
another language can be diffed against this one.
-
+ .
`mains` is what is compiled besides the library: the checks are one
file each in test/, the reader is src/read/.
*/
@@ -44,9 +41,7 @@
mainProgram = binary;
};
};
-
ldc = { compiler = pkgs.ldc; compilerBin = "ldmd2"; };
-
ocda-read = mkTool (ldc // {
tool = "read"; binary = "ocda-read"; mains = "src/read/*.d";
what = "reads the abstraction in any of its forms and says what is in it";
@@ -72,7 +67,6 @@
};
inherit ocda-read ocda-agree ocda-dump;
};
-
devShells.${system}.default = pkgs.mkShell {
packages = [ pkgs.ldc pkgs.dub pkgs.dtools pkgs.sqlite pkgs.zstd ];
};
diff --git a/gleam/README.md b/gleam/README.md
index ed4a30b..3e6dc53 100644
--- a/gleam/README.md
+++ b/gleam/README.md
@@ -3,16 +3,16 @@
The abstraction's model and both its loaders, as a Gleam library, with its
reader and the two programs that put it to the question.
-| | |
-|---|---|
-| `ocda/model` | the document, the object, the image. Depends on nothing but the standard library |
-| `ocda/load_ssp` | text -> model |
-| `ocda/load_db` | sqlite -> model |
-| `ocda/load` | what a path is, and loading it |
+| | |
+|------------------|-----------------------------------------------------------------------------------------|
+| `ocda/model` | the document, the object, the image. Depends on nothing but the standard library |
+| `ocda/load_ssp` | text -> model |
+| `ocda/load_db` | sqlite -> model |
+| `ocda/load` | what a path is, and loading it |
| `ocda/inventory` | what a file holds beyond the one document: its languages, and what a `.ocda.db` carries |
-| `ocda/dump` | the model written out canonically, as an oracle |
-| `read/main` | `ocda-read`, the reader |
-| `ocda_ffi.erl` | a sha256, from OTP's crypto, which the standard library lacks |
+| `ocda/dump` | the model written out canonically, as an oracle |
+| `read/main` | `ocda-read`, the reader |
+| `ocda_ffi.erl` | a sha256, from OTP's crypto, which the standard library lacks |
## Use
@@ -60,12 +60,12 @@ the whole model out canonically, and is the stronger check of the two,
because it is **byte for byte what the D library's `ocda-dump` writes**.
So the four questions can all be asked with `diff`:
-| | |
-|---|---|
-| D: `.ssp` == `.ocda.db` | 36 of 36 document-languages |
-| Gleam: `.ssp` == `.ocda.db` | 36 of 36 |
-| `.ssp`: D == Gleam | 36 of 36 |
-| `.ocda.db`: D == Gleam | 36 of 36 |
+| | |
+|-----------------------------|-----------------------------|
+| D: `.ssp` == `.ocda.db` | 36 of 36 document-languages |
+| Gleam: `.ssp` == `.ocda.db` | 36 of 36 |
+| `.ssp`: D == Gleam | 36 of 36 |
+| `.ocda.db`: D == Gleam | 36 of 36 |
## Deliberately a mirror of the D library
diff --git a/gleam/flake.nix b/gleam/flake.nix
index eb77f86..5ffb74c 100644
--- a/gleam/flake.nix
+++ b/gleam/flake.nix
@@ -1,8 +1,6 @@
{
description = "ocda-lib: the object-centric document abstraction as a Gleam library";
-
- inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
-
+ inputs.nixpkgs.url = "https://channels.nixos.org/nixpkgs-unstable/nixexprs.tar.zst";
outputs = { self, nixpkgs }:
let
system = "x86_64-linux"; # adjust for aarch64-linux / darwin
@@ -11,7 +9,6 @@
name = "ocda_lib";
/* rebar3 with the port compiler already in it.
-
esqlite builds a NIF, and its rebar3 configuration asks for the
`pc` plugin, which rebar3 would otherwise fetch from Hex at build
time: impossible in a sandbox, and the build fails with
@@ -23,11 +20,11 @@
};
/* The Hex dependencies, as manifest.toml pins them.
-
+ .
There is no Gleam builder in nixpkgs and `gleam build` fetches from
Hex, so a sandboxed build cannot reach them. A fixed output
derivation may, because its result is pinned by hash.
-
+ .
When a dependency changes, manifest.toml changes and this hash must
change with it: build, take the hash nix reports, put it here.
*/
@@ -52,7 +49,6 @@
in {
packages.${system} = {
default = self.packages.${system}.${name};
-
/* Shipped as an Erlang shipment rather than as an escript, because
sqlite on the BEAM is a NIF: esqlite compiles its own sqlite into
a shared object, and a NIF cannot be loaded from inside an
@@ -98,7 +94,6 @@
};
};
};
-
devShells.${system}.default = pkgs.mkShell {
packages = [ pkgs.gleam pkgs.erlang pkgs.rebar3 pkgs.sqlite ];
};
diff --git a/gleam/src/ocda/dump.gleam b/gleam/src/ocda/dump.gleam
index e5563ca..19bd93d 100644
--- a/gleam/src/ocda/dump.gleam
+++ b/gleam/src/ocda/dump.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// The model, written out canonically.
////
//// Not an output format: an *oracle*. Two libraries in two languages
diff --git a/gleam/src/ocda/images.gleam b/gleam/src/ocda/images.gleam
index 2f3a56e..0639ff7 100644
--- a/gleam/src/ocda/images.gleam
+++ b/gleam/src/ocda/images.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// An image's bytes, from whichever source the document came from.
////
//// The model *names* images, with digest, byte size and pixel size; it does
diff --git a/gleam/src/ocda/inventory.gleam b/gleam/src/ocda/inventory.gleam
index 6047313..6bf1e99 100644
--- a/gleam/src/ocda/inventory.gleam
+++ b/gleam/src/ocda/inventory.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// What a source holds, beyond the one document the model is loaded from.
////
//// The model is one language of one document. Two questions a reader asks
diff --git a/gleam/src/ocda/load.gleam b/gleam/src/ocda/load.gleam
index 2a82ac9..579a4dd 100644
--- a/gleam/src/ocda/load.gleam
+++ b/gleam/src/ocda/load.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// What a path is, and loading it.
////
//// One place that decides, so that every reader and every writer accepts
diff --git a/gleam/src/ocda/load_db.gleam b/gleam/src/ocda/load_db.gleam
index 18a5335..5373be8 100644
--- a/gleam/src/ocda/load_db.gleam
+++ b/gleam/src/ocda/load_db.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// Loading a `.ocda.db` into the model.
////
//// The point of this module is that it returns *the same value*
diff --git a/gleam/src/ocda/load_pod.gleam b/gleam/src/ocda/load_pod.gleam
index 05f7add..4f8ee67 100644
--- a/gleam/src/ocda/load_pod.gleam
+++ b/gleam/src/ocda/load_pod.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// The abstraction out of a pod archive.
////
//// A pod built with `--pod2` carries the abstraction inside it, at
diff --git a/gleam/src/ocda/load_ssp.gleam b/gleam/src/ocda/load_ssp.gleam
index ec93ef5..1a546d4 100644
--- a/gleam/src/ocda/load_ssp.gleam
+++ b/gleam/src/ocda/load_ssp.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// Loading a `.ssp` into the model.
////
//// The format is line oriented and every line is one of eight shapes told
diff --git a/gleam/src/ocda/model.gleam b/gleam/src/ocda/model.gleam
index 48cc76e..7161308 100644
--- a/gleam/src/ocda/model.gleam
+++ b/gleam/src/ocda/model.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// The object-centric document abstraction, as Gleam types.
////
//// A deliberate mirror of the D library's `ocda.model`, field for field.
diff --git a/gleam/src/ocda/zip.gleam b/gleam/src/ocda/zip.gleam
index 5ce6e09..6917796 100644
--- a/gleam/src/ocda/zip.gleam
+++ b/gleam/src/ocda/zip.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// Enough of the zip format to find one file in a pod and read it.
////
//// Not a zip library. A pod archive holds a handful of entries, spine writes
diff --git a/gleam/src/ocda_ffi.erl b/gleam/src/ocda_ffi.erl
index 2e395cd..608b5b9 100644
--- a/gleam/src/ocda_ffi.erl
+++ b/gleam/src/ocda_ffi.erl
@@ -1,3 +1,52 @@
+%%/+
+%%- Name: SisuDoc Spine, Doc Reform [a part of]
+%% - Description: documents, structuring, processing, publishing, search
+%% - static content generator
+%%
+%% - Author: Ralph Amissah
+%% [ralph.amissah@gmail.com]
+%%
+%% - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+%%
+%% - License: AGPL 3 or later:
+%%
+%% Spine (SiSU), a framework for document structuring, publishing and
+%% search
+%%
+%% Copyright (C) Ralph Amissah
+%%
+%% This program is free software: you can redistribute it and/or modify it
+%% under the terms of the GNU AFERO General Public License as published by the
+%% Free Software Foundation, either version 3 of the License, or (at your
+%% option) any later version.
+%%
+%% This program is distributed in the hope that it will be useful, but WITHOUT
+%% ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+%% FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+%% more details.
+%%
+%% You should have received a copy of the GNU General Public License along with
+%% this program. If not, see [https://www.gnu.org/licenses/].
+%%
+%% If you have Internet connection, the latest version of the AGPL should be
+%% available at these locations:
+%% [https://www.fsf.org/licensing/licenses/agpl.html]
+%% [https://www.gnu.org/licenses/agpl.html]
+%%
+%% - Spine (by Doc Reform, related to SiSU) uses standard:
+%% - docReform markup syntax
+%% - standard SiSU markup syntax with modified headers and minor modifications
+%% - docReform object numbering
+%% - standard SiSU object citation numbering & system
+%%
+%% - Homepages:
+%% [https://www.sisudoc.org]
+%% [https://www.doc-reform.org]
+%%
+%% - Git
+%% [https://git.sisudoc.org/]
+%%
+%%+/
%% The one thing ocda-lib asks of Erlang directly: a sha256, which OTP's
%% crypto has and the Gleam standard library does not. Upper case hex, as
%% the database stores it and the D library writes it.
diff --git a/gleam/src/ocda_lib.gleam b/gleam/src/ocda_lib.gleam
index 6eccaff..62d8d87 100644
--- a/gleam/src/ocda_lib.gleam
+++ b/gleam/src/ocda_lib.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// ocda-lib, in Gleam: the abstraction as a library, put to the question.
////
//// A library has no binary, so this is the programs over it: the reader,
diff --git a/gleam/src/read/main.gleam b/gleam/src/read/main.gleam
index 307b8d8..f474a26 100644
--- a/gleam/src/read/main.gleam
+++ b/gleam/src/read/main.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
//// ocda-read, in Gleam: reads the abstraction, in any of the forms the
//// library reads, and says what is in it.
////
diff --git a/gleam/test/ocda_lib_test.gleam b/gleam/test/ocda_lib_test.gleam
index 02fe463..ecd1a6a 100644
--- a/gleam/test/ocda_lib_test.gleam
+++ b/gleam/test/ocda_lib_test.gleam
@@ -1,3 +1,52 @@
+/////+
+////- Name: SisuDoc Spine, Doc Reform [a part of]
+//// - Description: documents, structuring, processing, publishing, search
+//// - static content generator
+////
+//// - Author: Ralph Amissah
+//// [ralph.amissah@gmail.com]
+////
+//// - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+////
+//// - License: AGPL 3 or later:
+////
+//// Spine (SiSU), a framework for document structuring, publishing and
+//// search
+////
+//// Copyright (C) Ralph Amissah
+////
+//// This program is free software: you can redistribute it and/or modify it
+//// under the terms of the GNU AFERO General Public License as published by the
+//// Free Software Foundation, either version 3 of the License, or (at your
+//// option) any later version.
+////
+//// This program is distributed in the hope that it will be useful, but WITHOUT
+//// ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+//// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+//// more details.
+////
+//// You should have received a copy of the GNU General Public License along with
+//// this program. If not, see [https://www.gnu.org/licenses/].
+////
+//// If you have Internet connection, the latest version of the AGPL should be
+//// available at these locations:
+//// [https://www.fsf.org/licensing/licenses/agpl.html]
+//// [https://www.gnu.org/licenses/agpl.html]
+////
+//// - Spine (by Doc Reform, related to SiSU) uses standard:
+//// - docReform markup syntax
+//// - standard SiSU markup syntax with modified headers and minor modifications
+//// - docReform object numbering
+//// - standard SiSU object citation numbering & system
+////
+//// - Homepages:
+//// [https://www.sisudoc.org]
+//// [https://www.doc-reform.org]
+////
+//// - Git
+//// [https://git.sisudoc.org/]
+////
+////+/
import gleam/dict
import gleam/list
import simplifile
@@ -36,7 +85,7 @@ const minimal = "% SiSU Document Abstraction v2.0
.part: body
.parent: 1
| First line
-|
+|
| Third line, after a blank one
}
@@ -139,7 +188,7 @@ pub fn pod_source_test() {
/// only zstd on the BEAM is a NIF that fetches zstd in its own build.
pub fn zstd_refusal_test() {
let path = "test/fixture-zstd-frame.sisupod"
- let assert Ok(_) = simplifile.write_bits(path, <<0x28, 0xB5, 0x2F, 0xFD, 0>>)
+ let assert Ok(_) = simplifile.write_bits(path, )
let assert Error(message) = load.load(path, "", False)
message |> string.contains(contain: "zstd") |> should.be_true
message |> string.contains(contain: "zstd -d") |> should.be_true
diff --git a/org/config_git.org b/org/config_git.org
new file mode 100644
index 0000000..11cb940
--- /dev/null
+++ b/org/config_git.org
@@ -0,0 +1,101 @@
+-*- mode: org -*-
+#+TITLE: configuration git
+#+DESCRIPTION: documents - structuring, various output representations & search
+#+FILETAGS: :spine:hub:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+- [[./doc-reform.org][doc-reform.org]] [[./][org/]]
+
+* git
+** .gitignore :gitignore:
+
+#+HEADER: :tangle "../.gitignore"
+#+BEGIN_SRC gitignore
+# git ls-files --others --exclude-from=.git/info/exclude
+# git ls-files --others --ignored --exclude-standard --directory
+# git check-ignore -v flake.lock
+# git clean -ix
+# find . -name "*_" | xargs rm -rf
+# find . -name "*.o" | xargs rm -rf
+
+# Deny everything, then whitelist what belongs in the repository.
+#
+# The practice is deliberate: a repository should hold what was decided to
+# put in it, not whatever happened to be in the directory when someone ran
+# "git add .". Build output, editor droppings, downloaded dependencies and
+# exported binaries are numerous, are regenerated, and differ between
+# machines; source is few and is chosen. So the default is no, and every
+# yes below is a decision that can be read.
+#
+# To see what is being excluded, and why:
+# git status --ignored
+# git check-ignore -v <path>
+# git ls-files --others --ignored --exclude-standard --directory
+
+*
+
+# ── the repository itself ────────────────────────────────────────────────
+!.gitignore
+!README.md
+!LICENSE
+
+# ── directories have to be walked into before a file inside can match ────
+!*/
+# but not these, whatever they contain
+build/
+.build/
+.direnv/
+.gleam/
+bin/
+result
+result-*
+
+# ── documentation and notes, in any language directory ───────────────────
+!**/*.md
+!**/*.org
+
+# ── nix ──────────────────────────────────────────────────────────────────
+!**/*.nix
+!**/flake.lock
+!**/.envrc
+
+# ── dlang ────────────────────────────────────────────────────────────────
+!**/*.d
+!**/dub.json
+!**/dub.sdl
+!**/dub.selections.json
+*.o
+*.a
+*.lst
+
+# ── gleam ────────────────────────────────────────────────────────────────
+!**/*.gleam
+# Erlang, for what Gleam reaches through FFI (a sha256, from OTP crypto)
+!**/src/*.erl
+# gleam.toml is the manifest, manifest.toml is the lockfile: both belong
+!**/*.toml
+*.beam
+*.ez
+erl_crash.dump
+
+# ── ruby, when it arrives ────────────────────────────────────────────────
+!**/*.rb
+!**/Gemfile
+!**/Gemfile.lock
+!**/*.gemspec
+
+# ── continuous integration ───────────────────────────────────────────────
+!**/*.yml
+!**/*.yaml
+#+END_SRC
diff --git a/org/config_nix.org b/org/config_nix.org
new file mode 100644
index 0000000..fcf53f3
--- /dev/null
+++ b/org/config_nix.org
@@ -0,0 +1,287 @@
+-*- mode: org -*-
+#+TITLE: configuration nix
+#+DESCRIPTION: configuration nix
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* nix :nix:
+** dlang flake.nix :flake:
+
+#+HEADER: :tangle ../dlang/flake.nix
+#+HEADER: :noweb yes
+#+BEGIN_SRC nix
+{
+ description = "ocda-lib: the object-centric document abstraction as a D library";
+ inputs.nixpkgs.url = "<<nixpkgs_unstable>>";
+ outputs = { self, nixpkgs }:
+ let
+ system = "x86_64-linux"; # adjust for aarch64-linux / darwin
+ pkgs = import nixpkgs { inherit system; };
+ /* A library, so the library itself is not a build product: a program
+ that uses it compiles its modules alongside its own, as the writer's
+ flake does, or takes it as a dub dependency. What is built here are
+ programs over it: the reader, and the two that put the library to
+ the question - the agreement check, which loads a document's .ssp
+ and its .ocda.db and compares the two models, and the dump, which
+ writes a loaded model out canonically so that an implementation in
+ another language can be diffed against this one.
+ .
+ `mains` is what is compiled besides the library: the checks are one
+ file each in test/, the reader is src/read/.
+ */
+ mkTool = { tool, binary, what, mains, compiler, compilerBin }:
+ pkgs.stdenv.mkDerivation {
+ pname = "ocda-lib-${tool}";
+ version = "0.1.0";
+ src = self;
+ nativeBuildInputs = [ compiler ];
+ buildInputs = [ pkgs.sqlite pkgs.zstd ];
+ buildPhase = ''
+ runHook preBuild
+ export HOME=$TMPDIR
+ ${compilerBin} -I=src -of=${tool} src/ocda/*.d ${mains} -L-lsqlite3 -L-lzstd
+ runHook postBuild
+ '';
+ installPhase = ''
+ runHook preInstall
+ mkdir -p $out/bin
+ install -m755 ${tool} $out/bin/${binary}
+ runHook postInstall
+ '';
+ meta = {
+ description = what;
+ mainProgram = binary;
+ };
+ };
+ ldc = { compiler = pkgs.ldc; compilerBin = "ldmd2"; };
+ ocda-read = mkTool (ldc // {
+ tool = "read"; binary = "ocda-read"; mains = "src/read/*.d";
+ what = "reads the abstraction in any of its forms and says what is in it";
+ });
+ ocda-agree = mkTool (ldc // {
+ tool = "agree"; binary = "ocda-agree"; mains = "test/agree.d";
+ what = "checks that the .ssp and .ocda.db loaders give one model";
+ });
+ ocda-dump = mkTool (ldc // {
+ tool = "dump"; binary = "ocda-dump"; mains = "test/dump.d";
+ what = "writes a loaded abstraction out canonically, as an oracle";
+ });
+ in {
+ packages.${system} = {
+ /* All three in one bin/, so that one build leaves every program in
+ one result link. Built separately into the same link, each build
+ replaced the one before, and a script looking for ocda-dump beside
+ ocda-agree found nothing.
+ */
+ default = pkgs.symlinkJoin {
+ name = "ocda-lib-programs-0.1.0";
+ paths = [ ocda-read ocda-agree ocda-dump ];
+ };
+ inherit ocda-read ocda-agree ocda-dump;
+ };
+ devShells.${system}.default = pkgs.mkShell {
+ packages = [ pkgs.ldc pkgs.dub pkgs.dtools pkgs.sqlite pkgs.zstd ];
+ };
+ };
+}
+#+END_SRC
+
+** gleam flake.nix :flake:
+
+#+HEADER: :tangle ../gleam/flake.nix
+#+HEADER: :noweb yes
+#+BEGIN_SRC nix
+{
+ description = "ocda-lib: the object-centric document abstraction as a Gleam library";
+ inputs.nixpkgs.url = "<<nixpkgs_unstable>>";
+ outputs = { self, nixpkgs }:
+ let
+ system = "x86_64-linux"; # adjust for aarch64-linux / darwin
+ pkgs = import nixpkgs { inherit system; };
+
+ name = "ocda_lib";
+
+ /* rebar3 with the port compiler already in it.
+ esqlite builds a NIF, and its rebar3 configuration asks for the
+ `pc` plugin, which rebar3 would otherwise fetch from Hex at build
+ time: impossible in a sandbox, and the build fails with
+ "missing_package pc". nixpkgs carries the plugin, so the answer is
+ a rebar3 that has it before it starts.
+ */
+ rebar3-pc = pkgs.beamPackages.rebar3WithPlugins {
+ plugins = [ pkgs.beamPackages.pc ];
+ };
+
+ /* The Hex dependencies, as manifest.toml pins them.
+ .
+ There is no Gleam builder in nixpkgs and `gleam build` fetches from
+ Hex, so a sandboxed build cannot reach them. A fixed output
+ derivation may, because its result is pinned by hash.
+ .
+ When a dependency changes, manifest.toml changes and this hash must
+ change with it: build, take the hash nix reports, put it here.
+ */
+ deps = pkgs.stdenv.mkDerivation {
+ pname = "${name}-deps";
+ version = "0.1.0";
+ src = ./.;
+ nativeBuildInputs = [ pkgs.gleam pkgs.cacert ];
+ buildPhase = ''
+ export HOME=$TMPDIR
+ gleam deps download
+ '';
+ installPhase = ''
+ mkdir -p $out
+ cp -r build/packages/* $out/
+ '';
+ dontFixup = true;
+ outputHashMode = "recursive";
+ outputHashAlgo = "sha256";
+ outputHash = "sha256-nWU753YRsC28gsaGyy+A3aZaWf15ctULYiUlvsfyB1c=";
+ };
+ in {
+ packages.${system} = {
+ default = self.packages.${system}.${name};
+ /* Shipped as an Erlang shipment rather than as an escript, because
+ sqlite on the BEAM is a NIF: esqlite compiles its own sqlite into
+ a shared object, and a NIF cannot be loaded from inside an
+ escript archive, the runtime needing a real file on disk. The
+ shipment keeps priv/ on disk, so it loads.
+ */
+ ${name} = pkgs.stdenv.mkDerivation {
+ pname = "ocda-lib";
+ version = "0.1.0";
+ src = ./.;
+ nativeBuildInputs = [
+ pkgs.gleam
+ pkgs.erlang
+ rebar3-pc
+ pkgs.makeWrapper
+ ];
+ buildPhase = ''
+ runHook preBuild
+ export HOME=$TMPDIR
+ # the dependencies, in place of the fetch gleam would do
+ mkdir -p build/packages
+ cp -r ${deps}/* build/packages/
+ chmod -R u+w build/packages
+ gleam export erlang-shipment
+ runHook postBuild
+ '';
+ installPhase = ''
+ runHook preInstall
+ mkdir -p $out/lib $out/bin
+ cp -r build/erlang-shipment $out/lib/${name}
+ makeWrapper $out/lib/${name}/entrypoint.sh $out/bin/ocda-lib \
+ --add-flags run \
+ --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.erlang pkgs.coreutils ]}
+ # the reader under its own name, as the D flake installs it
+ makeWrapper $out/lib/${name}/entrypoint.sh $out/bin/ocda-read \
+ --add-flags "run read" \
+ --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.erlang pkgs.coreutils ]}
+ runHook postInstall
+ '';
+ meta = {
+ description = "the abstraction as a Gleam library, with its reader, dump and agreement check";
+ mainProgram = "ocda-lib";
+ };
+ };
+ };
+ devShells.${system}.default = pkgs.mkShell {
+ packages = [ pkgs.gleam pkgs.erlang pkgs.rebar3 pkgs.sqlite ];
+ };
+ };
+}
+#+END_SRC
+
+* versions GET
+** dlang overlays
+*** ldc
+
+#+NAME: ldc_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./nix-develop-dlang-shared.org:ldc-version()>>
+#+END_SRC
+
+*** dmd
+
+#+NAME: dmd_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./nix-develop-dlang-shared.org:dmd-version()>>
+#+END_SRC
+
+*** dub
+
+#+NAME: dub_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./nix-develop-dlang-shared.org:dub-version()>>
+#+END_SRC
+
+*** dtools
+
+#+NAME: dtools_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./nix-develop-dlang-shared.org:dtools-version()>>
+#+END_SRC
+
+** sha256 blank hash
+
+#+NAME: sha256-blank
+#+BEGIN_SRC nix
+<<./nix-develop-dlang-shared.org:sha256-blank()>>
+#+END_SRC
+
+#+NAME: blank_hash
+#+BEGIN_SRC nix
+<<./nix-develop-dlang-shared.org:sha256-blank()>>
+#+END_SRC
+
+* org includes
+** spine project VERSION
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* nixpkgs source
+
+#+NAME: nixpkgs_unstable
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:nixpkgs_unstable()>>
+#+END_SRC
+
+* __END__
diff --git a/org/gleam-ocda-load.org b/org/gleam-ocda-load.org
new file mode 100644
index 0000000..dbf5611
--- /dev/null
+++ b/org/gleam-ocda-load.org
@@ -0,0 +1,1484 @@
+-*- mode: org -*-
+#+TITLE: gleam lib
+#+DESCRIPTION: gleam lib
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :tangle-mode :noweb no :padline no :tangle-mode :no-expand
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* gleam :gleam:
+** load gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/load.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<load>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: load
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// What a path is, and loading it.
+////
+//// One place that decides, so that every reader and every writer accepts
+//// the same inputs and a new input form is added once. This mirrors
+//// spine's own `ocda/abstraction/load.d`, which names the sources it can
+//// enter the pipeline from and tells them apart by suffix.
+////
+//// Deliberately *not* here: `.sst` and `.ssm` markup. Reading markup means
+//// implementing the parser, which is spine's work and a far larger thing
+//// than a reader or a sample writer needs. This library reads
+//// abstractions.
+
+import gleam/list
+import gleam/string
+import ocda/load_db
+import ocda/load_pod
+import ocda/load_ssp
+import ocda/model.{type Document}
+
+pub type Source {
+ /// `.ssp`, the abstraction as text
+ Ssp
+ /// `.ocda.db`, the abstraction as sqlite
+ OcdaDb
+ /// `.sisupod` or `.zip`, an archive carrying a `.ssp`
+ Pod
+ Unknown
+}
+
+/// `.sisupod` is the archive spine writes now; `.zip` is what it wrote
+/// before, and every pod already published carries that name, so both are
+/// recognised. What is inside decides how it is opened, not the name.
+const suffixes = [
+ #(".ocda.db", OcdaDb),
+ #(".ssp", Ssp),
+ #(".sisupod", Pod),
+ #(".zip", Pod),
+ #(".db", OcdaDb),
+]
+
+pub fn source_of(path: String) -> Source {
+ case list.find(suffixes, fn(s) { string.ends_with(path, s.0) }) {
+ Ok(#(_, source)) -> source
+ Error(_) -> Unknown
+ }
+}
+
+pub fn source_name(source: Source) -> String {
+ case source {
+ Ssp -> ".ssp (the abstraction as text)"
+ OcdaDb -> ".ocda.db (the abstraction as sqlite)"
+ Pod -> "a pod archive carrying a .ssp"
+ Unknown -> "not an abstraction this library reads"
+ }
+}
+
+/// Load whatever the path is into the model.
+///
+/// `lang` names which language to take from a `.ocda.db`, which holds
+/// every language of its document, and from a pod, which may carry more than
+/// one `.ssp`. A `.ssp` is one language by construction and ignores it.
+pub fn load(
+ path: String,
+ lang: String,
+ any_version: Bool,
+) -> Result(Document, String) {
+ case source_of(path) {
+ Ssp -> load_ssp.load_file(path)
+ OcdaDb -> load_db.load_file(path, lang, any_version)
+ Pod -> load_pod.load_file(path, lang)
+ Unknown ->
+ Error(
+ path
+ <> ": "
+ <> source_name(Unknown)
+ <> " (expected .ssp, .ocda.db, .sisupod or .zip)",
+ )
+ }
+}
+#+END_SRC
+
+** load pod
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/load_pod.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<load_pod>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: load_pod
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// The abstraction out of a pod archive.
+////
+//// A pod built with `--pod2` carries the abstraction inside it, at
+//// `media/abstraction/<uid>.<lang>.ssp`, and that is what is read. So this is
+//// a third input form and *not* a third model: the `.ssp` inside is parsed by
+//// `ocda/load_ssp` and comes back as the same value. A writer over the
+//// library gains pods without being touched, which is the whole reason the
+//// dispatch is in one place.
+////
+//// ## The one thing this cannot do
+////
+//// A `.sisupod` is **one zstd frame wrapping** the zip. This reads the zip
+//// and **not the frame**, because there is no zstd on the BEAM without a NIF,
+//// and the one that exists, `ezstd`, clones zstd from github in its build, so
+//// it cannot be built in a sandbox without patching it. The D library has
+//// both, through a declared `libzstd`.
+////
+//// So: a `.zip` pod is read, a `.sisupod` is refused, and the refusal says
+//// why rather than pretending the file is malformed. It is the only place the
+//// two libraries differ in what they accept, and it is a packaging limit
+//// rather than anything about the format.
+
+import gleam/list
+import gleam/result
+import gleam/string
+import ocda/load_ssp
+import ocda/model.{type Document}
+import ocda/zip
+import simplifile
+
+/// The magic number of a zstd frame, little endian 0xFD2FB528.
+const zstd_magic = <<0x28, 0xB5, 0x2F, 0xFD>>
+
+/// Which languages of the abstraction the pod carries, sorted.
+pub fn languages(path: String) -> Result(List(String), String) {
+ use bytes <- result.try(archive_bytes(path))
+ use entries <- result.try(zip.entries(bytes))
+ zip.find_suffix(entries, ".ssp")
+ |> list.filter_map(fn(e) { lang_of(e.name) })
+ |> list.sort(string.compare)
+ |> Ok
+}
+
+/// Load one language of the abstraction a pod carries.
+///
+/// `lang` == "" takes the only one there is, and where there is a choice it
+/// says so rather than choosing, as the `.ocda.db` loader does.
+pub fn load_file(path: String, lang: String) -> Result(Document, String) {
+ use bytes <- result.try(archive_bytes(path))
+ use entries <- result.try(zip.entries(bytes))
+ let ssps = zip.find_suffix(entries, ".ssp")
+ use chosen <- result.try(case ssps, lang {
+ [], _ ->
+ Error(
+ path
+ <> ": carries no .ssp. A pod carries the abstraction only when spine"
+ <> " was asked for it: spine --pod2 puts it in, spine --source does not.",
+ )
+ [only], "" -> Ok(only)
+ _, "" ->
+ Error(
+ path
+ <> ": holds more than one language ("
+ <> string.join(langs(ssps), " ")
+ <> "); name the one wanted",
+ )
+ _, wanted ->
+ case list.find(ssps, fn(e) { lang_of(e.name) == Ok(wanted) }) {
+ Ok(found) -> Ok(found)
+ Error(_) ->
+ Error(
+ path
+ <> ": no language '"
+ <> wanted
+ <> "'; it holds: "
+ <> string.join(langs(ssps), " "),
+ )
+ }
+ })
+ use content <- result.try(zip.read(bytes, chosen))
+ case bit_array_to_string(content) {
+ Ok(text) -> Ok(load_ssp.parse(text))
+ Error(_) -> Error(chosen.name <> ": not valid UTF-8")
+ }
+}
+
+fn langs(entries: List(zip.Entry)) -> List(String) {
+ list.filter_map(entries, fn(e) { lang_of(e.name) })
+}
+
+/// `<uid>.<lang>.ssp` -> `<lang>`
+pub fn lang_of(name: String) -> Result(String, Nil) {
+ case string.ends_with(name, ".ssp") {
+ False -> Error(Nil)
+ True -> {
+ let stem = string.drop_end(name, 4)
+ case string.split(stem, on: ".") |> list.last {
+ Ok(lang) if lang != stem -> Ok(lang)
+ _ -> Error(Nil)
+ }
+ }
+ }
+}
+
+/// The bytes decide, not the name: a file misnamed either way is treated by
+/// what it holds.
+pub fn archive_bytes(path: String) -> Result(BitArray, String) {
+ case simplifile.read_bits(path) {
+ Error(e) -> Error(path <> ": " <> simplifile.describe_error(e))
+ Ok(bytes) ->
+ case is_zstd_frame(bytes) {
+ False -> Ok(bytes)
+ True ->
+ Error(
+ path
+ <> " is a zstd frame wrapping the archive, and this library has no"
+ <> " zstd: the only one on the BEAM is a NIF that fetches zstd in"
+ <> " its own build, so it cannot be built sandboxed. Unwrap it"
+ <> " first (zstd -d) and read the archive, or use the D library,"
+ <> " which reads both.",
+ )
+ }
+ }
+}
+
+fn is_zstd_frame(bytes: BitArray) -> Bool {
+ case bytes {
+ <<head:bytes-size(4), _:bits>> -> head == zstd_magic
+ _ -> False
+ }
+}
+
+@external(erlang, "unicode", "characters_to_binary")
+fn bit_array_to_string_ffi(b: BitArray) -> String
+
+pub fn bit_array_to_string(b: BitArray) -> Result(String, Nil) {
+ Ok(bit_array_to_string_ffi(b))
+}
+#+END_SRC
+
+** load ssp
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/load_ssp.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<load_ssp>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: load_ssp
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// Loading a `.ssp` into the model.
+////
+//// The format is line oriented and every line is one of eight shapes told
+//// apart by its first character, so the loader is a fold over the lines
+//// carrying the state the shapes need.
+////
+//// Written against `ssp-format-specification.md`. Where it and the D
+//// library differ the D library is the bug, or this is: they are meant to
+//// give the same model, and `ocda-dump` says whether they do.
+
+import gleam/dict.{type Dict}
+import gleam/int
+import gleam/list
+import gleam/option
+import gleam/result
+import gleam/string
+import ocda/model.{type Document, type Image, type Object, Image, Object}
+import simplifile
+
+pub fn load_file(path: String) -> Result(Document, String) {
+ case simplifile.read(path) {
+ Ok(content) -> Ok(parse(content))
+ Error(e) -> Error(path <> ": " <> simplifile.describe_error(e))
+ }
+}
+
+type State {
+ State(
+ document: Document,
+ /// the block currently open, "" when none
+ block: String,
+ /// the section currently open, "" when in the preamble
+ section: String,
+ /// objects of the current section, newest first
+ objects: List(Object),
+ /// the object being read, if any
+ current: Result(Object, Nil),
+ /// text lines of the object being read, newest first
+ text: List(String),
+ /// whether a text line has been seen for the object being read.
+ /// Not "is the text non-empty": an object whose first text line is
+ /// blank, which the reference collection has seven of, would then lose
+ /// it.
+ seen_text: Bool,
+ )
+}
+
+pub fn parse(content: String) -> Document {
+ content
+ |> string.split(on: "\n")
+ |> list.fold(from: initial(), with: line)
+ |> finish
+}
+
+fn initial() -> State {
+ State(
+ document: model.new_document(),
+ block: "",
+ section: "",
+ objects: [],
+ current: Error(Nil),
+ text: [],
+ seen_text: False,
+ )
+}
+
+fn line(state: State, raw: String) -> State {
+ case raw {
+ "% SiSU Document Abstraction v" <> version ->
+ case state.document.format_version {
+ "" ->
+ State(
+ ..state,
+ document: model.Document(
+ ..state.document,
+ format_version: string.trim(version),
+ ),
+ )
+ _ -> state
+ }
+ "% Source: " <> filename ->
+ case state.document.source_filename {
+ "" ->
+ State(
+ ..state,
+ document: model.Document(
+ ..state.document,
+ source_filename: string.trim(filename),
+ ),
+ )
+ _ -> state
+ }
+ "%" <> _ -> state
+ "@" <> rest -> open_block(state, rest)
+ "}" <> _ -> close_block(state)
+ "[" <> rest -> declaration(state, rest)
+ "." <> rest -> property(state, rest)
+ "|" <> rest -> text_line(state, rest)
+ _ ->
+ case string.trim(raw) {
+ // a blank line ends the object being read
+ "" -> commit(state)
+ _ ->
+ case model.is_preamble_block(state.block) {
+ True -> preamble_pair(state, raw)
+ False -> unknown_line(state, raw)
+ }
+ }
+ }
+}
+
+fn open_block(state: State, rest: String) -> State {
+ let name = case string.split_once(rest, "{") {
+ Ok(#(name, _)) -> string.trim(name)
+ Error(_) -> string.trim(rest)
+ }
+ case model.is_preamble_block(name) {
+ True -> State(..flush(state), block: name)
+ False -> {
+ let state = flush(state)
+ let document = case model.is_section(name) {
+ True -> state.document
+ // a block that is neither preamble nor one of the nine sections:
+ // counted, so that a reader can say the file held something it did
+ // not expect
+ False ->
+ model.Document(
+ ..state.document,
+ unknown: model.Unknown(
+ ..state.document.unknown,
+ block: bump(state.document.unknown.block, name),
+ ),
+ )
+ }
+ State(..state, block: name, section: name, document: document)
+ }
+ }
+}
+
+fn close_block(state: State) -> State {
+ State(..flush(state), block: "")
+}
+
+/// Commit the object being read, then put the section's objects into the
+/// document and close the section.
+fn flush(state: State) -> State {
+ let state = commit(state)
+ case state.section, state.objects {
+ "", _ -> state
+ _, [] -> State(..state, section: "", objects: [])
+ name, objects ->
+ State(
+ ..state,
+ section: "",
+ objects: [],
+ document: model.Document(
+ ..state.document,
+ sections: add_section(
+ state.document.sections,
+ name,
+ list.reverse(objects),
+ ),
+ ),
+ )
+ }
+}
+
+fn preamble_pair(state: State, raw: String) -> State {
+ case string.split_once(string.trim_start(raw), ":") {
+ Error(_) -> state
+ Ok(#(key, value)) -> {
+ let key = string.trim(key)
+ let value = string.trim(value)
+ let document = state.document
+ let document = case state.block {
+ "meta" ->
+ model.Document(
+ ..document,
+ metadata: dict.insert(document.metadata, key, value),
+ )
+ "make" ->
+ model.Document(..document, make: dict.insert(document.make, key, value))
+ "source" ->
+ model.Document(
+ ..document,
+ source: dict.insert(document.source, key, value),
+ )
+ _ ->
+ model.Document(
+ ..document,
+ doc_has: dict.insert(document.doc_has, key, value),
+ )
+ }
+ State(..state, document: document)
+ }
+ }
+}
+
+/// `[<ocn>] <kind>`, and for a heading `[<ocn>] heading :<level>
+/// [<identifier>]`.
+fn declaration(state: State, rest: String) -> State {
+ let state = commit(state)
+ case string.split_once(rest, "]") {
+ Error(_) -> state
+ Ok(#(ocn_text, tail)) -> {
+ let ocn = parse_int(string.trim(ocn_text))
+ let object =
+ Object(..model.new_object(), ocn: ocn, section: state.section)
+ let object = case string.split(string.trim(tail), on: " ") {
+ ["heading", level, ..identifier] ->
+ Object(
+ ..object,
+ is_a: "heading",
+ heading_level: string.drop_start(level, 1),
+ identifier: string.join(identifier, " "),
+ )
+ [""] -> object
+ [kind, ..] -> Object(..object, is_a: kind)
+ [] -> object
+ }
+ State(..state, current: Ok(object), text: [], seen_text: False)
+ }
+ }
+}
+
+fn property(state: State, rest: String) -> State {
+ case state.current, string.split_once(rest, ":") {
+ Ok(object), Ok(#(name, raw_value)) -> {
+ let name = string.trim(name)
+ let value = string.trim(raw_value)
+ let #(object, known) = apply_property(object, name, value)
+ let document = case known {
+ True -> state.document
+ False ->
+ model.Document(
+ ..state.document,
+ unknown: model.Unknown(
+ ..state.document.unknown,
+ property: bump(state.document.unknown.property, name),
+ ),
+ )
+ }
+ State(..state, current: Ok(object), document: document)
+ }
+ _, _ -> state
+ }
+}
+
+/// The properties this library reads, and what each sets.
+///
+/// A property it has no use for is counted rather than passed over:
+/// ignoring one is conforming within a major version, doing so in silence
+/// is what stops a reader being a check on the specification.
+fn apply_property(
+ object: Object,
+ name: String,
+ value: String,
+) -> #(Object, Bool) {
+ case name {
+ "part" -> #(Object(..object, part: value), True)
+ "is_of_type" -> #(Object(..object, is_of_type: value), True)
+ "section" -> #(Object(..object, own_section: value), True)
+ "identifier" -> #(Object(..object, identifier: value), True)
+ "segment_html_is" -> #(Object(..object, segment_html_is: value), True)
+ "segment_epub" -> #(Object(..object, segment_epub: value), True)
+ "segment_epub_is" -> #(Object(..object, segment_epub_is: value), True)
+ "table_aligns" -> #(Object(..object, table_aligns: value), True)
+ "stow_link" -> #(
+ Object(..object, stow_link: list.append(object.stow_link, [value])),
+ True,
+ )
+ "anchor_tag" -> #(
+ Object(..object, anchor_tags: list.append(object.anchor_tags, [value])),
+ True,
+ )
+ "parent" -> #(Object(..object, parent_ocn: parse_int(value)), True)
+ "last_descendant" -> #(
+ Object(..object, last_descendant_ocn: parse_int(value)),
+ True,
+ )
+ "ocn_range" ->
+ case numbers(value) {
+ [first, last] -> #(
+ Object(..object, ocn_range_first: first, ocn_range_last: last),
+ True,
+ )
+ _ -> #(object, True)
+ }
+ "children" -> #(Object(..object, children: numbers(value)), True)
+ "ancestors" -> #(Object(..object, ancestors: numbers(value)), True)
+ "dummy" -> #(Object(..object, dummy_heading: value == "true"), True)
+ "ocn_off" -> #(Object(..object, ocn_off: value == "true"), True)
+ "indent" -> {
+ let parts = string.split(value, on: " ")
+ let base = case parts {
+ [first, ..] -> parse_int(first)
+ [] -> 0
+ }
+ let hang = case parts {
+ [_, second, ..] -> parse_int(second)
+ _ -> 0
+ }
+ #(Object(..object, indent_base: base, indent_hang: hang), True)
+ }
+ "bullet" -> #(Object(..object, bullet: value == "true"), True)
+ "lang" -> #(Object(..object, lang: value), True)
+ "has" -> #(has_flags(object, value), True)
+ "segment" -> #(Object(..object, segment: value), True)
+ "segment_prev" -> #(Object(..object, segment_prev: value), True)
+ "segment_next" -> #(Object(..object, segment_next: value), True)
+ "anchor" -> #(Object(..object, anchor: value), True)
+ "heading_lev_anchor" -> #(
+ Object(..object, heading_lev_anchor: value),
+ True,
+ )
+ "table_cols" -> #(Object(..object, table_cols: parse_int(value)), True)
+ "table_widths" -> #(Object(..object, table_widths: value), True)
+ "table_header" -> #(Object(..object, table_header: value == "true"), True)
+ "code_syntax" -> #(Object(..object, code_syntax: value), True)
+ "code_linenumbers" -> #(
+ Object(..object, code_linenumbers: value == "true"),
+ True,
+ )
+ "heading_ancestors_text" -> #(
+ Object(..object, heading_ancestors_text: value),
+ True,
+ )
+ "sha256" -> #(Object(..object, sha256: value), True)
+ "image" -> #(
+ Object(..object, images: list.append(object.images, [image(value)])),
+ True,
+ )
+ _ -> #(object, False)
+ }
+}
+
+fn has_flags(object: Object, value: String) -> Object {
+ value
+ |> string.split(on: " ")
+ |> list.fold(from: object, with: fn(obj, flag) {
+ case flag {
+ "links" -> Object(..obj, has_links: True)
+ "notes_reg" -> Object(..obj, has_notes_reg: True)
+ "notes_star" -> Object(..obj, has_notes_star: True)
+ "images" -> Object(..obj, has_images: True)
+ _ -> obj
+ }
+ })
+}
+
+/// `<filename> sha256:<HEX> bytes:<n> [px:<w>x<h>] [missing:true] [...]`
+///
+/// The filename comes first, the rest are order-independent, and a field
+/// this library does not recognise is kept rather than discarded.
+fn image(value: String) -> Image {
+ case string.split(value, on: " ") {
+ [] -> model.new_image()
+ [name, ..fields] ->
+ list.fold(fields, model.new_image(), fn(img, field) {
+ case string.split_once(field, ":") {
+ Error(_) -> img
+ Ok(#(key, val)) ->
+ case key {
+ "sha256" -> Image(..img, sha256: val)
+ "bytes" -> Image(..img, bytes: parse_int(val))
+ "px" ->
+ case string.split_once(val, "x") {
+ Ok(#(w, h)) ->
+ Image(..img, width: parse_int(w), height: parse_int(h))
+ Error(_) -> img
+ }
+ "missing" -> Image(..img, missing: val == "true")
+ _ -> Image(..img, fields: dict.insert(img.fields, key, val))
+ }
+ }
+ })
+ |> fn(img) { Image(..img, name: name) }
+ }
+}
+
+/// `| <text>`, and `| ` alone for a blank line. Never a bare `|`: the
+/// emitter writes the space.
+fn text_line(state: State, rest: String) -> State {
+ case state.current {
+ Error(_) -> state
+ Ok(_) -> {
+ let content = string.drop_start(rest, 1)
+ State(..state, text: [content, ..state.text], seen_text: True)
+ }
+ }
+}
+
+fn unknown_line(state: State, raw: String) -> State {
+ let first = string.slice(raw, 0, 1)
+ State(
+ ..state,
+ document: model.Document(
+ ..state.document,
+ unknown: model.Unknown(
+ ..state.document.unknown,
+ line: bump(state.document.unknown.line, first),
+ ),
+ ),
+ )
+}
+
+fn commit(state: State) -> State {
+ case state.current {
+ Error(_) -> state
+ Ok(object) -> {
+ let text = case state.seen_text {
+ True -> string.join(list.reverse(state.text), "\n")
+ False -> ""
+ }
+ let object = derive_omitted(Object(..object, text: text))
+ let objects = case model.is_section(state.section) {
+ True -> [object, ..state.objects]
+ False -> state.objects
+ }
+ State(
+ ..state,
+ current: Error(Nil),
+ text: [],
+ seen_text: False,
+ objects: objects,
+ )
+ }
+ }
+}
+
+/// Two values the text form leaves out because they can be worked out.
+///
+/// `.identifier` is absent when it is simply the ocn, `.anchor` when it is
+/// the segment name. The `.ocda.db`, being columns rather than lines,
+/// stores both unconditionally. So a loader that takes the text form
+/// literally gives a different model from the one the database gives, and
+/// over one document that is 1,314 identifiers and 69 anchors.
+fn derive_omitted(object: Object) -> Object {
+ let object = case object.identifier, object.ocn {
+ "", 0 -> object
+ "", ocn -> Object(..object, identifier: int.to_string(ocn))
+ _, _ -> object
+ }
+ case object.anchor, object.segment {
+ "", "" -> object
+ "", segment -> Object(..object, anchor: segment)
+ _, _ -> object
+ }
+}
+
+fn finish(state: State) -> Document {
+ flush(state).document
+}
+
+fn add_section(
+ sections: Dict(String, List(Object)),
+ name: String,
+ objects: List(Object),
+) -> Dict(String, List(Object)) {
+ dict.upsert(sections, name, fn(existing) {
+ case existing {
+ option.Some(previous) -> list.append(previous, objects)
+ option.None -> objects
+ }
+ })
+}
+
+fn bump(counts: Dict(String, Int), key: String) -> Dict(String, Int) {
+ dict.upsert(counts, key, fn(existing) {
+ case existing {
+ option.Some(n) -> n + 1
+ option.None -> 1
+ }
+ })
+}
+
+fn parse_int(value: String) -> Int {
+ value |> string.trim |> int.parse |> result.unwrap(0)
+}
+
+fn numbers(value: String) -> List(Int) {
+ value
+ |> string.split(on: " ")
+ |> list.filter(fn(s) { s != "" })
+ |> list.map(parse_int)
+}
+#+END_SRC
+
+** load db
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/load_db.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<load_db>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: load_db
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// Loading a `.ocda.db` into the model.
+////
+//// The point of this module is that it returns *the same value*
+//// `ocda/load_ssp` returns from the same document's `.ssp`. A writer is
+//// then a function from the model to bytes and cannot tell which
+//// serialisation it was given, exactly as spine's own writers cannot.
+////
+//// Written against `ocda-db-format-specification.md`, whose section 4
+//// gives the column by column correspondence this module implements.
+
+import gleam/dict
+import gleam/dynamic/decode
+import gleam/float
+import gleam/int
+import gleam/json
+import gleam/list
+import gleam/option
+import gleam/result
+import gleam/string
+import ocda/model.{type Document, Image, Object}
+import sqlight
+
+/// The schema major version this loader knows.
+pub const schema_major_read = "2"
+
+/// One language of the database, as the manifest gives it.
+pub type Language {
+ Language(doc_id: Int, lang: String, source_filename: String, ssp_digest: String)
+}
+
+/// Load one language of a database as the model. `lang` == "" takes the
+/// first language the manifest gives.
+pub fn load_file(
+ path: String,
+ lang: String,
+ any_version: Bool,
+) -> Result(Document, String) {
+ // read only: a loader has no business writing, and a read-write open
+ // *creates* an empty database where a path is mistyped, which then reads
+ // as a document with nothing in it, a failure that looks like an answer
+ use conn <- with_connection("file:" <> path <> "?mode=ro", path)
+
+ use _ <- result.try(case
+ has(conn, "documents") && has(conn, "objects") && has(conn, "metadata")
+ {
+ True -> Ok(Nil)
+ False ->
+ Error(
+ path
+ <> ": no documents/objects/metadata tables; it does not look like a .ocda.db",
+ )
+ })
+
+ let schema_version =
+ one(conn, "select value from metadata where doc_id is null
+ and key = 'schema.version'")
+ use _ <- result.try(case schema_version, any_version {
+ "", _ -> Ok(Nil)
+ _, True -> Ok(Nil)
+ v, False ->
+ case major_of(v) == schema_major_read {
+ True -> Ok(Nil)
+ False ->
+ Error(
+ path
+ <> " is schema v"
+ <> v
+ <> ", this loader knows v"
+ <> schema_major_read
+ <> ".x",
+ )
+ }
+ })
+
+ let languages = languages(conn)
+ use chosen <- result.try(case languages, lang {
+ [], _ -> Error(path <> ": no documents rows")
+ [first, ..], "" -> Ok(first)
+ _, wanted ->
+ case list.find(languages, fn(l) { l.lang == wanted }) {
+ Ok(found) -> Ok(found)
+ Error(_) ->
+ Error(
+ path
+ <> ": no language '"
+ <> wanted
+ <> "'; it holds: "
+ <> string.join(list.map(languages, fn(l) { l.lang }), " "),
+ )
+ }
+ })
+
+ let source =
+ dict.new()
+ |> dict.insert("language", chosen.lang)
+ |> dict.insert(
+ "languages",
+ string.join(list.map(languages, fn(l) { l.lang }), " "),
+ )
+ let digest = meta_value(conn, chosen.doc_id, "source.digest")
+ let source = case digest {
+ "" -> source
+ d -> dict.insert(source, "digest", d)
+ }
+
+ let document =
+ model.Document(
+ ..model.new_document(),
+ format_version: schema_version,
+ source_filename: chosen.source_filename,
+ source: source,
+ )
+ let document = metadata(conn, chosen.doc_id, document)
+ let document = objects(conn, chosen.doc_id, document)
+ let document = count_unread(conn, chosen.doc_id, document)
+ Ok(document)
+}
+
+fn with_connection(
+ uri: String,
+ path: String,
+ then: fn(sqlight.Connection) -> Result(a, String),
+) -> Result(a, String) {
+ case sqlight.open(uri) {
+ Error(e) -> Error(path <> ": " <> e.message)
+ Ok(conn) -> {
+ let outcome = then(conn)
+ let _ = sqlight.close(conn)
+ outcome
+ }
+ }
+}
+
+/// Every language the database holds, in manifest order.
+pub fn languages(conn: sqlight.Connection) -> List(Language) {
+ let d = {
+ use id <- decode.field(0, decode.int)
+ use lang <- decode.field(1, decode.string)
+ use source <- decode.field(2, decode.string)
+ use digest <- decode.field(3, decode.string)
+ decode.success(Language(id, lang, source, digest))
+ }
+ sqlight.query(
+ "select id, lang, coalesce(source_filename,''), coalesce(ssp_digest,'')
+ from documents order by id",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+/// The three preamble blocks of the text form, out of the one metadata
+/// table the database keeps them in.
+fn metadata(
+ conn: sqlight.Connection,
+ doc_id: Int,
+ document: Document,
+) -> Document {
+ let d = {
+ use key <- decode.field(0, decode.string)
+ use value <- decode.field(1, decode.string)
+ decode.success(#(key, value))
+ }
+ let rows =
+ sqlight.query(
+ "select key, value from metadata where doc_id = "
+ <> int.to_string(doc_id)
+ <> " order by key",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+
+ list.fold(rows, document, fn(doc, row) {
+ let #(key, raw) = row
+ let value = string.trim(raw)
+ case key {
+ // source.* describes the artefact rather than the document, and is
+ // @source in the text form
+ "source." <> _ -> doc
+ // translation.ocn_aligned says whether the languages of this document
+ // share one numbering. A .ocda.db can know it because it holds every
+ // language; a .ssp is one language and cannot. It is a fact about the
+ // file, not the document, and carrying it here would let a writer
+ // tell which serialisation it was handed.
+ "translation." <> _ -> doc
+ // the database tells the three blocks apart by a key prefix.
+ // Unprefixing them is what makes the two loaders give one model: a
+ // writer asks make["footer"], never metadata["make.footer"].
+ "make." <> rest ->
+ model.Document(..doc, make: dict.insert(doc.make, rest, value))
+ "doc_has." <> rest ->
+ model.Document(..doc, doc_has: dict.insert(doc.doc_has, rest, value))
+ _ ->
+ model.Document(..doc, metadata: dict.insert(doc.metadata, key, value))
+ }
+ })
+}
+
+/// The objects, in document order within each section, the sections in the
+/// order the format gives rather than the alphabetical one sqlite would.
+fn objects(
+ conn: sqlight.Connection,
+ doc_id: Int,
+ document: Document,
+) -> Document {
+ let d = {
+ use section <- decode.field(0, decode.string)
+ use ocn <- decode.field(2, decode.int)
+ use is_a <- decode.field(3, decode.string)
+ use part <- decode.field(4, decode.string)
+ use is_of_section <- decode.field(5, decode.string)
+ use is_of_type <- decode.field(6, decode.string)
+ use heading_level <- decode.field(7, decode.int)
+ use identifier <- decode.field(8, decode.string)
+ use parent <- decode.field(9, decode.int)
+ use last_descendant <- decode.field(10, decode.int)
+ use children <- decode.field(11, decode.string)
+ use ancestors <- decode.field(12, decode.string)
+ use dummy <- decode.field(13, decode.int)
+ use ocn_off <- decode.field(14, decode.int)
+ use indent_base <- decode.field(15, decode.int)
+ use indent_hang <- decode.field(16, decode.int)
+ use bullet <- decode.field(17, decode.int)
+ use lang <- decode.field(18, decode.string)
+ use has_links <- decode.field(19, decode.int)
+ use has_notes_reg <- decode.field(20, decode.int)
+ use has_notes_star <- decode.field(21, decode.int)
+ use has_images <- decode.field(22, decode.int)
+ use segment <- decode.field(23, decode.string)
+ use segment_prev <- decode.field(24, decode.string)
+ use segment_next <- decode.field(25, decode.string)
+ use anchor <- decode.field(26, decode.string)
+ use segment_html_is <- decode.field(27, decode.string)
+ use table_cols <- decode.field(28, decode.int)
+ use table_widths <- decode.field(29, decode.string)
+ use table_aligns <- decode.field(30, decode.string)
+ use table_header <- decode.field(31, decode.int)
+ use code_linenumbers <- decode.field(32, decode.int)
+ use hat <- decode.field(33, decode.string)
+ use sha256 <- decode.field(34, decode.string)
+ use text <- decode.field(35, decode.string)
+ use id <- decode.field(36, decode.int)
+ use hla <- decode.field(37, decode.string)
+ use seg_epub <- decode.field(38, decode.string)
+ use seg_epub_is <- decode.field(39, decode.string)
+ use range_first <- decode.field(40, decode.int)
+ use range_last <- decode.field(41, decode.int)
+ decode.success(#(
+ id,
+ Object(
+ ocn: ocn,
+ is_a: is_a,
+ section: section,
+ heading_level: level_name(heading_level),
+ identifier: identifier,
+ part: part,
+ is_of_type: is_of_type,
+ own_section: case is_of_section == section {
+ True -> ""
+ False -> is_of_section
+ },
+ segment_html_is: segment_html_is,
+ table_aligns: json_words(table_aligns),
+ stow_link: [],
+ anchor_tags: [],
+ parent_ocn: parent,
+ last_descendant_ocn: last_descendant,
+ ocn_range_first: range_first,
+ ocn_range_last: range_last,
+ children: json_ints(children),
+ ancestors: json_ints(ancestors),
+ dummy_heading: dummy != 0,
+ ocn_off: ocn_off != 0,
+ indent_base: indent_base,
+ indent_hang: indent_hang,
+ bullet: bullet != 0,
+ lang: lang,
+ has_links: has_links != 0,
+ has_notes_reg: has_notes_reg != 0,
+ has_notes_star: has_notes_star != 0,
+ has_images: has_images != 0,
+ segment: segment,
+ segment_prev: segment_prev,
+ segment_next: segment_next,
+ anchor: anchor,
+ heading_lev_anchor: hla,
+ segment_epub: seg_epub,
+ segment_epub_is: seg_epub_is,
+ table_cols: table_cols,
+ table_widths: json_words(table_widths),
+ table_header: table_header != 0,
+ code_syntax: "",
+ code_linenumbers: code_linenumbers != 0,
+ heading_ancestors_text: json_joined(hat),
+ sha256: sha256,
+ images: [],
+ text: text,
+ ),
+ ))
+ }
+ let rows =
+ sqlight.query(
+ object_sql(doc_id, has_range_columns(conn)),
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+
+ // Built newest-first per section and reversed once at the end. Appending
+ // to the tail of each section instead would be quadratic, and the largest
+ // document in the reference collection has some thousands of objects in
+ // one section.
+ let sections =
+ list.fold(rows, dict.new(), fn(acc, row) {
+ let #(id, object) = row
+ // the four repeatable properties of the text form are child tables
+ // here, one query each
+ let object =
+ Object(
+ ..object,
+ images: images(conn, id),
+ stow_link: links(conn, id),
+ anchor_tags: anchors(conn, id),
+ )
+ case model.is_section(object.section) {
+ False -> acc
+ True ->
+ dict.upsert(acc, object.section, fn(existing) {
+ case existing {
+ option.Some(previous) -> [object, ..previous]
+ option.None -> [object]
+ }
+ })
+ }
+ })
+ |> dict.map_values(fn(_, objects) { list.reverse(objects) })
+ model.Document(..document, sections: sections)
+}
+
+/// 2.1 added the container range as two columns. A 2.0 database has
+/// neither, and reads as having no range, which is what 2.0 meant.
+fn has_range_columns(conn: sqlight.Connection) -> Bool {
+ let d = decode.field(0, decode.int, decode.success)
+ case
+ sqlight.query(
+ "select count(*) from pragma_table_info('objects')
+ where name = 'ocn_range_first'",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ {
+ Ok([n]) -> n > 0
+ _ -> False
+ }
+}
+
+fn object_sql(doc_id: Int, has_range: Bool) -> String {
+ let range_cols = case has_range {
+ True -> "coalesce(ocn_range_first,0), coalesce(ocn_range_last,0)"
+ False -> "0, 0"
+ }
+ "select section, seq, ocn, is_a, coalesce(is_of_part,''),
+ coalesce(is_of_section,''), coalesce(is_of_type,''),
+ coalesce(heading_level,-1), coalesce(identifier,''),
+ coalesce(parent_ocn,0), coalesce(last_descendant_ocn,0),
+ coalesce(children,''), coalesce(ancestors,''),
+ coalesce(dummy_heading,0), coalesce(object_number_off,0),
+ coalesce(indent_base,0), coalesce(indent_hang,0),
+ coalesce(bullet,0), coalesce(lang,''),
+ coalesce(has_links,0), coalesce(has_notes_reg,0),
+ coalesce(has_notes_star,0), coalesce(has_images,0),
+ coalesce(segment,''), coalesce(segment_prev,''),
+ coalesce(segment_next,''), coalesce(anchor,''),
+ coalesce(segment_html_is,''),
+ coalesce(table_cols,0), coalesce(table_widths,''),
+ coalesce(table_aligns,''), coalesce(table_header,0),
+ coalesce(code_linenumbers,0),
+ coalesce(heading_ancestors_text,''), coalesce(sha256,''),
+ coalesce(text,''), id, coalesce(heading_lev_anchor,''),
+ coalesce(segment_epub,''), coalesce(segment_epub_is,''), " <> range_cols <> "
+ from objects where doc_id = " <> int.to_string(doc_id) <> "
+ order by case section
+ when 'head' then 1 when 'toc' then 2 when 'body' then 3
+ when 'endnotes' then 4 when 'glossary' then 5
+ when 'bibliography' then 6 when 'bookindex' then 7
+ when 'blurb' then 8 when 'tail' then 9 else 10 end, seq"
+}
+
+fn images(conn: sqlight.Connection, object_id: Int) -> List(model.Image) {
+ let d = {
+ use name <- decode.field(0, decode.string)
+ use sha256 <- decode.field(1, decode.string)
+ use bytes <- decode.field(2, decode.int)
+ use width <- decode.field(3, decode.int)
+ use height <- decode.field(4, decode.int)
+ use missing <- decode.field(5, decode.int)
+ decode.success(Image(
+ name: name,
+ sha256: sha256,
+ bytes: bytes,
+ width: width,
+ height: height,
+ missing: missing != 0,
+ fields: dict.new(),
+ ))
+ }
+ sqlight.query(
+ "select name, coalesce(sha256,''), coalesce(bytes,0),
+ coalesce(width,0), coalesce(height,0), coalesce(missing,0)
+ from object_images where object_id = "
+ <> int.to_string(object_id)
+ <> " order by seq",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+fn links(conn: sqlight.Connection, object_id: Int) -> List(String) {
+ let d = {
+ use url <- decode.field(0, decode.string)
+ decode.success(url)
+ }
+ sqlight.query(
+ "select url from object_links where object_id = "
+ <> int.to_string(object_id)
+ <> " order by seq",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+fn anchors(conn: sqlight.Connection, object_id: Int) -> List(String) {
+ let d = {
+ use anchor <- decode.field(0, decode.string)
+ decode.success(anchor)
+ }
+ sqlight.query(
+ "select anchor from object_anchors where object_id = "
+ <> int.to_string(object_id)
+ <> " order by seq",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+fn one(conn: sqlight.Connection, sql: String) -> String {
+ let d = {
+ use v <- decode.field(0, decode.string)
+ decode.success(v)
+ }
+ case sqlight.query(sql, on: conn, with: [], expecting: d) {
+ Ok([v, ..]) -> v
+ _ -> ""
+ }
+}
+
+fn has(conn: sqlight.Connection, name: String) -> Bool {
+ one(
+ conn,
+ "select cast(count(*) as text) from sqlite_master where name = '"
+ <> name
+ <> "'",
+ )
+ != "0"
+}
+
+fn meta_value(conn: sqlight.Connection, doc_id: Int, key: String) -> String {
+ one(
+ conn,
+ "select value from metadata where doc_id = "
+ <> int.to_string(doc_id)
+ <> " and key = '"
+ <> key
+ <> "'",
+ )
+}
+
+fn major_of(version: String) -> String {
+ case string.split_once(version, ".") {
+ Ok(#(major, _)) -> major
+ Error(_) -> version
+ }
+}
+
+/// The eight heading levels and their indices.
+fn level_name(index: Int) -> String {
+ case index {
+ 0 -> "A"
+ 1 -> "B"
+ 2 -> "C"
+ 3 -> "D"
+ 4 -> "1"
+ 5 -> "2"
+ 6 -> "3"
+ 7 -> "4"
+ _ -> ""
+ }
+}
+
+/// The JSON columns, read as JSON.
+///
+/// Not as a character scan that strips brackets and quotes: that is wrong
+/// the moment a value holds an escape, and spine's encoder writes "/" as
+/// "\/", so every heading containing a slash comes back with a backslash in
+/// it. The D library had exactly that bug and all 36 document-languages
+/// differed on heading_ancestors_text because of it.
+fn json_strings(value: String) -> List(String) {
+ case value {
+ "" -> []
+ v ->
+ case json.parse(v, decode.list(of: loose_string())) {
+ Ok(items) -> items
+ Error(_) -> []
+ }
+ }
+}
+
+/// An element that may be a string or a number, which is what these columns
+/// hold: children and ancestors are int arrays, table_aligns and
+/// heading_ancestors_text string arrays, and table_widths an int array
+/// except where the columns divide evenly, when it is [33.3333,...]. One
+/// table in the reference collection does that, and without the float
+/// branch the widths came back as three empty strings: the fallback turned
+/// a value it could not read into a value that looked read.
+fn loose_string() -> decode.Decoder(String) {
+ decode.one_of(decode.string, [
+ decode.int |> decode.map(int.to_string),
+ decode.float |> decode.map(float.to_string),
+ decode.success(""),
+ ])
+}
+
+fn json_ints(value: String) -> List(Int) {
+ value
+ |> json_strings
+ |> list.filter(fn(s) { s != "" })
+ |> list.map(fn(s) { s |> int.parse |> result.unwrap(0) })
+}
+
+/// Space separated in the text form: table_widths, table_aligns.
+fn json_words(value: String) -> String {
+ value |> json_strings |> string.join(" ")
+}
+
+/// heading_ancestors_text is eight slots, joined with "|" in the text form
+/// and a JSON array here. The empty slots are significant: they say which
+/// levels the heading sits under.
+fn json_joined(value: String) -> String {
+ case json_strings(value) {
+ [] -> ""
+ parts -> string.join(parts, "|")
+ }
+}
+
+/// What the database holds and the model does not.
+///
+/// The text loader meets every property line and counts those it does not
+/// keep, so a reader can say what was read and not understood. This loader
+/// asks for columns by name, and would never see the ones it did not ask
+/// for: it would report that it had understood everything, which is the
+/// silence the count exists to prevent. So it counts them, on the
+/// conditions under which the `.ssp` writes the same property
+/// (specification section 6.3), and the report is the same from either
+/// form, as the D loader's is.
+fn count_unread(
+ conn: sqlight.Connection,
+ doc_id: Int,
+ document: Document,
+) -> Document {
+ let id = int.to_string(doc_id)
+ let counts = [
+ #("ancestors_collapsed", "coalesce(ancestors_collapsed,'') != ''"),
+ #("dom_status", "coalesce(dom_status,'') != ''"),
+ #("dom_status_collapsed", "coalesce(dom_status_collapsed,'') != ''"),
+ #("heading_lev_collapsed", "heading_lev_collapsed is not null"),
+ #("parent_lev", "coalesce(parent_lev,0) != 0"),
+ #("segment_lv4_is", "coalesce(segment_lv4_is,'') != ''"),
+ #("attrib", "coalesce(attrib,'') != ''"),
+ ]
+ let property =
+ list.fold(counts, document.unknown.property, fn(acc, c) {
+ let n =
+ count(
+ conn,
+ "select count(*) from objects where doc_id = " <> id <> " and " <> c.1,
+ )
+ case n > 0 {
+ True -> dict.insert(acc, c.0, n)
+ False -> acc
+ }
+ })
+ let property = case has(conn, "object_subtoc") {
+ False -> property
+ True -> {
+ let n =
+ count(
+ conn,
+ "select count(*) from object_subtoc s join objects o
+ on o.id = s.object_id where o.doc_id = " <> id,
+ )
+ case n > 0 {
+ True -> dict.insert(property, "lev4_subtoc", n)
+ False -> property
+ }
+ }
+ }
+ model.Document(
+ ..document,
+ unknown: model.Unknown(..document.unknown, property: property),
+ )
+}
+
+fn count(conn: sqlight.Connection, sql: String) -> Int {
+ let d = decode.field(0, decode.int, decode.success)
+ case sqlight.query(sql, on: conn, with: [], expecting: d) {
+ Ok([n, ..]) -> n
+ _ -> 0
+ }
+}
+
+/// Every language the database at `path` holds, in manifest order.
+pub fn languages_of_file(path: String) -> Result(List(Language), String) {
+ use conn <- with_connection("file:" <> path <> "?mode=ro", path)
+ Ok(languages(conn))
+}
+
+/// What the database at `path` carries besides the abstraction, by role
+/// and storage: role, compression, count, original bytes, stored bytes.
+pub fn carried_of_file(path: String) -> Result(List(List(String)), String) {
+ use conn <- with_connection("file:" <> path <> "?mode=ro", path)
+ case has(conn, "files") {
+ False -> Ok([])
+ True -> {
+ let d = {
+ use role <- decode.field(0, decode.string)
+ use compression <- decode.field(1, decode.string)
+ use n <- decode.field(2, decode.int)
+ use bytes <- decode.field(3, decode.int)
+ use stored <- decode.field(4, decode.int)
+ decode.success([
+ role,
+ compression,
+ int.to_string(n),
+ int.to_string(bytes),
+ int.to_string(stored),
+ ])
+ }
+ sqlight.query(
+ "select role, compression, count(*), sum(bytes), sum(length(data))
+ from files group by role, compression order by role",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.map_error(fn(e) { path <> ": " <> e.message })
+ }
+ }
+}
+#+END_SRC
+
+* org includes
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* __END__
diff --git a/org/gleam-ocda-misc.org b/org/gleam-ocda-misc.org
new file mode 100644
index 0000000..cc32dcb
--- /dev/null
+++ b/org/gleam-ocda-misc.org
@@ -0,0 +1,649 @@
+-*- mode: org -*-
+#+TITLE: gleam lib
+#+DESCRIPTION: gleam lib
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :tangle-mode :noweb no :padline no :tangle-mode :no-expand
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* gleam :gleam:
+** zip gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/zip.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<zip>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: zip
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// Enough of the zip format to find one file in a pod and read it.
+////
+//// Not a zip library. A pod archive holds a handful of entries, spine writes
+//// them **stored** (uncompressed) so that the outer zstd frame does the
+//// compressing once over the whole thing, and what is wanted from it here is
+//// one named entry.
+////
+//// Deflate is *refused rather than mishandled*. Erlang's `zlib` could
+//// inflate it, but a library that says it read an entry it in fact skipped is
+//// worse than one that says it cannot, and spine's own archives do not need
+//// it.
+
+import gleam/bit_array
+import gleam/int
+import gleam/list
+import gleam/result
+import gleam/string
+
+pub type Entry {
+ Entry(
+ name: String,
+ /// 0 stored, 8 deflate
+ method: Int,
+ compressed: Int,
+ uncompressed: Int,
+ local_offset: Int,
+ )
+}
+
+const eocd_sig = 0x06054b50
+
+const cd_sig = 0x02014b50
+
+const local_sig = 0x04034b50
+
+/// The entries, from the central directory at the end of the file.
+pub fn entries(bytes: BitArray) -> Result(List(Entry), String) {
+ let size = bit_array.byte_size(bytes)
+ use _ <- result.try(case size >= 22 {
+ True -> Ok(Nil)
+ False -> Error("too short to be a zip archive")
+ })
+ use eocd <- result.try(find_eocd(bytes, size - 22, lowest(size)))
+ use count <- result.try(u16(bytes, eocd + 10))
+ use offset <- result.try(u32(bytes, eocd + 16))
+ read_central(bytes, offset, count, [])
+}
+
+/// The end of central directory record is last, and may be followed by a
+/// comment of up to 65535 bytes, so it is searched for backwards.
+fn find_eocd(bytes: BitArray, at: Int, lowest: Int) -> Result(Int, String) {
+ case at < lowest {
+ True ->
+ Error(
+ "no end of central directory record; it does not look like a zip archive",
+ )
+ False ->
+ case u32(bytes, at) {
+ Ok(sig) if sig == eocd_sig -> Ok(at)
+ _ -> find_eocd(bytes, at - 1, lowest)
+ }
+ }
+}
+
+fn lowest(size: Int) -> Int {
+ case size > 22 + 65_535 {
+ True -> size - 22 - 65_535
+ False -> 0
+ }
+}
+
+fn read_central(
+ bytes: BitArray,
+ at: Int,
+ remaining: Int,
+ acc: List(Entry),
+) -> Result(List(Entry), String) {
+ case remaining {
+ 0 -> Ok(list.reverse(acc))
+ _ -> {
+ use sig <- result.try(u32(bytes, at))
+ use _ <- result.try(case sig == cd_sig {
+ True -> Ok(Nil)
+ False ->
+ Error(
+ "central directory entry "
+ <> int.to_string(list.length(acc))
+ <> " is malformed",
+ )
+ })
+ use method <- result.try(u16(bytes, at + 10))
+ use compressed <- result.try(u32(bytes, at + 20))
+ use uncompressed <- result.try(u32(bytes, at + 24))
+ use name_len <- result.try(u16(bytes, at + 28))
+ use extra_len <- result.try(u16(bytes, at + 30))
+ use cmnt_len <- result.try(u16(bytes, at + 32))
+ use local_offset <- result.try(u32(bytes, at + 42))
+ use name <- result.try(text(bytes, at + 46, name_len))
+ let entry =
+ Entry(
+ name: name,
+ method: method,
+ compressed: compressed,
+ uncompressed: uncompressed,
+ local_offset: local_offset,
+ )
+ read_central(
+ bytes,
+ at + 46 + name_len + extra_len + cmnt_len,
+ remaining - 1,
+ [entry, ..acc],
+ )
+ }
+ }
+}
+
+/// One entry's bytes, from its local header.
+pub fn read(bytes: BitArray, entry: Entry) -> Result(BitArray, String) {
+ use sig <- result.try(u32(bytes, entry.local_offset))
+ use _ <- result.try(case sig == local_sig {
+ True -> Ok(Nil)
+ False -> Error(entry.name <> ": local header is malformed")
+ })
+ use name_len <- result.try(u16(bytes, entry.local_offset + 26))
+ use extra_len <- result.try(u16(bytes, entry.local_offset + 28))
+ use _ <- result.try(case entry.method {
+ 0 -> Ok(Nil)
+ m ->
+ Error(
+ entry.name
+ <> ": compression method "
+ <> int.to_string(m)
+ <> "; this reader takes only stored entries, which is what spine writes",
+ )
+ })
+ let start = entry.local_offset + 30 + name_len + extra_len
+ case bit_array.slice(bytes, start, entry.compressed) {
+ Ok(slice) -> Ok(slice)
+ Error(_) -> Error(entry.name <> ": runs past the end of the archive")
+ }
+}
+
+/// The entries whose names end this way, in archive order. Used to find
+/// media/abstraction/<uid>.<lang>.ssp without knowing the uid.
+pub fn find_suffix(entries: List(Entry), suffix: String) -> List(Entry) {
+ list.filter(entries, fn(e) { string.ends_with(e.name, suffix) })
+}
+
+fn u16(bytes: BitArray, at: Int) -> Result(Int, String) {
+ case bit_array.slice(bytes, at, 2) {
+ Ok(<<v:little-size(16)>>) -> Ok(v)
+ _ -> Error("read past the end of the archive at " <> int.to_string(at))
+ }
+}
+
+fn u32(bytes: BitArray, at: Int) -> Result(Int, String) {
+ case bit_array.slice(bytes, at, 4) {
+ Ok(<<v:little-size(32)>>) -> Ok(v)
+ _ -> Error("read past the end of the archive at " <> int.to_string(at))
+ }
+}
+
+fn text(bytes: BitArray, at: Int, len: Int) -> Result(String, String) {
+ case bit_array.slice(bytes, at, len) {
+ Ok(slice) ->
+ bit_array.to_string(slice)
+ |> result.replace_error("an entry name is not valid UTF-8")
+ Error(_) -> Error("an entry name runs past the end of the archive")
+ }
+}
+#+END_SRC
+
+** imapges gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/images.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<images>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: images
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// An image's bytes, from whichever source the document came from.
+////
+//// The model *names* images, with digest, byte size and pixel size; it does
+//// not hold them. That is right - an abstraction of a document is not a
+//// bundle of its assets - but a writer of any format that shows images needs
+//// the bytes, so something has to know where they are. This is that
+//// something, and it is in the library rather than in a writer because the
+//// answer depends on the *input form* and not on the output.
+////
+//// | `.ocda.db` | the `files` table, role 'image', by name |
+//// | `.zip` | the archive entry whose path ends in `/image/<name>` |
+//// | `.sisupod` | the same, if the frame can be unwrapped, which on the BEAM it cannot |
+//// | `.ssp` | beside it: a pod puts the abstraction in `media/abstraction/` and its images in `media/image/` |
+
+import gleam/bit_array
+import gleam/dynamic/decode
+import gleam/list
+import gleam/result
+import gleam/string
+import ocda/load
+import ocda/zip
+import simplifile
+import sqlight
+
+/// The bytes, or an error saying why not.
+pub fn image_bytes(source: String, name: String) -> Result(BitArray, String) {
+ case load.source_of(source) {
+ load.OcdaDb -> from_db(source, name)
+ load.Pod -> from_archive(source, name)
+ load.Ssp -> from_beside(source, name)
+ load.Unknown -> Error(source <> ": not a source this library reads")
+ }
+}
+
+/// Can this source give images at all?
+///
+/// A writer asks before it decides whether to emit them, because an `<img>`
+/// pointing at a file it did not include is worse than no image.
+pub fn can_give_images(source: String) -> Bool {
+ case load.source_of(source) {
+ load.OcdaDb -> True
+ load.Pod -> zip_readable(source)
+ load.Ssp -> image_dir_beside(source) != ""
+ load.Unknown -> False
+ }
+}
+
+fn zip_readable(path: String) -> Bool {
+ case simplifile.read_bits(path) {
+ Error(_) -> False
+ Ok(bytes) ->
+ case zip.entries(bytes) {
+ Ok(_) -> True
+ Error(_) -> False
+ }
+ }
+}
+
+fn from_db(path: String, name: String) -> Result(BitArray, String) {
+ case sqlight.open("file:" <> path <> "?mode=ro") {
+ Error(e) -> Error(path <> ": " <> e.message)
+ Ok(conn) -> {
+ let out =
+ sqlight.query(
+ "select data from files where role = 'image' and name = ?1",
+ on: conn,
+ with: [sqlight.text(name)],
+ expecting: blob(),
+ )
+ |> result.replace_error(path <> ": could not read " <> name)
+ let _ = sqlight.close(conn)
+ case out {
+ Ok([data, ..]) -> Ok(data)
+ Ok([]) -> Error(name <> ": not in this database")
+ Error(e) -> Error(e)
+ }
+ }
+ }
+}
+
+/// A blob column, read as bytes rather than as text: an image read as text
+/// stops at the first zero byte, and a png's ninth byte is zero.
+fn blob() -> decode.Decoder(BitArray) {
+ use v <- decode.field(0, decode.bit_array)
+ decode.success(v)
+}
+
+fn from_archive(path: String, name: String) -> Result(BitArray, String) {
+ use bytes <- result.try(
+ simplifile.read_bits(path)
+ |> result.replace_error(path <> ": cannot read"),
+ )
+ use entries <- result.try(zip.entries(bytes))
+ case list.find(entries, fn(e) { string.ends_with(e.name, "/image/" <> name) }) {
+ Error(_) -> Error(name <> ": not in this archive")
+ Ok(entry) -> zip.read(bytes, entry)
+ }
+}
+
+fn from_beside(path: String, name: String) -> Result(BitArray, String) {
+ case image_dir_beside(path) {
+ "" -> Error(name <> ": a .ssp outside a pod has no image directory beside it")
+ dir ->
+ simplifile.read_bits(dir <> "/" <> name)
+ |> result.replace_error(dir <> "/" <> name <> ": cannot read")
+ }
+}
+
+/// `media/abstraction/<x>.ssp` -> `media/image/`
+fn image_dir_beside(path: String) -> String {
+ let parts = string.split(path, on: "/")
+ let dir = case list.reverse(parts) {
+ [_file, ..rest] -> string.join(list.reverse(rest), "/")
+ [] -> ""
+ }
+ case string.ends_with(dir, "abstraction") {
+ False -> ""
+ True -> {
+ let parent = case list.reverse(string.split(dir, on: "/")) {
+ [_, ..rest] -> string.join(list.reverse(rest), "/")
+ [] -> ""
+ }
+ let image_dir = parent <> "/image"
+ case simplifile.is_directory(image_dir) {
+ Ok(True) -> image_dir
+ _ -> ""
+ }
+ }
+ }
+}
+
+/// Unused here, kept so that a caller can size a read before making it.
+pub fn byte_size(b: BitArray) -> Int {
+ bit_array.byte_size(b)
+}
+#+END_SRC
+
+** dump gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/dump.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<dump>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: dump
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// The model, written out canonically.
+////
+//// Not an output format: an *oracle*. Two libraries in two languages
+//// implement the same model from the same specification, and the cheap way
+//// to ask whether they agree is to have each write the whole loaded
+//// document as text and diff the two. Byte for byte identical to the D
+//// library's `ocda/dump.d`, which is the whole point of it.
+////
+//// The rules, which both implementations follow:
+////
+//// - every line is `<key> <value>`, the value running to end of line,
+//// and only one free-form value per line;
+//// - a field at its default is *omitted*, so the output says what the
+//// document has rather than what the type declares;
+//// - dictionaries come in sorted key order, because neither language's
+//// map has an order worth relying on;
+//// - sections come in the fixed order of the format.
+////
+//// Anything that reads this to reconstruct a document has misunderstood
+//// it. Load the `.ssp` or the `.ocda.db`.
+
+import gleam/dict.{type Dict}
+import gleam/int
+import gleam/list
+import gleam/string
+import ocda/model.{type Document, type Object}
+
+pub fn document(doc: Document) -> String {
+ let lines =
+ []
+ |> kv("version", doc.format_version)
+ |> kv("source_filename", doc.source_filename)
+ |> entries("source", doc.source)
+ |> entries("meta", doc.metadata)
+ |> entries("make", doc.make)
+ |> entries("doc_has", doc.doc_has)
+ |> counts(doc)
+ |> objects(doc)
+ lines |> list.reverse |> join
+}
+
+/// What the loader met and did not understand, sorted.
+///
+/// Separate from the document dump: it describes the loader, not the
+/// document, and the `.ocda.db` loader has nothing to report where the
+/// `.ssp` loader does.
+pub fn unknown(doc: Document) -> String {
+ []
+ |> numbered("property", doc.unknown.property)
+ |> numbered("block", doc.unknown.block)
+ |> numbered("line", doc.unknown.line)
+ |> list.reverse
+ |> join
+}
+
+fn join(lines: List(String)) -> String {
+ case lines {
+ [] -> ""
+ _ -> string.join(lines, "\n") <> "\n"
+ }
+}
+
+/// Accumulated newest-first and reversed once, so that nothing appends to
+/// the tail of a list that grows to a few hundred thousand lines.
+fn kv(acc: List(String), key: String, value: String) -> List(String) {
+ case value {
+ "" -> acc
+ v -> [key <> " " <> v, ..acc]
+ }
+}
+
+fn kn(acc: List(String), key: String, value: Int) -> List(String) {
+ case value {
+ 0 -> acc
+ n -> [key <> " " <> int.to_string(n), ..acc]
+ }
+}
+
+fn kb(acc: List(String), key: String, value: Bool) -> List(String) {
+ case value {
+ False -> acc
+ True -> [key <> " 1", ..acc]
+ }
+}
+
+fn entries(
+ acc: List(String),
+ prefix: String,
+ d: Dict(String, String),
+) -> List(String) {
+ d
+ |> dict.to_list
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.fold(acc, fn(lines, pair) {
+ [prefix <> " " <> pair.0 <> " " <> pair.1, ..lines]
+ })
+}
+
+fn numbered(
+ acc: List(String),
+ prefix: String,
+ d: Dict(String, Int),
+) -> List(String) {
+ d
+ |> dict.to_list
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.fold(acc, fn(lines, pair) {
+ [prefix <> " " <> pair.0 <> " " <> int.to_string(pair.1), ..lines]
+ })
+}
+
+fn counts(acc: List(String), doc: Document) -> List(String) {
+ list.fold(model.section_names, acc, fn(lines, name) {
+ case model.section(doc, name) {
+ [] -> lines
+ objects ->
+ [
+ "section " <> name <> " " <> int.to_string(list.length(objects)),
+ ..lines
+ ]
+ }
+ })
+}
+
+fn objects(acc: List(String), doc: Document) -> List(String) {
+ list.fold(model.section_names, acc, fn(lines, name) {
+ model.section(doc, name)
+ |> list.index_fold(lines, fn(l, obj, i) {
+ object(l, name, i, obj)
+ })
+ })
+}
+
+fn object(
+ acc: List(String),
+ section: String,
+ index: Int,
+ obj: Object,
+) -> List(String) {
+ let acc = ["object " <> section <> " " <> int.to_string(index), ..acc]
+ acc
+ |> kn("ocn", obj.ocn)
+ |> kv("is_a", obj.is_a)
+ |> kv("own_section", obj.own_section)
+ |> kv("level", obj.heading_level)
+ |> kv("identifier", obj.identifier)
+ |> kv("part", obj.part)
+ |> kv("is_of_type", obj.is_of_type)
+ |> kv("segment_html_is", obj.segment_html_is)
+ |> kv("segment_epub", obj.segment_epub)
+ |> kv("segment_epub_is", obj.segment_epub_is)
+ |> kn("parent", obj.parent_ocn)
+ |> kn("last_descendant", obj.last_descendant_ocn)
+ |> kn("ocn_range_first", obj.ocn_range_first)
+ |> kn("ocn_range_last", obj.ocn_range_last)
+ |> kv("children", ints(obj.children))
+ |> kv("ancestors", ints(obj.ancestors))
+ |> kb("dummy", obj.dummy_heading)
+ |> kb("ocn_off", obj.ocn_off)
+ |> kn("indent_base", obj.indent_base)
+ |> kn("indent_hang", obj.indent_hang)
+ |> kb("bullet", obj.bullet)
+ |> kv("lang", obj.lang)
+ |> kb("has_links", obj.has_links)
+ |> kb("has_notes_reg", obj.has_notes_reg)
+ |> kb("has_notes_star", obj.has_notes_star)
+ |> kb("has_images", obj.has_images)
+ |> kv("segment", obj.segment)
+ |> kv("segment_prev", obj.segment_prev)
+ |> kv("segment_next", obj.segment_next)
+ |> kv("anchor", obj.anchor)
+ |> kv("heading_lev_anchor", obj.heading_lev_anchor)
+ |> kn("table_cols", obj.table_cols)
+ |> kv("table_widths", obj.table_widths)
+ |> kv("table_aligns", obj.table_aligns)
+ |> kb("table_header", obj.table_header)
+ |> kv("code_syntax", obj.code_syntax)
+ |> kb("code_linenumbers", obj.code_linenumbers)
+ |> kv("heading_ancestors_text", obj.heading_ancestors_text)
+ |> kv("sha256", obj.sha256)
+ |> links(obj)
+ |> anchor_tags(obj)
+ |> images(obj)
+ |> text(obj)
+}
+
+fn links(acc: List(String), obj: Object) -> List(String) {
+ list.fold(obj.stow_link, acc, fn(lines, url) { kv(lines, "stow_link", url) })
+}
+
+fn anchor_tags(acc: List(String), obj: Object) -> List(String) {
+ list.fold(obj.anchor_tags, acc, fn(lines, a) { kv(lines, "anchor_tag", a) })
+}
+
+fn images(acc: List(String), obj: Object) -> List(String) {
+ list.fold(obj.images, acc, fn(lines, img) {
+ // the name last, because a filename may hold a space where none of the
+ // other fields can
+ let lines =
+ kv(
+ lines,
+ "image",
+ img.sha256
+ <> " "
+ <> int.to_string(img.bytes)
+ <> " "
+ <> int.to_string(img.width)
+ <> "x"
+ <> int.to_string(img.height)
+ <> " "
+ <> case img.missing {
+ True -> "1"
+ False -> "0"
+ }
+ <> " "
+ <> img.name,
+ )
+ img.fields
+ |> dict.to_list
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.fold(lines, fn(l, pair) {
+ kv(l, "image_field", pair.0 <> " " <> pair.1)
+ })
+ })
+}
+
+/// Text, when there is any.
+///
+/// An object whose only text line is blank holds "" here, indistinguishable
+/// from no text at all, because the model joins the lines into one string.
+/// Both libraries lose the same distinction in the same place, which is
+/// what makes them comparable; it is recorded rather than hidden.
+fn text(acc: List(String), obj: Object) -> List(String) {
+ case obj.text {
+ "" -> acc
+ t -> {
+ let lines = string.split(t, on: "\n")
+ let acc = kv(acc, "text", int.to_string(list.length(lines)))
+ list.fold(lines, acc, fn(l, line) { ["| " <> line, ..l] })
+ }
+ }
+}
+
+fn ints(xs: List(Int)) -> String {
+ case xs {
+ [] -> ""
+ _ -> xs |> list.map(int.to_string) |> string.join(",")
+ }
+}
+#+END_SRC
+
+* org includes
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* __END__
diff --git a/org/gleam-ocda-model_and_inventory.org b/org/gleam-ocda-model_and_inventory.org
new file mode 100644
index 0000000..0e091a4
--- /dev/null
+++ b/org/gleam-ocda-model_and_inventory.org
@@ -0,0 +1,494 @@
+-*- mode: org -*-
+#+TITLE: gleam lib
+#+DESCRIPTION: gleam lib
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :tangle-mode :noweb no :padline no :tangle-mode :no-expand
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* gleam :gleam:
+** model gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/model.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<model>>
+#+END_SRC
+
+#+NAME: model
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// The object-centric document abstraction, as Gleam types.
+////
+//// A deliberate mirror of the D library's `ocda.model`, field for field.
+//// The two are meant to agree, and the cheapest way to make that testable
+//// is for them to hold the same things under the same names: the dump of a
+//// loaded document is then identical by construction rather than by
+//// negotiation.
+////
+//// Written against specs/sisudoc-abstraction-format: `ocda-model.md` for
+//// the model, the two format specifications for what each serialisation
+//// says about it.
+
+import gleam/dict.{type Dict}
+import gleam/list
+import gleam/string
+
+/// The format major version this library knows.
+///
+/// The `.ssp` and the `.ocda.db` carry one version between them, declared
+/// in band. The minor part rises when a property is added and a reader of
+/// the newer format still reads the older, so only the major is checked.
+pub const format_major_read = "2"
+
+/// The nine sections, in the order the format gives them, which is not the
+/// order a sorted key list or a `group by` would give.
+pub const section_names = [
+ "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex",
+ "blurb", "tail",
+]
+
+/// The four blocks that are document preamble rather than sections of
+/// objects.
+pub const preamble_blocks = ["source", "meta", "make", "doc_has"]
+
+/// A repeatable `.image` record: the file as the abstraction found it.
+pub type Image {
+ Image(
+ /// filename as the object refers to it
+ name: String,
+ /// hex, uppercase
+ sha256: String,
+ bytes: Int,
+ /// the file's own pixel dimensions, not any display width
+ width: Int,
+ height: Int,
+ /// absent or unreadable, so no digest and no size
+ missing: Bool,
+ /// any further key:value fields, kept rather than dropped
+ fields: Dict(String, String),
+ )
+}
+
+pub fn new_image() -> Image {
+ Image(
+ name: "",
+ sha256: "",
+ bytes: 0,
+ width: 0,
+ height: 0,
+ missing: False,
+ fields: dict.new(),
+ )
+}
+
+/// One object of the abstraction.
+///
+/// `text` is one string with the lines joined by newline, as in the D
+/// library and for the same reason: an object whose only text line is
+/// blank is then indistinguishable from an object with no text, in both
+/// libraries, in the same place. A difference that exists in one language
+/// and not the other would be worse than a loss both share.
+pub type Object {
+ Object(
+ ocn: Int,
+ /// one of the sixteen kinds
+ is_a: String,
+ /// the section this object was found in
+ section: String,
+ /// headings: A B C D 1 2 3 4
+ heading_level: String,
+ identifier: String,
+ part: String,
+ /// how the text behaves: para, block, comment
+ is_of_type: String,
+ /// the object's own section, when it differs from the block it sits in
+ own_section: String,
+ segment_html_is: String,
+ segment_epub: String,
+ segment_epub_is: String,
+ table_aligns: String,
+ stow_link: List(String),
+ anchor_tags: List(String),
+ parent_ocn: Int,
+ last_descendant_ocn: Int,
+ ocn_range_first: Int,
+ ocn_range_last: Int,
+ children: List(Int),
+ ancestors: List(Int),
+ dummy_heading: Bool,
+ ocn_off: Bool,
+ indent_base: Int,
+ indent_hang: Int,
+ bullet: Bool,
+ lang: String,
+ has_links: Bool,
+ has_notes_reg: Bool,
+ has_notes_star: Bool,
+ has_images: Bool,
+ segment: String,
+ segment_prev: String,
+ segment_next: String,
+ anchor: String,
+ heading_lev_anchor: String,
+ table_cols: Int,
+ table_widths: String,
+ table_header: Bool,
+ code_syntax: String,
+ code_linenumbers: Bool,
+ heading_ancestors_text: String,
+ sha256: String,
+ images: List(Image),
+ text: String,
+ )
+}
+
+pub fn new_object() -> Object {
+ Object(
+ ocn: 0,
+ is_a: "",
+ section: "",
+ heading_level: "",
+ identifier: "",
+ part: "",
+ is_of_type: "",
+ own_section: "",
+ segment_html_is: "",
+ segment_epub: "",
+ segment_epub_is: "",
+ table_aligns: "",
+ stow_link: [],
+ anchor_tags: [],
+ parent_ocn: 0,
+ last_descendant_ocn: 0,
+ ocn_range_first: 0,
+ ocn_range_last: 0,
+ children: [],
+ ancestors: [],
+ dummy_heading: False,
+ ocn_off: False,
+ indent_base: 0,
+ indent_hang: 0,
+ bullet: False,
+ lang: "",
+ has_links: False,
+ has_notes_reg: False,
+ has_notes_star: False,
+ has_images: False,
+ segment: "",
+ segment_prev: "",
+ segment_next: "",
+ anchor: "",
+ heading_lev_anchor: "",
+ table_cols: 0,
+ table_widths: "",
+ table_header: False,
+ code_syntax: "",
+ code_linenumbers: False,
+ heading_ancestors_text: "",
+ sha256: "",
+ images: [],
+ text: "",
+ )
+}
+
+/// Whether this object is rendered at all.
+///
+/// `is_of_type` "comment" is markup commentary: carried by the abstraction
+/// so that nothing is lost, and not put into output. An output that
+/// renders it leaks the author's notes into the document, which is what
+/// spine itself does not do.
+pub fn renders(object: Object) -> Bool {
+ object.is_of_type != "comment"
+}
+
+/// The section this object belongs to: the block it sits in, unless it
+/// says otherwise.
+pub fn section_of(object: Object) -> String {
+ case object.own_section {
+ "" -> object.section
+ own -> own
+ }
+}
+
+/// What the loader met and did not understand.
+///
+/// A loader that silently drops what it does not know cannot say whether
+/// it understood the document, and so cannot be used to check a
+/// specification against an implementation: silence and understanding look
+/// alike.
+pub type Unknown {
+ Unknown(
+ /// `.name` -> times seen
+ property: Dict(String, Int),
+ /// `@name` -> times seen
+ block: Dict(String, Int),
+ /// first character -> times seen, for lines of no known shape
+ line: Dict(String, Int),
+ )
+}
+
+pub fn new_unknown() -> Unknown {
+ Unknown(property: dict.new(), block: dict.new(), line: dict.new())
+}
+
+pub fn any_unknown(unknown: Unknown) -> Bool {
+ dict.size(unknown.property) > 0
+ || dict.size(unknown.block) > 0
+ || dict.size(unknown.line) > 0
+}
+
+pub type Document {
+ Document(
+ /// as declared on line 1 of a `.ssp` or in `schema.version`, "" when
+ /// absent
+ format_version: String,
+ source_filename: String,
+ /// language, languages, digest
+ source: Dict(String, String),
+ unknown: Unknown,
+ metadata: Dict(String, String),
+ make: Dict(String, String),
+ doc_has: Dict(String, String),
+ /// section name -> objects, in document order
+ sections: Dict(String, List(Object)),
+ )
+}
+
+pub fn new_document() -> Document {
+ Document(
+ format_version: "",
+ source_filename: "",
+ source: dict.new(),
+ unknown: new_unknown(),
+ metadata: dict.new(),
+ make: dict.new(),
+ doc_has: dict.new(),
+ sections: dict.new(),
+ )
+}
+
+/// The major part of the declared version, "" when none was declared.
+pub fn version_major(document: Document) -> String {
+ case string.split_once(document.format_version, ".") {
+ Ok(#(major, _)) -> major
+ Error(_) -> document.format_version
+ }
+}
+
+pub fn meta(document: Document, key: String) -> String {
+ case dict.get(document.metadata, key) {
+ Ok(value) -> value
+ Error(_) -> ""
+ }
+}
+
+/// The objects of a named section, empty when the document has none.
+pub fn section(document: Document, name: String) -> List(Object) {
+ case dict.get(document.sections, name) {
+ Ok(objects) -> objects
+ Error(_) -> []
+ }
+}
+
+/// Is this one of the nine sections of the format?
+pub fn is_section(name: String) -> Bool {
+ list.contains(section_names, name)
+}
+
+/// Is this one of the four preamble blocks?
+pub fn is_preamble_block(name: String) -> Bool {
+ list.contains(preamble_blocks, name)
+}
+#+END_SRC
+
+** inventory gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda/inventory.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<inventory>>
+#+END_SRC
+
+#+NAME: inventory
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// What a source holds, beyond the one document the model is loaded from.
+////
+//// The model is one language of one document. Two questions a reader asks
+//// are about the file instead: which languages it holds, and, for a
+//// database, what else it carries. They are here rather than in a reader
+//// because the answer depends on the input form, which is the library's to
+//// know, and so that a reader asking them does not learn which form it was
+//// handed. A mirror of the D library's `ocda.inventory`.
+////
+//// `languages` answers for every form, and answers the same: the languages
+//// in the order the manifest gave them (which every `.ssp` records, in
+//// `@source languages`), each with the markup file it was made from and the
+//// sha256 of its `.ssp`. The database stores that digest; for a `.ssp` and a
+//// pod it is taken from the bytes, which are the same bytes.
+////
+//// `carried` is the database's: a `.ocda.db` holds the markup, images,
+//// configuration and catalogues it was built from, a `.ssp` holds none of
+//// them, and a pod's are its archive entries rather than anything the
+//// abstraction says.
+
+import gleam/bit_array
+import gleam/dict
+import gleam/list
+import gleam/result
+import gleam/string
+import ocda/load
+import ocda/load_db
+import ocda/load_pod
+import ocda/load_ssp
+import ocda/zip
+import simplifile
+
+pub type Language {
+ Language(lang: String, source_filename: String, ssp_digest: String)
+}
+
+pub type Carried {
+ Carried(
+ role: String,
+ compression: String,
+ count: String,
+ bytes: String,
+ stored: String,
+ )
+}
+
+pub fn languages(path: String) -> Result(List(Language), String) {
+ case load.source_of(path) {
+ load.OcdaDb -> {
+ use held <- result.try(load_db.languages_of_file(path))
+ Ok(
+ list.map(held, fn(l) {
+ Language(l.lang, l.source_filename, l.ssp_digest)
+ }),
+ )
+ }
+ load.Ssp -> {
+ use bytes <- result.try(
+ simplifile.read_bits(path)
+ |> result.map_error(fn(e) { path <> ": " <> simplifile.describe_error(e) }),
+ )
+ use text <- result.try(
+ bit_array.to_string(bytes)
+ |> result.replace_error(path <> ": not valid UTF-8"),
+ )
+ let doc = load_ssp.parse(text)
+ Ok([
+ Language(
+ source_value(doc.source, "language"),
+ doc.source_filename,
+ sha256_hex(bytes),
+ ),
+ ])
+ }
+ load.Pod -> {
+ use archive <- result.try(load_pod.archive_bytes(path))
+ use entries <- result.try(zip.entries(archive))
+ use found <- result.try(
+ zip.find_suffix(entries, ".ssp")
+ |> list.try_map(fn(e) {
+ use bytes <- result.try(zip.read(archive, e))
+ use text <- result.try(
+ bit_array.to_string(bytes)
+ |> result.replace_error(e.name <> ": not valid UTF-8"),
+ )
+ let doc = load_ssp.parse(text)
+ let lang = result.unwrap(load_pod.lang_of(e.name), "")
+ Ok(#(
+ Language(lang, doc.source_filename, sha256_hex(bytes)),
+ string.split(source_value(doc.source, "languages"), " "),
+ ))
+ }),
+ )
+ let order = case found {
+ [#(_, o), ..] -> list.filter(o, fn(l) { l != "" })
+ [] -> []
+ }
+ let by_lang = list.map(found, fn(f) { #({ f.0 }.lang, f.0) }) |> dict.from_list
+ // the manifest's order, and anything it did not name after it
+ let named = list.filter_map(order, fn(l) { dict.get(by_lang, l) })
+ let rest =
+ list.filter(found, fn(f) { !list.contains(order, { f.0 }.lang) })
+ |> list.map(fn(f) { f.0 })
+ Ok(list.append(named, rest))
+ }
+ load.Unknown -> Error(path <> ": " <> load.source_name(load.Unknown))
+ }
+}
+
+/// Only a `.ocda.db` carries files.
+pub fn is_carrier(path: String) -> Bool {
+ load.source_of(path) == load.OcdaDb
+}
+
+pub fn carried(path: String) -> Result(List(Carried), String) {
+ case is_carrier(path) {
+ False -> Ok([])
+ True -> {
+ use rows <- result.try(load_db.carried_of_file(path))
+ Ok(
+ list.filter_map(rows, fn(r) {
+ case r {
+ [role, compression, n, bytes, stored] ->
+ Ok(Carried(role, compression, n, bytes, stored))
+ _ -> Error(Nil)
+ }
+ }),
+ )
+ }
+ }
+}
+
+fn source_value(source: dict.Dict(String, String), key: String) -> String {
+ result.unwrap(dict.get(source, key), "")
+}
+
+@external(erlang, "ocda_ffi", "sha256_hex")
+fn sha256_hex(bytes: BitArray) -> String
+#+END_SRC
+
+* org includes
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* __END__
diff --git a/org/gleam-ocda_lib.org b/org/gleam-ocda_lib.org
new file mode 100644
index 0000000..a359818
--- /dev/null
+++ b/org/gleam-ocda_lib.org
@@ -0,0 +1,381 @@
+-*- mode: org -*-
+#+TITLE: gleam lib
+#+DESCRIPTION: gleam lib
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* gleam :gleam:
+** README.md
+*** tangle
+
+#+HEADER: :tangle ../gleam/README.md
+#+HEADER: :noweb yes
+#+BEGIN_SRC md
+# ocda-lib, in Gleam
+
+The abstraction's model and both its loaders, as a Gleam library, with its
+reader and the two programs that put it to the question.
+
+| | |
+|------------------|-----------------------------------------------------------------------------------------|
+| `ocda/model` | the document, the object, the image. Depends on nothing but the standard library |
+| `ocda/load_ssp` | text -> model |
+| `ocda/load_db` | sqlite -> model |
+| `ocda/load` | what a path is, and loading it |
+| `ocda/inventory` | what a file holds beyond the one document: its languages, and what a `.ocda.db` carries |
+| `ocda/dump` | the model written out canonically, as an oracle |
+| `read/main` | `ocda-read`, the reader |
+| `ocda_ffi.erl` | a sha256, from OTP's crypto, which the standard library lacks |
+
+## Use
+
+```gleam
+import ocda/load
+import ocda/model
+
+case load.load(path, "en", False) {
+ Error(why) -> io.println_error(why)
+ Ok(doc) -> {
+ // and from here a writer neither knows nor cares which of the two
+ // files it was handed
+ list.each(model.section(doc, "body"), fn(obj) { ... })
+ }
+}
+```
+
+## The reader
+
+```sh
+nix build
+./result/bin/ocda-read [--tree|--outline|--languages|--files|--report-unknown]
+ <file>.ssp|<file>.ocda.db|<file>.zip [lang]
+```
+
+`ocda-lib read`, installed also as `ocda-read`. The modes the Gleam readers
+had, over the library, so each reads every form this library reads; the
+others are the D reader's. **Its output is the D `ocda-read`'s, byte for
+byte**, in every mode both have. In `src/read/`, because Gleam compiles
+nothing outside `src/`; a program depending on the library compiles it too,
+and does not use it.
+
+A `.sisupod` it refuses, saying why: the zstd frame, below.
+
+## The two programs
+
+```sh
+nix build
+./result/bin/ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
+./result/bin/ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
+```
+
+`agree` asks whether the two loaders give the same model. `dump` writes
+the whole model out canonically, and is the stronger check of the two,
+because it is **byte for byte what the D library's `ocda-dump` writes**.
+So the four questions can all be asked with `diff`:
+
+| | |
+|-----------------------------|-----------------------------|
+| D: `.ssp` == `.ocda.db` | 36 of 36 document-languages |
+| Gleam: `.ssp` == `.ocda.db` | 36 of 36 |
+| `.ssp`: D == Gleam | 36 of 36 |
+| `.ocda.db`: D == Gleam | 36 of 36 |
+
+## Deliberately a mirror of the D library
+
+Field for field, name for name, including where that is not the most
+Gleamish choice:
+
+- `text` is one `String` with the lines joined by newline, not a
+ `List(String)`. So an object whose only text line is blank is
+ indistinguishable from an object with no text, *in both languages, in the
+ same place*. A difference that existed in one and not the other would be
+ worse than a loss both share, because it would show up in every dump and
+ mean nothing.
+- the properties are typed fields rather than a list of pairs. A writer
+ wants `obj.parent_ocn`, not a lookup that might miss.
+
+The point is that the two agree. Making them agree is much cheaper when
+they are shaped alike, and the shape is the specification's, not either
+language's.
+
+## What the agreement found
+
+Four disagreements, none of which a comparison of section counts or of a
+handful of fields would have shown. All four are recorded in the D
+library's history; the short version:
+
+1. `.identifier` and `.anchor` are omitted by the text form when they can
+ be worked out (the identifier is the ocn, the anchor is the segment).
+ Both loaders now derive them. Over one document that is 1,314
+ identifiers.
+2. `make` and `doc_has` are one metadata table in the database,
+ distinguished by a key prefix. Both loaders unprefix them.
+3. The JSON columns must be *parsed*, not scanned: the encoder writes `/`
+ as `\/` and every heading with a slash in it came back wrong.
+4. `translation.ocn_aligned` is in the database and cannot be in a `.ssp`,
+ which is one language. It is a fact about the file, so neither loader
+ puts it in the document.
+
+## Build
+
+```sh
+nix build # sandboxed, no network
+nix develop -c gleam test
+```
+
+Shipped as an Erlang shipment rather than an escript, because sqlite on the
+BEAM is a NIF and a NIF cannot be loaded from inside an escript archive.
+The dependency hash in `flake.nix` pins what `manifest.toml` pins; change
+one and the other must change with it.
+
+## Format
+
+Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model,
+`ssp-format-specification.md` and `ocda-db-format-specification.md` for the
+two serialisations. At format **2.x**.
+#+END_SRC
+
+** gleam.toml
+*** tangle
+
+#+HEADER: :tangle ../gleam/gleam.toml
+#+HEADER: :noweb yes
+#+BEGIN_SRC toml
+name = "ocda_lib"
+version = "0.1.0"
+description = "the object-centric document abstraction as a library: the model, and loaders for its two serialisations"
+licences = ["AGPL-3.0-or-later"]
+
+[dependencies]
+gleam_stdlib = ">= 1.0.0 and < 2.0.0"
+argv = ">= 1.1.0 and < 2.0.0"
+simplifile = ">= 2.7.0 and < 3.0.0"
+sqlight = ">= 1.0.0 and < 2.0.0"
+gleam_json = ">= 3.0.0 and < 4.0.0"
+
+[dev-dependencies]
+gleeunit = ">= 1.0.0 and < 2.0.0"
+#+END_SRC
+
+** gleam ocda_lib
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda_lib.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<ocda_lib>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: ocda_lib
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// ocda-lib, in Gleam: the abstraction as a library, put to the question.
+////
+//// A library has no binary, so this is the programs over it: the reader,
+//// and the two that test its claims.
+////
+//// ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
+//// ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
+//// ocda-lib read [options] <file> [lang] (also installed as ocda-read)
+////
+//// `dump` writes the whole loaded model out canonically, byte for byte as
+//// the D library's `ocda-dump` writes it, so that the two implementations
+//// can be diffed. `agree` asks the same question the D library's
+//// `ocda-agree` asks, within this language: does loading a document's
+//// `.ssp` give what loading its `.ocda.db` gives?
+
+import argv
+import gleam/int
+import gleam/io
+import gleam/list
+import gleam/string
+import ocda/dump
+import ocda/load
+import ocda/model.{type Document}
+import read/main as reader
+
+pub fn main() {
+ case argv.load().arguments {
+ ["dump", "--unknown", path] -> dump_file(path, "", True)
+ ["dump", "--unknown", path, lang] -> dump_file(path, lang, True)
+ ["dump", path] -> dump_file(path, "", False)
+ ["dump", path, lang] -> dump_file(path, lang, False)
+ ["agree", ssp, db] -> agree(ssp, db, "")
+ ["agree", ssp, db, lang] -> agree(ssp, db, lang)
+ ["read", ..rest] ->
+ case reader.run(rest) {
+ 0 -> Nil
+ code -> halt(code)
+ }
+ _ -> usage()
+ }
+}
+
+fn usage() -> Nil {
+ io.println_error(
+ "usage: ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
+ ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
+ ocda-lib read [--help] [options] <file> [lang]",
+ )
+ halt(2)
+}
+
+fn dump_file(path: String, lang: String, unknown: Bool) -> Nil {
+ case load.load(path, lang, False) {
+ Error(message) -> fail(message)
+ Ok(doc) ->
+ case unknown {
+ True -> io.print(dump.unknown(doc))
+ False -> io.print(dump.document(doc))
+ }
+ }
+}
+
+/// The library's whole purpose is that a writer cannot tell which
+/// serialisation it was handed. That is only true if the two loaders give
+/// the same model, so this asks, and says where they part company rather
+/// than only that they did.
+fn agree(ssp_path: String, db_path: String, lang: String) -> Nil {
+ case load.load(ssp_path, "", False), load.load(db_path, lang, False) {
+ Error(message), _ -> fail(message)
+ _, Error(message) -> fail(message)
+ Ok(from_ssp), Ok(from_db) -> {
+ let differences = compare(from_ssp, from_db)
+ case differences {
+ [] -> io.println("agree: the two loaders give the same model")
+ _ -> {
+ list.each(list.take(differences, 12), io.println)
+ case list.length(differences) > 12 {
+ True -> io.println(" ... stopping after 12")
+ False -> Nil
+ }
+ io.println(int.to_string(list.length(differences)) <> " difference(s)")
+ halt(1)
+ }
+ }
+ }
+ }
+}
+
+/// Compared through the dump rather than field by field, which is both
+/// shorter and stricter: the dump carries every field, so nothing can
+/// differ without showing here.
+fn compare(from_ssp: Document, from_db: Document) -> List(String) {
+ let a = dump.document(from_ssp)
+ let b = dump.document(from_db)
+ case a == b {
+ True -> []
+ False -> diff_lines(a, b)
+ }
+}
+
+fn diff_lines(a: String, b: String) -> List(String) {
+ let la = split_lines(a)
+ let lb = split_lines(b)
+ case list.length(la) == list.length(lb) {
+ False -> [
+ " differ: line count",
+ " .ssp: " <> int.to_string(list.length(la)),
+ " .ocda.db: " <> int.to_string(list.length(lb)),
+ ..pairwise(la, lb)
+ ]
+ True -> pairwise(la, lb)
+ }
+}
+
+fn pairwise(la: List(String), lb: List(String)) -> List(String) {
+ list.zip(la, lb)
+ |> list.index_map(fn(pair, i) { #(i, pair.0, pair.1) })
+ |> list.filter(fn(t) { t.1 != t.2 })
+ |> list.map(fn(t) {
+ " differ: line "
+ <> int.to_string(t.0 + 1)
+ <> "\n .ssp: "
+ <> t.1
+ <> "\n .ocda.db: "
+ <> t.2
+ })
+}
+
+fn split_lines(text: String) -> List(String) {
+ case text {
+ "" -> []
+ _ -> string.split(text, on: "\n")
+ }
+}
+
+fn fail(message: String) -> Nil {
+ io.println_error("error: " <> message)
+ halt(2)
+}
+
+/// Gleam has no standard exit, and the exit status is what a sweep script
+/// reads.
+@external(erlang, "erlang", "halt")
+fn halt(code: Int) -> Nil
+#+END_SRC
+
+** erlang
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/ocda_ffi.erl
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+%%<<doc_header_including_copyright_and_license>>
+<<erlang_ffi>>
+#+END_SRC
+
+*** erlang source
+
+#+NAME: erlang_ffi
+#+HEADER: :no-expand
+#+BEGIN_SRC erlang
+%% The one thing ocda-lib asks of Erlang directly: a sha256, which OTP's
+%% crypto has and the Gleam standard library does not. Upper case hex, as
+%% the database stores it and the D library writes it.
+-module(ocda_ffi).
+-export([sha256_hex/1]).
+
+sha256_hex(Bin) -> binary:encode_hex(crypto:hash(sha256, Bin)).
+#+END_SRC
+
+* org includes
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* __END__
diff --git a/org/gleam-read-main.org b/org/gleam-read-main.org
new file mode 100644
index 0000000..f968fd1
--- /dev/null
+++ b/org/gleam-read-main.org
@@ -0,0 +1,520 @@
+-*- mode: org -*-
+#+TITLE: gleam lib
+#+DESCRIPTION: gleam lib
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* gleam :gleam:
+** gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/src/read/main.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<read>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: read
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+//// ocda-read, in Gleam: reads the abstraction, in any of the forms the
+//// library reads, and says what is in it.
+////
+//// ocda-read [--tree|--outline|--languages|--files|--report-unknown]
+//// <file>.ssp | <file>.ocda.db | <file>.zip [lang]
+////
+//// The modes the Gleam readers had (the summary, the outline, the unknown
+//// report, and the database's languages and files), over the library, so
+//// that each reads every form. Their output is the D ocda-read's, byte for
+//// byte: two readers in unrelated languages, neither written from the other
+//// line by line, agreeing is what says each has read the model and not its
+//// own idea of it. A `.sisupod` is the one form the Gleam library cannot
+//// open (its zstd frame; see `ocda/load_pod`), and it says so.
+
+import gleam/dict
+import gleam/int
+import gleam/io
+import gleam/list
+import gleam/result
+import gleam/string
+import ocda/inventory.{type Language}
+import ocda/load
+import ocda/model.{type Document, type Object}
+
+/// The format major this reader knows.
+const major_read = "2"
+
+type Options {
+ Options(
+ tree: Bool,
+ languages: Bool,
+ files: Bool,
+ report_unknown: Bool,
+ any_version: Bool,
+ lang: String,
+ positional: List(String),
+ )
+}
+
+/// Run with the arguments after the program name; returns the exit status.
+pub fn run(args: List(String)) -> Int {
+ case parse(args, Options(False, False, False, False, False, "", [])) {
+ Error(message) -> {
+ io.println_error("ocda-read: " <> message <> "; --help lists the options")
+ 2
+ }
+ Ok(Options(positional: [], ..)) -> {
+ help()
+ 0
+ }
+ Ok(options) -> read(options)
+ }
+}
+
+fn parse(args: List(String), o: Options) -> Result(Options, String) {
+ case args {
+ [] -> Ok(Options(..o, positional: list.reverse(o.positional)))
+ ["--help", ..] | ["-h", ..] -> Ok(Options(..o, positional: []))
+ ["--tree", ..rest] | ["--outline", ..rest] ->
+ parse(rest, Options(..o, tree: True))
+ ["--languages", ..rest] -> parse(rest, Options(..o, languages: True))
+ ["--files", ..rest] -> parse(rest, Options(..o, files: True))
+ ["--report-unknown", ..rest] ->
+ parse(rest, Options(..o, report_unknown: True))
+ ["--any-version", ..rest] -> parse(rest, Options(..o, any_version: True))
+ ["--lang", code, ..rest] -> parse(rest, Options(..o, lang: code))
+ [flag, ..rest] ->
+ case string.starts_with(flag, "--lang=") {
+ True -> parse(rest, Options(..o, lang: string.drop_start(flag, 7)))
+ False ->
+ case string.starts_with(flag, "-") {
+ True ->
+ case
+ list.contains(
+ [
+ "--html", "--llm-text", "--llm-jsonl", "--images", "--verify",
+ "--validate",
+ ],
+ flag,
+ )
+ {
+ True ->
+ Error(flag <> " is a mode of the D ocda-read, not of this one")
+ False -> Error("Unrecognized option " <> flag)
+ }
+ False ->
+ parse(rest, Options(..o, positional: [flag, ..o.positional]))
+ }
+ }
+ }
+}
+
+fn help() -> Nil {
+ io.println("ocda-read - reads the document abstraction, in any of its forms")
+ io.println("")
+ io.println(
+ "Usage: ocda-read [options] <file>.ssp|<file>.ocda.db|<file>.zip [lang]",
+ )
+ io.println("")
+ io.println(" --tree the headings, as a tree")
+ io.println(" --outline the same as --tree")
+ io.println(" --languages the languages the file holds")
+ io.println(" --files what a .ocda.db carries besides the abstraction")
+ io.println(" --report-unknown what was read and not understood")
+ io.println(" --lang=<code> the language to read (default: the first)")
+ io.println(" --any-version read a format major this reader does not know")
+}
+
+fn read(o: Options) -> Int {
+ let assert [path, ..more] = o.positional
+ let lang = case o.lang, more {
+ "", [given, ..] -> given
+ l, _ -> l
+ }
+ let lang_named = lang != ""
+ let loaded = {
+ use held <- result.try(inventory.languages(path))
+ let lang = case lang, held {
+ "", [first, ..] -> first.lang
+ l, _ -> l
+ }
+ use _ <- result.try(case
+ held != [] && !list.any(held, fn(l) { l.lang == lang })
+ {
+ True ->
+ Error(
+ "no language '"
+ <> lang
+ <> "' in "
+ <> path
+ <> "; it holds: "
+ <> string.join(list.map(held, fn(l) { l.lang }), " "),
+ )
+ False -> Ok(Nil)
+ })
+ use doc <- result.try(load.load(path, lang, o.any_version))
+ Ok(#(held, lang, doc))
+ }
+ case loaded {
+ Error(message) -> {
+ io.println_error("Error reading " <> path <> ": " <> message)
+ 2
+ }
+ Ok(#(held, lang, doc)) ->
+ case gate(path, doc, o.any_version) {
+ False -> 3
+ True -> {
+ summary(path, doc, held, lang, lang_named)
+ case o.languages {
+ True -> print_languages(held)
+ False -> Nil
+ }
+ case o.files {
+ True -> print_files(path)
+ False -> Nil
+ }
+ case o.tree {
+ True -> tree(doc)
+ False -> Nil
+ }
+ case o.report_unknown {
+ True -> report_unknown(doc)
+ False -> Nil
+ }
+ case o.tree || o.languages || o.files || o.report_unknown {
+ True -> Nil
+ False ->
+ io.println(
+ "\n(this is the summary: --help lists the modes, each adding its output)",
+ )
+ }
+ 0
+ }
+ }
+ }
+}
+
+/// The major part, and refuse what is not known.
+fn gate(path: String, doc: Document, any_version: Bool) -> Bool {
+ case doc.format_version, model.version_major(doc) == major_read, any_version {
+ "", _, _ -> {
+ io.println_error("warning: " <> path <> " declares no format version")
+ True
+ }
+ _, True, _ -> True
+ v, False, True -> {
+ io.println_error(
+ "warning: "
+ <> path
+ <> " is format v"
+ <> v
+ <> ", this reader knows v"
+ <> major_read
+ <> ".x; reading anyway",
+ )
+ True
+ }
+ v, False, False -> {
+ io.println_error(
+ "error: "
+ <> path
+ <> " is format v"
+ <> v
+ <> ", this reader knows v"
+ <> major_read
+ <> ".x",
+ )
+ io.println_error(
+ " a major version differs: a property may have changed meaning or gone away.",
+ )
+ io.println_error(" --any-version reads it regardless.")
+ False
+ }
+ }
+}
+
+const sections = [
+ "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex",
+ "blurb", "tail",
+]
+
+/// The sections whose images are counted: every one but the tail, as the D
+/// reader counts them.
+const image_sections = [
+ "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex",
+ "blurb",
+]
+
+fn summary(
+ path: String,
+ doc: Document,
+ held: List(Language),
+ lang: String,
+ lang_named: Bool,
+) -> Nil {
+ io.println("Read: " <> path)
+ case doc.format_version {
+ "" -> Nil
+ v -> io.println(" Format: v" <> v)
+ }
+ case dict.get(doc.source, "language") {
+ Error(_) -> Nil
+ Ok(language) -> {
+ let of = case dict.get(doc.source, "languages") {
+ Ok(ls) -> " of " <> ls
+ Error(_) -> ""
+ }
+ let digest = case dict.get(doc.source, "digest") {
+ Ok(d) -> ", markup digest " <> string.slice(d, 0, 16) <> "..."
+ Error(_) -> ""
+ }
+ io.println(" Source: " <> language <> of <> digest)
+ }
+ }
+ case list.length(held) > 1 && !lang_named {
+ True -> io.println(" (reading " <> lang <> "; --lang=<code> for another)")
+ False -> Nil
+ }
+ io.println(" Title: " <> model.meta(doc, "title.main"))
+ io.println(" Author: " <> model.meta(doc, "creator.author"))
+ io.println(
+ " Sections: "
+ <> string.join(
+ list.map(sections, fn(s) {
+ s <> "=" <> int.to_string(list.length(model.section(doc, s)))
+ }),
+ " ",
+ ),
+ )
+ let records =
+ list.fold(image_sections, 0, fn(n, s) {
+ list.fold(model.section(doc, s), n, fn(n, o) {
+ n + list.length(o.images)
+ })
+ })
+ case records {
+ 0 -> Nil
+ 1 -> io.println(" Images: 1 record")
+ n -> io.println(" Images: " <> int.to_string(n) <> " records")
+ }
+}
+
+fn print_languages(held: List(Language)) -> Nil {
+ io.println("")
+ io.println(" lang source ssp digest")
+ list.each(held, fn(l) {
+ let digest = case string.length(l.ssp_digest) > 16 {
+ True -> string.slice(l.ssp_digest, 0, 16) <> "..."
+ False -> l.ssp_digest
+ }
+ io.println(
+ " "
+ <> string.pad_end(l.lang, 7, " ")
+ <> " "
+ <> string.pad_end(l.source_filename, 31, " ")
+ <> " "
+ <> digest,
+ )
+ })
+}
+
+fn print_files(path: String) -> Nil {
+ io.println("")
+ case inventory.is_carrier(path) {
+ False ->
+ io.println(
+ " carried files: only a .ocda.db carries files; this is "
+ <> load.source_name(load.source_of(path)),
+ )
+ True ->
+ case inventory.carried(path) {
+ Error(message) -> io.println_error("error: " <> message)
+ Ok([]) -> io.println(" carries no files")
+ Ok(rows) -> {
+ io.println(" carried files:")
+ io.println(" role stored n original in the file")
+ list.each(rows, fn(r) {
+ io.println(
+ " "
+ <> string.pad_end(r.role, 9, " ")
+ <> " "
+ <> string.pad_end(r.compression, 7, " ")
+ <> " "
+ <> string.pad_end(r.count, 5, " ")
+ <> " "
+ <> string.pad_end(r.bytes, 13, " ")
+ <> " "
+ <> r.stored,
+ )
+ })
+ }
+ }
+ }
+}
+
+/// The headings, as a tree: from the document's root heading, down the
+/// children each heading lists, as the D reader walks it.
+fn tree(doc: Document) -> Nil {
+ io.println("Document: " <> model.meta(doc, "title.main"))
+ case model.meta(doc, "title.subtitle") {
+ "" -> Nil
+ sub -> io.println(" " <> sub)
+ }
+ io.println("Author: " <> model.meta(doc, "creator.author"))
+ io.println("")
+ let headings =
+ model.section(doc, "body")
+ |> list.filter(fn(o) { o.is_a == "heading" && o.ocn > 0 })
+ |> list.map(fn(o) { #(o.ocn, o) })
+ |> dict.from_list
+ case model.section(doc, "head") {
+ [root, ..] if root.is_a == "heading" -> {
+ io.println(heading_line(root, 0))
+ walk(root, headings, 1)
+ }
+ _ -> Nil
+ }
+}
+
+fn walk(parent: Object, headings: dict.Dict(Int, Object), depth: Int) -> Nil {
+ list.each(parent.children, fn(ocn) {
+ case dict.get(headings, ocn) {
+ Ok(child) -> {
+ io.println(heading_line(child, depth))
+ walk(child, headings, depth + 1)
+ }
+ Error(_) -> Nil
+ }
+ })
+}
+
+fn heading_line(o: Object, depth: Int) -> String {
+ let level = case o.heading_level {
+ "" -> "?"
+ l -> l
+ }
+ let ocn = case o.ocn > 0 {
+ True -> " [" <> int.to_string(o.ocn) <> "]"
+ False -> ""
+ }
+ string.repeat(" ", depth) <> level <> "~" <> ocn <> " " <> strip_markup(o.text)
+}
+
+/// The internal markers a heading's text may carry, removed for display, as
+/// the D reader removes them.
+fn strip_markup(text: String) -> String {
+ text
+ |> string.replace("⑆*", "")
+ |> string.replace("⑆!", "")
+ |> string.replace("⑆/", "")
+ |> string.replace("⑆_", "")
+ |> string.replace("⑆^", "")
+ |> string.replace("⑆,", "")
+ |> string.replace("┨", "")
+ |> string.replace("┣", "")
+ |> string.replace("┙", " ")
+ |> string.replace("┘", "\n")
+}
+
+/// Say what was read and not understood.
+fn report_unknown(doc: Document) -> Nil {
+ let u = doc.unknown
+ case model.any_unknown(u) {
+ False ->
+ io.println(
+ "\nread and understood: every property, block and line was known",
+ )
+ True -> {
+ io.println("\nread and NOT understood:")
+ section_of_unknown(u.property, " properties ignored (", fn(name, n) {
+ " ."
+ <> string.pad_end(name, 26, " ")
+ <> " "
+ <> string.pad_start(int.to_string(n), 8, " ")
+ })
+ section_of_unknown(u.block, " blocks ignored (", fn(name, n) {
+ " @"
+ <> string.pad_end(name, 26, " ")
+ <> " "
+ <> string.pad_start(int.to_string(n), 8, " ")
+ })
+ case dict.size(u.line) {
+ 0 -> Nil
+ _ -> {
+ io.println(" lines of no known shape, by first character:")
+ dict.to_list(u.line)
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.each(fn(e) {
+ io.println(
+ " "
+ <> string.pad_end("\"" <> e.0 <> "\"", 27, " ")
+ <> " "
+ <> string.pad_start(int.to_string(e.1), 8, " "),
+ )
+ })
+ }
+ }
+ io.println(
+ " (ignoring these is conforming within a major version; the count is so that it is not silent)",
+ )
+ }
+ }
+}
+
+fn section_of_unknown(
+ counts: dict.Dict(String, Int),
+ heading: String,
+ line: fn(String, Int) -> String,
+) -> Nil {
+ case dict.size(counts) {
+ 0 -> Nil
+ n -> {
+ io.println(heading <> int.to_string(n) <> " distinct):")
+ dict.to_list(counts)
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.each(fn(e) { io.println(line(e.0, e.1)) })
+ }
+ }
+}
+#+END_SRC
+
+* org includes
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* __END__
diff --git a/org/gleam-test.org b/org/gleam-test.org
new file mode 100644
index 0000000..402c6e0
--- /dev/null
+++ b/org/gleam-test.org
@@ -0,0 +1,209 @@
+-*- mode: org -*-
+#+TITLE: gleam lib
+#+DESCRIPTION: gleam lib
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+* gleam :gleam:
+** gleam
+*** tangle
+
+#+HEADER: :tangle ../gleam/test/ocda_lib_test.gleam
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+////<<doc_header_including_copyright_and_license>>
+<<test>>
+#+END_SRC
+
+*** gleam source
+
+#+NAME: test
+#+HEADER: :no-expand
+#+BEGIN_SRC gleam
+import gleam/dict
+import gleam/list
+import simplifile
+import gleam/string
+import gleeunit
+import gleeunit/should
+import ocda/dump
+import ocda/load
+import ocda/load_ssp
+import ocda/model
+
+pub fn main() {
+ gleeunit.main()
+}
+
+const minimal = "% SiSU Document Abstraction v2.0
+% Source: example.sst
+
+@source {
+ language: en
+ languages: en
+ digest: ABC
+}
+
+@meta {
+ title.main: An Example
+}
+
+@body {
+[1] heading :A the_title
+.part: frontmatter
+.sha256: DEAD
+| An Example
+
+[2] para
+.part: body
+.parent: 1
+| First line
+|
+| Third line, after a blank one
+
+}
+"
+
+pub fn version_test() {
+ let doc = load_ssp.parse(minimal)
+ doc.format_version |> should.equal("2.0")
+ model.version_major(doc) |> should.equal("2")
+ doc.source_filename |> should.equal("example.sst")
+}
+
+pub fn preamble_test() {
+ let doc = load_ssp.parse(minimal)
+ model.meta(doc, "title.main") |> should.equal("An Example")
+}
+
+pub fn objects_test() {
+ let doc = load_ssp.parse(minimal)
+ let body = model.section(doc, "body")
+ body |> list.length |> should.equal(2)
+}
+
+/// The blank line in the middle of an object's text is kept, and so is the
+/// count: three lines, not two.
+pub fn blank_text_line_test() {
+ let doc = load_ssp.parse(minimal)
+ case model.section(doc, "body") {
+ [_, second] ->
+ second.text
+ |> should.equal("First line\n\nThird line, after a blank one")
+ _ -> should.fail()
+ }
+}
+
+/// A heading's identifier comes from its declaration line; an object whose
+/// identifier is simply its ocn has it derived, because the text form does
+/// not write what it can work out.
+pub fn derived_identifier_test() {
+ let doc = load_ssp.parse(minimal)
+ case model.section(doc, "body") {
+ [first, second] -> {
+ first.identifier |> should.equal("the_title")
+ second.identifier |> should.equal("2")
+ }
+ _ -> should.fail()
+ }
+}
+
+pub fn source_of_test() {
+ load.source_of("a/b.ssp") |> should.equal(load.Ssp)
+ load.source_of("a/b.ocda.db") |> should.equal(load.OcdaDb)
+ load.source_of("a/b.sst") |> should.equal(load.Unknown)
+}
+
+/// The dump omits a field at its default, so a minimal document gives a
+/// short dump and every line in it is a fact.
+pub fn dump_test() {
+ let text = dump.document(load_ssp.parse(minimal))
+ text |> string.contains(contain: "version 2.0") |> should.be_true
+ text |> string.contains(contain: "meta title.main An Example") |> should.be_true
+ text |> string.contains(contain: "object body 0") |> should.be_true
+ text |> string.contains(contain: "level A") |> should.be_true
+ // a false boolean is not written at all
+ text |> string.contains(contain: "bullet") |> should.be_false
+}
+
+
+/// A property is read, which the fixture above had to be corrected to show:
+/// a property line is `.name: value` at the start of the line, and an
+/// indented one is a line of no known shape. The fixture indented them, so
+/// every property in it was being counted as malformed and the tests around
+/// it passed for the wrong reason.
+pub fn property_test() {
+ let doc = load_ssp.parse(minimal)
+ case model.section(doc, "body") {
+ [first, second] -> {
+ first.part |> should.equal("frontmatter")
+ first.sha256 |> should.equal("DEAD")
+ second.parent_ocn |> should.equal(1)
+ }
+ _ -> should.fail()
+ }
+ // and nothing in the fixture is malformed any more
+ dict_size(doc.unknown.line) |> should.equal(0)
+}
+
+@external(erlang, "maps", "size")
+fn dict_size(d: dict.Dict(a, b)) -> Int
+
+/// What a path is, including the two archive forms.
+pub fn pod_source_test() {
+ load.source_of("a/b.sisupod") |> should.equal(load.Pod)
+ load.source_of("a/b.zip") |> should.equal(load.Pod)
+}
+
+/// A zstd-wrapped archive is refused, and the refusal says why rather than
+/// claiming the file is malformed. This is the one place the D and Gleam
+/// libraries differ in what they accept, and it is a packaging limit: the
+/// only zstd on the BEAM is a NIF that fetches zstd in its own build.
+pub fn zstd_refusal_test() {
+ let path = "test/fixture-zstd-frame.sisupod"
+ let assert Ok(_) = simplifile.write_bits(path, <<0x28, 0xB5, 0x2F, 0xFD, 0>>)
+ let assert Error(message) = load.load(path, "", False)
+ message |> string.contains(contain: "zstd") |> should.be_true
+ message |> string.contains(contain: "zstd -d") |> should.be_true
+ let _ = simplifile.delete(path)
+ Nil
+}
+#+END_SRC
+
+* org includes
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+** document header including copyright & license
+
+#+NAME: doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_doc_header_including_copyright_and_license()>>
+#+END_SRC
+
+* __END__
diff --git a/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org b/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
new file mode 100644
index 0000000..dbd4bab
--- /dev/null
+++ b/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
@@ -0,0 +1,181 @@
+-*- mode: org -*-
+#+TITLE: sisudoc spine (doc_reform) object-centric document abstraction
+#+DESCRIPTION: documents - structuring, publishing in multiple formats & search
+#+FILETAGS: :spine:abstraction:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2015 (continuously updated, current 2026) Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+- magic single double-quote → " ← FIX changes hilighting behavior (occuring
+ after it) in org document. INVESTIGATE (org-mode CONFIG?) FIND & FIX
+
+- [[./doc-reform.org][doc-reform.org]] [[./][org/]]
+
+* spine doc header including copyright & license
+
+#+NAME: spine_doc_header_including_copyright_and_license
+#+BEGIN_SRC org
+/+
+- Name: SisuDoc Spine, Doc Reform [a part of]
+ - Description: documents, structuring, processing, publishing, search
+ - static content generator
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2015 (continuously updated, current 2026) Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later:
+
+ Spine (SiSU), a framework for document structuring, publishing and
+ search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU AFERO General Public License as published by the
+ Free Software Foundation, either version 3 of the License, or (at your
+ option) any later version.
+
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ more details.
+
+ You should have received a copy of the GNU General Public License along with
+ this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at these locations:
+ [https://www.fsf.org/licensing/licenses/agpl.html]
+ [https://www.gnu.org/licenses/agpl.html]
+
+ - Spine (by Doc Reform, related to SiSU) uses standard:
+ - docReform markup syntax
+ - standard SiSU markup syntax with modified headers and minor modifications
+ - docReform object numbering
+ - standard SiSU object citation numbering & system
+
+ - Homepages:
+ [https://www.sisudoc.org]
+ [https://www.doc-reform.org]
+
+ - Git
+ [https://git.sisudoc.org/]
+
++/
+#+END_SRC
+
+* spine version info SET VERSION
+** version in full
+
+#+NAME: spine_project_version
+#+BEGIN_SRC org
+<<spine_project_version_part_major>>.<<spine_project_version_part_minor>>.<<spine_project_version_part_patch>>
+#+END_SRC
+
+** version parts (major, minor, patch)
+
+#+NAME: spine_project_version_major_minor_patch
+#+BEGIN_SRC org
+<<spine_project_version_part_major>>, <<spine_project_version_part_minor>>, <<spine_project_version_part_patch>>
+#+END_SRC
+
+*** major
+
+#+NAME: spine_project_version_part_major
+#+BEGIN_SRC org
+0
+#+END_SRC
+
+*** minor
+
+#+NAME: spine_project_version_part_minor
+#+BEGIN_SRC org
+25
+#+END_SRC
+
+*** patch
+
+#+NAME: spine_project_version_part_patch
+#+BEGIN_SRC org
+1
+#+END_SRC
+
+* year
+
+#+NAME: year
+#+BEGIN_SRC org
+2026
+#+END_SRC
+
+* sisudoc spine git source SET
+** sisudoc-spine src
+
+#+NAME: sisudoc_spine_nix_src
+#+BEGIN_SRC org
+git://git.sisudoc.org/software/sisudoc-spine
+#+END_SRC
+
+#+BEGIN_SRC org
+git+file:///home/ralph/grotto/repo/git.repo/projects/doc-reform/code/software/sisudoc-spine
+#+END_SRC
+
+** dlang overlay src
+
+#+NAME: sisudoc_dlang_nix_overlay_src
+#+BEGIN_SRC org
+git://git.sisudoc.org/package/dlang-nix-overlays-and-flake
+#+END_SRC
+
+#+BEGIN_SRC org
+git+file:///home/ralph/grotto/repo/git.repo/projects/doc-reform/packages/nix-flakes/dlang/dlang-nix-flakes"
+#+END_SRC
+
+* ocda ssp version format
+
+#+NAME: ocda_ssp_format_version
+#+BEGIN_SRC org
+<<ocda_ssp_format_version_part_major>>.<<ocda_ssp_format_version_part_minor>>
+#+END_SRC
+
+*** major
+
+#+NAME: ocda_ssp_format_version_part_major
+#+BEGIN_SRC org
+2
+#+END_SRC
+
+*** minor
+
+#+NAME: ocda_ssp_format_version_part_minor
+#+BEGIN_SRC org
+0
+#+END_SRC
+
+* sqlite collection search db schema version
+
+#+NAME: sqlite_collection_search_db_schema_version
+#+BEGIN_SRC org
+1
+#+END_SRC
+
+* nipkgs source
+
+- github:NixOS/nixpkgs/nixpkgs-unstable
+- https://channels.nixos.org/nixpkgs-unstable/nixexprs.tar.zst
+
+#+NAME: nixpkgs_unstable
+#+BEGIN_SRC org
+https://channels.nixos.org/nixpkgs-unstable/nixexprs.tar.zst
+#+END_SRC
+
+#+BEGIN_SRC org
+github:NixOS/nixpkgs/nixpkgs-unstable
+#+END_SRC