summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
-rw-r--r--.gitignore76
-rw-r--r--COPYRIGHT45
-rw-r--r--LICENSE660
-rw-r--r--README.md150
-rw-r--r--dlang/COPYRIGHT45
-rw-r--r--dlang/README.md62
-rw-r--r--dlang/dub.json13
-rw-r--r--dlang/dub.selections.json1
-rw-r--r--dlang/flake.lock27
-rw-r--r--dlang/flake.nix63
-rw-r--r--dlang/src/COPYRIGHT45
-rw-r--r--dlang/src/ocda/dump.d164
-rw-r--r--dlang/src/ocda/images.d100
-rw-r--r--dlang/src/ocda/load.d69
-rw-r--r--dlang/src/ocda/load_db.d310
-rw-r--r--dlang/src/ocda/load_pod.d115
-rw-r--r--dlang/src/ocda/load_ssp.d337
-rw-r--r--dlang/src/ocda/model.d147
-rw-r--r--dlang/src/ocda/sqlite.d122
-rw-r--r--dlang/src/ocda/zip.d115
-rw-r--r--dlang/src/ocda/zstd.d79
-rw-r--r--dlang/test/agree.d84
-rw-r--r--dlang/test/dump.d35
-rw-r--r--gleam/COPYRIGHT48
-rw-r--r--gleam/README.md102
-rw-r--r--gleam/flake.lock27
-rw-r--r--gleam/flake.nix66
-rw-r--r--gleam/gleam.toml14
-rw-r--r--gleam/manifest.toml26
-rw-r--r--gleam/nix-gleam/LICENSE21
-rw-r--r--gleam/nix-gleam/README.md32
-rw-r--r--gleam/nix-gleam/default.nix280
-rw-r--r--gleam/src/COPYRIGHT45
-rw-r--r--gleam/src/ocda/dump.gleam247
-rw-r--r--gleam/src/ocda/images.gleam135
-rw-r--r--gleam/src/ocda/load.gleam79
-rw-r--r--gleam/src/ocda/load_db.gleam532
-rw-r--r--gleam/src/ocda/load_pod.gleam138
-rw-r--r--gleam/src/ocda/load_ssp.gleam478
-rw-r--r--gleam/src/ocda/model.gleam278
-rw-r--r--gleam/src/ocda/zip.gleam170
-rw-r--r--gleam/src/ocda_lib.gleam136
-rw-r--r--gleam/test/ocda_lib_test.gleam148
-rw-r--r--org/COPYRIGHT19
-rw-r--r--org/config-lib.org279
-rw-r--r--org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org972
46 files changed, 7136 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..12a3507
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,76 @@
+# Deny everything, then whitelist what belongs in the repository.
+#
+# The practice is deliberate: a repository should hold what was decided to
+# put in it, not whatever happened to be in the directory when someone ran
+# "git add .". Build output, editor droppings, downloaded dependencies and
+# exported binaries are numerous, are regenerated, and differ between
+# machines; source is few and is chosen. So the default is no, and every
+# yes below is a decision that can be read.
+#
+# To see what is being excluded, and why:
+# git status --ignored
+# git check-ignore -v <path>
+# git ls-files --others --ignored --exclude-standard --directory
+
+*
+
+# ── the repository itself ────────────────────────────────────────────────
+!.gitignore
+!README.md
+!LICENSE
+!**/LICENSE
+
+# ── directories have to be walked into before a file inside can match ────
+!gleam/
+!dlang/
+!org/
+!gleam/**/
+!dlang/**/
+!org/**/
+# but not these, whatever they contain
+build/
+.build/
+.direnv/
+.gleam/
+bin/
+result
+result-*
+
+# ── documentation and notes, in any language directory ───────────────────
+!**/COPYRIGHT
+!**/*.md
+!**/*.org
+
+# ── nix ──────────────────────────────────────────────────────────────────
+!**/*.nix
+!**/flake.lock
+!**/.envrc
+
+# ── dlang ────────────────────────────────────────────────────────────────
+!**/*.d
+!**/dub.json
+!**/dub.sdl
+!**/dub.selections.json
+*.o
+*.a
+*.lst
+
+# ── gleam ────────────────────────────────────────────────────────────────
+!**/*.gleam
+# Erlang, for what Gleam reaches through FFI (a sha256, from OTP crypto)
+!**/src/*.erl
+# gleam.toml is the manifest, manifest.toml is the lockfile: both belong
+!**/*.toml
+*.beam
+*.ez
+erl_crash.dump
+
+# ── ruby, when it arrives ────────────────────────────────────────────────
+!**/*.rb
+!**/Gemfile
+!**/Gemfile.lock
+!**/*.gemspec
+
+# ── continuous integration ───────────────────────────────────────────────
+!**/*.yml
+!**/*.yaml
diff --git a/COPYRIGHT b/COPYRIGHT
new file mode 100644
index 0000000..7e7dd56
--- /dev/null
+++ b/COPYRIGHT
@@ -0,0 +1,45 @@
+- Name: sisudoc-ocda-lib
+ - Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2026 Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ sisudoc-ocda-lib
+
+ part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation, either version 3 of the License,
+ or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+
+ - Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+ - Homepages:
+ [https://www.sisudoc.org]
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..c6f01c6
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,660 @@
+# GNU AFFERO GENERAL PUBLIC LICENSE
+
+Version 3, 19 November 2007
+
+Copyright (C) 2007 Free Software Foundation, Inc.
+<https://fsf.org/>
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+## Preamble
+
+The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains
+free software for all its users.
+
+When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing
+under this license.
+
+The precise terms and conditions for copying, distribution and
+modification follow.
+
+## TERMS AND CONDITIONS
+
+### 0. Definitions.
+
+"This License" refers to version 3 of the GNU Affero General Public
+License.
+
+"Copyright" also means copyright-like laws that apply to other kinds
+of works, such as semiconductor masks.
+
+"The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of
+an exact copy. The resulting work is called a "modified version" of
+the earlier work or a work "based on" the earlier work.
+
+A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user
+through a computer network, with no transfer of a copy, is not
+conveying.
+
+An interactive user interface displays "Appropriate Legal Notices" to
+the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+### 1. Source Code.
+
+The "source code" for a work means the preferred form of the work for
+making modifications to it. "Object code" means any non-source form of
+a work.
+
+A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+The Corresponding Source need not include anything that users can
+regenerate automatically from other parts of the Corresponding Source.
+
+The Corresponding Source for a work in source code form is that same
+work.
+
+### 2. Basic Permissions.
+
+All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+You may make, run and propagate covered works that you do not convey,
+without conditions so long as your license otherwise remains in force.
+You may convey covered works to others for the sole purpose of having
+them make modifications exclusively for you, or provide you with
+facilities for running those works, provided that you comply with the
+terms of this License in conveying all material for which you do not
+control copyright. Those thus making or running the covered works for
+you must do so exclusively on your behalf, under your direction and
+control, on terms that prohibit them from making any copies of your
+copyrighted material outside their relationship with you.
+
+Conveying under any other circumstances is permitted solely under the
+conditions stated below. Sublicensing is not allowed; section 10 makes
+it unnecessary.
+
+### 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such
+circumvention is effected by exercising rights under this License with
+respect to the covered work, and you disclaim any intention to limit
+operation or modification of the work as a means of enforcing, against
+the work's users, your or third parties' legal rights to forbid
+circumvention of technological measures.
+
+### 4. Conveying Verbatim Copies.
+
+You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+### 5. Conveying Modified Source Versions.
+
+You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these
+conditions:
+
+- a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+- b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under
+ section 7. This requirement modifies the requirement in section 4
+ to "keep intact all notices".
+- c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+- d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+### 6. Conveying Non-Source Forms.
+
+You may convey a covered work in object code form under the terms of
+sections 4 and 5, provided that you also convey the machine-readable
+Corresponding Source under the terms of this License, in one of these
+ways:
+
+- a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+- b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the Corresponding
+ Source from a network server at no charge.
+- c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+- d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+- e) Convey the object code using peer-to-peer transmission,
+ provided you inform other peers where the object code and
+ Corresponding Source of the work are being offered to the general
+ public at no charge under subsection 6d.
+
+A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal,
+family, or household purposes, or (2) anything designed or sold for
+incorporation into a dwelling. In determining whether a product is a
+consumer product, doubtful cases shall be resolved in favor of
+coverage. For a particular product received by a particular user,
+"normally used" refers to a typical or common use of that class of
+product, regardless of the status of the particular user or of the way
+in which the particular user actually uses, or expects or is expected
+to use, the product. A product is a consumer product regardless of
+whether the product has substantial commercial, industrial or
+non-consumer uses, unless such uses represent the only significant
+mode of use of the product.
+
+"Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to
+install and execute modified versions of a covered work in that User
+Product from a modified version of its Corresponding Source. The
+information must suffice to ensure that the continued functioning of
+the modified object code is in no case prevented or interfered with
+solely because modification has been made.
+
+If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or
+updates for a work that has been modified or installed by the
+recipient, or for the User Product in which it has been modified or
+installed. Access to a network may be denied when the modification
+itself materially and adversely affects the operation of the network
+or violates the rules and protocols for communication across the
+network.
+
+Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+### 7. Additional Terms.
+
+"Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders
+of that material) supplement the terms of this License with terms:
+
+- a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+- b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+- c) Prohibiting misrepresentation of the origin of that material,
+ or requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+- d) Limiting the use for publicity purposes of names of licensors
+ or authors of the material; or
+- e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+- f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions
+ of it) with contractual assumptions of liability to the recipient,
+ for any liability that these contractual assumptions directly
+ impose on those licensors and authors.
+
+All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions; the
+above requirements apply either way.
+
+### 8. Termination.
+
+You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+However, if you cease all violation of this License, then your license
+from a particular copyright holder is reinstated (a) provisionally,
+unless and until the copyright holder explicitly and finally
+terminates your license, and (b) permanently, if the copyright holder
+fails to notify you of the violation by some reasonable means prior to
+60 days after the cessation.
+
+Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+### 9. Acceptance Not Required for Having Copies.
+
+You are not required to accept this License in order to receive or run
+a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+### 10. Automatic Licensing of Downstream Recipients.
+
+Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+### 11. Patents.
+
+A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+A contributor's "essential patent claims" are all patent claims owned
+or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+A patent license is "discriminatory" if it does not include within the
+scope of its coverage, prohibits the exercise of, or is conditioned on
+the non-exercise of one or more of the rights that are specifically
+granted under this License. You may not convey a covered work if you
+are a party to an arrangement with a third party that is in the
+business of distributing software, under which you make payment to the
+third party based on the extent of your activity of conveying the
+work, and under which the third party grants, to any of the parties
+who would receive the covered work from you, a discriminatory patent
+license (a) in connection with copies of the covered work conveyed by
+you (or copies made from those copies), or (b) primarily for and in
+connection with specific products or compilations that contain the
+covered work, unless you entered into that arrangement, or that patent
+license was granted, prior to 28 March 2007.
+
+Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+### 12. No Surrender of Others' Freedom.
+
+If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under
+this License and any other pertinent obligations, then as a
+consequence you may not convey it at all. For example, if you agree to
+terms that obligate you to collect a royalty for further conveying
+from those to whom you convey the Program, the only way you could
+satisfy both those terms and this License would be to refrain entirely
+from conveying the Program.
+
+### 13. Remote Network Interaction; Use with the GNU General Public License.
+
+Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your
+version supports such interaction) an opportunity to receive the
+Corresponding Source of your version by providing access to the
+Corresponding Source from a network server at no charge, through some
+standard or customary means of facilitating copying of software. This
+Corresponding Source shall include the Corresponding Source for any
+work covered by version 3 of the GNU General Public License that is
+incorporated pursuant to the following paragraph.
+
+Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+### 14. Revised Versions of this License.
+
+The Free Software Foundation may publish revised and/or new versions
+of the GNU Affero General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever
+published by the Free Software Foundation.
+
+If the Program specifies that a proxy can decide which future versions
+of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+### 15. Disclaimer of Warranty.
+
+THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT
+WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND
+PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE
+DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
+CORRECTION.
+
+### 16. Limitation of Liability.
+
+IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR
+CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES
+ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT
+NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR
+LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM
+TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER
+PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+### 17. Interpretation of Sections 15 and 16.
+
+If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+END OF TERMS AND CONDITIONS
+
+## How to Apply These Terms to Your New Programs
+
+If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these
+terms.
+
+To do so, attach the following notices to the program. It is safest to
+attach them to the start of each source file to most effectively state
+the exclusion of warranty; and each file should have at least the
+"copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as
+ published by the Free Software Foundation, either version 3 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper
+mail.
+
+If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for
+the specific requirements.
+
+You should also get your employer (if you work as a programmer) or
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. For more information on this, and how to apply and follow
+the GNU AGPL, see <https://www.gnu.org/licenses/>.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..5e328c0
--- /dev/null
+++ b/README.md
@@ -0,0 +1,150 @@
+# sisudoc-ocda-lib
+
+The **object-centric document abstraction as a library**: the model, and
+loaders for both of the forms it is written in.
+
+Written by Claude (Anthropic), working in Claude Code from the specification,
+at the direction of and reviewed by Ralph Amissah.
+
+**One reader per language, in a directory named for the language.**
+
+| | |
+|--------------------|---------|
+| [`dlang/`](dlang/) | D |
+| [`gleam/`](gleam/) | Gleam |
+| `ruby/` | to come |
+
+## What it is for
+
+The readers in `sisudoc-ssp-reader` and `sisudoc-ocda-db-reader` each read
+one serialisation and print. A **writer** needs neither: it needs the document.
+
+```
+ .ssp ─┐
+ ├─> the model ─> a writer
+ .ocda.db ─┘
+```
+
+So this library holds the model and both loaders, and **a writer cannot
+tell which serialisation it was handed** - exactly as spine's own writers
+cannot, since spine 0.25.0.
+
+That is the whole of the idea. The readers become thin programs over this,
+and the writers are functions from the model to bytes.
+
+## The checks that matter
+
+The claim above is only true if the loaders agree, so the library asks. Two
+programs, in each language:
+
+```sh
+ocda-agree <doc>.ssp <doc>.ocda.db [lang] # do the two loaders agree?
+ocda-dump <file>.ssp|<file>.ocda.db [lang] # the whole model, canonically
+```
+
+`agree` compares two models within one language. **`dump` is the stronger
+of the two**, and the reason it exists: it writes the entire loaded model
+out in a fully determined form, and the D and the Gleam library write it
+*byte for byte alike*. So four questions can be asked, all of them with
+`diff`:
+
+| | |
+|-----------------------------|---------------------------------|
+| D: `.ssp` == `.ocda.db` | **36 of 36** document-languages |
+| Gleam: `.ssp` == `.ocda.db` | **36 of 36** |
+| `.ssp`: D == Gleam | **36 of 36** |
+| `.ocda.db`: D == Gleam | **36 of 36** |
+
+None of those held at first.
+
+## What the checks found
+
+### the ten-field check: a blank first text line
+
+An object whose **first text line is blank** lost it. The loader decided
+whether to start a new line by asking whether the accumulated text was
+empty, which is not the same question as whether a line has been seen. Of
+the sample collection **seven documents have verse opening on a blank
+line**, and in every one of them the blank was dropped. Eight of
+thirty-six documents disagreed.
+
+Nothing else had caught it. The two `.ssp` readers had been compared on
+*section counts*, which this does not change. It took loading the same
+document two ways and comparing the objects.
+
+Fixed here and in `sisudoc-ssp-reader/dlang`, which carried the same code.
+
+### the whole-model dump: four more
+
+The ten-field check was enough for that and not enough for anything
+subtler. Dumping the whole model found four further disagreements between
+the two loaders:
+
+1. **`.identifier` and `.anchor`.** The `.ssp` emits a property only when it
+ carries information, so the identifier is absent when it is simply the
+ ocn and the anchor is absent when it is the segment name; the database
+ stores both unconditionally. Over one document that is **1,314
+ identifiers and 69 anchors**. The loaders now derive them, which is what
+ the model means.
+2. **`make` and `doc_has`.** The database has one metadata table where the
+ text form has three blocks, and tells them apart by a key prefix. The
+ loaders unprefix them, so a writer asks `make["footer"]` whichever
+ serialisation it was handed.
+3. **The JSON columns were scanned rather than parsed.** Phobos writes `/`
+ as `\/`, so every heading containing a slash came back with a backslash
+ in it: **all 36 document-languages** differed on
+ `heading_ancestors_text` and nothing else had shown it.
+4. **`translation.ocn_aligned`** is in the database and cannot be in a
+ `.ssp`, which is one language and cannot know whether the languages
+ share a numbering. It is a fact about the *file*, so the loaders leave
+ it out rather than letting a writer tell which serialisation it was
+ given.
+
+The first three are defects. The fourth is a real asymmetry between the two
+serialisations, and naming it is the useful part.
+
+## What it reads, and what it does not
+
+| | | |
+|-----------------------|------------------------------------------------------------------|-------------------|
+| `.ssp` | yes | |
+| `.ocda.db` | yes, any language of it | |
+| `.sisupod` | yes, any language it carries | D only, see below |
+| `.zip` pod | yes, any language it carries | |
+| `.sst`, `.ssm` markup | **no, and not planned**: that is a parser, which is spine's work | |
+| a url | no | |
+
+A pod carries the abstraction only when spine was asked for it: `--pod2`
+puts it in, `--source` does not. A pod without one is refused with a message
+saying which flag makes a pod this library can read, because that is a fact
+about the pod rather than about the library.
+
+**Over the reference collection, a `.sisupod` gives the same model as the
+`.ssp` beside it: 36 of 36 document-languages, byte identical dumps.** A
+writer over the library therefore gained a third input form without being
+touched, which is the point of the dispatch being in one place.
+
+### the one thing the Gleam library cannot do
+
+A `.sisupod` is **one zstd frame wrapping** the zip. The D library reads both
+through a declared `libzstd`. The Gleam library reads the **zip and not the
+frame**, because the only zstd on the BEAM is a NIF that clones zstd from
+github in its own build and so cannot be built in a sandbox without patching
+it.
+
+So a `.zip` pod is read in both languages, a `.sisupod` in D only, and the
+Gleam refusal says why and says `zstd -d` rather than claiming the file is
+malformed. It is a packaging limit, not anything about the format, and it is
+the only place the two libraries differ in what they accept.
+
+`ocda/load` is the one place in each language that decides what a path is,
+so a new input form is added once and every reader and writer gains it.
+
+## The format
+
+Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model
+this library implements, and the two format specifications for the loaders.
+Section 4 of `ocda-db-format-specification.md` gives the column by column
+correspondence the `load_db` of each language follows.
+
+At format **2.x**.
diff --git a/dlang/COPYRIGHT b/dlang/COPYRIGHT
new file mode 100644
index 0000000..7e7dd56
--- /dev/null
+++ b/dlang/COPYRIGHT
@@ -0,0 +1,45 @@
+- Name: sisudoc-ocda-lib
+ - Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2026 Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ sisudoc-ocda-lib
+
+ part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation, either version 3 of the License,
+ or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+
+ - Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+ - Homepages:
+ [https://www.sisudoc.org]
diff --git a/dlang/README.md b/dlang/README.md
new file mode 100644
index 0000000..124e4d6
--- /dev/null
+++ b/dlang/README.md
@@ -0,0 +1,62 @@
+# ocda-lib, in D
+
+The abstraction's model and both its loaders, as a dub library.
+
+## Modules
+
+| | |
+|-----------------|---------------------------------------------------------------|
+| `ocda.model` | the document, the object, the eight sections. Imports nothing |
+| `ocda.load_ssp` | text -> model |
+| `ocda.load_db` | sqlite -> model |
+| `ocda.load` | what a path is, and loading it |
+| `ocda.dump` | the model written out canonically, as an oracle |
+| `ocda.sqlite` | the sqlite3 C API, declared rather than depended upon |
+
+`ocda.model` importing nothing is the property that matters: a writer
+depends on the model and on neither loader.
+
+## Use
+
+```json
+"dependencies": { "ocda-lib": { "path": "../../sisudoc-ocda-lib/dlang" } }
+```
+
+```d
+import ocda.load;
+auto doc = load("document.ocda.db", "de"); // or "document.ssp"
+foreach (obj; doc.body) { ... }
+```
+
+Link `sqlite3`; the library declares the dozen functions it needs rather
+than taking a binding from the registry, so there is nothing to fetch.
+
+## The two checks
+
+```sh
+nix build .#ocda-agree && ./result/bin/ocda-agree <doc>.ssp <doc>.ocda.db [lang]
+nix build .#ocda-dump && ./result/bin/ocda-dump [--unknown] <file> [lang]
+```
+
+`agree` loads both and compares ten fields of each object, naming the field
+where they part company. `agree` found the blank-first-text-line defect and
+was not enough for anything subtler.
+
+**`ocda-dump` is the stronger check**: the whole loaded model, every field,
+written out in a form fully determined by the model, so that two
+implementations can be compared with `diff`. It is an oracle, not an output
+format; anything reading it to reconstruct a document has misunderstood it.
+It agrees byte for byte with the Gleam library's `ocda-lib dump`, which is
+the point of having it.
+
+It found four disagreements `agree` could not see. They are in the
+repository README and in the history.
+
+## Build
+
+```sh
+nix build .#ocda-agree # the agreement check
+nix build .#ocda-dump # the canonical dump
+nix develop # ldc, dub, dtools, sqlite
+dub build # the library
+```
diff --git a/dlang/dub.json b/dlang/dub.json
new file mode 100644
index 0000000..4f59aec
--- /dev/null
+++ b/dlang/dub.json
@@ -0,0 +1,13 @@
+{
+ "authors": [ "Ralph Amissah" ],
+ "copyright": "Copyright (C) 2026 Ralph Amissah",
+ "name": "ocda-lib",
+ "version": "2.0.0",
+ "description": "the object-centric document abstraction as a library: the model, and loaders for its two serialisations",
+ "homepage": "https://sisudoc.org",
+ "license": "AGPL-3.0+",
+ "targetType": "library",
+ "sourcePaths": [ "./src" ],
+ "importPaths": [ "./src" ],
+ "libs": [ "sqlite3", "zstd" ]
+}
diff --git a/dlang/dub.selections.json b/dlang/dub.selections.json
new file mode 100644
index 0000000..84a60aa
--- /dev/null
+++ b/dlang/dub.selections.json
@@ -0,0 +1 @@
+{ "fileVersion": 1, "versions": {} }
diff --git a/dlang/flake.lock b/dlang/flake.lock
new file mode 100644
index 0000000..5f236ab
--- /dev/null
+++ b/dlang/flake.lock
@@ -0,0 +1,27 @@
+{
+ "nodes": {
+ "nixpkgs": {
+ "locked": {
+ "lastModified": 1790185690,
+ "narHash": "sha256-xJ+X4hBtOcAFGBOe5nAMyMUeF9foJBmIOu3NjBqBycU=",
+ "owner": "NixOS",
+ "repo": "nixpkgs",
+ "rev": "4975466d324710c576dc11ad614684e6bd8cad8e",
+ "type": "github"
+ },
+ "original": {
+ "owner": "NixOS",
+ "ref": "nixos-unstable",
+ "repo": "nixpkgs",
+ "type": "github"
+ }
+ },
+ "root": {
+ "inputs": {
+ "nixpkgs": "nixpkgs"
+ }
+ }
+ },
+ "root": "root",
+ "version": 7
+}
diff --git a/dlang/flake.nix b/dlang/flake.nix
new file mode 100644
index 0000000..3aa9a60
--- /dev/null
+++ b/dlang/flake.nix
@@ -0,0 +1,63 @@
+{
+ description = "ocda-lib: the object-centric document abstraction as a D library";
+
+ inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
+
+ outputs = { self, nixpkgs }:
+ let
+ system = "x86_64-linux"; # adjust for aarch64-linux / darwin
+ pkgs = import nixpkgs { inherit system; };
+
+ /* A library, so there is nothing to install as a binary. What is
+ built here are the two programs that put the library to the
+ question: the agreement check, which loads a document's .ssp and
+ its .ocda.db and compares the two models, and the dump, which
+ writes a loaded model out canonically so that an implementation in
+ another language can be diffed against this one.
+ */
+ mkTool = { tool, binary, what, compiler, compilerBin }:
+ pkgs.stdenv.mkDerivation {
+ pname = "ocda-lib-${tool}";
+ version = "2.0.0";
+ src = self;
+ nativeBuildInputs = [ compiler ];
+ buildInputs = [ pkgs.sqlite pkgs.zstd ];
+ buildPhase = ''
+ runHook preBuild
+ export HOME=$TMPDIR
+ ${compilerBin} -I=src -of=${tool} src/ocda/*.d test/${tool}.d -L-lsqlite3 -L-lzstd
+ runHook postBuild
+ '';
+ installPhase = ''
+ runHook preInstall
+ mkdir -p $out/bin
+ install -m755 ${tool} $out/bin/${binary}
+ runHook postInstall
+ '';
+ meta = {
+ description = what;
+ mainProgram = binary;
+ };
+ };
+
+ ldc = { compiler = pkgs.ldc; compilerBin = "ldmd2"; };
+ in {
+ packages.${system} = {
+ default = self.packages.${system}.ocda-agree;
+
+ ocda-agree = mkTool (ldc // {
+ tool = "agree"; binary = "ocda-agree";
+ what = "checks that the .ssp and .ocda.db loaders give one model";
+ });
+
+ ocda-dump = mkTool (ldc // {
+ tool = "dump"; binary = "ocda-dump";
+ what = "writes a loaded abstraction out canonically, as an oracle";
+ });
+ };
+
+ devShells.${system}.default = pkgs.mkShell {
+ packages = [ pkgs.ldc pkgs.dub pkgs.dtools pkgs.sqlite pkgs.zstd ];
+ };
+ };
+}
diff --git a/dlang/src/COPYRIGHT b/dlang/src/COPYRIGHT
new file mode 100644
index 0000000..7e7dd56
--- /dev/null
+++ b/dlang/src/COPYRIGHT
@@ -0,0 +1,45 @@
+- Name: sisudoc-ocda-lib
+ - Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2026 Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ sisudoc-ocda-lib
+
+ part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation, either version 3 of the License,
+ or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+
+ - Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+ - Homepages:
+ [https://www.sisudoc.org]
diff --git a/dlang/src/ocda/dump.d b/dlang/src/ocda/dump.d
new file mode 100644
index 0000000..7a4f11a
--- /dev/null
+++ b/dlang/src/ocda/dump.d
@@ -0,0 +1,164 @@
+module ocda.dump;
+
+/+ ↓ the model, written out canonically.
+ .
+ Not an output format: an *oracle*. Two libraries in two languages
+ implement the same model from the same specification, and the cheap
+ way to ask whether they agree is to have each write the whole loaded
+ document as text and diff the two. That is only useful if the text is
+ fully determined by the model, so:
+ .
+ - every line is "<key> <value>", the value running to end of line,
+ and only one free-form value per line;
+ - a field at its default is *omitted*, so the output says what the
+ document has rather than what the struct declares;
+ - dictionaries are emitted in sorted key order, because neither
+ language's hash map has an order worth relying on;
+ - sections come in the fixed order of the format, not
+ alphabetically.
+ .
+ Anything that reads this to reconstruct a document has misunderstood
+ it. Load the .ssp or the .ocda.db.
++/
+
+import std.algorithm : sort, splitter;
+import std.array : appender, array, join;
+import std.conv : to;
+import ocda.model;
+
+/+ ↓ the nine sections, in format order +/
+static immutable string[] dump_sections = ["head", "toc", "body", "endnotes",
+ "glossary", "bibliography", "bookindex", "blurb", "tail"];
+
+string dumpDocument(ref OcdaDocument doc) {
+ auto out_ = appender!string;
+
+ void kv(string key, string value) {
+ if (value.length == 0) { return; }
+ out_ ~= key; out_ ~= " "; out_ ~= value; out_ ~= "\n";
+ }
+ void kn(string key, long value) {
+ if (value == 0) { return; }
+ kv(key, value.to!string);
+ }
+ void kb(string key, bool value) {
+ if (!value) { return; }
+ kv(key, "1");
+ }
+ void dict(string prefix, string[string] d) {
+ foreach (key; d.keys.dup.sort) {
+ out_ ~= prefix; out_ ~= " "; out_ ~= key; out_ ~= " ";
+ out_ ~= d[key]; out_ ~= "\n";
+ }
+ }
+
+ kv("version", doc.format_version);
+ kv("source_filename", doc.source_filename);
+ dict("source", doc.source);
+ dict("meta", doc.metadata);
+ dict("make", doc.make);
+ dict("doc_has", doc.doc_has);
+
+ foreach (name; dump_sections) {
+ auto sec = doc.sectionByName(name);
+ if (sec is null || (*sec).length == 0) { continue; }
+ kv("section", name ~ " " ~ (*sec).length.to!string);
+ }
+
+ foreach (name; dump_sections) {
+ auto sec = doc.sectionByName(name);
+ if (sec is null) { continue; }
+ foreach (i, obj; *sec) {
+ out_ ~= "object "; out_ ~= name; out_ ~= " ";
+ out_ ~= i.to!string; out_ ~= "\n";
+ kn("ocn", obj.ocn);
+ kv("is_a", obj.is_a);
+ kv("own_section", obj.own_section);
+ kv("level", obj.heading_level);
+ kv("identifier", obj.identifier);
+ kv("part", obj.part);
+ kv("is_of_type", obj.is_of_type);
+ kv("segment_html_is", obj.segment_html_is);
+ kv("segment_epub", obj.segment_epub);
+ kv("segment_epub_is", obj.segment_epub_is);
+ kn("parent", obj.parent_ocn);
+ kn("last_descendant", obj.last_descendant_ocn);
+ kv("children", ints(obj.children));
+ kv("ancestors", ints(obj.ancestors));
+ kb("dummy", obj.dummy_heading);
+ kb("ocn_off", obj.ocn_off);
+ kn("indent_base", obj.indent_base);
+ kn("indent_hang", obj.indent_hang);
+ kb("bullet", obj.bullet);
+ kv("lang", obj.lang);
+ kb("has_links", obj.has_links);
+ kb("has_notes_reg", obj.has_notes_reg);
+ kb("has_notes_star", obj.has_notes_star);
+ kb("has_images", obj.has_images);
+ kv("segment", obj.segment);
+ kv("segment_prev", obj.segment_prev);
+ kv("segment_next", obj.segment_next);
+ kv("anchor", obj.anchor);
+ kv("heading_lev_anchor", obj.heading_lev_anchor);
+ kn("table_cols", obj.table_cols);
+ kv("table_widths", obj.table_widths);
+ kv("table_aligns", obj.table_aligns);
+ kb("table_header", obj.table_header);
+ kv("code_syntax", obj.code_syntax);
+ kb("code_linenumbers", obj.code_linenumbers);
+ kv("heading_ancestors_text", obj.heading_ancestors_text);
+ kv("sha256", obj.sha256);
+ foreach (url; obj.stow_link) { kv("stow_link", url); }
+ foreach (a; obj.anchor_tags) { kv("anchor_tag", a); }
+ foreach (img; obj.images) {
+ /+ ↓ the name last, because a filename may hold a space where
+ none of the other fields can +/
+ kv("image", img.sha256 ~ " " ~ img.bytes.to!string ~ " "
+ ~ img.width.to!string ~ "x" ~ img.height.to!string ~ " "
+ ~ (img.missing ? "1" : "0") ~ " " ~ img.name);
+ foreach (key; img.fields.keys.dup.sort) {
+ kv("image_field", key ~ " " ~ img.fields[key]);
+ }
+ }
+ /+ ↓ text, when there is any. An object whose only text line is
+ blank holds "" here, indistinguishable from no text at all,
+ because the model joins the lines into one string. Both
+ libraries lose the same distinction in the same place, which is
+ what makes them comparable; it is recorded rather than hidden.
+ +/
+ if (obj.text.length > 0) {
+ auto lines = obj.text.splitter("\n").array;
+ kv("text", lines.length.to!string);
+ foreach (line; lines) {
+ out_ ~= "| "; out_ ~= line; out_ ~= "\n";
+ }
+ }
+ }
+ }
+ return out_.data;
+}
+
+/+ ↓ what the reader met and did not understand, sorted. Separate from the
+ document dump: it describes the reader, not the document, and the
+ .ocda.db loader has nothing to report where the .ssp loader does.
++/
+string dumpUnknown(ref OcdaDocument doc) {
+ auto out_ = appender!string;
+ void counts(string prefix, size_t[string] d) {
+ foreach (key; d.keys.dup.sort) {
+ out_ ~= prefix; out_ ~= " "; out_ ~= key; out_ ~= " ";
+ out_ ~= d[key].to!string; out_ ~= "\n";
+ }
+ }
+ counts("property", doc.unknown.property);
+ counts("block", doc.unknown.block);
+ counts("line", doc.unknown.line);
+ return out_.data;
+}
+
+private string ints(int[] xs) {
+ if (xs.length == 0) { return ""; }
+ string[] parts;
+ foreach (x; xs) { parts ~= x.to!string; }
+ return parts.join(",");
+}
diff --git a/dlang/src/ocda/images.d b/dlang/src/ocda/images.d
new file mode 100644
index 0000000..b3a9652
--- /dev/null
+++ b/dlang/src/ocda/images.d
@@ -0,0 +1,100 @@
+module ocda.images;
+
+/+ ↓ an image's bytes, from whichever source the document came from.
+ .
+ The model *names* images, with digest, byte size and pixel size; it does
+ not hold them. That is right - an abstraction of a document is not a
+ bundle of its assets - but a writer of any format that shows images
+ needs the bytes, so something has to know where they are. This is that
+ something, and it is in the library rather than in a writer because the
+ answer depends on the *input form* and not on the output.
+ .
+ Three answers, and one refusal:
+ .
+ .ocda.db the files table, role 'image', by name
+ .sisupod the archive entry whose path ends in /image/<name>
+ .zip the same
+ .ssp beside it: a pod puts the abstraction in
+ media/abstraction/ and its images in media/image/, so the
+ path is documented rather than guessed
+ .
+ A .ssp that is not in a pod has nowhere to look, and that is reported as
+ absent rather than as an error: a reader that cannot find an image
+ should say so and carry on, because the document is still readable.
++/
+
+import std.algorithm : endsWith;
+import std.array : array;
+import std.file : exists, isFile, read;
+import std.path : buildPath, dirName;
+import ocda.load;
+import ocda.sqlite;
+import ocda.zip;
+import ocda.zstd;
+
+/+ ↓ the bytes, or an empty slice when the source has no such image +/
+ubyte[] imageBytes(string source, string name) {
+ final switch (sourceOf(source)) {
+ case Source.ocda_db: return fromDb(source, name);
+ case Source.pod: return fromArchive(source, name);
+ case Source.ssp: return fromBeside(source, name);
+ case Source.unknown: return [];
+ }
+}
+
+/+ ↓ can this source give images at all?
+ A writer asks before it decides whether to emit them, because an <img>
+ pointing at a file it did not include is worse than no image.
++/
+bool canGiveImages(string source) {
+ final switch (sourceOf(source)) {
+ case Source.ocda_db: return true;
+ case Source.pod: return true;
+ case Source.ssp: return imageDirBeside(source).length > 0;
+ case Source.unknown: return false;
+ }
+}
+
+private ubyte[] fromDb(string path, string name) {
+ auto db = Db.open(path);
+ scope(exit) { db.close(); }
+ if (!db.has("files")) { return []; }
+ /+ ↓ the name is from the abstraction, not from a caller, and the quote
+ doubling is what keeps it a value rather than syntax +/
+ return db.blob("SELECT data FROM files WHERE role = 'image' AND name = '"
+ ~ sqlQuote(name) ~ "'");
+}
+
+private ubyte[] fromArchive(string path, string name) {
+ auto bytes = cast(ubyte[]) read(path);
+ if (zstdIsFrame(bytes)) { bytes = zstdDecompress(bytes, 512 * 1024 * 1024); }
+ auto entries = zipEntries(bytes);
+ foreach (e; entries) {
+ if (e.name.endsWith("/image/" ~ name)) { return zipRead(bytes, e); }
+ }
+ return [];
+}
+
+private ubyte[] fromBeside(string path, string name) {
+ string dir = imageDirBeside(path);
+ if (dir.length == 0) { return []; }
+ string file = buildPath(dir, name);
+ return file.exists && file.isFile ? cast(ubyte[]) read(file) : [];
+}
+
+/+ ↓ media/abstraction/<x>.ssp -> media/image/ +/
+private string imageDirBeside(string path) {
+ string dir = path.dirName;
+ if (!dir.endsWith("abstraction")) { return ""; }
+ string image_dir = buildPath(dir.dirName, "image");
+ return image_dir.exists ? image_dir : "";
+}
+
+/+ ↓ a single quote doubled is how sql says a single quote +/
+private string sqlQuote(string s) {
+ string out_;
+ foreach (c; s) {
+ out_ ~= (c == '\'') ? "''" : [c];
+ }
+ return out_;
+}
diff --git a/dlang/src/ocda/load.d b/dlang/src/ocda/load.d
new file mode 100644
index 0000000..89c8dcc
--- /dev/null
+++ b/dlang/src/ocda/load.d
@@ -0,0 +1,69 @@
+module ocda.load;
+
+/+ ↓ what a path is, and loading it.
+ .
+ One place that decides, so that every reader and every writer accepts
+ the same inputs and a new input type is added once. This mirrors
+ spine's own ocda/abstraction/load.d, which names the sources it can
+ enter the pipeline from and tells them apart by suffix.
+ .
+ Deliberately *not* here: .sst and .ssm markup. Reading markup means
+ implementing the parser, which is spine's work and a far larger thing
+ than a reader or a sample writer needs. This library reads
+ abstractions.
++/
+
+import std.algorithm : endsWith;
+import ocda.model;
+import ocda.load_ssp;
+import ocda.load_db;
+import ocda.load_pod;
+
+enum Source {
+ ssp, // .ssp, the abstraction as text
+ ocda_db, // .ocda.db, the abstraction as sqlite
+ pod, // .sisupod or .zip, an archive carrying a .ssp
+ unknown,
+}
+
+Source sourceOf(string path) {
+ if (path.endsWith(".ocda.db")) { return Source.ocda_db; }
+ if (path.endsWith(".ssp")) { return Source.ssp; }
+ /+ ↓ .sisupod is the archive spine writes now; .zip is what it wrote
+ before, and every pod already published carries that name, so both
+ are read. What is inside decides how it is opened, not the name.
+ +/
+ if (path.endsWith(".sisupod")) { return Source.pod; }
+ if (path.endsWith(".zip")) { return Source.pod; }
+ if (path.endsWith(".db")) { return Source.ocda_db; }
+ return Source.unknown;
+}
+
+string sourceName(Source s) {
+ final switch (s) {
+ case Source.ssp: return ".ssp (the abstraction as text)";
+ case Source.ocda_db: return ".ocda.db (the abstraction as sqlite)";
+ case Source.pod: return "a pod archive carrying a .ssp";
+ case Source.unknown: return "not an abstraction this library reads";
+ }
+}
+
+class OcdaLoadError : Exception {
+ this(string msg) { super(msg); }
+}
+
+/+ ↓ load whatever the path is into the model.
+ lang names which language to take from a .ocda.db, which holds
+ every language of its document, and from a pod, which may carry more
+ than one .ssp. A .ssp is one language by construction and ignores it.
++/
+OcdaDocument load(string path, string lang = "", bool any_version = false) {
+ final switch (sourceOf(path)) {
+ case Source.ssp: return parseSspFile(path);
+ case Source.ocda_db: return loadDbFile(path, lang, any_version);
+ case Source.pod: return loadPodFile(path, lang);
+ case Source.unknown:
+ throw new OcdaLoadError(path ~ ": " ~ sourceName(Source.unknown)
+ ~ " (expected .ssp, .ocda.db, .sisupod or .zip)");
+ }
+}
diff --git a/dlang/src/ocda/load_db.d b/dlang/src/ocda/load_db.d
new file mode 100644
index 0000000..dec0626
--- /dev/null
+++ b/dlang/src/ocda/load_db.d
@@ -0,0 +1,310 @@
+module ocda.load_db;
+
+/+ ↓ build the model from a .ocda.db.
+ .
+ The point of this module is that it returns *the same value*
+ ocda.load_ssp returns from the same document's .ssp. A writer is then
+ a function from the model to bytes, and cannot tell which
+ serialisation it was given, exactly as spine's own writers cannot.
+ .
+ Written from the sisudoc-abstraction-format specification,
+ ocda-db-format-specification.md, whose section 4 gives the column by
+ column correspondence this module implements.
++/
+
+import std.array : join, split;
+import std.json : JSONType, parseJSON;
+import std.conv : to;
+import std.algorithm : startsWith;
+import std.string : strip;
+import ocda.model;
+import ocda.sqlite;
+
+/+ ↓ the schema major this loader knows (specification section 2) +/
+enum schema_major_read = "2";
+
+class OcdaDbError : Exception {
+ this(string msg) { super(msg); }
+}
+
+/+ ↓ every language the database holds, in manifest order +/
+struct DbLanguage {
+ int doc_id;
+ string lang;
+ string source_filename;
+ string ssp_digest;
+}
+
+DbLanguage[] dbLanguages(ref Db db) {
+ DbLanguage[] out_;
+ foreach (r; db.rows(
+ "SELECT id, lang, coalesce(source_filename,''), coalesce(ssp_digest,'')
+ FROM documents ORDER BY id")) {
+ out_ ~= DbLanguage(r[0].to!int, r[1], r[2], r[3]);
+ }
+ return out_;
+}
+
+/+ ↓ load one language of a database as the model.
+ lang == "" takes the first language the manifest gave.
++/
+OcdaDocument loadDbFile(string path, string lang = "", bool any_version = false) {
+ auto db = Db.open(path);
+ scope(exit) { db.close(); }
+
+ if (!db.has("documents") || !db.has("objects") || !db.has("metadata")) {
+ throw new OcdaDbError(path ~ ": no documents/objects/metadata tables;"
+ ~ " it does not look like a .ocda.db");
+ }
+
+ /+ ↓ specification sections 2 and 3.2: the rows describing the file rather
+ than a document in it are the only ones with doc_id null +/
+ string schema_version = db.one(
+ "SELECT value FROM metadata WHERE doc_id IS NULL AND key = 'schema.version'");
+ if (schema_version.length > 0 && !any_version) {
+ auto major = schema_version.split(".")[0];
+ if (major != schema_major_read) {
+ throw new OcdaDbError(path ~ " is schema v" ~ schema_version
+ ~ ", this loader knows v" ~ schema_major_read ~ ".x");
+ }
+ }
+
+ auto languages = dbLanguages(db);
+ if (languages.length == 0) {
+ throw new OcdaDbError(path ~ ": no documents rows");
+ }
+ DbLanguage chosen = languages[0];
+ if (lang.length > 0) {
+ bool found = false;
+ foreach (l; languages) { if (l.lang == lang) { chosen = l; found = true; break; } }
+ if (!found) {
+ string have;
+ foreach (i, l; languages) { have ~= (i > 0 ? " " : "") ~ l.lang; }
+ throw new OcdaDbError(path ~ ": no language '" ~ lang ~ "'; it holds: " ~ have);
+ }
+ }
+
+ OcdaDocument doc;
+ doc.format_version = schema_version;
+ doc.source_filename = chosen.source_filename;
+
+ /+ ↓ @source of the .ssp corresponds to the documents row and the
+ source.digest metadata (specification section 5.1 of the .ssp, and
+ section 7 here) +/
+ doc.source["language"] = chosen.lang;
+ string languages_joined;
+ foreach (i, l; languages) { languages_joined ~= (i > 0 ? " " : "") ~ l.lang; }
+ doc.source["languages"] = languages_joined;
+ auto digest = db.one("SELECT value FROM metadata WHERE doc_id = "
+ ~ chosen.doc_id.to!string ~ " AND key = 'source.digest'");
+ if (digest.length > 0) { doc.source["digest"] = digest; }
+
+ /+ ↓ @meta: every metadata row of this language that is not a source.*
+ property. source.* describes the artefact rather than the document
+ and lives in @source in the text form.
+ +/
+ foreach (r; db.rows("SELECT key, value FROM metadata WHERE doc_id = "
+ ~ chosen.doc_id.to!string ~ " ORDER BY key")) {
+ string key = r[0];
+ if (key.startsWith("source.")) { continue; }
+ /+ ↓ translation.ocn_aligned says whether the languages of this
+ document share one numbering. A .ocda.db can know it because it
+ holds every language; a .ssp is one language and cannot. So it is
+ a fact about the *file* rather than about the document, and putting
+ it in the document's metadata would let a writer tell which
+ serialisation it was handed, which is the one thing this library
+ exists to prevent. Left out until the model has a place for
+ collection-wide facts.
+ +/
+ if (key.startsWith("translation.")) { continue; }
+ /+ ↓ the database has one metadata table where the text form has three
+ blocks, and tells them apart by a key prefix. Unprefixing them
+ here is what makes the two loaders give one model: a writer asks
+ doc.make["footer"] whichever serialisation it was handed, and
+ never doc.metadata["make.footer"].
+ +/
+ if (key.startsWith("make.")) {
+ doc.make[key["make.".length .. $]] = r[1].strip;
+ } else if (key.startsWith("doc_has.")) {
+ doc.doc_has[key["doc_has.".length .. $]] = r[1].strip;
+ } else {
+ doc.metadata[key] = r[1].strip;
+ }
+ }
+
+ /+ ↓ the objects, in document order within each section +/
+ foreach (r; db.rows(
+ "SELECT section, seq, ocn, is_a, coalesce(is_of_part,''),
+ coalesce(is_of_section,''), coalesce(is_of_type,''),
+ coalesce(heading_level,-1), coalesce(identifier,''),
+ coalesce(parent_ocn,0), coalesce(last_descendant_ocn,0),
+ coalesce(children,''), coalesce(ancestors,''),
+ coalesce(dummy_heading,0), coalesce(object_number_off,0),
+ coalesce(indent_base,0), coalesce(indent_hang,0),
+ coalesce(bullet,0), coalesce(lang,''),
+ coalesce(has_links,0), coalesce(has_notes_reg,0),
+ coalesce(has_notes_star,0), coalesce(has_images,0),
+ coalesce(segment,''), coalesce(segment_prev,''),
+ coalesce(segment_next,''), coalesce(anchor,''),
+ coalesce(segment_html_is,''),
+ coalesce(table_cols,0), coalesce(table_widths,''),
+ coalesce(table_aligns,''), coalesce(table_header,0),
+ coalesce(code_linenumbers,0),
+ coalesce(heading_ancestors_text,''), coalesce(sha256,''),
+ coalesce(text,''), id, coalesce(heading_lev_anchor,''),
+ coalesce(segment_epub,''), coalesce(segment_epub_is,'')
+ FROM objects WHERE doc_id = " ~ chosen.doc_id.to!string
+ ~ " ORDER BY CASE section
+ WHEN 'head' THEN 1 WHEN 'toc' THEN 2 WHEN 'body' THEN 3
+ WHEN 'endnotes' THEN 4 WHEN 'glossary' THEN 5
+ WHEN 'bibliography' THEN 6 WHEN 'bookindex' THEN 7
+ WHEN 'blurb' THEN 8 WHEN 'tail' THEN 9 ELSE 10 END, seq")) {
+ OcdaObject obj;
+ obj.section = r[0];
+ obj.ocn = r[2].to!int;
+ obj.is_a = r[3];
+ obj.part = r[4];
+ obj.own_section = (r[5] == r[0]) ? "" : r[5];
+ obj.is_of_type = r[6];
+ /+ ↓ the database holds the level as a number, the text form as the
+ letter or digit the markup wrote. Model section 4 is the mapping. +/
+ obj.heading_level = levelName(r[7].to!int);
+ obj.identifier = r[8];
+ obj.parent_ocn = r[9].to!int;
+ obj.last_descendant_ocn = r[10].to!int;
+ obj.children = jsonInts(r[11]);
+ obj.ancestors = jsonInts(r[12]);
+ obj.dummy_heading = (r[13] != "0");
+ obj.ocn_off = (r[14] != "0");
+ obj.indent_base = r[15].to!int;
+ obj.indent_hang = r[16].to!int;
+ obj.bullet = (r[17] != "0");
+ obj.lang = r[18];
+ obj.has_links = (r[19] != "0");
+ obj.has_notes_reg = (r[20] != "0");
+ obj.has_notes_star = (r[21] != "0");
+ obj.has_images = (r[22] != "0");
+ obj.segment = r[23];
+ obj.segment_prev = r[24];
+ obj.segment_next = r[25];
+ obj.anchor = r[26];
+ obj.segment_html_is = r[27];
+ obj.heading_lev_anchor = r[37];
+ obj.segment_epub = r[38];
+ obj.segment_epub_is = r[39];
+ obj.table_cols = r[28].to!int;
+ obj.table_widths = jsonWords(r[29]);
+ obj.table_aligns = jsonWords(r[30]);
+ obj.table_header = (r[31] != "0");
+ obj.code_linenumbers = (r[32] != "0");
+ obj.heading_ancestors_text = jsonJoined(r[33]);
+ obj.sha256 = r[34];
+ obj.text = r[35];
+
+ /+ ↓ the four repeatable properties of the text form are child tables
+ here (specification section 4) +/
+ string object_id = r[36];
+ foreach (i; db.rows("SELECT name, coalesce(sha256,''), coalesce(bytes,0),
+ coalesce(width,0), coalesce(height,0),
+ coalesce(missing,0)
+ FROM object_images WHERE object_id = " ~ object_id
+ ~ " ORDER BY seq")) {
+ OcdaImage img;
+ img.name = i[0];
+ img.sha256 = i[1];
+ img.bytes = i[2].to!ulong;
+ img.width = i[3].to!int;
+ img.height = i[4].to!int;
+ img.missing = (i[5] != "0");
+ obj.images ~= img;
+ }
+ foreach (l; db.rows("SELECT url FROM object_links WHERE object_id = "
+ ~ object_id ~ " ORDER BY seq")) {
+ obj.stow_link ~= l[0];
+ }
+ foreach (a; db.rows("SELECT anchor FROM object_anchors WHERE object_id = "
+ ~ object_id ~ " ORDER BY seq")) {
+ obj.anchor_tags ~= a[0];
+ }
+
+ auto section = doc.sectionByName(r[0]);
+ if (section !is null) { *section ~= obj; }
+ }
+ return doc;
+}
+
+/+ ↓ model section 4: the eight levels and their indices +/
+private string levelName(int index) {
+ switch (index) {
+ case 0: return "A";
+ case 1: return "B";
+ case 2: return "C";
+ case 3: return "D";
+ case 4: return "1";
+ case 5: return "2";
+ case 6: return "3";
+ case 7: return "4";
+ default: return "";
+ }
+}
+
+/+ ↓ the JSON columns, read as JSON.
+ .
+ The fixed width arrays are JSON in the database and space separated
+ (or "|" separated) in the text form. This was first written as a
+ character scan that threw away brackets and quotes, which is wrong the
+ moment a value holds an escape: Phobos' JSONValue.toString escapes
+ "/" as "\/", so every heading containing a slash came back with a
+ backslash in it. 36 of 36 documents differed on
+ heading_ancestors_text and nothing else had shown it.
+ .
+ So: parse. The values are small, there are few of them, and a real
+ parser is shorter than the scan it replaces.
++/
+private string[] jsonStrings(string v) {
+ if (v.length == 0) { return []; }
+ string[] out_;
+ try {
+ auto j = parseJSON(v);
+ if (j.type != JSONType.array) { return []; }
+ foreach (e; j.array) {
+ switch (e.type) {
+ case JSONType.string: out_ ~= e.str; break;
+ case JSONType.integer: out_ ~= e.integer.to!string; break;
+ case JSONType.uinteger: out_ ~= e.uinteger.to!string; break;
+ case JSONType.float_: out_ ~= e.floating.to!string; break;
+ case JSONType.true_: out_ ~= "true"; break;
+ case JSONType.false_: out_ ~= "false"; break;
+ default: out_ ~= ""; break;
+ }
+ }
+ } catch (Exception) {
+ return [];
+ }
+ return out_;
+}
+
+private int[] jsonInts(string v) {
+ int[] out_;
+ foreach (s; jsonStrings(v)) {
+ if (s.length == 0) { continue; }
+ try { out_ ~= s.to!int; } catch (Exception) {}
+ }
+ return out_;
+}
+
+/+ ↓ space separated in the text form: table_widths, table_aligns +/
+private string jsonWords(string v) {
+ return jsonStrings(v).join(" ");
+}
+
+/+ ↓ heading_ancestors_text is eight slots, joined with "|" in the text
+ form and a JSON array here. The empty slots are significant: they say
+ which levels the heading sits under, so the separators carry as much
+ as the text does.
++/
+private string jsonJoined(string v) {
+ auto parts = jsonStrings(v);
+ if (parts.length == 0) { return ""; }
+ return parts.join("|");
+}
diff --git a/dlang/src/ocda/load_pod.d b/dlang/src/ocda/load_pod.d
new file mode 100644
index 0000000..75993be
--- /dev/null
+++ b/dlang/src/ocda/load_pod.d
@@ -0,0 +1,115 @@
+module ocda.load_pod;
+
+/+ ↓ the abstraction out of a pod archive.
+ .
+ A .sisupod is one zstd frame wrapping a stored zip; a .zip is the same
+ archive unwrapped, which is what spine published before it wrapped them.
+ Either way, a pod built with --pod2 carries the abstraction inside it,
+ at media/abstraction/<uid>.<lang>.ssp, and that is what is read.
+ .
+ So this is a third input form and *not* a third model: the .ssp inside
+ is parsed by ocda.load_ssp and comes back as the same value. A writer
+ over the library gains .sisupod without being touched, which is the
+ whole reason the dispatch is in one place.
+ .
+ A pod built without --pod2 carries markup and no abstraction. That is
+ refused, and the refusal says which flag makes a pod this reader can
+ read, because "no abstraction in it" is a fact about the pod rather
+ than about the reader.
++/
+
+import std.algorithm : endsWith, sort;
+import std.array : array;
+import std.file : read;
+import std.string : indexOf;
+import ocda.model;
+import ocda.load_ssp;
+import ocda.zip;
+import ocda.zstd;
+
+/+ ↓ a limit on what a frame may expand to.
+ A reader of an artefact it did not make declines to be told how much
+ memory to allocate. 512 MiB is far above the largest pod in the
+ reference collection (4 MiB) and far below anything that matters.
++/
+enum size_t max_pod_size = 512 * 1024 * 1024;
+
+class OcdaPodError : Exception {
+ this(string msg) { super(msg); }
+}
+
+/+ ↓ which languages of the abstraction the pod carries, in sorted order +/
+string[] podLanguages(string path) {
+ auto bytes = archiveBytes(path);
+ string[] out_;
+ foreach (e; zipFindSuffix(zipEntries(bytes), ".ssp")) {
+ string lang = langOfSspName(e.name);
+ if (lang.length > 0) { out_ ~= lang; }
+ }
+ return out_.sort.array;
+}
+
+/+ ↓ load one language of the abstraction a pod carries.
+ lang == "" takes the only one there is, and where there is a choice it
+ says so rather than choosing, as the .ocda.db loader does.
++/
+OcdaDocument loadPodFile(string path, string lang = "") {
+ auto bytes = archiveBytes(path);
+ auto entries = zipEntries(bytes);
+ auto ssps = zipFindSuffix(entries, ".ssp");
+ if (ssps.length == 0) {
+ throw new OcdaPodError(path ~ ": carries no .ssp."
+ ~ " A pod carries the abstraction only when spine was asked for it:"
+ ~ " spine --pod2 puts it in, spine --source does not.");
+ }
+ ZipEntry chosen = ssps[0];
+ if (lang.length > 0) {
+ bool found = false;
+ foreach (e; ssps) {
+ if (langOfSspName(e.name) == lang) { chosen = e; found = true; break; }
+ }
+ if (!found) {
+ string have;
+ foreach (i, e; ssps) {
+ have ~= (i > 0 ? " " : "") ~ langOfSspName(e.name);
+ }
+ throw new OcdaPodError(path ~ ": no language '" ~ lang
+ ~ "'; it holds: " ~ have);
+ }
+ } else if (ssps.length > 1) {
+ string have;
+ foreach (i, e; ssps) {
+ have ~= (i > 0 ? " " : "") ~ langOfSspName(e.name);
+ }
+ throw new OcdaPodError(path ~ ": holds more than one language ("
+ ~ have ~ "); name the one wanted");
+ }
+ auto ssp = zipRead(bytes, chosen);
+ return parseSsp(cast(string) ssp.idup);
+}
+
+/+ ↓ the archive, unwrapped if it was wrapped.
+ The bytes decide, not the name: a .sisupod is a zstd frame and a .zip
+ is not, and a file misnamed either way still reads.
++/
+private ubyte[] archiveBytes(string path) {
+ auto bytes = cast(ubyte[]) read(path);
+ return zstdIsFrame(bytes)
+ ? zstdDecompress(bytes, max_pod_size)
+ : bytes;
+}
+
+/+ ↓ <uid>.<lang>.ssp -> <lang> +/
+private string langOfSspName(string name) {
+ if (!name.endsWith(".ssp")) { return ""; }
+ string stem = name[0 .. $ - 4];
+ auto dot = stem.lastIndexOf('.');
+ return (dot > 0) ? stem[dot + 1 .. $] : "";
+}
+
+private ptrdiff_t lastIndexOf(string s, char c) {
+ for (ptrdiff_t i = s.length - 1; i >= 0; i--) {
+ if (s[i] == c) { return i; }
+ }
+ return -1;
+}
diff --git a/dlang/src/ocda/load_ssp.d b/dlang/src/ocda/load_ssp.d
new file mode 100644
index 0000000..34d156c
--- /dev/null
+++ b/dlang/src/ocda/load_ssp.d
@@ -0,0 +1,337 @@
+module ocda.load_ssp;
+
+import std.algorithm : splitter, startsWith;
+import std.array : array, split;
+import std.conv : to, ConvException;
+import std.file : readText;
+import std.stdio : writeln;
+import std.string : strip, stripLeft, indexOf;
+import ocda.model;
+
+OcdaDocument parseSspFile(string filename) {
+ string content = readText(filename);
+ return parseSsp(content);
+}
+
+OcdaDocument parseSsp(string content) {
+ OcdaDocument doc;
+ OcdaObject current_obj;
+ bool in_object = false;
+ /+ ↓ whether a text line has been seen for the object being read.
+ Not "is the text non-empty": an object whose first text line is blank
+ (a verse opening on an empty line, which the sample set has seven of)
+ would then lose it, the next line overwriting rather than appending.
+ +/
+ bool seen_text = false;
+ string current_section = "";
+ string current_block = ""; // meta, make, doc_has, or section name
+
+ foreach (line; content.splitter("\n")) {
+ /+ ↓ comment. The first line of a .ssp declares the format version, and
+ it is the one comment that carries meaning: a reader checks the
+ major part and refuses what it does not know rather than loading
+ an unfamiliar artefact half populated in silence.
+ +/
+ if (line.length > 0 && line[0] == '%') {
+ enum _banner = "% SiSU Document Abstraction v";
+ enum _source = "% Source: ";
+ if (doc.format_version.length == 0 && line.startsWith(_banner)) {
+ doc.format_version = line[_banner.length .. $].strip;
+ } else if (doc.source_filename.length == 0 && line.startsWith(_source)) {
+ doc.source_filename = line[_source.length .. $].strip;
+ }
+ continue;
+ }
+
+ /+ ↓ blank line - finalize current object +/
+ if (line.length == 0 || line.strip.length == 0) {
+ if (in_object) {
+ commitObject(doc, current_section, current_obj);
+ current_obj = OcdaObject.init;
+ in_object = false;
+ seen_text = false;
+ }
+ continue;
+ }
+
+ /+ ↓ section/block open +/
+ if (line[0] == '@') {
+ auto brace_pos = line.indexOf('{');
+ if (brace_pos > 0) {
+ string name = line[1 .. brace_pos].strip;
+ current_block = name;
+ if (!isPreambleBlock(name)) {
+ current_section = name;
+ /+ ↓ a block that is neither preamble nor a section this reader
+ knows: counted, so that --report-unknown can say so +/
+ if (doc.sectionByName(name) is null) {
+ doc.unknown.block[name]++;
+ }
+ }
+ }
+ continue;
+ }
+
+ /+ ↓ section/block close +/
+ if (line[0] == '}') {
+ if (in_object) {
+ commitObject(doc, current_section, current_obj);
+ current_obj = OcdaObject.init;
+ in_object = false;
+ seen_text = false;
+ }
+ current_block = "";
+ continue;
+ }
+
+ /+ ↓ inside meta/make/doc_has blocks: key-value pairs +/
+ if (isPreambleBlock(current_block)) {
+ string stripped = line.stripLeft;
+ auto colon_pos = stripped.indexOf(':');
+ if (colon_pos > 0) {
+ string key = stripped[0 .. colon_pos].strip;
+ string value = (colon_pos + 1 < stripped.length)
+ ? stripped[colon_pos + 1 .. $].strip : "";
+ if (current_block == "meta") { doc.metadata[key] = value; }
+ else if (current_block == "make") { doc.make[key] = value; }
+ else if (current_block == "source") { doc.source[key] = value; }
+ else { doc.doc_has[key] = value; }
+ }
+ continue;
+ }
+
+ /+ ↓ object declaration: [N] type ... +/
+ if (line[0] == '[') {
+ if (in_object) {
+ commitObject(doc, current_section, current_obj);
+ }
+ current_obj = OcdaObject.init;
+ current_obj.section = current_section;
+ in_object = true;
+ seen_text = false;
+ parseObjectDeclaration(line, current_obj);
+ continue;
+ }
+
+ /+ ↓ property: .name: value +/
+ if (line[0] == '.') {
+ if (in_object) {
+ parseProperty(line, current_obj, doc.unknown);
+ }
+ continue;
+ }
+
+ /+ ↓ text content: | content +/
+ if (line.length >= 1 && line[0] == '|') {
+ if (in_object) {
+ string text_content = (line.length > 2) ? line[2 .. $] : "";
+ if (seen_text) {
+ current_obj.text ~= "\n" ~ text_content;
+ } else {
+ current_obj.text = text_content;
+ seen_text = true;
+ }
+ }
+ continue;
+ }
+
+ /+ ↓ no known shape. Counted by its first character so that a reader
+ can be told the file held something this one did not expect
+ +/
+ doc.unknown.line[line[0 .. 1].idup]++;
+ }
+
+ /+ ↓ finalize last object +/
+ if (in_object) {
+ commitObject(doc, current_section, current_obj);
+ }
+
+ return doc;
+}
+
+private void parseObjectDeclaration(string line, ref OcdaObject obj) {
+ /+ parse: [N] type :level identifier +/
+ auto bracket_close = line.indexOf(']');
+ if (bracket_close < 0) return;
+ try {
+ obj.ocn = line[1 .. bracket_close].strip.to!int;
+ } catch (ConvException) {
+ obj.ocn = 0;
+ }
+ string rest = (bracket_close + 1 < line.length)
+ ? line[bracket_close + 1 .. $].strip : "";
+ if (rest.length == 0) return;
+
+ auto parts = rest.split(" ");
+ if (parts.length == 0) return;
+ obj.is_a = parts[0];
+
+ if (obj.is_a == "heading" && parts.length >= 2) {
+ /+ parse :A, :B, :1, etc. +/
+ string lev = parts[1];
+ if (lev.length >= 2 && lev[0] == ':') {
+ obj.heading_level = lev[1 .. $];
+ }
+ /+ optional identifier after level +/
+ if (parts.length >= 3) {
+ obj.identifier = parts[2];
+ }
+ }
+}
+
+/+ ↓ the blocks that are document preamble rather than sections of objects +/
+private bool isPreambleBlock(string name) {
+ return name == "source" || name == "meta" || name == "make" || name == "doc_has";
+}
+
+private void parseProperty(string line, ref OcdaObject obj, ref OcdaUnknown unknown) {
+ /+ parse: .name: value +/
+ auto colon_pos = line.indexOf(':');
+ if (colon_pos < 1) return;
+ string name = line[1 .. colon_pos].strip;
+ string value = (colon_pos + 1 < line.length)
+ ? line[colon_pos + 1 .. $].strip : "";
+
+ switch (name) {
+ case "part": obj.part = value; break;
+ case "is_of_type": obj.is_of_type = value; break;
+ case "section": obj.own_section = value; break;
+ case "identifier": obj.identifier = value; break;
+ case "segment_html_is": obj.segment_html_is = value; break;
+ /+ ↓ the epub segments at heading levels 1 to 4 where the html
+ segments at level 4 alone, so the two are different files and
+ the abstraction records both +/
+ case "segment_epub": obj.segment_epub = value; break;
+ case "segment_epub_is": obj.segment_epub_is = value; break;
+ case "table_aligns": obj.table_aligns = value; break;
+ case "stow_link": obj.stow_link ~= value; break;
+ case "anchor_tag": obj.anchor_tags ~= value; break;
+ case "parent": obj.parent_ocn = toIntSafe(value); break;
+ case "last_descendant": obj.last_descendant_ocn = toIntSafe(value); break;
+ case "children": obj.children = parseIntArray(value); break;
+ case "ancestors": obj.ancestors = parseIntArray(value); break;
+ case "dummy": obj.dummy_heading = (value == "true"); break;
+ case "ocn_off": obj.ocn_off = (value == "true"); break;
+ case "indent":
+ auto parts = value.split(" ");
+ if (parts.length >= 1) obj.indent_base = toIntSafe(parts[0]);
+ if (parts.length >= 2) obj.indent_hang = toIntSafe(parts[1]);
+ break;
+ case "bullet": obj.bullet = (value == "true"); break;
+ case "lang": obj.lang = value; break;
+ case "has":
+ foreach (flag; value.split(" ")) {
+ switch (flag) {
+ case "links": obj.has_links = true; break;
+ case "notes_reg": obj.has_notes_reg = true; break;
+ case "notes_star": obj.has_notes_star = true; break;
+ case "images": obj.has_images = true; break;
+ default: break;
+ }
+ }
+ break;
+ case "segment": obj.segment = value; break;
+ case "segment_prev": obj.segment_prev = value; break;
+ case "segment_next": obj.segment_next = value; break;
+ case "anchor": obj.anchor = value; break;
+ case "heading_lev_anchor": obj.heading_lev_anchor = value; break;
+ case "table_cols": obj.table_cols = toIntSafe(value); break;
+ case "table_widths": obj.table_widths = value; break;
+ case "table_header": obj.table_header = (value == "true"); break;
+ case "code_syntax": obj.code_syntax = value; break;
+ case "code_linenumbers": obj.code_linenumbers = (value == "true"); break;
+ case "heading_ancestors_text": obj.heading_ancestors_text = value; break;
+ case "sha256": obj.sha256 = value; break;
+ case "image": obj.images ~= parseImageRecord(value); break;
+ /+ ↓ a property this reader has no use for, or has never heard of.
+ Ignored, which is the conforming behaviour within a major
+ version, but counted rather than passed over in silence
+ +/
+ default: unknown.property[name]++; break;
+ }
+}
+
+private OcdaImage parseImageRecord(string value) {
+ /+ parse: <filename> sha256:<HEX> bytes:<n> [key:value ...]
+ filename first, remaining fields order-independent; unrecognised
+ fields are kept rather than discarded +/
+ OcdaImage img;
+ bool first = true;
+ foreach (field; value.split(" ")) {
+ if (field.length == 0) continue;
+ if (first) {
+ img.name = field;
+ first = false;
+ continue;
+ }
+ auto colon_pos = field.indexOf(':');
+ if (colon_pos < 1) continue;
+ string key = field[0 .. colon_pos];
+ string val = (colon_pos + 1 < field.length) ? field[colon_pos + 1 .. $] : "";
+ switch (key) {
+ case "sha256": img.sha256 = val; break;
+ case "bytes": img.bytes = toUlongSafe(val); break;
+ case "px":
+ auto dims = val.split("x");
+ if (dims.length == 2) {
+ img.width = toIntSafe(dims[0]);
+ img.height = toIntSafe(dims[1]);
+ }
+ break;
+ case "missing": img.missing = (val == "true"); break;
+ default: img.fields[key] = val; break;
+ }
+ }
+ return img;
+}
+
+/+ ↓ two values the text form leaves out because they can be worked out.
+ .
+ The .ssp emits a property only when it carries information, so
+ .identifier is absent when it is simply the ocn, and .anchor is absent
+ when it is the segment name. The .ocda.db, being columns rather than
+ lines, stores both unconditionally. So a loader that takes the text
+ form literally gives a *different model* from the one the database
+ gives, and the difference is not small: in the reference collection
+ the identifier is omitted on all but a handful of objects.
+ .
+ The model is the thing both serialisations describe, so the loader
+ fills them in. Found by dumping both models and diffing them, which no
+ amount of comparing section counts would have shown.
++/
+private void deriveOmitted(ref OcdaObject obj) {
+ import std.conv : to;
+ if (obj.identifier.length == 0 && obj.ocn != 0) {
+ obj.identifier = obj.ocn.to!string;
+ }
+ if (obj.anchor.length == 0 && obj.segment.length > 0) {
+ obj.anchor = obj.segment;
+ }
+}
+
+private void commitObject(ref OcdaDocument doc, string section, ref OcdaObject obj) {
+ deriveOmitted(obj);
+ auto sec = doc.sectionByName(section);
+ if (sec !is null) {
+ *sec ~= obj;
+ }
+}
+
+private int toIntSafe(string s) {
+ try { return s.strip.to!int; }
+ catch (ConvException) { return 0; }
+}
+
+private ulong toUlongSafe(string s) {
+ try { return s.strip.to!ulong; }
+ catch (ConvException) { return 0; }
+}
+
+private int[] parseIntArray(string s) {
+ int[] result;
+ foreach (part; s.split(" ")) {
+ auto v = toIntSafe(part);
+ result ~= v;
+ }
+ return result;
+}
diff --git a/dlang/src/ocda/model.d b/dlang/src/ocda/model.d
new file mode 100644
index 0000000..403502a
--- /dev/null
+++ b/dlang/src/ocda/model.d
@@ -0,0 +1,147 @@
+module ocda.model;
+
+struct OcdaImage {
+ string name; // image filename as referenced by the object
+ string sha256; // hex-encoded (uppercase) digest of the file
+ ulong bytes; // file size
+ int width; // file's own pixel width (not display width)
+ int height; // file's own pixel height
+ bool missing; // file absent or unreadable, no sha256/bytes
+ string[string] fields; // any further/unrecognised key:value fields
+}
+
+struct OcdaObject {
+ int ocn;
+ string is_a; // heading, para, toc, table, code, group, block, verse, quote, endnote, glossary, bibliography, bookindex, blurb
+ string section; // head, toc, body, endnotes, glossary, bibliography, bookindex, blurb
+ string heading_level; // A, B, C, D, 1, 2, 3, 4 (for headings)
+ string identifier; // heading: from the declaration line; otherwise .identifier
+ string part; // frontmatter, body, backmatter
+ string is_of_type; // para, block, comment: how the text behaves
+ string own_section; // .section, when it differs from the block it sits in
+ string segment_html_is;// the html segment this object sits in
+ string segment_epub; // the epub segment this heading opens
+ string segment_epub_is;// the epub segment this object sits in
+ string table_aligns; // l c r per column
+ string[] stow_link; // links extracted from the object
+ string[] anchor_tags; // named targets the markup placed on this object
+ int parent_ocn;
+ int last_descendant_ocn;
+ int[] children; // direct child heading OCNs
+ int[] ancestors; // markedup_ancestors[8]
+ bool dummy_heading;
+ bool ocn_off;
+ int indent_base;
+ int indent_hang;
+ bool bullet;
+ string lang;
+ bool has_links;
+ bool has_notes_reg;
+ bool has_notes_star;
+ bool has_images;
+ string segment;
+ string segment_prev;
+ string segment_next;
+ string anchor;
+ string heading_lev_anchor; // a heading's own named target
+ int table_cols;
+ string table_widths;
+ bool table_header;
+ string code_syntax;
+ bool code_linenumbers;
+ string heading_ancestors_text;
+ string sha256;
+ OcdaImage[] images; // repeatable .image records, in order
+ string text; // content (lines joined with newline)
+
+ /+ ↓ whether this object is rendered at all.
+ is_of_type "comment" is markup commentary: carried by the
+ abstraction so that nothing is lost, and not put into output. An
+ output that renders it leaks the author notes into the document,
+ which is what spine itself does not do.
+ +/
+ bool renders() const {
+ return is_of_type != "comment";
+ }
+
+ /+ ↓ the section this object belongs to, which is the block it sits in
+ unless it says otherwise +/
+ string sectionOf() const {
+ return (own_section.length > 0) ? own_section : section;
+ }
+}
+
+/+ ↓ what the reader met and did not understand.
+ A reader that silently drops what it does not know cannot say whether
+ it understood the document, and cannot be used to check a
+ specification against an implementation: silence and understanding
+ look alike. So the parser counts what it ignored, and
+ --report-unknown prints it.
++/
+struct OcdaUnknown {
+ size_t[string] property; // .name -> times seen
+ size_t[string] block; // @name -> times seen
+ size_t[string] line; // first char -> times seen, lines of no known shape
+
+ bool any() const {
+ return property.length > 0 || block.length > 0 || line.length > 0;
+ }
+}
+
+struct OcdaDocument {
+ string format_version; // as declared on line 1, "" when absent
+ string source_filename; // from the "% Source:" line
+ string[string] source; // @source: language, languages, digest
+ OcdaUnknown unknown; // what was met and not understood
+ string[string] metadata; // key -> value
+ string[string] make; // key -> value
+ string[string] doc_has; // key -> value
+ OcdaObject[] head;
+ OcdaObject[] toc;
+ OcdaObject[] body;
+ OcdaObject[] endnotes;
+ OcdaObject[] glossary;
+ OcdaObject[] bibliography;
+ OcdaObject[] bookindex;
+ OcdaObject[] blurb;
+ OcdaObject[] tail;
+
+ /+ ↓ the major part of the declared version, "" when none was declared +/
+ string formatVersionMajor() const {
+ import std.string : indexOf;
+ if (format_version.length == 0) { return ""; }
+ auto dot = format_version.indexOf('.');
+ return (dot > 0) ? format_version[0 .. dot] : format_version;
+ }
+
+ string meta(string key) {
+ if (auto p = key in metadata) return *p;
+ return "";
+ }
+
+ OcdaObject[]* sectionByName(string name) {
+ switch (name) {
+ case "head": return &head;
+ case "toc": return &toc;
+ case "body": return &body;
+ case "endnotes": return &endnotes;
+ case "glossary": return &glossary;
+ case "bibliography": return &bibliography;
+ case "bookindex": return &bookindex;
+ case "blurb": return &blurb;
+ case "tail": return &tail;
+ default: return null;
+ }
+ }
+}
+
+/+ ↓ the format major version this reader knows.
+ The .ssp and the .ocda.db carry one version between them, declared in
+ band: the .ssp on its first line, the database in its schema.version
+ row. The minor part is raised when a property is added, and a reader of
+ the newer format still reads the older one, so only the major part is
+ checked here. Raise this when the reader has been brought to a new
+ major, not before: the point of the gate is to refuse what has not been
+ looked at.
++/
+enum ssp_format_major_read = "2";
diff --git a/dlang/src/ocda/sqlite.d b/dlang/src/ocda/sqlite.d
new file mode 100644
index 0000000..6f3753c
--- /dev/null
+++ b/dlang/src/ocda/sqlite.d
@@ -0,0 +1,122 @@
+module ocda.sqlite;
+
+/+ ↓ the sqlite3 C API, declared rather than depended upon.
+ .
+ A binding from the dub registry would be a dependency to fetch, and
+ the point of this reader is that it builds in a sandbox with no
+ network. The dozen functions a reader needs are declared here and the
+ library is linked directly, which costs a page of extern(C) and saves
+ a dependency.
++/
+
+import std.conv : to;
+import std.string : fromStringz, toStringz;
+
+extern (C) {
+ struct sqlite3;
+ struct sqlite3_stmt;
+
+ int sqlite3_open_v2(const(char)* filename, sqlite3** db, int flags, const(char)* vfs);
+ int sqlite3_close(sqlite3* db);
+ int sqlite3_prepare_v2(sqlite3* db, const(char)* sql, int nbyte,
+ sqlite3_stmt** stmt, const(char)** tail);
+ int sqlite3_step(sqlite3_stmt* stmt);
+ int sqlite3_finalize(sqlite3_stmt* stmt);
+ int sqlite3_column_count(sqlite3_stmt* stmt);
+ const(char)* sqlite3_column_name(sqlite3_stmt* stmt, int col);
+ const(ubyte)* sqlite3_column_text(sqlite3_stmt* stmt, int col);
+ int sqlite3_column_int(sqlite3_stmt* stmt, int col);
+ int sqlite3_column_type(sqlite3_stmt* stmt, int col);
+ const(char)* sqlite3_errmsg(sqlite3* db);
+ const(void)* sqlite3_column_blob(sqlite3_stmt* stmt, int col);
+ int sqlite3_column_bytes(sqlite3_stmt* stmt, int col);
+}
+
+enum SQLITE_OK = 0;
+enum SQLITE_ROW = 100;
+enum SQLITE_DONE = 101;
+enum SQLITE_NULL = 5;
+enum SQLITE_OPEN_READONLY = 0x00000001;
+
+class SqliteError : Exception {
+ this(string msg) { super(msg); }
+}
+
+/+ ↓ a database, opened read only.
+ Read only is deliberate: a reader has no business writing, and a
+ request that opens a database read-write can create an empty one where
+ a path is mistyped.
++/
+struct Db {
+ private sqlite3* _handle;
+
+ static Db open(string path) {
+ sqlite3* handle;
+ auto rc = sqlite3_open_v2(path.toStringz, &handle, SQLITE_OPEN_READONLY, null);
+ if (rc != SQLITE_OK) {
+ string msg = (handle is null)
+ ? "cannot open " ~ path
+ : "cannot open " ~ path ~ ": " ~ sqlite3_errmsg(handle).fromStringz.idup;
+ if (handle !is null) { sqlite3_close(handle); }
+ throw new SqliteError(msg);
+ }
+ Db db;
+ db._handle = handle;
+ return db;
+ }
+
+ void close() {
+ if (_handle !is null) { sqlite3_close(_handle); _handle = null; }
+ }
+
+ /+ ↓ every row of a query, each row as its columns in order +/
+ string[][] rows(string sql) {
+ sqlite3_stmt* stmt;
+ auto rc = sqlite3_prepare_v2(_handle, sql.toStringz, -1, &stmt, null);
+ if (rc != SQLITE_OK) {
+ throw new SqliteError(sqlite3_errmsg(_handle).fromStringz.idup ~ " in: " ~ sql);
+ }
+ scope(exit) { sqlite3_finalize(stmt); }
+ string[][] out_rows;
+ int cols = sqlite3_column_count(stmt);
+ while (sqlite3_step(stmt) == SQLITE_ROW) {
+ string[] row;
+ foreach (c; 0 .. cols) {
+ row ~= (sqlite3_column_type(stmt, c) == SQLITE_NULL)
+ ? "" : (cast(const(char)*) sqlite3_column_text(stmt, c)).fromStringz.idup;
+ }
+ out_rows ~= row;
+ }
+ return out_rows;
+ }
+
+ /+ ↓ the first column of the first row, "" when there is none +/
+ string one(string sql) {
+ auto r = rows(sql);
+ return (r.length > 0 && r[0].length > 0) ? r[0][0] : "";
+ }
+
+ /+ ↓ the first column of the first row as bytes, for a blob.
+ Separate from one(), because a blob is not text: an image read as text
+ stops at the first zero byte, and a png's ninth byte is zero.
+ +/
+ ubyte[] blob(string sql) {
+ sqlite3_stmt* stmt;
+ if (sqlite3_prepare_v2(_handle, sql.toStringz, -1, &stmt, null) != SQLITE_OK) {
+ throw new SqliteError("prepare: " ~ sqlite3_errmsg(_handle).fromStringz.idup
+ ~ " [" ~ sql ~ "]");
+ }
+ scope(exit) { sqlite3_finalize(stmt); }
+ if (sqlite3_step(stmt) != SQLITE_ROW) { return []; }
+ int n = sqlite3_column_bytes(stmt, 0);
+ if (n <= 0) { return []; }
+ auto p = cast(const(ubyte)*) sqlite3_column_blob(stmt, 0);
+ if (p is null) { return []; }
+ return p[0 .. n].dup;
+ }
+
+ /+ ↓ does a table or view of this name exist +/
+ bool has(string name) {
+ return one("SELECT count(*) FROM sqlite_master WHERE name = '" ~ name ~ "'") != "0";
+ }
+}
diff --git a/dlang/src/ocda/zip.d b/dlang/src/ocda/zip.d
new file mode 100644
index 0000000..9dff6f4
--- /dev/null
+++ b/dlang/src/ocda/zip.d
@@ -0,0 +1,115 @@
+module ocda.zip;
+
+/+ ↓ enough of the zip format to find one file in a pod and read it.
+ .
+ Not a zip library. A pod archive holds a handful of entries, spine
+ writes them **stored** (uncompressed) so that the outer zstd frame does
+ the compressing once over the whole thing, and what is wanted from it
+ here is one named entry. So: read the central directory, find the
+ entry, return its bytes.
+ .
+ Deflate is *refused rather than mishandled*. Phobos can inflate, but a
+ library that says it read an entry it in fact skipped is worse than one
+ that says it cannot, and spine's own archives do not need it.
++/
+
+import std.algorithm : endsWith;
+import std.conv : to;
+
+class OcdaZipError : Exception {
+ this(string msg) { super(msg); }
+}
+
+struct ZipEntry {
+ string name;
+ uint method; // 0 stored, 8 deflate
+ uint compressed;
+ uint uncompressed;
+ uint local_offset;
+}
+
+/+ ↓ the entries, from the central directory at the end of the file +/
+ZipEntry[] zipEntries(const(ubyte)[] bytes) {
+ enum uint eocd_sig = 0x06054b50;
+ enum uint cd_sig = 0x02014b50;
+ if (bytes.length < 22) {
+ throw new OcdaZipError("too short to be a zip archive");
+ }
+ /+ ↓ the end of central directory record is last, and may be followed by a
+ comment of up to 65535 bytes, so it is searched for backwards +/
+ ptrdiff_t eocd = -1;
+ ptrdiff_t lowest = bytes.length > (22 + 65535) ? bytes.length - (22 + 65535) : 0;
+ for (ptrdiff_t i = bytes.length - 22; i >= lowest; i--) {
+ if (u32(bytes, i) == eocd_sig) { eocd = i; break; }
+ }
+ if (eocd < 0) {
+ throw new OcdaZipError("no end of central directory record;"
+ ~ " it does not look like a zip archive");
+ }
+ uint count = u16(bytes, eocd + 10);
+ uint offset = u32(bytes, eocd + 16);
+ ZipEntry[] out_;
+ size_t p = offset;
+ foreach (_; 0 .. count) {
+ if (p + 46 > bytes.length || u32(bytes, p) != cd_sig) {
+ throw new OcdaZipError("central directory entry "
+ ~ out_.length.to!string ~ " is malformed");
+ }
+ ZipEntry e;
+ e.method = u16(bytes, p + 10);
+ e.compressed = u32(bytes, p + 20);
+ e.uncompressed = u32(bytes, p + 24);
+ uint name_len = u16(bytes, p + 28);
+ uint extra_len = u16(bytes, p + 30);
+ uint cmnt_len = u16(bytes, p + 32);
+ e.local_offset = u32(bytes, p + 42);
+ e.name = cast(string) bytes[p + 46 .. p + 46 + name_len].idup;
+ out_ ~= e;
+ p += 46 + name_len + extra_len + cmnt_len;
+ }
+ return out_;
+}
+
+/+ ↓ one entry's bytes, from its local header +/
+ubyte[] zipRead(const(ubyte)[] bytes, ZipEntry e) {
+ enum uint local_sig = 0x04034b50;
+ size_t p = e.local_offset;
+ if (p + 30 > bytes.length || u32(bytes, p) != local_sig) {
+ throw new OcdaZipError(e.name ~ ": local header is malformed");
+ }
+ uint name_len = u16(bytes, p + 26);
+ uint extra_len = u16(bytes, p + 28);
+ size_t start = p + 30 + name_len + extra_len;
+ size_t end = start + e.compressed;
+ if (end > bytes.length) {
+ throw new OcdaZipError(e.name ~ ": runs past the end of the archive");
+ }
+ if (e.method != 0) {
+ throw new OcdaZipError(e.name ~ ": compression method "
+ ~ e.method.to!string ~ "; this reader takes only stored entries,"
+ ~ " which is what spine writes");
+ }
+ return bytes[start .. end].dup;
+}
+
+/+ ↓ the first entry whose name ends this way, or none.
+ Used to find media/abstraction/<uid>.<lang>.ssp without knowing the uid.
++/
+ZipEntry[] zipFindSuffix(ZipEntry[] entries, string suffix) {
+ ZipEntry[] out_;
+ foreach (e; entries) {
+ if (e.name.endsWith(suffix)) { out_ ~= e; }
+ }
+ return out_;
+}
+
+private uint u16(const(ubyte)[] b, size_t i) {
+ return cast(uint) b[i] | (cast(uint) b[i + 1] << 8);
+}
+
+private uint u32(const(ubyte)[] b, size_t i) {
+ return cast(uint) b[i]
+ | (cast(uint) b[i + 1] << 8)
+ | (cast(uint) b[i + 2] << 16)
+ | (cast(uint) b[i + 3] << 24);
+}
diff --git a/dlang/src/ocda/zstd.d b/dlang/src/ocda/zstd.d
new file mode 100644
index 0000000..401f1a3
--- /dev/null
+++ b/dlang/src/ocda/zstd.d
@@ -0,0 +1,79 @@
+module ocda.zstd;
+
+/+ ↓ libzstd, declared rather than bound.
+ .
+ A .sisupod is one zstd frame wrapping a zip archive, so reading one
+ needs exactly two things from the library: is this a frame, and what is
+ in it. Both are declared here and the C library is linked, which is the
+ same arrangement ocda.sqlite uses: no vendored tree, no generated
+ header, and the prototypes say what is being called.
++/
+
+extern (C) @nogc nothrow private {
+ size_t ZSTD_decompress(void* dst, size_t dstCapacity,
+ const(void)* src, size_t compressedSize);
+ ulong ZSTD_getFrameContentSize(const(void)* src, size_t srcSize);
+ uint ZSTD_isError(size_t code);
+ const(char)* ZSTD_getErrorName(size_t code);
+}
+
+/+ ↓ the two values ZSTD_getFrameContentSize returns in place of a size +/
+private enum ulong contentSizeUnknown = ulong.max;
+private enum ulong contentSizeError = ulong.max - 1;
+
+/+ ↓ the magic number of a zstd frame, little endian 0xFD2FB528 +/
+private static immutable ubyte[4] zstd_magic = [0x28, 0xB5, 0x2F, 0xFD];
+
+class OcdaZstdError : Exception {
+ this(string msg) { super(msg); }
+}
+
+/+ ↓ does this begin with a zstd frame?
+ Asked of the bytes rather than of the filename, because what is inside
+ decides how a thing is opened. A .sisupod is a frame; a .zip published
+ before spine wrapped them is not, and both are read.
++/
+bool zstdIsFrame(const(ubyte)[] bytes) {
+ if (bytes.length < zstd_magic.length) { return false; }
+ foreach (i, b; zstd_magic) {
+ if (bytes[i] != b) { return false; }
+ }
+ return true;
+}
+
+/+ ↓ one frame in, its contents out.
+ max_size is a refusal rather than a buffer size: a frame declares how
+ much it expands to, and a frame that claims more than the caller is
+ willing to hold is not decompressed at all. A reader of an artefact it
+ did not make wants that limit.
++/
+ubyte[] zstdDecompress(const(ubyte)[] bytes, size_t max_size) {
+ if (!zstdIsFrame(bytes)) {
+ throw new OcdaZstdError("not a zstd frame");
+ }
+ ulong declared = ZSTD_getFrameContentSize(bytes.ptr, bytes.length);
+ if (declared == contentSizeError) {
+ throw new OcdaZstdError("zstd frame: could not read its content size");
+ }
+ if (declared == contentSizeUnknown) {
+ throw new OcdaZstdError("zstd frame declares no content size;"
+ ~ " this reader takes only frames that do");
+ }
+ if (declared > max_size) {
+ import std.conv : to;
+ throw new OcdaZstdError("zstd frame expands to " ~ declared.to!string
+ ~ " bytes, over the " ~ max_size.to!string ~ " byte limit");
+ }
+ auto out_ = new ubyte[](cast(size_t) declared);
+ size_t got = ZSTD_decompress(out_.ptr, out_.length, bytes.ptr, bytes.length);
+ if (ZSTD_isError(got)) {
+ import std.string : fromStringz;
+ throw new OcdaZstdError("zstd: "
+ ~ ZSTD_getErrorName(got).fromStringz.idup);
+ }
+ if (got != declared) {
+ throw new OcdaZstdError("zstd frame expanded to a different size"
+ ~ " than it declared");
+ }
+ return out_;
+}
diff --git a/dlang/test/agree.d b/dlang/test/agree.d
new file mode 100644
index 0000000..bdf1b2e
--- /dev/null
+++ b/dlang/test/agree.d
@@ -0,0 +1,84 @@
+/++
+ Do the two loaders agree?
+
+ The library's whole purpose is that a writer cannot tell which
+ serialisation it was handed. That is only true if loading a document's
+ .ssp and loading its .ocda.db give the same model, so this asks.
+
+ Usage: agree <doc>.ssp <doc>.ocda.db [lang]
++/
+import std.conv : to;
+import std.stdio;
+import ocda.model;
+import ocda.load;
+
+int main(string[] args) {
+ if (args.length < 3) {
+ stderr.writeln("usage: agree <doc>.ssp <doc>.ocda.db [lang]");
+ return 2;
+ }
+ string lang = (args.length > 3) ? args[3] : "";
+ OcdaDocument a, b;
+ try {
+ a = load(args[1]);
+ b = load(args[2], lang);
+ } catch (Exception e) {
+ stderr.writeln("error: ", e.msg);
+ return 2;
+ }
+
+ int differences = 0;
+ void differ(string what, string from_ssp, string from_db) {
+ writeln(" differ: ", what);
+ writeln(" .ssp: ", from_ssp);
+ writeln(" .ocda.db: ", from_db);
+ differences++;
+ }
+
+ if (a.format_version != b.format_version) {
+ differ("format_version", a.format_version, b.format_version);
+ }
+ if (a.source_filename != b.source_filename) {
+ differ("source_filename", a.source_filename, b.source_filename);
+ }
+
+ /+ ↓ the sections, object by object. Position within a section is the
+ identity to compare on: an ocn is not unique and 0 is shared +/
+ static immutable sections = ["head", "toc", "body", "endnotes", "glossary",
+ "bibliography", "bookindex", "blurb", "tail"];
+ foreach (name; sections) {
+ auto sa = a.sectionByName(name);
+ auto sb = b.sectionByName(name);
+ if (sa is null || sb is null) { continue; }
+ if ((*sa).length != (*sb).length) {
+ differ(name ~ " object count", (*sa).length.to!string, (*sb).length.to!string);
+ continue;
+ }
+ foreach (i, oa; *sa) {
+ auto ob = (*sb)[i];
+ string at = name ~ "[" ~ i.to!string ~ "]";
+ if (oa.ocn != ob.ocn) { differ(at ~ " ocn", oa.ocn.to!string, ob.ocn.to!string); }
+ if (oa.is_a != ob.is_a) { differ(at ~ " is_a", oa.is_a, ob.is_a); }
+ if (oa.is_of_type != ob.is_of_type) { differ(at ~ " is_of_type", oa.is_of_type, ob.is_of_type); }
+ if (oa.part != ob.part) { differ(at ~ " part", oa.part, ob.part); }
+ if (oa.parent_ocn != ob.parent_ocn) { differ(at ~ " parent", oa.parent_ocn.to!string, ob.parent_ocn.to!string); }
+ if (oa.heading_level != ob.heading_level) { differ(at ~ " level", oa.heading_level, ob.heading_level); }
+ if (oa.sha256 != ob.sha256) { differ(at ~ " sha256", oa.sha256, ob.sha256); }
+ if (oa.text != ob.text) { differ(at ~ " text", oa.text, ob.text); }
+ if (oa.ancestors != ob.ancestors) {
+ differ(at ~ " ancestors", oa.ancestors.to!string, ob.ancestors.to!string);
+ }
+ if (oa.images.length != ob.images.length) {
+ differ(at ~ " image count", oa.images.length.to!string, ob.images.length.to!string);
+ }
+ if (differences > 12) { writeln(" ... stopping after 12"); return 1; }
+ }
+ }
+
+ if (differences == 0) {
+ writeln("agree: the two loaders give the same model");
+ return 0;
+ }
+ writeln(differences, " difference(s)");
+ return 1;
+}
diff --git a/dlang/test/dump.d b/dlang/test/dump.d
new file mode 100644
index 0000000..17aa9bb
--- /dev/null
+++ b/dlang/test/dump.d
@@ -0,0 +1,35 @@
+/++
+ Write a loaded abstraction out canonically.
+
+ The oracle for everything that claims to implement this model: load a
+ .ssp or a .ocda.db and print the model, deterministically, so that two
+ implementations can be compared with diff.
+
+ Usage: ocda-dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
++/
+import std.stdio;
+import ocda.model;
+import ocda.load;
+import ocda.dump;
+
+int main(string[] args) {
+ bool unknown = false;
+ string[] rest;
+ foreach (a; args[1 .. $]) {
+ if (a == "--unknown") { unknown = true; } else { rest ~= a; }
+ }
+ if (rest.length < 1) {
+ stderr.writeln("usage: ocda-dump [--unknown] <file>.ssp|<file>.ocda.db [lang]");
+ return 2;
+ }
+ string lang = (rest.length > 1) ? rest[1] : "";
+ OcdaDocument doc;
+ try {
+ doc = load(rest[0], lang);
+ } catch (Exception e) {
+ stderr.writeln("error: ", e.msg);
+ return 2;
+ }
+ write(unknown ? dumpUnknown(doc) : dumpDocument(doc));
+ return 0;
+}
diff --git a/gleam/COPYRIGHT b/gleam/COPYRIGHT
new file mode 100644
index 0000000..adc5446
--- /dev/null
+++ b/gleam/COPYRIGHT
@@ -0,0 +1,48 @@
+- Name: sisudoc-ocda-lib
+ - Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2026 Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ sisudoc-ocda-lib
+
+ part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation, either version 3 of the License,
+ or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+
+ - Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+ - Homepages:
+ [https://www.sisudoc.org]
+
+ - Copyright for ./nix-gleam/ 2023 Arnar Gauti Ingason
+ - License for ./nix-gleam/ MIT License
diff --git a/gleam/README.md b/gleam/README.md
new file mode 100644
index 0000000..d586bd3
--- /dev/null
+++ b/gleam/README.md
@@ -0,0 +1,102 @@
+# ocda-lib, in Gleam
+
+The abstraction's model and both its loaders, as a Gleam library, and the
+two programs that put it to the question.
+
+| | |
+|-----------------|----------------------------------------------------------------------------------|
+| `ocda/model` | the document, the object, the image. Depends on nothing but the standard library |
+| `ocda/load_ssp` | text -> model |
+| `ocda/load_db` | sqlite -> model |
+| `ocda/load` | what a path is, and loading it |
+| `ocda/dump` | the model written out canonically, as an oracle |
+
+## Use
+
+```gleam
+import ocda/load
+import ocda/model
+
+case load.load(path, "en", False) {
+ Error(why) -> io.println_error(why)
+ Ok(doc) -> {
+ // and from here a writer neither knows nor cares which of the two
+ // files it was handed
+ list.each(model.section(doc, "body"), fn(obj) { ... })
+ }
+}
+```
+
+## The two programs
+
+```sh
+nix build
+./result/bin/ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
+./result/bin/ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
+```
+
+`agree` asks whether the two loaders give the same model. `dump` writes
+the whole model out canonically, and is the stronger check of the two,
+because it is **byte for byte what the D library's `ocda-dump` writes**.
+So the four questions can all be asked with `diff`:
+
+| | |
+|-----------------------------|-----------------------------|
+| D: `.ssp` == `.ocda.db` | 36 of 36 document-languages |
+| Gleam: `.ssp` == `.ocda.db` | 36 of 36 |
+| `.ssp`: D == Gleam | 36 of 36 |
+| `.ocda.db`: D == Gleam | 36 of 36 |
+
+## Deliberately a mirror of the D library
+
+Field for field, name for name, including where that is not the most
+Gleamish choice:
+
+- `text` is one `String` with the lines joined by newline, not a
+ `List(String)`. So an object whose only text line is blank is
+ indistinguishable from an object with no text, *in both languages, in the
+ same place*. A difference that existed in one and not the other would be
+ worse than a loss both share, because it would show up in every dump and
+ mean nothing.
+- the properties are typed fields rather than a list of pairs. A writer
+ wants `obj.parent_ocn`, not a lookup that might miss.
+
+The point is that the two agree. Making them agree is much cheaper when
+they are shaped alike, and the shape is the specification's, not either
+language's.
+
+## What the agreement found
+
+Four disagreements, none of which a comparison of section counts or of a
+handful of fields would have shown. All four are recorded in the D
+library's history; the short version:
+
+1. `.identifier` and `.anchor` are omitted by the text form when they can
+ be worked out (the identifier is the ocn, the anchor is the segment).
+ Both loaders now derive them. Over one document that is 1,314
+ identifiers.
+2. `make` and `doc_has` are one metadata table in the database,
+ distinguished by a key prefix. Both loaders unprefix them.
+3. The JSON columns must be *parsed*, not scanned: the encoder writes `/`
+ as `\/` and every heading with a slash in it came back wrong.
+4. `translation.ocn_aligned` is in the database and cannot be in a `.ssp`,
+ which is one language. It is a fact about the file, so neither loader
+ puts it in the document.
+
+## Build
+
+```sh
+nix build # sandboxed, no network
+nix develop -c gleam test
+```
+
+Shipped as an Erlang shipment rather than an escript, because sqlite on the
+BEAM is a NIF and a NIF cannot be loaded from inside an escript archive.
+The dependency hash in `flake.nix` pins what `manifest.toml` pins; change
+one and the other must change with it.
+
+## Format
+
+Specified in **sisudoc-abstraction-format**: `ocda-model.md` for the model,
+`ssp-format-specification.md` and `ocda-db-format-specification.md` for the
+two serialisations. At format **2.x**.
diff --git a/gleam/flake.lock b/gleam/flake.lock
new file mode 100644
index 0000000..5f236ab
--- /dev/null
+++ b/gleam/flake.lock
@@ -0,0 +1,27 @@
+{
+ "nodes": {
+ "nixpkgs": {
+ "locked": {
+ "lastModified": 1790185690,
+ "narHash": "sha256-xJ+X4hBtOcAFGBOe5nAMyMUeF9foJBmIOu3NjBqBycU=",
+ "owner": "NixOS",
+ "repo": "nixpkgs",
+ "rev": "4975466d324710c576dc11ad614684e6bd8cad8e",
+ "type": "github"
+ },
+ "original": {
+ "owner": "NixOS",
+ "ref": "nixos-unstable",
+ "repo": "nixpkgs",
+ "type": "github"
+ }
+ },
+ "root": {
+ "inputs": {
+ "nixpkgs": "nixpkgs"
+ }
+ }
+ },
+ "root": "root",
+ "version": 7
+}
diff --git a/gleam/flake.nix b/gleam/flake.nix
new file mode 100644
index 0000000..a5a3407
--- /dev/null
+++ b/gleam/flake.nix
@@ -0,0 +1,66 @@
+{
+ description = "ocda-lib: the object-centric document abstraction as a Gleam library";
+ inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
+ outputs = { self, nixpkgs }:
+ let
+ system = "x86_64-linux"; # adjust for aarch64-linux / darwin
+ pkgs = import nixpkgs { inherit system; };
+
+ name = "ocda_lib";
+
+ /* The builder, nix-gleam's, kept here in nix-gleam/ (see the README
+ there). It fetches each Hex dependency on its own, with the
+ checksum gleam has already written into manifest.toml, so there is
+ no hash in this file to keep up to date, and nothing a run of gleam
+ could make come out differently.
+ */
+ inherit (pkgs.callPackage ./nix-gleam { }) buildGleamApplication;
+
+ /* rebar3 with the port compiler already in it.
+ esqlite builds a NIF, and its rebar3 configuration asks for the
+ `pc` plugin, which rebar3 would otherwise fetch from Hex at build
+ time: impossible in a sandbox, and the build fails with
+ "missing_package pc". nixpkgs carries the plugin, so the answer is
+ a rebar3 that has it before it starts.
+ */
+ rebar3-pc = pkgs.beamPackages.rebar3WithPlugins {
+ plugins = [ pkgs.beamPackages.pc ];
+ };
+ in {
+ packages.${system} = {
+ default = self.packages.${system}.${name};
+ /* Shipped as an Erlang shipment rather than as an escript, because
+ sqlite on the BEAM is a NIF: esqlite compiles its own sqlite into
+ a shared object, and a NIF cannot be loaded from inside an
+ escript archive, the runtime needing a real file on disk. The
+ shipment keeps priv/ on disk, so it loads.
+ */
+ ${name} = buildGleamApplication {
+ pname = "ocda-lib";
+ version = "2.0.0";
+ src = ./.;
+ rebar3Package = rebar3-pc;
+ nativeBuildInputs = [ pkgs.makeWrapper ];
+ preBuild = ''
+ export HOME=$TMPDIR
+ '';
+ installPhase = ''
+ runHook preInstall
+ mkdir -p $out/lib $out/bin
+ cp -r build/erlang-shipment $out/lib/${name}
+ makeWrapper $out/lib/${name}/entrypoint.sh $out/bin/ocda-lib \
+ --add-flags run \
+ --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.beamPackages.erlang pkgs.coreutils ]}
+ runHook postInstall
+ '';
+ meta = {
+ description = "the abstraction as a Gleam library";
+ mainProgram = "ocda-lib";
+ };
+ };
+ };
+ devShells.${system}.default = pkgs.mkShell {
+ packages = [ pkgs.gleam pkgs.beamPackages.erlang pkgs.rebar3 pkgs.sqlite ];
+ };
+ };
+}
diff --git a/gleam/gleam.toml b/gleam/gleam.toml
new file mode 100644
index 0000000..7b474eb
--- /dev/null
+++ b/gleam/gleam.toml
@@ -0,0 +1,14 @@
+name = "ocda_lib"
+version = "2.0.0"
+description = "the object-centric document abstraction as a library: the model, and loaders for its two serialisations"
+licences = ["AGPL-3.0-or-later"]
+
+[dependencies]
+gleam_stdlib = ">= 1.0.0 and < 2.0.0"
+argv = ">= 1.1.0 and < 2.0.0"
+simplifile = ">= 2.7.0 and < 3.0.0"
+sqlight = ">= 1.0.0 and < 2.0.0"
+gleam_json = ">= 3.0.0 and < 4.0.0"
+
+[dev-dependencies]
+gleeunit = ">= 1.0.0 and < 2.0.0"
diff --git a/gleam/manifest.toml b/gleam/manifest.toml
new file mode 100644
index 0000000..2b0cac1
--- /dev/null
+++ b/gleam/manifest.toml
@@ -0,0 +1,26 @@
+# Do not manually edit this file, it is managed by Gleam.
+#
+# This file locks the dependency versions used, to make your build
+# deterministic and to prevent unexpected versions from being included
+# in your application.
+#
+# You should check this file into your source control repository.
+
+packages = [
+ { name = "argv", version = "1.1.0", build_tools = ["gleam"], requirements = [], otp_app = "argv", source = "hex", outer_checksum = "3277D100448BDB4A29B6D58C0F36F631CBC349E8BDD09766C6309DF202831140" },
+ { name = "esqlite", version = "0.9.0", build_tools = ["rebar3"], requirements = [], otp_app = "esqlite", source = "hex", outer_checksum = "CCF72258A4EE152EC7AD92AA9A03552EB6CA1B06B65C93AD5B6E55C302E05855" },
+ { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" },
+ { name = "gleam_json", version = "3.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_json", source = "hex", outer_checksum = "44FDAA8847BE8FC48CA7A1C089706BD54BADCC4C45B237A992EDDF9F2CDB2836" },
+ { name = "gleam_stdlib", version = "1.0.5", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "CEE5B6C076A85B45F60C585F4316C63EC8B7127C119D5738C3958A9C4D50404E" },
+ { name = "gleeunit", version = "1.11.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleeunit", source = "hex", outer_checksum = "EC31ABA74256AEA531EDF8169931D775BBB384FED0A8A1BDC4DD9354E3E21826" },
+ { name = "simplifile", version = "2.7.0", build_tools = ["gleam"], requirements = ["filepath", "gleam_stdlib"], otp_app = "simplifile", source = "hex", outer_checksum = "A2727627B063E87351934C7F7F008F2D1FDB16F6DE0B8C79F9E46459CFC9C164" },
+ { name = "sqlight", version = "1.2.0", build_tools = ["gleam"], requirements = ["esqlite", "gleam_stdlib"], otp_app = "sqlight", source = "hex", outer_checksum = "841768D0A81107EE2DB46B949354F284A68B2E8098315C28A1FE3C07D206C17A" },
+]
+
+[requirements]
+argv = { version = ">= 1.1.0 and < 2.0.0" }
+gleam_json = { version = ">= 3.0.0 and < 4.0.0" }
+gleam_stdlib = { version = ">= 1.0.0 and < 2.0.0" }
+gleeunit = { version = ">= 1.0.0 and < 2.0.0" }
+simplifile = { version = ">= 2.7.0 and < 3.0.0" }
+sqlight = { version = ">= 1.0.0 and < 2.0.0" }
diff --git a/gleam/nix-gleam/LICENSE b/gleam/nix-gleam/LICENSE
new file mode 100644
index 0000000..c35ad09
--- /dev/null
+++ b/gleam/nix-gleam/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Arnar Gauti Ingason
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/gleam/nix-gleam/README.md b/gleam/nix-gleam/README.md
new file mode 100644
index 0000000..39bd1b6
--- /dev/null
+++ b/gleam/nix-gleam/README.md
@@ -0,0 +1,32 @@
+# nix-gleam, kept here
+
+`default.nix` is the builder of **nix-gleam**, by Arnar Gauti Ingason,
+copied unchanged from
+
+- https://github.com/arnarg/nix-gleam
+- `builder/default.nix` at commit `1f38154` (2026-10-06)
+
+under the MIT licence, which is in `LICENSE` beside it.
+
+It is kept here rather than taken as a flake input so that the build does
+not depend on a third repository being reachable, and so that what builds
+this package is in the package.
+
+## Why it is used
+
+There is no Gleam builder in nixpkgs, and a sandboxed build cannot fetch
+from Hex. The usual answer, one fixed output derivation that runs
+`gleam deps download` and is pinned by a hash of everything it fetched, has
+two faults: the hash has to be updated by hand whenever a dependency
+changes, and gleam writes `packages.toml` in a different order on each run,
+so the same dependencies do not always give the same hash.
+
+nix-gleam fetches each Hex package on its own with `fetchHex`, using the
+checksum gleam has already written for it into `manifest.toml`, and writes
+`packages.toml` itself. So there is no hash to maintain, and nothing a run
+of gleam could make come out differently.
+
+## Updating it
+
+Copy `builder/default.nix` from a newer commit over `default.nix`, update
+the commit above, and build.
diff --git a/gleam/nix-gleam/default.nix b/gleam/nix-gleam/default.nix
new file mode 100644
index 0000000..d074263
--- /dev/null
+++ b/gleam/nix-gleam/default.nix
@@ -0,0 +1,280 @@
+{
+ stdenv,
+ lib,
+ git,
+ xxhash,
+ fetchHex,
+ gleam,
+ beamPackages,
+ rsync,
+ nodejs,
+}: let
+ inherit (builtins) fromTOML readFile;
+in {
+ buildGleamApplication = {
+ src,
+ nativeBuildInputs ? [],
+ localPackages ? [],
+ erlangPackage ? beamPackages.erlang,
+ rebar3Package ? beamPackages.rebar3,
+ ...
+ } @ attrs: let
+ # gleam.toml contains an application name and version.
+ gleamToml = lib.importTOML (src + "/gleam.toml");
+
+ # manifest.toml contains a list of required packages including a sha256 checksum
+ # that can be used by nix fetchHex fetcher.
+ manifestToml = lib.importTOML (src + "/manifest.toml");
+
+ # Specify which target to build for.
+ buildTarget = attrs.target or gleamToml.target or "erlang";
+
+ # Generates a packages.toml expected by gleam compiler.
+ packagesTOML = with lib;
+ concatStringsSep "\n" (
+ ["[packages]"]
+ ++ (map
+ (p: "${p.name} = \"${p.version}\"")
+ manifestToml.packages)
+ ++ (lib.optional (gitRecordsTOML != "") "")
+ ++ (lib.optional (gitRecordsTOML != "") gitRecordsTOML)
+ );
+
+ # Gleam records a git dependency twice: once in `[packages]`,
+ # and once in a `[git.<name>]` table with the commit.
+ # Without the second, Gleam will try to download it.
+ gitRecordsTOML = with lib;
+ concatMapStringsSep "\n" (
+ p: "[git.${p.name}]\ncommit = \"${p.commit}\""
+ ) (filterPackagesBySource "git" manifestToml.packages);
+
+ # Helper function to filter manifest.toml packages
+ filterPackagesBySource = type: packages: lib.lists.filter (p: p.source == type) packages;
+
+ gitDerivs =
+ map
+ (p: {
+ name = p.name;
+ derivation = fetchGit {
+ url = p.repo;
+ rev = p.commit;
+ };
+ })
+ (filterPackagesBySource "git" manifestToml.packages);
+
+ # Fetch all dependencies
+ depsDerivs =
+ map
+ (p: {
+ name = p.name;
+ derivation = fetchHex {
+ inherit (p) version;
+ pkg = p.name;
+ sha256 = p.outer_checksum;
+ };
+ })
+ (filterPackagesBySource "hex" manifestToml.packages);
+
+ # Find replacement paths for `local` package dependencies
+ # from `localPackages` list.
+ localDeps = let
+ # Build a lookup attrset for local packages.
+ localDerivs = lib.mergeAttrsList (map (
+ p: let
+ name = (fromTOML (readFile (p + "/gleam.toml"))).name;
+ in {
+ "${name}" = p;
+ }
+ )
+ localPackages);
+ in
+ map (
+ p: {
+ inherit (p) name path;
+ localSrc =
+ if localDerivs ? "${p.name}"
+ then localDerivs.${p.name}
+ else throw "Local dependency \"${p.name}\" not found in `localPackages`.";
+ # Keep local packages in a writable location during build.
+ newPath = p.path;
+ }
+ ) (filterPackagesBySource "local" manifestToml.packages);
+
+ # Check if elixir is needed in nativeBuildInputs by checking if "mix" is in
+ # required build_tools.
+ isElixirProject = with lib; p: any (t: t == "mix") p.build_tools;
+ needsElixir = with lib; any isElixirProject manifestToml.packages;
+
+ # nativeBuildInputs needed for both targets.
+ defaultNativeBuildInputs = [gleam beamPackages.hex rsync git xxhash];
+ in
+ # Base common mkDerivation attributes
+ stdenv.mkDerivation (attrs
+ // {
+ pname = attrs.pname or gleamToml.name;
+ version = attrs.version or gleamToml.version;
+
+ src = lib.cleanSource attrs.src;
+
+ # Here we must copy the dependencies into the right spot and
+ # create a packages.toml file so the gleam compiler does not
+ # attempt to pull the dependencies from the internet.
+ configurePhase =
+ attrs.configurePhase
+ or ''
+ runHook preConfigure
+
+ mkdir -p build/packages
+
+ # Write the packages.toml file
+ cat <<EOF > build/packages/packages.toml
+ ${packagesTOML}
+ EOF
+
+ ${
+ # Gleam writes build output into the local dependency's own source
+ # directory, so it cannot be the read-only store path.
+ lib.concatStringsSep "\n" (
+ lib.forEach localDeps (
+ d: ''
+ mkdir -p "${d.newPath}"
+ rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r ${d.localSrc}/* "${d.newPath}/"
+ ''
+ )
+ )
+ }
+
+ ${
+ lib.concatStringsSep "\n" (
+ lib.forEach gitDerivs (
+ d: ''
+ rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r ${d.derivation}/* build/packages/${d.name}/
+ ''
+ )
+ )
+ }
+
+ # Copy all the dependencies into place
+ ${lib.concatStringsSep "\n" (
+ lib.forEach depsDerivs (
+ # gleam outputs files inside the dependency's source directory
+ # and therefor it needs to have permissive permissions.
+ d: ''
+ rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r ${d.derivation}/* build/packages/${d.name}/
+ ''
+ )
+ )}
+
+ # To prevent dependency resolution in Gleam 1.15+, local packages
+ # need to have their fingerprint up-to-date.
+ ${lib.concatStringsSep "\n" (
+ lib.forEach localDeps (d: ''
+ printf "%u" 0x$(xxhsum -H3 ${d.newPath}/gleam.toml | cut -d' ' -f1 | cut -d '_' -f2) > build/packages/${d.name}.config_fingerprint
+ '')
+ )}
+
+ ${
+ # Prime local package dependency caches so they do not try to fetch.
+ #
+ # Gleam compiles a local dependency inside that dependency's own
+ # directory, using the package cache found there.
+ lib.concatStringsSep "\n" (
+ lib.forEach localDeps (
+ d: ''
+ mkdir -p "${d.newPath}/build/packages"
+ cp build/packages/packages.toml "${d.newPath}/build/packages/packages.toml"
+ rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r build/packages/* "${d.newPath}/build/packages/"
+ ''
+ )
+ )
+ }
+
+ runHook postConfigure
+ '';
+ }
+ # When the build target is erlang
+ // lib.optionalAttrs (buildTarget == "erlang") {
+ nativeBuildInputs =
+ defaultNativeBuildInputs
+ ++ [erlangPackage rebar3Package]
+ ++ (lib.optional needsElixir beamPackages.elixir)
+ ++ nativeBuildInputs;
+
+ # The gleam compiler has a nice export function for erlang shipment.
+ buildPhase =
+ attrs.buildPhase
+ or ''
+ runHook preBuild
+
+ export REBAR_CACHE_DIR="$TMP/.rebar-cache"
+
+ gleam export erlang-shipment
+
+ runHook postBuild
+ '';
+
+ # Install all built packages into lib and create an entrypoint script
+ # that starts the application.
+ installPhase =
+ attrs.installPhase
+ or ''
+ runHook preInstall
+
+ mkdir -p $out/{bin,lib}
+
+ rsync --exclude=entrypoint.sh -r build/erlang-shipment/* $out/lib/
+
+ cat <<EOF > $out/bin/${gleamToml.name}
+ #!/usr/bin/env sh
+ ${erlangPackage}/bin/erl \
+ -pa $out/lib/*/ebin \
+ -eval "${gleamToml.name}@@main:run(${gleamToml.name})" \
+ -noshell \
+ -extra "\$@"
+ EOF
+ chmod +x $out/bin/${gleamToml.name}
+
+ runHook postInstall
+ '';
+ }
+ # When the build target is javascript
+ // lib.optionalAttrs (buildTarget == "javascript") {
+ nativeBuildInputs = defaultNativeBuildInputs ++ nativeBuildInputs;
+
+ # The gleam compiler doesn't provide an export mechanism for javascript target.
+ buildPhase =
+ attrs.buildPhase
+ or ''
+ runHook prebuild
+
+ gleam build --target javascript
+
+ runHook postBuild
+ '';
+
+ # Install all built packages into lib and create an entrypoint script
+ # that starts the application.
+ installPhase =
+ attrs.installPhase
+ or ''
+ runHook preInstall
+
+ mkdir -p $out/{bin,lib}
+
+ rsync --exclude=gleam.lock --exclude=gleam_version -r build/dev/javascript/* $out/lib/
+
+ cat <<EOF > $out/lib/${gleamToml.name}/main.mjs
+ import { main } from "./${gleamToml.name}.mjs";
+ main();
+ EOF
+
+ cat <<EOF > $out/bin/${gleamToml.name}
+ #!/usr/bin/env sh
+ ${nodejs}/bin/node $out/lib/${gleamToml.name}/main.mjs "\$@"
+ EOF
+ chmod +x $out/bin/${gleamToml.name}
+
+ runHook postInstall
+ '';
+ });
+}
diff --git a/gleam/src/COPYRIGHT b/gleam/src/COPYRIGHT
new file mode 100644
index 0000000..7e7dd56
--- /dev/null
+++ b/gleam/src/COPYRIGHT
@@ -0,0 +1,45 @@
+- Name: sisudoc-ocda-lib
+ - Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2026 Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ sisudoc-ocda-lib
+
+ part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation, either version 3 of the License,
+ or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+
+ - Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+ - Homepages:
+ [https://www.sisudoc.org]
diff --git a/gleam/src/ocda/dump.gleam b/gleam/src/ocda/dump.gleam
new file mode 100644
index 0000000..a75da67
--- /dev/null
+++ b/gleam/src/ocda/dump.gleam
@@ -0,0 +1,247 @@
+//// The model, written out canonically.
+////
+//// Not an output format: an *oracle*. Two libraries in two languages
+//// implement the same model from the same specification, and the cheap way
+//// to ask whether they agree is to have each write the whole loaded
+//// document as text and diff the two. Byte for byte identical to the D
+//// library's `ocda/dump.d`, which is the whole point of it.
+////
+//// The rules, which both implementations follow:
+////
+//// - every line is `<key> <value>`, the value running to end of line,
+//// and only one free-form value per line;
+//// - a field at its default is *omitted*, so the output says what the
+//// document has rather than what the type declares;
+//// - dictionaries come in sorted key order, because neither language's
+//// map has an order worth relying on;
+//// - sections come in the fixed order of the format.
+////
+//// Anything that reads this to reconstruct a document has misunderstood
+//// it. Load the `.ssp` or the `.ocda.db`.
+
+import gleam/dict.{type Dict}
+import gleam/int
+import gleam/list
+import gleam/string
+import ocda/model.{type Document, type Object}
+
+pub fn document(doc: Document) -> String {
+ let lines =
+ []
+ |> kv("version", doc.format_version)
+ |> kv("source_filename", doc.source_filename)
+ |> entries("source", doc.source)
+ |> entries("meta", doc.metadata)
+ |> entries("make", doc.make)
+ |> entries("doc_has", doc.doc_has)
+ |> counts(doc)
+ |> objects(doc)
+ lines |> list.reverse |> join
+}
+
+/// What the loader met and did not understand, sorted.
+///
+/// Separate from the document dump: it describes the loader, not the
+/// document, and the `.ocda.db` loader has nothing to report where the
+/// `.ssp` loader does.
+pub fn unknown(doc: Document) -> String {
+ []
+ |> numbered("property", doc.unknown.property)
+ |> numbered("block", doc.unknown.block)
+ |> numbered("line", doc.unknown.line)
+ |> list.reverse
+ |> join
+}
+
+fn join(lines: List(String)) -> String {
+ case lines {
+ [] -> ""
+ _ -> string.join(lines, "\n") <> "\n"
+ }
+}
+
+/// Accumulated newest-first and reversed once, so that nothing appends to
+/// the tail of a list that grows to a few hundred thousand lines.
+fn kv(acc: List(String), key: String, value: String) -> List(String) {
+ case value {
+ "" -> acc
+ v -> [key <> " " <> v, ..acc]
+ }
+}
+
+fn kn(acc: List(String), key: String, value: Int) -> List(String) {
+ case value {
+ 0 -> acc
+ n -> [key <> " " <> int.to_string(n), ..acc]
+ }
+}
+
+fn kb(acc: List(String), key: String, value: Bool) -> List(String) {
+ case value {
+ False -> acc
+ True -> [key <> " 1", ..acc]
+ }
+}
+
+fn entries(
+ acc: List(String),
+ prefix: String,
+ d: Dict(String, String),
+) -> List(String) {
+ d
+ |> dict.to_list
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.fold(acc, fn(lines, pair) {
+ [prefix <> " " <> pair.0 <> " " <> pair.1, ..lines]
+ })
+}
+
+fn numbered(
+ acc: List(String),
+ prefix: String,
+ d: Dict(String, Int),
+) -> List(String) {
+ d
+ |> dict.to_list
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.fold(acc, fn(lines, pair) {
+ [prefix <> " " <> pair.0 <> " " <> int.to_string(pair.1), ..lines]
+ })
+}
+
+fn counts(acc: List(String), doc: Document) -> List(String) {
+ list.fold(model.section_names, acc, fn(lines, name) {
+ case model.section(doc, name) {
+ [] -> lines
+ objects ->
+ [
+ "section " <> name <> " " <> int.to_string(list.length(objects)),
+ ..lines
+ ]
+ }
+ })
+}
+
+fn objects(acc: List(String), doc: Document) -> List(String) {
+ list.fold(model.section_names, acc, fn(lines, name) {
+ model.section(doc, name)
+ |> list.index_fold(lines, fn(l, obj, i) {
+ object(l, name, i, obj)
+ })
+ })
+}
+
+fn object(
+ acc: List(String),
+ section: String,
+ index: Int,
+ obj: Object,
+) -> List(String) {
+ let acc = ["object " <> section <> " " <> int.to_string(index), ..acc]
+ acc
+ |> kn("ocn", obj.ocn)
+ |> kv("is_a", obj.is_a)
+ |> kv("own_section", obj.own_section)
+ |> kv("level", obj.heading_level)
+ |> kv("identifier", obj.identifier)
+ |> kv("part", obj.part)
+ |> kv("is_of_type", obj.is_of_type)
+ |> kv("segment_html_is", obj.segment_html_is)
+ |> kv("segment_epub", obj.segment_epub)
+ |> kv("segment_epub_is", obj.segment_epub_is)
+ |> kn("parent", obj.parent_ocn)
+ |> kn("last_descendant", obj.last_descendant_ocn)
+ |> kv("children", ints(obj.children))
+ |> kv("ancestors", ints(obj.ancestors))
+ |> kb("dummy", obj.dummy_heading)
+ |> kb("ocn_off", obj.ocn_off)
+ |> kn("indent_base", obj.indent_base)
+ |> kn("indent_hang", obj.indent_hang)
+ |> kb("bullet", obj.bullet)
+ |> kv("lang", obj.lang)
+ |> kb("has_links", obj.has_links)
+ |> kb("has_notes_reg", obj.has_notes_reg)
+ |> kb("has_notes_star", obj.has_notes_star)
+ |> kb("has_images", obj.has_images)
+ |> kv("segment", obj.segment)
+ |> kv("segment_prev", obj.segment_prev)
+ |> kv("segment_next", obj.segment_next)
+ |> kv("anchor", obj.anchor)
+ |> kv("heading_lev_anchor", obj.heading_lev_anchor)
+ |> kn("table_cols", obj.table_cols)
+ |> kv("table_widths", obj.table_widths)
+ |> kv("table_aligns", obj.table_aligns)
+ |> kb("table_header", obj.table_header)
+ |> kv("code_syntax", obj.code_syntax)
+ |> kb("code_linenumbers", obj.code_linenumbers)
+ |> kv("heading_ancestors_text", obj.heading_ancestors_text)
+ |> kv("sha256", obj.sha256)
+ |> links(obj)
+ |> anchor_tags(obj)
+ |> images(obj)
+ |> text(obj)
+}
+
+fn links(acc: List(String), obj: Object) -> List(String) {
+ list.fold(obj.stow_link, acc, fn(lines, url) { kv(lines, "stow_link", url) })
+}
+
+fn anchor_tags(acc: List(String), obj: Object) -> List(String) {
+ list.fold(obj.anchor_tags, acc, fn(lines, a) { kv(lines, "anchor_tag", a) })
+}
+
+fn images(acc: List(String), obj: Object) -> List(String) {
+ list.fold(obj.images, acc, fn(lines, img) {
+ // the name last, because a filename may hold a space where none of the
+ // other fields can
+ let lines =
+ kv(
+ lines,
+ "image",
+ img.sha256
+ <> " "
+ <> int.to_string(img.bytes)
+ <> " "
+ <> int.to_string(img.width)
+ <> "x"
+ <> int.to_string(img.height)
+ <> " "
+ <> case img.missing {
+ True -> "1"
+ False -> "0"
+ }
+ <> " "
+ <> img.name,
+ )
+ img.fields
+ |> dict.to_list
+ |> list.sort(fn(a, b) { string.compare(a.0, b.0) })
+ |> list.fold(lines, fn(l, pair) {
+ kv(l, "image_field", pair.0 <> " " <> pair.1)
+ })
+ })
+}
+
+/// Text, when there is any.
+///
+/// An object whose only text line is blank holds "" here, indistinguishable
+/// from no text at all, because the model joins the lines into one string.
+/// Both libraries lose the same distinction in the same place, which is
+/// what makes them comparable; it is recorded rather than hidden.
+fn text(acc: List(String), obj: Object) -> List(String) {
+ case obj.text {
+ "" -> acc
+ t -> {
+ let lines = string.split(t, on: "\n")
+ let acc = kv(acc, "text", int.to_string(list.length(lines)))
+ list.fold(lines, acc, fn(l, line) { ["| " <> line, ..l] })
+ }
+ }
+}
+
+fn ints(xs: List(Int)) -> String {
+ case xs {
+ [] -> ""
+ _ -> xs |> list.map(int.to_string) |> string.join(",")
+ }
+}
diff --git a/gleam/src/ocda/images.gleam b/gleam/src/ocda/images.gleam
new file mode 100644
index 0000000..2f3a56e
--- /dev/null
+++ b/gleam/src/ocda/images.gleam
@@ -0,0 +1,135 @@
+//// An image's bytes, from whichever source the document came from.
+////
+//// The model *names* images, with digest, byte size and pixel size; it does
+//// not hold them. That is right - an abstraction of a document is not a
+//// bundle of its assets - but a writer of any format that shows images needs
+//// the bytes, so something has to know where they are. This is that
+//// something, and it is in the library rather than in a writer because the
+//// answer depends on the *input form* and not on the output.
+////
+//// | `.ocda.db` | the `files` table, role 'image', by name |
+//// | `.zip` | the archive entry whose path ends in `/image/<name>` |
+//// | `.sisupod` | the same, if the frame can be unwrapped, which on the BEAM it cannot |
+//// | `.ssp` | beside it: a pod puts the abstraction in `media/abstraction/` and its images in `media/image/` |
+
+import gleam/bit_array
+import gleam/dynamic/decode
+import gleam/list
+import gleam/result
+import gleam/string
+import ocda/load
+import ocda/zip
+import simplifile
+import sqlight
+
+/// The bytes, or an error saying why not.
+pub fn image_bytes(source: String, name: String) -> Result(BitArray, String) {
+ case load.source_of(source) {
+ load.OcdaDb -> from_db(source, name)
+ load.Pod -> from_archive(source, name)
+ load.Ssp -> from_beside(source, name)
+ load.Unknown -> Error(source <> ": not a source this library reads")
+ }
+}
+
+/// Can this source give images at all?
+///
+/// A writer asks before it decides whether to emit them, because an `<img>`
+/// pointing at a file it did not include is worse than no image.
+pub fn can_give_images(source: String) -> Bool {
+ case load.source_of(source) {
+ load.OcdaDb -> True
+ load.Pod -> zip_readable(source)
+ load.Ssp -> image_dir_beside(source) != ""
+ load.Unknown -> False
+ }
+}
+
+fn zip_readable(path: String) -> Bool {
+ case simplifile.read_bits(path) {
+ Error(_) -> False
+ Ok(bytes) ->
+ case zip.entries(bytes) {
+ Ok(_) -> True
+ Error(_) -> False
+ }
+ }
+}
+
+fn from_db(path: String, name: String) -> Result(BitArray, String) {
+ case sqlight.open("file:" <> path <> "?mode=ro") {
+ Error(e) -> Error(path <> ": " <> e.message)
+ Ok(conn) -> {
+ let out =
+ sqlight.query(
+ "select data from files where role = 'image' and name = ?1",
+ on: conn,
+ with: [sqlight.text(name)],
+ expecting: blob(),
+ )
+ |> result.replace_error(path <> ": could not read " <> name)
+ let _ = sqlight.close(conn)
+ case out {
+ Ok([data, ..]) -> Ok(data)
+ Ok([]) -> Error(name <> ": not in this database")
+ Error(e) -> Error(e)
+ }
+ }
+ }
+}
+
+/// A blob column, read as bytes rather than as text: an image read as text
+/// stops at the first zero byte, and a png's ninth byte is zero.
+fn blob() -> decode.Decoder(BitArray) {
+ use v <- decode.field(0, decode.bit_array)
+ decode.success(v)
+}
+
+fn from_archive(path: String, name: String) -> Result(BitArray, String) {
+ use bytes <- result.try(
+ simplifile.read_bits(path)
+ |> result.replace_error(path <> ": cannot read"),
+ )
+ use entries <- result.try(zip.entries(bytes))
+ case list.find(entries, fn(e) { string.ends_with(e.name, "/image/" <> name) }) {
+ Error(_) -> Error(name <> ": not in this archive")
+ Ok(entry) -> zip.read(bytes, entry)
+ }
+}
+
+fn from_beside(path: String, name: String) -> Result(BitArray, String) {
+ case image_dir_beside(path) {
+ "" -> Error(name <> ": a .ssp outside a pod has no image directory beside it")
+ dir ->
+ simplifile.read_bits(dir <> "/" <> name)
+ |> result.replace_error(dir <> "/" <> name <> ": cannot read")
+ }
+}
+
+/// `media/abstraction/<x>.ssp` -> `media/image/`
+fn image_dir_beside(path: String) -> String {
+ let parts = string.split(path, on: "/")
+ let dir = case list.reverse(parts) {
+ [_file, ..rest] -> string.join(list.reverse(rest), "/")
+ [] -> ""
+ }
+ case string.ends_with(dir, "abstraction") {
+ False -> ""
+ True -> {
+ let parent = case list.reverse(string.split(dir, on: "/")) {
+ [_, ..rest] -> string.join(list.reverse(rest), "/")
+ [] -> ""
+ }
+ let image_dir = parent <> "/image"
+ case simplifile.is_directory(image_dir) {
+ Ok(True) -> image_dir
+ _ -> ""
+ }
+ }
+ }
+}
+
+/// Unused here, kept so that a caller can size a read before making it.
+pub fn byte_size(b: BitArray) -> Int {
+ bit_array.byte_size(b)
+}
diff --git a/gleam/src/ocda/load.gleam b/gleam/src/ocda/load.gleam
new file mode 100644
index 0000000..2a82ac9
--- /dev/null
+++ b/gleam/src/ocda/load.gleam
@@ -0,0 +1,79 @@
+//// What a path is, and loading it.
+////
+//// One place that decides, so that every reader and every writer accepts
+//// the same inputs and a new input form is added once. This mirrors
+//// spine's own `ocda/abstraction/load.d`, which names the sources it can
+//// enter the pipeline from and tells them apart by suffix.
+////
+//// Deliberately *not* here: `.sst` and `.ssm` markup. Reading markup means
+//// implementing the parser, which is spine's work and a far larger thing
+//// than a reader or a sample writer needs. This library reads
+//// abstractions.
+
+import gleam/list
+import gleam/string
+import ocda/load_db
+import ocda/load_pod
+import ocda/load_ssp
+import ocda/model.{type Document}
+
+pub type Source {
+ /// `.ssp`, the abstraction as text
+ Ssp
+ /// `.ocda.db`, the abstraction as sqlite
+ OcdaDb
+ /// `.sisupod` or `.zip`, an archive carrying a `.ssp`
+ Pod
+ Unknown
+}
+
+/// `.sisupod` is the archive spine writes now; `.zip` is what it wrote
+/// before, and every pod already published carries that name, so both are
+/// recognised. What is inside decides how it is opened, not the name.
+const suffixes = [
+ #(".ocda.db", OcdaDb),
+ #(".ssp", Ssp),
+ #(".sisupod", Pod),
+ #(".zip", Pod),
+ #(".db", OcdaDb),
+]
+
+pub fn source_of(path: String) -> Source {
+ case list.find(suffixes, fn(s) { string.ends_with(path, s.0) }) {
+ Ok(#(_, source)) -> source
+ Error(_) -> Unknown
+ }
+}
+
+pub fn source_name(source: Source) -> String {
+ case source {
+ Ssp -> ".ssp (the abstraction as text)"
+ OcdaDb -> ".ocda.db (the abstraction as sqlite)"
+ Pod -> "a pod archive carrying a .ssp"
+ Unknown -> "not an abstraction this library reads"
+ }
+}
+
+/// Load whatever the path is into the model.
+///
+/// `lang` names which language to take from a `.ocda.db`, which holds
+/// every language of its document, and from a pod, which may carry more than
+/// one `.ssp`. A `.ssp` is one language by construction and ignores it.
+pub fn load(
+ path: String,
+ lang: String,
+ any_version: Bool,
+) -> Result(Document, String) {
+ case source_of(path) {
+ Ssp -> load_ssp.load_file(path)
+ OcdaDb -> load_db.load_file(path, lang, any_version)
+ Pod -> load_pod.load_file(path, lang)
+ Unknown ->
+ Error(
+ path
+ <> ": "
+ <> source_name(Unknown)
+ <> " (expected .ssp, .ocda.db, .sisupod or .zip)",
+ )
+ }
+}
diff --git a/gleam/src/ocda/load_db.gleam b/gleam/src/ocda/load_db.gleam
new file mode 100644
index 0000000..a5f044b
--- /dev/null
+++ b/gleam/src/ocda/load_db.gleam
@@ -0,0 +1,532 @@
+//// Loading a `.ocda.db` into the model.
+////
+//// The point of this module is that it returns *the same value*
+//// `ocda/load_ssp` returns from the same document's `.ssp`. A writer is
+//// then a function from the model to bytes and cannot tell which
+//// serialisation it was given, exactly as spine's own writers cannot.
+////
+//// Written against `ocda-db-format-specification.md`, whose section 4
+//// gives the column by column correspondence this module implements.
+
+import gleam/dict
+import gleam/dynamic/decode
+import gleam/float
+import gleam/int
+import gleam/json
+import gleam/list
+import gleam/option
+import gleam/result
+import gleam/string
+import ocda/model.{type Document, Image, Object}
+import sqlight
+
+/// The schema major version this loader knows.
+pub const schema_major_read = "2"
+
+/// One language of the database, as the manifest gives it.
+pub type Language {
+ Language(doc_id: Int, lang: String, source_filename: String, ssp_digest: String)
+}
+
+/// Load one language of a database as the model. `lang` == "" takes the
+/// first language the manifest gives.
+pub fn load_file(
+ path: String,
+ lang: String,
+ any_version: Bool,
+) -> Result(Document, String) {
+ // read only: a loader has no business writing, and a read-write open
+ // *creates* an empty database where a path is mistyped, which then reads
+ // as a document with nothing in it, a failure that looks like an answer
+ use conn <- with_connection("file:" <> path <> "?mode=ro", path)
+
+ use _ <- result.try(case
+ has(conn, "documents") && has(conn, "objects") && has(conn, "metadata")
+ {
+ True -> Ok(Nil)
+ False ->
+ Error(
+ path
+ <> ": no documents/objects/metadata tables; it does not look like a .ocda.db",
+ )
+ })
+
+ let schema_version =
+ one(conn, "select value from metadata where doc_id is null
+ and key = 'schema.version'")
+ use _ <- result.try(case schema_version, any_version {
+ "", _ -> Ok(Nil)
+ _, True -> Ok(Nil)
+ v, False ->
+ case major_of(v) == schema_major_read {
+ True -> Ok(Nil)
+ False ->
+ Error(
+ path
+ <> " is schema v"
+ <> v
+ <> ", this loader knows v"
+ <> schema_major_read
+ <> ".x",
+ )
+ }
+ })
+
+ let languages = languages(conn)
+ use chosen <- result.try(case languages, lang {
+ [], _ -> Error(path <> ": no documents rows")
+ [first, ..], "" -> Ok(first)
+ _, wanted ->
+ case list.find(languages, fn(l) { l.lang == wanted }) {
+ Ok(found) -> Ok(found)
+ Error(_) ->
+ Error(
+ path
+ <> ": no language '"
+ <> wanted
+ <> "'; it holds: "
+ <> string.join(list.map(languages, fn(l) { l.lang }), " "),
+ )
+ }
+ })
+
+ let source =
+ dict.new()
+ |> dict.insert("language", chosen.lang)
+ |> dict.insert(
+ "languages",
+ string.join(list.map(languages, fn(l) { l.lang }), " "),
+ )
+ let digest = meta_value(conn, chosen.doc_id, "source.digest")
+ let source = case digest {
+ "" -> source
+ d -> dict.insert(source, "digest", d)
+ }
+
+ let document =
+ model.Document(
+ ..model.new_document(),
+ format_version: schema_version,
+ source_filename: chosen.source_filename,
+ source: source,
+ )
+ let document = metadata(conn, chosen.doc_id, document)
+ let document = objects(conn, chosen.doc_id, document)
+ Ok(document)
+}
+
+fn with_connection(
+ uri: String,
+ path: String,
+ then: fn(sqlight.Connection) -> Result(a, String),
+) -> Result(a, String) {
+ case sqlight.open(uri) {
+ Error(e) -> Error(path <> ": " <> e.message)
+ Ok(conn) -> {
+ let outcome = then(conn)
+ let _ = sqlight.close(conn)
+ outcome
+ }
+ }
+}
+
+/// Every language the database holds, in manifest order.
+pub fn languages(conn: sqlight.Connection) -> List(Language) {
+ let d = {
+ use id <- decode.field(0, decode.int)
+ use lang <- decode.field(1, decode.string)
+ use source <- decode.field(2, decode.string)
+ use digest <- decode.field(3, decode.string)
+ decode.success(Language(id, lang, source, digest))
+ }
+ sqlight.query(
+ "select id, lang, coalesce(source_filename,''), coalesce(ssp_digest,'')
+ from documents order by id",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+/// The three preamble blocks of the text form, out of the one metadata
+/// table the database keeps them in.
+fn metadata(
+ conn: sqlight.Connection,
+ doc_id: Int,
+ document: Document,
+) -> Document {
+ let d = {
+ use key <- decode.field(0, decode.string)
+ use value <- decode.field(1, decode.string)
+ decode.success(#(key, value))
+ }
+ let rows =
+ sqlight.query(
+ "select key, value from metadata where doc_id = "
+ <> int.to_string(doc_id)
+ <> " order by key",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+
+ list.fold(rows, document, fn(doc, row) {
+ let #(key, raw) = row
+ let value = string.trim(raw)
+ case key {
+ // source.* describes the artefact rather than the document, and is
+ // @source in the text form
+ "source." <> _ -> doc
+ // translation.ocn_aligned says whether the languages of this document
+ // share one numbering. A .ocda.db can know it because it holds every
+ // language; a .ssp is one language and cannot. It is a fact about the
+ // file, not the document, and carrying it here would let a writer
+ // tell which serialisation it was handed.
+ "translation." <> _ -> doc
+ // the database tells the three blocks apart by a key prefix.
+ // Unprefixing them is what makes the two loaders give one model: a
+ // writer asks make["footer"], never metadata["make.footer"].
+ "make." <> rest ->
+ model.Document(..doc, make: dict.insert(doc.make, rest, value))
+ "doc_has." <> rest ->
+ model.Document(..doc, doc_has: dict.insert(doc.doc_has, rest, value))
+ _ ->
+ model.Document(..doc, metadata: dict.insert(doc.metadata, key, value))
+ }
+ })
+}
+
+/// The objects, in document order within each section, the sections in the
+/// order the format gives rather than the alphabetical one sqlite would.
+fn objects(
+ conn: sqlight.Connection,
+ doc_id: Int,
+ document: Document,
+) -> Document {
+ let d = {
+ use section <- decode.field(0, decode.string)
+ use ocn <- decode.field(2, decode.int)
+ use is_a <- decode.field(3, decode.string)
+ use part <- decode.field(4, decode.string)
+ use is_of_section <- decode.field(5, decode.string)
+ use is_of_type <- decode.field(6, decode.string)
+ use heading_level <- decode.field(7, decode.int)
+ use identifier <- decode.field(8, decode.string)
+ use parent <- decode.field(9, decode.int)
+ use last_descendant <- decode.field(10, decode.int)
+ use children <- decode.field(11, decode.string)
+ use ancestors <- decode.field(12, decode.string)
+ use dummy <- decode.field(13, decode.int)
+ use ocn_off <- decode.field(14, decode.int)
+ use indent_base <- decode.field(15, decode.int)
+ use indent_hang <- decode.field(16, decode.int)
+ use bullet <- decode.field(17, decode.int)
+ use lang <- decode.field(18, decode.string)
+ use has_links <- decode.field(19, decode.int)
+ use has_notes_reg <- decode.field(20, decode.int)
+ use has_notes_star <- decode.field(21, decode.int)
+ use has_images <- decode.field(22, decode.int)
+ use segment <- decode.field(23, decode.string)
+ use segment_prev <- decode.field(24, decode.string)
+ use segment_next <- decode.field(25, decode.string)
+ use anchor <- decode.field(26, decode.string)
+ use segment_html_is <- decode.field(27, decode.string)
+ use table_cols <- decode.field(28, decode.int)
+ use table_widths <- decode.field(29, decode.string)
+ use table_aligns <- decode.field(30, decode.string)
+ use table_header <- decode.field(31, decode.int)
+ use code_linenumbers <- decode.field(32, decode.int)
+ use hat <- decode.field(33, decode.string)
+ use sha256 <- decode.field(34, decode.string)
+ use text <- decode.field(35, decode.string)
+ use id <- decode.field(36, decode.int)
+ use hla <- decode.field(37, decode.string)
+ use seg_epub <- decode.field(38, decode.string)
+ use seg_epub_is <- decode.field(39, decode.string)
+ decode.success(#(
+ id,
+ Object(
+ ocn: ocn,
+ is_a: is_a,
+ section: section,
+ heading_level: level_name(heading_level),
+ identifier: identifier,
+ part: part,
+ is_of_type: is_of_type,
+ own_section: case is_of_section == section {
+ True -> ""
+ False -> is_of_section
+ },
+ segment_html_is: segment_html_is,
+ table_aligns: json_words(table_aligns),
+ stow_link: [],
+ anchor_tags: [],
+ parent_ocn: parent,
+ last_descendant_ocn: last_descendant,
+ children: json_ints(children),
+ ancestors: json_ints(ancestors),
+ dummy_heading: dummy != 0,
+ ocn_off: ocn_off != 0,
+ indent_base: indent_base,
+ indent_hang: indent_hang,
+ bullet: bullet != 0,
+ lang: lang,
+ has_links: has_links != 0,
+ has_notes_reg: has_notes_reg != 0,
+ has_notes_star: has_notes_star != 0,
+ has_images: has_images != 0,
+ segment: segment,
+ segment_prev: segment_prev,
+ segment_next: segment_next,
+ anchor: anchor,
+ heading_lev_anchor: hla,
+ segment_epub: seg_epub,
+ segment_epub_is: seg_epub_is,
+ table_cols: table_cols,
+ table_widths: json_words(table_widths),
+ table_header: table_header != 0,
+ code_syntax: "",
+ code_linenumbers: code_linenumbers != 0,
+ heading_ancestors_text: json_joined(hat),
+ sha256: sha256,
+ images: [],
+ text: text,
+ ),
+ ))
+ }
+ let rows =
+ sqlight.query(object_sql(doc_id), on: conn, with: [], expecting: d)
+ |> result.unwrap([])
+
+ // Built newest-first per section and reversed once at the end. Appending
+ // to the tail of each section instead would be quadratic, and the largest
+ // document in the reference collection has some thousands of objects in
+ // one section.
+ let sections =
+ list.fold(rows, dict.new(), fn(acc, row) {
+ let #(id, object) = row
+ // the four repeatable properties of the text form are child tables
+ // here, one query each
+ let object =
+ Object(
+ ..object,
+ images: images(conn, id),
+ stow_link: links(conn, id),
+ anchor_tags: anchors(conn, id),
+ )
+ case model.is_section(object.section) {
+ False -> acc
+ True ->
+ dict.upsert(acc, object.section, fn(existing) {
+ case existing {
+ option.Some(previous) -> [object, ..previous]
+ option.None -> [object]
+ }
+ })
+ }
+ })
+ |> dict.map_values(fn(_, objects) { list.reverse(objects) })
+ model.Document(..document, sections: sections)
+}
+
+fn object_sql(doc_id: Int) -> String {
+ "select section, seq, ocn, is_a, coalesce(is_of_part,''),
+ coalesce(is_of_section,''), coalesce(is_of_type,''),
+ coalesce(heading_level,-1), coalesce(identifier,''),
+ coalesce(parent_ocn,0), coalesce(last_descendant_ocn,0),
+ coalesce(children,''), coalesce(ancestors,''),
+ coalesce(dummy_heading,0), coalesce(object_number_off,0),
+ coalesce(indent_base,0), coalesce(indent_hang,0),
+ coalesce(bullet,0), coalesce(lang,''),
+ coalesce(has_links,0), coalesce(has_notes_reg,0),
+ coalesce(has_notes_star,0), coalesce(has_images,0),
+ coalesce(segment,''), coalesce(segment_prev,''),
+ coalesce(segment_next,''), coalesce(anchor,''),
+ coalesce(segment_html_is,''),
+ coalesce(table_cols,0), coalesce(table_widths,''),
+ coalesce(table_aligns,''), coalesce(table_header,0),
+ coalesce(code_linenumbers,0),
+ coalesce(heading_ancestors_text,''), coalesce(sha256,''),
+ coalesce(text,''), id, coalesce(heading_lev_anchor,''),
+ coalesce(segment_epub,''), coalesce(segment_epub_is,'')
+ from objects where doc_id = " <> int.to_string(doc_id) <> "
+ order by case section
+ when 'head' then 1 when 'toc' then 2 when 'body' then 3
+ when 'endnotes' then 4 when 'glossary' then 5
+ when 'bibliography' then 6 when 'bookindex' then 7
+ when 'blurb' then 8 when 'tail' then 9 else 10 end, seq"
+}
+
+fn images(conn: sqlight.Connection, object_id: Int) -> List(model.Image) {
+ let d = {
+ use name <- decode.field(0, decode.string)
+ use sha256 <- decode.field(1, decode.string)
+ use bytes <- decode.field(2, decode.int)
+ use width <- decode.field(3, decode.int)
+ use height <- decode.field(4, decode.int)
+ use missing <- decode.field(5, decode.int)
+ decode.success(Image(
+ name: name,
+ sha256: sha256,
+ bytes: bytes,
+ width: width,
+ height: height,
+ missing: missing != 0,
+ fields: dict.new(),
+ ))
+ }
+ sqlight.query(
+ "select name, coalesce(sha256,''), coalesce(bytes,0),
+ coalesce(width,0), coalesce(height,0), coalesce(missing,0)
+ from object_images where object_id = "
+ <> int.to_string(object_id)
+ <> " order by seq",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+fn links(conn: sqlight.Connection, object_id: Int) -> List(String) {
+ let d = {
+ use url <- decode.field(0, decode.string)
+ decode.success(url)
+ }
+ sqlight.query(
+ "select url from object_links where object_id = "
+ <> int.to_string(object_id)
+ <> " order by seq",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+fn anchors(conn: sqlight.Connection, object_id: Int) -> List(String) {
+ let d = {
+ use anchor <- decode.field(0, decode.string)
+ decode.success(anchor)
+ }
+ sqlight.query(
+ "select anchor from object_anchors where object_id = "
+ <> int.to_string(object_id)
+ <> " order by seq",
+ on: conn,
+ with: [],
+ expecting: d,
+ )
+ |> result.unwrap([])
+}
+
+fn one(conn: sqlight.Connection, sql: String) -> String {
+ let d = {
+ use v <- decode.field(0, decode.string)
+ decode.success(v)
+ }
+ case sqlight.query(sql, on: conn, with: [], expecting: d) {
+ Ok([v, ..]) -> v
+ _ -> ""
+ }
+}
+
+fn has(conn: sqlight.Connection, name: String) -> Bool {
+ one(
+ conn,
+ "select cast(count(*) as text) from sqlite_master where name = '"
+ <> name
+ <> "'",
+ )
+ != "0"
+}
+
+fn meta_value(conn: sqlight.Connection, doc_id: Int, key: String) -> String {
+ one(
+ conn,
+ "select value from metadata where doc_id = "
+ <> int.to_string(doc_id)
+ <> " and key = '"
+ <> key
+ <> "'",
+ )
+}
+
+fn major_of(version: String) -> String {
+ case string.split_once(version, ".") {
+ Ok(#(major, _)) -> major
+ Error(_) -> version
+ }
+}
+
+/// The eight heading levels and their indices.
+fn level_name(index: Int) -> String {
+ case index {
+ 0 -> "A"
+ 1 -> "B"
+ 2 -> "C"
+ 3 -> "D"
+ 4 -> "1"
+ 5 -> "2"
+ 6 -> "3"
+ 7 -> "4"
+ _ -> ""
+ }
+}
+
+/// The JSON columns, read as JSON.
+///
+/// Not as a character scan that strips brackets and quotes: that is wrong
+/// the moment a value holds an escape, and spine's encoder writes "/" as
+/// "\/", so every heading containing a slash comes back with a backslash in
+/// it. The D library had exactly that bug and all 36 document-languages
+/// differed on heading_ancestors_text because of it.
+fn json_strings(value: String) -> List(String) {
+ case value {
+ "" -> []
+ v ->
+ case json.parse(v, decode.list(of: loose_string())) {
+ Ok(items) -> items
+ Error(_) -> []
+ }
+ }
+}
+
+/// An element that may be a string or a number, which is what these columns
+/// hold: children and ancestors are int arrays, table_aligns and
+/// heading_ancestors_text string arrays, and table_widths an int array
+/// except where the columns divide evenly, when it is [33.3333,...]. One
+/// table in the reference collection does that, and without the float
+/// branch the widths came back as three empty strings: the fallback turned
+/// a value it could not read into a value that looked read.
+fn loose_string() -> decode.Decoder(String) {
+ decode.one_of(decode.string, [
+ decode.int |> decode.map(int.to_string),
+ decode.float |> decode.map(float.to_string),
+ decode.success(""),
+ ])
+}
+
+fn json_ints(value: String) -> List(Int) {
+ value
+ |> json_strings
+ |> list.filter(fn(s) { s != "" })
+ |> list.map(fn(s) { s |> int.parse |> result.unwrap(0) })
+}
+
+/// Space separated in the text form: table_widths, table_aligns.
+fn json_words(value: String) -> String {
+ value |> json_strings |> string.join(" ")
+}
+
+/// heading_ancestors_text is eight slots, joined with "|" in the text form
+/// and a JSON array here. The empty slots are significant: they say which
+/// levels the heading sits under.
+fn json_joined(value: String) -> String {
+ case json_strings(value) {
+ [] -> ""
+ parts -> string.join(parts, "|")
+ }
+}
diff --git a/gleam/src/ocda/load_pod.gleam b/gleam/src/ocda/load_pod.gleam
new file mode 100644
index 0000000..35483c6
--- /dev/null
+++ b/gleam/src/ocda/load_pod.gleam
@@ -0,0 +1,138 @@
+//// The abstraction out of a pod archive.
+////
+//// A pod built with `--pod2` carries the abstraction inside it, at
+//// `media/abstraction/<uid>.<lang>.ssp`, and that is what is read. So this is
+//// a third input form and *not* a third model: the `.ssp` inside is parsed by
+//// `ocda/load_ssp` and comes back as the same value. A writer over the
+//// library gains pods without being touched, which is the whole reason the
+//// dispatch is in one place.
+////
+//// ## The one thing this cannot do
+////
+//// A `.sisupod` is **one zstd frame wrapping** the zip. This reads the zip
+//// and **not the frame**, because there is no zstd on the BEAM without a NIF,
+//// and the one that exists, `ezstd`, clones zstd from github in its build, so
+//// it cannot be built in a sandbox without patching it. The D library has
+//// both, through a declared `libzstd`.
+////
+//// So: a `.zip` pod is read, a `.sisupod` is refused, and the refusal says
+//// why rather than pretending the file is malformed. It is the only place the
+//// two libraries differ in what they accept, and it is a packaging limit
+//// rather than anything about the format.
+
+import gleam/list
+import gleam/result
+import gleam/string
+import ocda/load_ssp
+import ocda/model.{type Document}
+import ocda/zip
+import simplifile
+
+/// The magic number of a zstd frame, little endian 0xFD2FB528.
+const zstd_magic = <<0x28, 0xB5, 0x2F, 0xFD>>
+
+/// Which languages of the abstraction the pod carries, sorted.
+pub fn languages(path: String) -> Result(List(String), String) {
+ use bytes <- result.try(archive_bytes(path))
+ use entries <- result.try(zip.entries(bytes))
+ zip.find_suffix(entries, ".ssp")
+ |> list.filter_map(fn(e) { lang_of(e.name) })
+ |> list.sort(string.compare)
+ |> Ok
+}
+
+/// Load one language of the abstraction a pod carries.
+///
+/// `lang` == "" takes the only one there is, and where there is a choice it
+/// says so rather than choosing, as the `.ocda.db` loader does.
+pub fn load_file(path: String, lang: String) -> Result(Document, String) {
+ use bytes <- result.try(archive_bytes(path))
+ use entries <- result.try(zip.entries(bytes))
+ let ssps = zip.find_suffix(entries, ".ssp")
+ use chosen <- result.try(case ssps, lang {
+ [], _ ->
+ Error(
+ path
+ <> ": carries no .ssp. A pod carries the abstraction only when spine"
+ <> " was asked for it: spine --pod2 puts it in, spine --source does not.",
+ )
+ [only], "" -> Ok(only)
+ _, "" ->
+ Error(
+ path
+ <> ": holds more than one language ("
+ <> string.join(langs(ssps), " ")
+ <> "); name the one wanted",
+ )
+ _, wanted ->
+ case list.find(ssps, fn(e) { lang_of(e.name) == Ok(wanted) }) {
+ Ok(found) -> Ok(found)
+ Error(_) ->
+ Error(
+ path
+ <> ": no language '"
+ <> wanted
+ <> "'; it holds: "
+ <> string.join(langs(ssps), " "),
+ )
+ }
+ })
+ use content <- result.try(zip.read(bytes, chosen))
+ case bit_array_to_string(content) {
+ Ok(text) -> Ok(load_ssp.parse(text))
+ Error(_) -> Error(chosen.name <> ": not valid UTF-8")
+ }
+}
+
+fn langs(entries: List(zip.Entry)) -> List(String) {
+ list.filter_map(entries, fn(e) { lang_of(e.name) })
+}
+
+/// `<uid>.<lang>.ssp` -> `<lang>`
+fn lang_of(name: String) -> Result(String, Nil) {
+ case string.ends_with(name, ".ssp") {
+ False -> Error(Nil)
+ True -> {
+ let stem = string.drop_end(name, 4)
+ case string.split(stem, on: ".") |> list.last {
+ Ok(lang) if lang != stem -> Ok(lang)
+ _ -> Error(Nil)
+ }
+ }
+ }
+}
+
+/// The bytes decide, not the name: a file misnamed either way is treated by
+/// what it holds.
+fn archive_bytes(path: String) -> Result(BitArray, String) {
+ case simplifile.read_bits(path) {
+ Error(e) -> Error(path <> ": " <> simplifile.describe_error(e))
+ Ok(bytes) ->
+ case is_zstd_frame(bytes) {
+ False -> Ok(bytes)
+ True ->
+ Error(
+ path
+ <> " is a zstd frame wrapping the archive, and this library has no"
+ <> " zstd: the only one on the BEAM is a NIF that fetches zstd in"
+ <> " its own build, so it cannot be built sandboxed. Unwrap it"
+ <> " first (zstd -d) and read the archive, or use the D library,"
+ <> " which reads both.",
+ )
+ }
+ }
+}
+
+fn is_zstd_frame(bytes: BitArray) -> Bool {
+ case bytes {
+ <<head:bytes-size(4), _:bits>> -> head == zstd_magic
+ _ -> False
+ }
+}
+
+@external(erlang, "unicode", "characters_to_binary")
+fn bit_array_to_string_ffi(b: BitArray) -> String
+
+fn bit_array_to_string(b: BitArray) -> Result(String, Nil) {
+ Ok(bit_array_to_string_ffi(b))
+}
diff --git a/gleam/src/ocda/load_ssp.gleam b/gleam/src/ocda/load_ssp.gleam
new file mode 100644
index 0000000..b48e1c5
--- /dev/null
+++ b/gleam/src/ocda/load_ssp.gleam
@@ -0,0 +1,478 @@
+//// Loading a `.ssp` into the model.
+////
+//// The format is line oriented and every line is one of eight shapes told
+//// apart by its first character, so the loader is a fold over the lines
+//// carrying the state the shapes need.
+////
+//// Written against `ssp-format-specification.md`. Where it and the D
+//// library differ the D library is the bug, or this is: they are meant to
+//// give the same model, and `ocda-dump` says whether they do.
+
+import gleam/dict.{type Dict}
+import gleam/int
+import gleam/list
+import gleam/option
+import gleam/result
+import gleam/string
+import ocda/model.{type Document, type Image, type Object, Image, Object}
+import simplifile
+
+pub fn load_file(path: String) -> Result(Document, String) {
+ case simplifile.read(path) {
+ Ok(content) -> Ok(parse(content))
+ Error(e) -> Error(path <> ": " <> simplifile.describe_error(e))
+ }
+}
+
+type State {
+ State(
+ document: Document,
+ /// the block currently open, "" when none
+ block: String,
+ /// the section currently open, "" when in the preamble
+ section: String,
+ /// objects of the current section, newest first
+ objects: List(Object),
+ /// the object being read, if any
+ current: Result(Object, Nil),
+ /// text lines of the object being read, newest first
+ text: List(String),
+ /// whether a text line has been seen for the object being read.
+ /// Not "is the text non-empty": an object whose first text line is
+ /// blank, which the reference collection has seven of, would then lose
+ /// it.
+ seen_text: Bool,
+ )
+}
+
+pub fn parse(content: String) -> Document {
+ content
+ |> string.split(on: "\n")
+ |> list.fold(from: initial(), with: line)
+ |> finish
+}
+
+fn initial() -> State {
+ State(
+ document: model.new_document(),
+ block: "",
+ section: "",
+ objects: [],
+ current: Error(Nil),
+ text: [],
+ seen_text: False,
+ )
+}
+
+fn line(state: State, raw: String) -> State {
+ case raw {
+ "% SiSU Document Abstraction v" <> version ->
+ case state.document.format_version {
+ "" ->
+ State(
+ ..state,
+ document: model.Document(
+ ..state.document,
+ format_version: string.trim(version),
+ ),
+ )
+ _ -> state
+ }
+ "% Source: " <> filename ->
+ case state.document.source_filename {
+ "" ->
+ State(
+ ..state,
+ document: model.Document(
+ ..state.document,
+ source_filename: string.trim(filename),
+ ),
+ )
+ _ -> state
+ }
+ "%" <> _ -> state
+ "@" <> rest -> open_block(state, rest)
+ "}" <> _ -> close_block(state)
+ "[" <> rest -> declaration(state, rest)
+ "." <> rest -> property(state, rest)
+ "|" <> rest -> text_line(state, rest)
+ _ ->
+ case string.trim(raw) {
+ // a blank line ends the object being read
+ "" -> commit(state)
+ _ ->
+ case model.is_preamble_block(state.block) {
+ True -> preamble_pair(state, raw)
+ False -> unknown_line(state, raw)
+ }
+ }
+ }
+}
+
+fn open_block(state: State, rest: String) -> State {
+ let name = case string.split_once(rest, "{") {
+ Ok(#(name, _)) -> string.trim(name)
+ Error(_) -> string.trim(rest)
+ }
+ case model.is_preamble_block(name) {
+ True -> State(..flush(state), block: name)
+ False -> {
+ let state = flush(state)
+ let document = case model.is_section(name) {
+ True -> state.document
+ // a block that is neither preamble nor one of the nine sections:
+ // counted, so that a reader can say the file held something it did
+ // not expect
+ False ->
+ model.Document(
+ ..state.document,
+ unknown: model.Unknown(
+ ..state.document.unknown,
+ block: bump(state.document.unknown.block, name),
+ ),
+ )
+ }
+ State(..state, block: name, section: name, document: document)
+ }
+ }
+}
+
+fn close_block(state: State) -> State {
+ State(..flush(state), block: "")
+}
+
+/// Commit the object being read, then put the section's objects into the
+/// document and close the section.
+fn flush(state: State) -> State {
+ let state = commit(state)
+ case state.section, state.objects {
+ "", _ -> state
+ _, [] -> State(..state, section: "", objects: [])
+ name, objects ->
+ State(
+ ..state,
+ section: "",
+ objects: [],
+ document: model.Document(
+ ..state.document,
+ sections: add_section(
+ state.document.sections,
+ name,
+ list.reverse(objects),
+ ),
+ ),
+ )
+ }
+}
+
+fn preamble_pair(state: State, raw: String) -> State {
+ case string.split_once(string.trim_start(raw), ":") {
+ Error(_) -> state
+ Ok(#(key, value)) -> {
+ let key = string.trim(key)
+ let value = string.trim(value)
+ let document = state.document
+ let document = case state.block {
+ "meta" ->
+ model.Document(
+ ..document,
+ metadata: dict.insert(document.metadata, key, value),
+ )
+ "make" ->
+ model.Document(..document, make: dict.insert(document.make, key, value))
+ "source" ->
+ model.Document(
+ ..document,
+ source: dict.insert(document.source, key, value),
+ )
+ _ ->
+ model.Document(
+ ..document,
+ doc_has: dict.insert(document.doc_has, key, value),
+ )
+ }
+ State(..state, document: document)
+ }
+ }
+}
+
+/// `[<ocn>] <kind>`, and for a heading `[<ocn>] heading :<level>
+/// [<identifier>]`.
+fn declaration(state: State, rest: String) -> State {
+ let state = commit(state)
+ case string.split_once(rest, "]") {
+ Error(_) -> state
+ Ok(#(ocn_text, tail)) -> {
+ let ocn = parse_int(string.trim(ocn_text))
+ let object =
+ Object(..model.new_object(), ocn: ocn, section: state.section)
+ let object = case string.split(string.trim(tail), on: " ") {
+ ["heading", level, ..identifier] ->
+ Object(
+ ..object,
+ is_a: "heading",
+ heading_level: string.drop_start(level, 1),
+ identifier: string.join(identifier, " "),
+ )
+ [""] -> object
+ [kind, ..] -> Object(..object, is_a: kind)
+ [] -> object
+ }
+ State(..state, current: Ok(object), text: [], seen_text: False)
+ }
+ }
+}
+
+fn property(state: State, rest: String) -> State {
+ case state.current, string.split_once(rest, ":") {
+ Ok(object), Ok(#(name, raw_value)) -> {
+ let name = string.trim(name)
+ let value = string.trim(raw_value)
+ let #(object, known) = apply_property(object, name, value)
+ let document = case known {
+ True -> state.document
+ False ->
+ model.Document(
+ ..state.document,
+ unknown: model.Unknown(
+ ..state.document.unknown,
+ property: bump(state.document.unknown.property, name),
+ ),
+ )
+ }
+ State(..state, current: Ok(object), document: document)
+ }
+ _, _ -> state
+ }
+}
+
+/// The properties this library reads, and what each sets.
+///
+/// A property it has no use for is counted rather than passed over:
+/// ignoring one is conforming within a major version, doing so in silence
+/// is what stops a reader being a check on the specification.
+fn apply_property(
+ object: Object,
+ name: String,
+ value: String,
+) -> #(Object, Bool) {
+ case name {
+ "part" -> #(Object(..object, part: value), True)
+ "is_of_type" -> #(Object(..object, is_of_type: value), True)
+ "section" -> #(Object(..object, own_section: value), True)
+ "identifier" -> #(Object(..object, identifier: value), True)
+ "segment_html_is" -> #(Object(..object, segment_html_is: value), True)
+ "segment_epub" -> #(Object(..object, segment_epub: value), True)
+ "segment_epub_is" -> #(Object(..object, segment_epub_is: value), True)
+ "table_aligns" -> #(Object(..object, table_aligns: value), True)
+ "stow_link" -> #(
+ Object(..object, stow_link: list.append(object.stow_link, [value])),
+ True,
+ )
+ "anchor_tag" -> #(
+ Object(..object, anchor_tags: list.append(object.anchor_tags, [value])),
+ True,
+ )
+ "parent" -> #(Object(..object, parent_ocn: parse_int(value)), True)
+ "last_descendant" -> #(
+ Object(..object, last_descendant_ocn: parse_int(value)),
+ True,
+ )
+ "children" -> #(Object(..object, children: numbers(value)), True)
+ "ancestors" -> #(Object(..object, ancestors: numbers(value)), True)
+ "dummy" -> #(Object(..object, dummy_heading: value == "true"), True)
+ "ocn_off" -> #(Object(..object, ocn_off: value == "true"), True)
+ "indent" -> {
+ let parts = string.split(value, on: " ")
+ let base = case parts {
+ [first, ..] -> parse_int(first)
+ [] -> 0
+ }
+ let hang = case parts {
+ [_, second, ..] -> parse_int(second)
+ _ -> 0
+ }
+ #(Object(..object, indent_base: base, indent_hang: hang), True)
+ }
+ "bullet" -> #(Object(..object, bullet: value == "true"), True)
+ "lang" -> #(Object(..object, lang: value), True)
+ "has" -> #(has_flags(object, value), True)
+ "segment" -> #(Object(..object, segment: value), True)
+ "segment_prev" -> #(Object(..object, segment_prev: value), True)
+ "segment_next" -> #(Object(..object, segment_next: value), True)
+ "anchor" -> #(Object(..object, anchor: value), True)
+ "heading_lev_anchor" -> #(
+ Object(..object, heading_lev_anchor: value),
+ True,
+ )
+ "table_cols" -> #(Object(..object, table_cols: parse_int(value)), True)
+ "table_widths" -> #(Object(..object, table_widths: value), True)
+ "table_header" -> #(Object(..object, table_header: value == "true"), True)
+ "code_syntax" -> #(Object(..object, code_syntax: value), True)
+ "code_linenumbers" -> #(
+ Object(..object, code_linenumbers: value == "true"),
+ True,
+ )
+ "heading_ancestors_text" -> #(
+ Object(..object, heading_ancestors_text: value),
+ True,
+ )
+ "sha256" -> #(Object(..object, sha256: value), True)
+ "image" -> #(
+ Object(..object, images: list.append(object.images, [image(value)])),
+ True,
+ )
+ _ -> #(object, False)
+ }
+}
+
+fn has_flags(object: Object, value: String) -> Object {
+ value
+ |> string.split(on: " ")
+ |> list.fold(from: object, with: fn(obj, flag) {
+ case flag {
+ "links" -> Object(..obj, has_links: True)
+ "notes_reg" -> Object(..obj, has_notes_reg: True)
+ "notes_star" -> Object(..obj, has_notes_star: True)
+ "images" -> Object(..obj, has_images: True)
+ _ -> obj
+ }
+ })
+}
+
+/// `<filename> sha256:<HEX> bytes:<n> [px:<w>x<h>] [missing:true] [...]`
+///
+/// The filename comes first, the rest are order-independent, and a field
+/// this library does not recognise is kept rather than discarded.
+fn image(value: String) -> Image {
+ case string.split(value, on: " ") {
+ [] -> model.new_image()
+ [name, ..fields] ->
+ list.fold(fields, model.new_image(), fn(img, field) {
+ case string.split_once(field, ":") {
+ Error(_) -> img
+ Ok(#(key, val)) ->
+ case key {
+ "sha256" -> Image(..img, sha256: val)
+ "bytes" -> Image(..img, bytes: parse_int(val))
+ "px" ->
+ case string.split_once(val, "x") {
+ Ok(#(w, h)) ->
+ Image(..img, width: parse_int(w), height: parse_int(h))
+ Error(_) -> img
+ }
+ "missing" -> Image(..img, missing: val == "true")
+ _ -> Image(..img, fields: dict.insert(img.fields, key, val))
+ }
+ }
+ })
+ |> fn(img) { Image(..img, name: name) }
+ }
+}
+
+/// `| <text>`, and `| ` alone for a blank line. Never a bare `|`: the
+/// emitter writes the space.
+fn text_line(state: State, rest: String) -> State {
+ case state.current {
+ Error(_) -> state
+ Ok(_) -> {
+ let content = string.drop_start(rest, 1)
+ State(..state, text: [content, ..state.text], seen_text: True)
+ }
+ }
+}
+
+fn unknown_line(state: State, raw: String) -> State {
+ let first = string.slice(raw, 0, 1)
+ State(
+ ..state,
+ document: model.Document(
+ ..state.document,
+ unknown: model.Unknown(
+ ..state.document.unknown,
+ line: bump(state.document.unknown.line, first),
+ ),
+ ),
+ )
+}
+
+fn commit(state: State) -> State {
+ case state.current {
+ Error(_) -> state
+ Ok(object) -> {
+ let text = case state.seen_text {
+ True -> string.join(list.reverse(state.text), "\n")
+ False -> ""
+ }
+ let object = derive_omitted(Object(..object, text: text))
+ let objects = case model.is_section(state.section) {
+ True -> [object, ..state.objects]
+ False -> state.objects
+ }
+ State(
+ ..state,
+ current: Error(Nil),
+ text: [],
+ seen_text: False,
+ objects: objects,
+ )
+ }
+ }
+}
+
+/// Two values the text form leaves out because they can be worked out.
+///
+/// `.identifier` is absent when it is simply the ocn, `.anchor` when it is
+/// the segment name. The `.ocda.db`, being columns rather than lines,
+/// stores both unconditionally. So a loader that takes the text form
+/// literally gives a different model from the one the database gives, and
+/// over one document that is 1,314 identifiers and 69 anchors.
+fn derive_omitted(object: Object) -> Object {
+ let object = case object.identifier, object.ocn {
+ "", 0 -> object
+ "", ocn -> Object(..object, identifier: int.to_string(ocn))
+ _, _ -> object
+ }
+ case object.anchor, object.segment {
+ "", "" -> object
+ "", segment -> Object(..object, anchor: segment)
+ _, _ -> object
+ }
+}
+
+fn finish(state: State) -> Document {
+ flush(state).document
+}
+
+fn add_section(
+ sections: Dict(String, List(Object)),
+ name: String,
+ objects: List(Object),
+) -> Dict(String, List(Object)) {
+ dict.upsert(sections, name, fn(existing) {
+ case existing {
+ option.Some(previous) -> list.append(previous, objects)
+ option.None -> objects
+ }
+ })
+}
+
+fn bump(counts: Dict(String, Int), key: String) -> Dict(String, Int) {
+ dict.upsert(counts, key, fn(existing) {
+ case existing {
+ option.Some(n) -> n + 1
+ option.None -> 1
+ }
+ })
+}
+
+fn parse_int(value: String) -> Int {
+ value |> string.trim |> int.parse |> result.unwrap(0)
+}
+
+fn numbers(value: String) -> List(Int) {
+ value
+ |> string.split(on: " ")
+ |> list.filter(fn(s) { s != "" })
+ |> list.map(parse_int)
+}
diff --git a/gleam/src/ocda/model.gleam b/gleam/src/ocda/model.gleam
new file mode 100644
index 0000000..050a97b
--- /dev/null
+++ b/gleam/src/ocda/model.gleam
@@ -0,0 +1,278 @@
+//// The object-centric document abstraction, as Gleam types.
+////
+//// A deliberate mirror of the D library's `ocda.model`, field for field.
+//// The two are meant to agree, and the cheapest way to make that testable
+//// is for them to hold the same things under the same names: the dump of a
+//// loaded document is then identical by construction rather than by
+//// negotiation.
+////
+//// Written against specs/sisudoc-abstraction-format: `ocda-model.md` for
+//// the model, the two format specifications for what each serialisation
+//// says about it.
+
+import gleam/dict.{type Dict}
+import gleam/list
+import gleam/string
+
+/// The format major version this library knows.
+///
+/// The `.ssp` and the `.ocda.db` carry one version between them, declared
+/// in band. The minor part rises when a property is added and a reader of
+/// the newer format still reads the older, so only the major is checked.
+pub const format_major_read = "2"
+
+/// The nine sections, in the order the format gives them, which is not the
+/// order a sorted key list or a `group by` would give.
+pub const section_names = [
+ "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex",
+ "blurb", "tail",
+]
+
+/// The four blocks that are document preamble rather than sections of
+/// objects.
+pub const preamble_blocks = ["source", "meta", "make", "doc_has"]
+
+/// A repeatable `.image` record: the file as the abstraction found it.
+pub type Image {
+ Image(
+ /// filename as the object refers to it
+ name: String,
+ /// hex, uppercase
+ sha256: String,
+ bytes: Int,
+ /// the file's own pixel dimensions, not any display width
+ width: Int,
+ height: Int,
+ /// absent or unreadable, so no digest and no size
+ missing: Bool,
+ /// any further key:value fields, kept rather than dropped
+ fields: Dict(String, String),
+ )
+}
+
+pub fn new_image() -> Image {
+ Image(
+ name: "",
+ sha256: "",
+ bytes: 0,
+ width: 0,
+ height: 0,
+ missing: False,
+ fields: dict.new(),
+ )
+}
+
+/// One object of the abstraction.
+///
+/// `text` is one string with the lines joined by newline, as in the D
+/// library and for the same reason: an object whose only text line is
+/// blank is then indistinguishable from an object with no text, in both
+/// libraries, in the same place. A difference that exists in one language
+/// and not the other would be worse than a loss both share.
+pub type Object {
+ Object(
+ ocn: Int,
+ /// one of the sixteen kinds
+ is_a: String,
+ /// the section this object was found in
+ section: String,
+ /// headings: A B C D 1 2 3 4
+ heading_level: String,
+ identifier: String,
+ part: String,
+ /// how the text behaves: para, block, comment
+ is_of_type: String,
+ /// the object's own section, when it differs from the block it sits in
+ own_section: String,
+ segment_html_is: String,
+ segment_epub: String,
+ segment_epub_is: String,
+ table_aligns: String,
+ stow_link: List(String),
+ anchor_tags: List(String),
+ parent_ocn: Int,
+ last_descendant_ocn: Int,
+ children: List(Int),
+ ancestors: List(Int),
+ dummy_heading: Bool,
+ ocn_off: Bool,
+ indent_base: Int,
+ indent_hang: Int,
+ bullet: Bool,
+ lang: String,
+ has_links: Bool,
+ has_notes_reg: Bool,
+ has_notes_star: Bool,
+ has_images: Bool,
+ segment: String,
+ segment_prev: String,
+ segment_next: String,
+ anchor: String,
+ heading_lev_anchor: String,
+ table_cols: Int,
+ table_widths: String,
+ table_header: Bool,
+ code_syntax: String,
+ code_linenumbers: Bool,
+ heading_ancestors_text: String,
+ sha256: String,
+ images: List(Image),
+ text: String,
+ )
+}
+
+pub fn new_object() -> Object {
+ Object(
+ ocn: 0,
+ is_a: "",
+ section: "",
+ heading_level: "",
+ identifier: "",
+ part: "",
+ is_of_type: "",
+ own_section: "",
+ segment_html_is: "",
+ segment_epub: "",
+ segment_epub_is: "",
+ table_aligns: "",
+ stow_link: [],
+ anchor_tags: [],
+ parent_ocn: 0,
+ last_descendant_ocn: 0,
+ children: [],
+ ancestors: [],
+ dummy_heading: False,
+ ocn_off: False,
+ indent_base: 0,
+ indent_hang: 0,
+ bullet: False,
+ lang: "",
+ has_links: False,
+ has_notes_reg: False,
+ has_notes_star: False,
+ has_images: False,
+ segment: "",
+ segment_prev: "",
+ segment_next: "",
+ anchor: "",
+ heading_lev_anchor: "",
+ table_cols: 0,
+ table_widths: "",
+ table_header: False,
+ code_syntax: "",
+ code_linenumbers: False,
+ heading_ancestors_text: "",
+ sha256: "",
+ images: [],
+ text: "",
+ )
+}
+
+/// Whether this object is rendered at all.
+///
+/// `is_of_type` "comment" is markup commentary: carried by the abstraction
+/// so that nothing is lost, and not put into output. An output that
+/// renders it leaks the author's notes into the document, which is what
+/// spine itself does not do.
+pub fn renders(object: Object) -> Bool {
+ object.is_of_type != "comment"
+}
+
+/// The section this object belongs to: the block it sits in, unless it
+/// says otherwise.
+pub fn section_of(object: Object) -> String {
+ case object.own_section {
+ "" -> object.section
+ own -> own
+ }
+}
+
+/// What the loader met and did not understand.
+///
+/// A loader that silently drops what it does not know cannot say whether
+/// it understood the document, and so cannot be used to check a
+/// specification against an implementation: silence and understanding look
+/// alike.
+pub type Unknown {
+ Unknown(
+ /// `.name` -> times seen
+ property: Dict(String, Int),
+ /// `@name` -> times seen
+ block: Dict(String, Int),
+ /// first character -> times seen, for lines of no known shape
+ line: Dict(String, Int),
+ )
+}
+
+pub fn new_unknown() -> Unknown {
+ Unknown(property: dict.new(), block: dict.new(), line: dict.new())
+}
+
+pub fn any_unknown(unknown: Unknown) -> Bool {
+ dict.size(unknown.property) > 0
+ || dict.size(unknown.block) > 0
+ || dict.size(unknown.line) > 0
+}
+
+pub type Document {
+ Document(
+ /// as declared on line 1 of a `.ssp` or in `schema.version`, "" when
+ /// absent
+ format_version: String,
+ source_filename: String,
+ /// language, languages, digest
+ source: Dict(String, String),
+ unknown: Unknown,
+ metadata: Dict(String, String),
+ make: Dict(String, String),
+ doc_has: Dict(String, String),
+ /// section name -> objects, in document order
+ sections: Dict(String, List(Object)),
+ )
+}
+
+pub fn new_document() -> Document {
+ Document(
+ format_version: "",
+ source_filename: "",
+ source: dict.new(),
+ unknown: new_unknown(),
+ metadata: dict.new(),
+ make: dict.new(),
+ doc_has: dict.new(),
+ sections: dict.new(),
+ )
+}
+
+/// The major part of the declared version, "" when none was declared.
+pub fn version_major(document: Document) -> String {
+ case string.split_once(document.format_version, ".") {
+ Ok(#(major, _)) -> major
+ Error(_) -> document.format_version
+ }
+}
+
+pub fn meta(document: Document, key: String) -> String {
+ case dict.get(document.metadata, key) {
+ Ok(value) -> value
+ Error(_) -> ""
+ }
+}
+
+/// The objects of a named section, empty when the document has none.
+pub fn section(document: Document, name: String) -> List(Object) {
+ case dict.get(document.sections, name) {
+ Ok(objects) -> objects
+ Error(_) -> []
+ }
+}
+
+/// Is this one of the nine sections of the format?
+pub fn is_section(name: String) -> Bool {
+ list.contains(section_names, name)
+}
+
+/// Is this one of the four preamble blocks?
+pub fn is_preamble_block(name: String) -> Bool {
+ list.contains(preamble_blocks, name)
+}
diff --git a/gleam/src/ocda/zip.gleam b/gleam/src/ocda/zip.gleam
new file mode 100644
index 0000000..5ce6e09
--- /dev/null
+++ b/gleam/src/ocda/zip.gleam
@@ -0,0 +1,170 @@
+//// Enough of the zip format to find one file in a pod and read it.
+////
+//// Not a zip library. A pod archive holds a handful of entries, spine writes
+//// them **stored** (uncompressed) so that the outer zstd frame does the
+//// compressing once over the whole thing, and what is wanted from it here is
+//// one named entry.
+////
+//// Deflate is *refused rather than mishandled*. Erlang's `zlib` could
+//// inflate it, but a library that says it read an entry it in fact skipped is
+//// worse than one that says it cannot, and spine's own archives do not need
+//// it.
+
+import gleam/bit_array
+import gleam/int
+import gleam/list
+import gleam/result
+import gleam/string
+
+pub type Entry {
+ Entry(
+ name: String,
+ /// 0 stored, 8 deflate
+ method: Int,
+ compressed: Int,
+ uncompressed: Int,
+ local_offset: Int,
+ )
+}
+
+const eocd_sig = 0x06054b50
+
+const cd_sig = 0x02014b50
+
+const local_sig = 0x04034b50
+
+/// The entries, from the central directory at the end of the file.
+pub fn entries(bytes: BitArray) -> Result(List(Entry), String) {
+ let size = bit_array.byte_size(bytes)
+ use _ <- result.try(case size >= 22 {
+ True -> Ok(Nil)
+ False -> Error("too short to be a zip archive")
+ })
+ use eocd <- result.try(find_eocd(bytes, size - 22, lowest(size)))
+ use count <- result.try(u16(bytes, eocd + 10))
+ use offset <- result.try(u32(bytes, eocd + 16))
+ read_central(bytes, offset, count, [])
+}
+
+/// The end of central directory record is last, and may be followed by a
+/// comment of up to 65535 bytes, so it is searched for backwards.
+fn find_eocd(bytes: BitArray, at: Int, lowest: Int) -> Result(Int, String) {
+ case at < lowest {
+ True ->
+ Error(
+ "no end of central directory record; it does not look like a zip archive",
+ )
+ False ->
+ case u32(bytes, at) {
+ Ok(sig) if sig == eocd_sig -> Ok(at)
+ _ -> find_eocd(bytes, at - 1, lowest)
+ }
+ }
+}
+
+fn lowest(size: Int) -> Int {
+ case size > 22 + 65_535 {
+ True -> size - 22 - 65_535
+ False -> 0
+ }
+}
+
+fn read_central(
+ bytes: BitArray,
+ at: Int,
+ remaining: Int,
+ acc: List(Entry),
+) -> Result(List(Entry), String) {
+ case remaining {
+ 0 -> Ok(list.reverse(acc))
+ _ -> {
+ use sig <- result.try(u32(bytes, at))
+ use _ <- result.try(case sig == cd_sig {
+ True -> Ok(Nil)
+ False ->
+ Error(
+ "central directory entry "
+ <> int.to_string(list.length(acc))
+ <> " is malformed",
+ )
+ })
+ use method <- result.try(u16(bytes, at + 10))
+ use compressed <- result.try(u32(bytes, at + 20))
+ use uncompressed <- result.try(u32(bytes, at + 24))
+ use name_len <- result.try(u16(bytes, at + 28))
+ use extra_len <- result.try(u16(bytes, at + 30))
+ use cmnt_len <- result.try(u16(bytes, at + 32))
+ use local_offset <- result.try(u32(bytes, at + 42))
+ use name <- result.try(text(bytes, at + 46, name_len))
+ let entry =
+ Entry(
+ name: name,
+ method: method,
+ compressed: compressed,
+ uncompressed: uncompressed,
+ local_offset: local_offset,
+ )
+ read_central(
+ bytes,
+ at + 46 + name_len + extra_len + cmnt_len,
+ remaining - 1,
+ [entry, ..acc],
+ )
+ }
+ }
+}
+
+/// One entry's bytes, from its local header.
+pub fn read(bytes: BitArray, entry: Entry) -> Result(BitArray, String) {
+ use sig <- result.try(u32(bytes, entry.local_offset))
+ use _ <- result.try(case sig == local_sig {
+ True -> Ok(Nil)
+ False -> Error(entry.name <> ": local header is malformed")
+ })
+ use name_len <- result.try(u16(bytes, entry.local_offset + 26))
+ use extra_len <- result.try(u16(bytes, entry.local_offset + 28))
+ use _ <- result.try(case entry.method {
+ 0 -> Ok(Nil)
+ m ->
+ Error(
+ entry.name
+ <> ": compression method "
+ <> int.to_string(m)
+ <> "; this reader takes only stored entries, which is what spine writes",
+ )
+ })
+ let start = entry.local_offset + 30 + name_len + extra_len
+ case bit_array.slice(bytes, start, entry.compressed) {
+ Ok(slice) -> Ok(slice)
+ Error(_) -> Error(entry.name <> ": runs past the end of the archive")
+ }
+}
+
+/// The entries whose names end this way, in archive order. Used to find
+/// media/abstraction/<uid>.<lang>.ssp without knowing the uid.
+pub fn find_suffix(entries: List(Entry), suffix: String) -> List(Entry) {
+ list.filter(entries, fn(e) { string.ends_with(e.name, suffix) })
+}
+
+fn u16(bytes: BitArray, at: Int) -> Result(Int, String) {
+ case bit_array.slice(bytes, at, 2) {
+ Ok(<<v:little-size(16)>>) -> Ok(v)
+ _ -> Error("read past the end of the archive at " <> int.to_string(at))
+ }
+}
+
+fn u32(bytes: BitArray, at: Int) -> Result(Int, String) {
+ case bit_array.slice(bytes, at, 4) {
+ Ok(<<v:little-size(32)>>) -> Ok(v)
+ _ -> Error("read past the end of the archive at " <> int.to_string(at))
+ }
+}
+
+fn text(bytes: BitArray, at: Int, len: Int) -> Result(String, String) {
+ case bit_array.slice(bytes, at, len) {
+ Ok(slice) ->
+ bit_array.to_string(slice)
+ |> result.replace_error("an entry name is not valid UTF-8")
+ Error(_) -> Error("an entry name runs past the end of the archive")
+ }
+}
diff --git a/gleam/src/ocda_lib.gleam b/gleam/src/ocda_lib.gleam
new file mode 100644
index 0000000..1e8a5ae
--- /dev/null
+++ b/gleam/src/ocda_lib.gleam
@@ -0,0 +1,136 @@
+//// ocda-lib, in Gleam: the abstraction as a library, put to the question.
+////
+//// A library has no binary, so this is the two programs that test its
+//// claims.
+////
+//// ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
+//// ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]
+////
+//// `dump` writes the whole loaded model out canonically, byte for byte as
+//// the D library's `ocda-dump` writes it, so that the two implementations
+//// can be diffed. `agree` asks the same question the D library's
+//// `ocda-agree` asks, within this language: does loading a document's
+//// `.ssp` give what loading its `.ocda.db` gives?
+
+import argv
+import gleam/int
+import gleam/io
+import gleam/list
+import gleam/string
+import ocda/dump
+import ocda/load
+import ocda/model.{type Document}
+
+pub fn main() {
+ case argv.load().arguments {
+ ["dump", "--unknown", path] -> dump_file(path, "", True)
+ ["dump", "--unknown", path, lang] -> dump_file(path, lang, True)
+ ["dump", path] -> dump_file(path, "", False)
+ ["dump", path, lang] -> dump_file(path, lang, False)
+ ["agree", ssp, db] -> agree(ssp, db, "")
+ ["agree", ssp, db, lang] -> agree(ssp, db, lang)
+ _ -> usage()
+ }
+}
+
+fn usage() -> Nil {
+ io.println_error(
+ "usage: ocda-lib dump [--unknown] <file>.ssp|<file>.ocda.db [lang]
+ ocda-lib agree <doc>.ssp <doc>.ocda.db [lang]",
+ )
+ halt(2)
+}
+
+fn dump_file(path: String, lang: String, unknown: Bool) -> Nil {
+ case load.load(path, lang, False) {
+ Error(message) -> fail(message)
+ Ok(doc) ->
+ case unknown {
+ True -> io.print(dump.unknown(doc))
+ False -> io.print(dump.document(doc))
+ }
+ }
+}
+
+/// The library's whole purpose is that a writer cannot tell which
+/// serialisation it was handed. That is only true if the two loaders give
+/// the same model, so this asks, and says where they part company rather
+/// than only that they did.
+fn agree(ssp_path: String, db_path: String, lang: String) -> Nil {
+ case load.load(ssp_path, "", False), load.load(db_path, lang, False) {
+ Error(message), _ -> fail(message)
+ _, Error(message) -> fail(message)
+ Ok(from_ssp), Ok(from_db) -> {
+ let differences = compare(from_ssp, from_db)
+ case differences {
+ [] -> io.println("agree: the two loaders give the same model")
+ _ -> {
+ list.each(list.take(differences, 12), io.println)
+ case list.length(differences) > 12 {
+ True -> io.println(" ... stopping after 12")
+ False -> Nil
+ }
+ io.println(int.to_string(list.length(differences)) <> " difference(s)")
+ halt(1)
+ }
+ }
+ }
+ }
+}
+
+/// Compared through the dump rather than field by field, which is both
+/// shorter and stricter: the dump carries every field, so nothing can
+/// differ without showing here.
+fn compare(from_ssp: Document, from_db: Document) -> List(String) {
+ let a = dump.document(from_ssp)
+ let b = dump.document(from_db)
+ case a == b {
+ True -> []
+ False -> diff_lines(a, b)
+ }
+}
+
+fn diff_lines(a: String, b: String) -> List(String) {
+ let la = split_lines(a)
+ let lb = split_lines(b)
+ case list.length(la) == list.length(lb) {
+ False -> [
+ " differ: line count",
+ " .ssp: " <> int.to_string(list.length(la)),
+ " .ocda.db: " <> int.to_string(list.length(lb)),
+ ..pairwise(la, lb)
+ ]
+ True -> pairwise(la, lb)
+ }
+}
+
+fn pairwise(la: List(String), lb: List(String)) -> List(String) {
+ list.zip(la, lb)
+ |> list.index_map(fn(pair, i) { #(i, pair.0, pair.1) })
+ |> list.filter(fn(t) { t.1 != t.2 })
+ |> list.map(fn(t) {
+ " differ: line "
+ <> int.to_string(t.0 + 1)
+ <> "\n .ssp: "
+ <> t.1
+ <> "\n .ocda.db: "
+ <> t.2
+ })
+}
+
+fn split_lines(text: String) -> List(String) {
+ case text {
+ "" -> []
+ _ -> string.split(text, on: "\n")
+ }
+}
+
+fn fail(message: String) -> Nil {
+ io.println_error("error: " <> message)
+ halt(2)
+}
+
+/// Gleam has no standard exit, and the exit status is what a sweep script
+/// reads.
+@external(erlang, "erlang", "halt")
+fn halt(code: Int) -> Nil
diff --git a/gleam/test/ocda_lib_test.gleam b/gleam/test/ocda_lib_test.gleam
new file mode 100644
index 0000000..02fe463
--- /dev/null
+++ b/gleam/test/ocda_lib_test.gleam
@@ -0,0 +1,148 @@
+import gleam/dict
+import gleam/list
+import simplifile
+import gleam/string
+import gleeunit
+import gleeunit/should
+import ocda/dump
+import ocda/load
+import ocda/load_ssp
+import ocda/model
+
+pub fn main() {
+ gleeunit.main()
+}
+
+const minimal = "% SiSU Document Abstraction v2.0
+% Source: example.sst
+
+@source {
+ language: en
+ languages: en
+ digest: ABC
+}
+
+@meta {
+ title.main: An Example
+}
+
+@body {
+[1] heading :A the_title
+.part: frontmatter
+.sha256: DEAD
+| An Example
+
+[2] para
+.part: body
+.parent: 1
+| First line
+|
+| Third line, after a blank one
+
+}
+"
+
+pub fn version_test() {
+ let doc = load_ssp.parse(minimal)
+ doc.format_version |> should.equal("2.0")
+ model.version_major(doc) |> should.equal("2")
+ doc.source_filename |> should.equal("example.sst")
+}
+
+pub fn preamble_test() {
+ let doc = load_ssp.parse(minimal)
+ model.meta(doc, "title.main") |> should.equal("An Example")
+}
+
+pub fn objects_test() {
+ let doc = load_ssp.parse(minimal)
+ let body = model.section(doc, "body")
+ body |> list.length |> should.equal(2)
+}
+
+/// The blank line in the middle of an object's text is kept, and so is the
+/// count: three lines, not two.
+pub fn blank_text_line_test() {
+ let doc = load_ssp.parse(minimal)
+ case model.section(doc, "body") {
+ [_, second] ->
+ second.text
+ |> should.equal("First line\n\nThird line, after a blank one")
+ _ -> should.fail()
+ }
+}
+
+/// A heading's identifier comes from its declaration line; an object whose
+/// identifier is simply its ocn has it derived, because the text form does
+/// not write what it can work out.
+pub fn derived_identifier_test() {
+ let doc = load_ssp.parse(minimal)
+ case model.section(doc, "body") {
+ [first, second] -> {
+ first.identifier |> should.equal("the_title")
+ second.identifier |> should.equal("2")
+ }
+ _ -> should.fail()
+ }
+}
+
+pub fn source_of_test() {
+ load.source_of("a/b.ssp") |> should.equal(load.Ssp)
+ load.source_of("a/b.ocda.db") |> should.equal(load.OcdaDb)
+ load.source_of("a/b.sst") |> should.equal(load.Unknown)
+}
+
+/// The dump omits a field at its default, so a minimal document gives a
+/// short dump and every line in it is a fact.
+pub fn dump_test() {
+ let text = dump.document(load_ssp.parse(minimal))
+ text |> string.contains(contain: "version 2.0") |> should.be_true
+ text |> string.contains(contain: "meta title.main An Example") |> should.be_true
+ text |> string.contains(contain: "object body 0") |> should.be_true
+ text |> string.contains(contain: "level A") |> should.be_true
+ // a false boolean is not written at all
+ text |> string.contains(contain: "bullet") |> should.be_false
+}
+
+
+/// A property is read, which the fixture above had to be corrected to show:
+/// a property line is `.name: value` at the start of the line, and an
+/// indented one is a line of no known shape. The fixture indented them, so
+/// every property in it was being counted as malformed and the tests around
+/// it passed for the wrong reason.
+pub fn property_test() {
+ let doc = load_ssp.parse(minimal)
+ case model.section(doc, "body") {
+ [first, second] -> {
+ first.part |> should.equal("frontmatter")
+ first.sha256 |> should.equal("DEAD")
+ second.parent_ocn |> should.equal(1)
+ }
+ _ -> should.fail()
+ }
+ // and nothing in the fixture is malformed any more
+ dict_size(doc.unknown.line) |> should.equal(0)
+}
+
+@external(erlang, "maps", "size")
+fn dict_size(d: dict.Dict(a, b)) -> Int
+
+/// What a path is, including the two archive forms.
+pub fn pod_source_test() {
+ load.source_of("a/b.sisupod") |> should.equal(load.Pod)
+ load.source_of("a/b.zip") |> should.equal(load.Pod)
+}
+
+/// A zstd-wrapped archive is refused, and the refusal says why rather than
+/// claiming the file is malformed. This is the one place the D and Gleam
+/// libraries differ in what they accept, and it is a packaging limit: the
+/// only zstd on the BEAM is a NIF that fetches zstd in its own build.
+pub fn zstd_refusal_test() {
+ let path = "test/fixture-zstd-frame.sisupod"
+ let assert Ok(_) = simplifile.write_bits(path, <<0x28, 0xB5, 0x2F, 0xFD, 0>>)
+ let assert Error(message) = load.load(path, "", False)
+ message |> string.contains(contain: "zstd") |> should.be_true
+ message |> string.contains(contain: "zstd -d") |> should.be_true
+ let _ = simplifile.delete(path)
+ Nil
+}
diff --git a/org/COPYRIGHT b/org/COPYRIGHT
new file mode 100644
index 0000000..23b22f5
--- /dev/null
+++ b/org/COPYRIGHT
@@ -0,0 +1,19 @@
+- Name: sisudoc-ocda-lib
+ - Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) 2026 Ralph Amissah, All Rights Reserved.
+
+ - Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+ - Homepages:
+ [https://www.sisudoc.org]
diff --git a/org/config-lib.org b/org/config-lib.org
new file mode 100644
index 0000000..7b7ba18
--- /dev/null
+++ b/org/config-lib.org
@@ -0,0 +1,279 @@
+-*- mode: org -*-
+#+TITLE: configuration: .gitignore and copyright
+#+DESCRIPTION: setup only; the code, READMEs and build files are kept as files
+#+FILETAGS: :spine:build:tools:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2026 Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: show3levels hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+
+* ./
+** git .gitignore :gitignore:
+
+- git ls-files --others --exclude-from=.git/info/exclude
+- git ls-files --others --ignored --exclude-standard --directory
+- git check-ignore -v flake.lock
+- git clean -ix
+- find . -name "*_" | xargs rm -rf
+- find . -name "*.o" | xargs rm -rf
+
+#+HEADER: :tangle "../.gitignore"
+#+BEGIN_SRC gitignore
+# Deny everything, then whitelist what belongs in the repository.
+#
+# The practice is deliberate: a repository should hold what was decided to
+# put in it, not whatever happened to be in the directory when someone ran
+# "git add .". Build output, editor droppings, downloaded dependencies and
+# exported binaries are numerous, are regenerated, and differ between
+# machines; source is few and is chosen. So the default is no, and every
+# yes below is a decision that can be read.
+#
+# To see what is being excluded, and why:
+# git status --ignored
+# git check-ignore -v <path>
+# git ls-files --others --ignored --exclude-standard --directory
+
+*
+
+# ── the repository itself ────────────────────────────────────────────────
+!.gitignore
+!README.md
+!LICENSE
+!**/LICENSE
+
+# ── directories have to be walked into before a file inside can match ────
+!gleam/
+!dlang/
+!org/
+!gleam/**/
+!dlang/**/
+!org/**/
+# but not these, whatever they contain
+build/
+.build/
+.direnv/
+.gleam/
+bin/
+result
+result-*
+
+# ── documentation and notes, in any language directory ───────────────────
+!**/COPYRIGHT
+!**/*.md
+!**/*.org
+
+# ── nix ──────────────────────────────────────────────────────────────────
+!**/*.nix
+!**/flake.lock
+!**/.envrc
+
+# ── dlang ────────────────────────────────────────────────────────────────
+!**/*.d
+!**/dub.json
+!**/dub.sdl
+!**/dub.selections.json
+*.o
+*.a
+*.lst
+
+# ── gleam ────────────────────────────────────────────────────────────────
+!**/*.gleam
+# Erlang, for what Gleam reaches through FFI (a sha256, from OTP crypto)
+!**/src/*.erl
+# gleam.toml is the manifest, manifest.toml is the lockfile: both belong
+!**/*.toml
+*.beam
+*.ez
+erl_crash.dump
+
+# ── ruby, when it arrives ────────────────────────────────────────────────
+!**/*.rb
+!**/Gemfile
+!**/Gemfile.lock
+!**/*.gemspec
+
+# ── continuous integration ───────────────────────────────────────────────
+!**/*.yml
+!**/*.yaml
+#+END_SRC
+
+* COPYRIGHT & LICENSE
+** notices
+*** project (project root) ./
+
+#+HEADER: :tangle "../COPYRIGHT"
+#+HEADER: :noweb yes
+#+BEGIN_SRC text
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_license_agpl3>>
+
+ <<sisudoc_spine_summary>>
+#+END_SRC
+
+#+HEADER: :tangle "../LICENSE"
+#+HEADER: :noweb yes
+#+BEGIN_SRC text
+<<sisudoc_spine_license_agpl3_the_license>>
+#+END_SRC
+
+*** code org ./org
+
+- .org literate programming:
+ - copyright asserted
+ - omitted
+ - license
+ - dependency information
+
+#+HEADER: :tangle "../org/COPYRIGHT"
+#+HEADER: :noweb yes
+#+BEGIN_SRC text
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_summary>>
+#+END_SRC
+
+
+#+HEADER: :noweb yes
+#+BEGIN_SRC text
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_license_agpl3>>
+
+ <<sisudoc_spine_summary>>
+#+END_SRC
+
+*** code source
+**** gleam
+
+#+HEADER: :tangle "../gleam/COPYRIGHT"
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_license_agpl3>>
+
+ <<sisudoc_spine_summary>>
+
+ - Copyright for ./nix-gleam/ 2023 Arnar Gauti Ingason
+ - License for ./nix-gleam/ MIT License
+#+END_SRC
+
+***** gleam src
+
+#+HEADER: :tangle "../gleam/src/COPYRIGHT"
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_license_agpl3>>
+
+ <<sisudoc_spine_summary>>
+#+END_SRC
+
+**** dlang
+***** dlang
+
+#+HEADER: :tangle "../dlang/COPYRIGHT"
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_license_agpl3>>
+
+ <<sisudoc_spine_summary>>
+#+END_SRC
+
+***** dlang src
+
+#+HEADER: :tangle "../dlang/src/COPYRIGHT"
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+- Name: <<project_name>>
+ <<sisudoc_spine_copyright>>
+
+ <<sisudoc_spine_license_agpl3>>
+
+ <<sisudoc_spine_summary>>
+#+END_SRC
+
+* org includes
+** project name
+
+#+NAME: project_name
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:project_name()>>
+#+END_SRC
+
+** project version
+
+#+NAME: spine_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:spine_project_version()>>
+#+END_SRC
+
+** ocda ssp format version
+
+#+NAME: ocda_ssp_format_version
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:ocda_ssp_format_version()>>
+#+END_SRC
+
+** copyright
+
+#+NAME: sisudoc_spine_copyright
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:sisudoc_spine_copyright()>>
+#+END_SRC
+
+** license agpl3
+
+#+NAME: sisudoc_spine_license_agpl3
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:sisudoc_spine_license_agpl3()>>
+#+END_SRC
+
+#+NAME: sisudoc_spine_license_agpl3_the_license
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:sisudoc_spine_license_agpl3_the_license()>>
+#+END_SRC
+
+** spine_summary
+
+#+NAME: sisudoc_spine_summary
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:sisudoc_spine_summary()>>
+#+END_SRC
+
+** year
+
+#+NAME: year
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:year()>>
+#+END_SRC
+
+* nixpkgs source
+
+#+NAME: nixpkgs_unstable
+#+HEADER: :noweb yes
+#+BEGIN_SRC emacs-lisp
+<<./sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org:nixpkgs_unstable()>>
+#+END_SRC
+
+* __END__
diff --git a/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org b/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
new file mode 100644
index 0000000..7b6fd66
--- /dev/null
+++ b/org/sisudoc_spine_version_info_and_doc_header_including_copyright_and_license.org
@@ -0,0 +1,972 @@
+-*- mode: org -*-
+#+TITLE: sisudoc spine (doc_reform) object-centric document abstraction
+#+DESCRIPTION: documents - structuring, publishing in multiple formats & search
+#+FILETAGS: :spine:abstraction:
+#+AUTHOR: Ralph Amissah
+#+EMAIL: [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
+#+COPYRIGHT: Copyright (C) 2026 Ralph Amissah
+#+LANGUAGE: en
+#+STARTUP: content hideblocks hidestars noindent entitiespretty
+#+PROPERTY: header-args+ :eval never-export :exports code
+#+PROPERTY: header-args+ :noweb yes :padline no
+#+PROPERTY: header-args+ :results silent :cache no
+#+PROPERTY: header-args+ :mkdirp yes
+#+OPTIONS: H:3 num:nil toc:t \n:t ::t |:t ^:nil -:t f:t *:t
+
+- [[./doc-reform.org][doc-reform.org]] [[./][org/]]
+
+* spine doc header including copyright & license
+
+#+NAME: spine_doc_header_including_copyright_and_license
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+/+
+- Name: <<project_name>>
+ - Description: <<project_description>>
+ - static content generator
+
+ - Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+ - Copyright: (C) <<project_year_started>> Ralph Amissah, All Rights Reserved.
+
+ - License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ Spine (SiSU), a framework for document structuring, processing, publishing
+ and search
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation (FSF), either version 3 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License
+ for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+
+ - Spine (by Doc Reform, related to SiSU) uses standard:
+ - docReform markup syntax
+ - standard SiSU markup syntax with modified headers and minor modifications
+ - docReform object numbering
+ - standard SiSU object citation numbering & system
+
+ - Homepages:
+ [https://www.sisudoc.org]
+ [https://www.doc-reform.org]
+
+ - Git
+ [https://git.sisudoc.org/]
+
++/
+#+END_SRC
+
+** incorporate
+*** copyright
+
+#+NAME: sisudoc_spine_copyright
+#+BEGIN_SRC org
+- Description: part of SiSU spine (Doc Reform): documents, structuring,
+ processing, publishing and search
+ - [https://www.sisudoc.org]
+
+- Author: Ralph Amissah
+ [ralph.amissah@gmail.com]
+
+- Copyright: (C) <<project_year_started>> Ralph Amissah, All Rights Reserved.
+#+END_SRC
+
+*** license
+**** AGPLv3 notice
+
+#+NAME: sisudoc_spine_license_agpl3
+#+HEADER: :noweb yes
+#+BEGIN_SRC org
+- License: AGPL 3 or later, for everything in this repository unless a file or
+ a directory specifies otherwise.:
+
+ <<project_name>>
+
+ <<project_description>>
+
+ Copyright (C) Ralph Amissah
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Affero General Public License (AGPLv3) as
+ published by the Free Software Foundation, either version 3 of the License,
+ or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see [https://www.gnu.org/licenses/].
+
+ If you have Internet connection, the latest version of the AGPL should be
+ available at: [https://www.gnu.org/licenses/agpl-3.0.html]
+#+END_SRC
+
+**** the AGPLv3 License
+
+#+NAME: sisudoc_spine_license_agpl3_the_license
+#+BEGIN_SRC org
+# GNU AFFERO GENERAL PUBLIC LICENSE
+
+Version 3, 19 November 2007
+
+Copyright (C) 2007 Free Software Foundation, Inc.
+<https://fsf.org/>
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+## Preamble
+
+The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains
+free software for all its users.
+
+When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate. Many developers of free software are heartened and
+encouraged by the resulting cooperation. However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community. It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server. Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals. This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing
+under this license.
+
+The precise terms and conditions for copying, distribution and
+modification follow.
+
+## TERMS AND CONDITIONS
+
+### 0. Definitions.
+
+"This License" refers to version 3 of the GNU Affero General Public
+License.
+
+"Copyright" also means copyright-like laws that apply to other kinds
+of works, such as semiconductor masks.
+
+"The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of
+an exact copy. The resulting work is called a "modified version" of
+the earlier work or a work "based on" the earlier work.
+
+A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user
+through a computer network, with no transfer of a copy, is not
+conveying.
+
+An interactive user interface displays "Appropriate Legal Notices" to
+the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+### 1. Source Code.
+
+The "source code" for a work means the preferred form of the work for
+making modifications to it. "Object code" means any non-source form of
+a work.
+
+A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+The Corresponding Source need not include anything that users can
+regenerate automatically from other parts of the Corresponding Source.
+
+The Corresponding Source for a work in source code form is that same
+work.
+
+### 2. Basic Permissions.
+
+All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+You may make, run and propagate covered works that you do not convey,
+without conditions so long as your license otherwise remains in force.
+You may convey covered works to others for the sole purpose of having
+them make modifications exclusively for you, or provide you with
+facilities for running those works, provided that you comply with the
+terms of this License in conveying all material for which you do not
+control copyright. Those thus making or running the covered works for
+you must do so exclusively on your behalf, under your direction and
+control, on terms that prohibit them from making any copies of your
+copyrighted material outside their relationship with you.
+
+Conveying under any other circumstances is permitted solely under the
+conditions stated below. Sublicensing is not allowed; section 10 makes
+it unnecessary.
+
+### 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such
+circumvention is effected by exercising rights under this License with
+respect to the covered work, and you disclaim any intention to limit
+operation or modification of the work as a means of enforcing, against
+the work's users, your or third parties' legal rights to forbid
+circumvention of technological measures.
+
+### 4. Conveying Verbatim Copies.
+
+You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+### 5. Conveying Modified Source Versions.
+
+You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these
+conditions:
+
+- a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+- b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under
+ section 7. This requirement modifies the requirement in section 4
+ to "keep intact all notices".
+- c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+- d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+### 6. Conveying Non-Source Forms.
+
+You may convey a covered work in object code form under the terms of
+sections 4 and 5, provided that you also convey the machine-readable
+Corresponding Source under the terms of this License, in one of these
+ways:
+
+- a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+- b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the Corresponding
+ Source from a network server at no charge.
+- c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+- d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+- e) Convey the object code using peer-to-peer transmission,
+ provided you inform other peers where the object code and
+ Corresponding Source of the work are being offered to the general
+ public at no charge under subsection 6d.
+
+A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal,
+family, or household purposes, or (2) anything designed or sold for
+incorporation into a dwelling. In determining whether a product is a
+consumer product, doubtful cases shall be resolved in favor of
+coverage. For a particular product received by a particular user,
+"normally used" refers to a typical or common use of that class of
+product, regardless of the status of the particular user or of the way
+in which the particular user actually uses, or expects or is expected
+to use, the product. A product is a consumer product regardless of
+whether the product has substantial commercial, industrial or
+non-consumer uses, unless such uses represent the only significant
+mode of use of the product.
+
+"Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to
+install and execute modified versions of a covered work in that User
+Product from a modified version of its Corresponding Source. The
+information must suffice to ensure that the continued functioning of
+the modified object code is in no case prevented or interfered with
+solely because modification has been made.
+
+If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or
+updates for a work that has been modified or installed by the
+recipient, or for the User Product in which it has been modified or
+installed. Access to a network may be denied when the modification
+itself materially and adversely affects the operation of the network
+or violates the rules and protocols for communication across the
+network.
+
+Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+### 7. Additional Terms.
+
+"Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders
+of that material) supplement the terms of this License with terms:
+
+- a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+- b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+- c) Prohibiting misrepresentation of the origin of that material,
+ or requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+- d) Limiting the use for publicity purposes of names of licensors
+ or authors of the material; or
+- e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+- f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions
+ of it) with contractual assumptions of liability to the recipient,
+ for any liability that these contractual assumptions directly
+ impose on those licensors and authors.
+
+All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions; the
+above requirements apply either way.
+
+### 8. Termination.
+
+You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+However, if you cease all violation of this License, then your license
+from a particular copyright holder is reinstated (a) provisionally,
+unless and until the copyright holder explicitly and finally
+terminates your license, and (b) permanently, if the copyright holder
+fails to notify you of the violation by some reasonable means prior to
+60 days after the cessation.
+
+Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+### 9. Acceptance Not Required for Having Copies.
+
+You are not required to accept this License in order to receive or run
+a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+### 10. Automatic Licensing of Downstream Recipients.
+
+Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+### 11. Patents.
+
+A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+A contributor's "essential patent claims" are all patent claims owned
+or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+A patent license is "discriminatory" if it does not include within the
+scope of its coverage, prohibits the exercise of, or is conditioned on
+the non-exercise of one or more of the rights that are specifically
+granted under this License. You may not convey a covered work if you
+are a party to an arrangement with a third party that is in the
+business of distributing software, under which you make payment to the
+third party based on the extent of your activity of conveying the
+work, and under which the third party grants, to any of the parties
+who would receive the covered work from you, a discriminatory patent
+license (a) in connection with copies of the covered work conveyed by
+you (or copies made from those copies), or (b) primarily for and in
+connection with specific products or compilations that contain the
+covered work, unless you entered into that arrangement, or that patent
+license was granted, prior to 28 March 2007.
+
+Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+### 12. No Surrender of Others' Freedom.
+
+If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under
+this License and any other pertinent obligations, then as a
+consequence you may not convey it at all. For example, if you agree to
+terms that obligate you to collect a royalty for further conveying
+from those to whom you convey the Program, the only way you could
+satisfy both those terms and this License would be to refrain entirely
+from conveying the Program.
+
+### 13. Remote Network Interaction; Use with the GNU General Public License.
+
+Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your
+version supports such interaction) an opportunity to receive the
+Corresponding Source of your version by providing access to the
+Corresponding Source from a network server at no charge, through some
+standard or customary means of facilitating copying of software. This
+Corresponding Source shall include the Corresponding Source for any
+work covered by version 3 of the GNU General Public License that is
+incorporated pursuant to the following paragraph.
+
+Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+### 14. Revised Versions of this License.
+
+The Free Software Foundation may publish revised and/or new versions
+of the GNU Affero General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever
+published by the Free Software Foundation.
+
+If the Program specifies that a proxy can decide which future versions
+of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+### 15. Disclaimer of Warranty.
+
+THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT
+WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND
+PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE
+DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
+CORRECTION.
+
+### 16. Limitation of Liability.
+
+IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR
+CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES
+ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT
+NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR
+LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM
+TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER
+PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+### 17. Interpretation of Sections 15 and 16.
+
+If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+END OF TERMS AND CONDITIONS
+
+## How to Apply These Terms to Your New Programs
+
+If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these
+terms.
+
+To do so, attach the following notices to the program. It is safest to
+attach them to the start of each source file to most effectively state
+the exclusion of warranty; and each file should have at least the
+"copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as
+ published by the Free Software Foundation, either version 3 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper
+mail.
+
+If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for
+the specific requirements.
+
+You should also get your employer (if you work as a programmer) or
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. For more information on this, and how to apply and follow
+the GNU AGPL, see <https://www.gnu.org/licenses/>.
+#+END_SRC
+
+*** spine_summary
+
+#+NAME: sisudoc_spine_summary
+#+BEGIN_SRC org
+- Spine, Doc Reform (related to SiSU) uses standard:
+ - docReform markup syntax (based on SiSU markup)
+ - standard SiSU markup syntax with modified headers and minor
+ modifications
+ - docReform object numbering (based on SiSU object citation numbering)
+ - standard SiSU document object numbering
+
+- Homepages:
+ [https://www.sisudoc.org]
+#+END_SRC
+
+#+BEGIN_SRC org
+ - Homepages:
+ [https://www.sisudoc.org]
+
+ - Git
+ []
+#+END_SRC
+
+*** markup_samples
+
+#+NAME: sisudoc_spine_markup_samples
+#+BEGIN_SRC org
+- Spine, Doc Reform (SiSU) markup samples
+ Individual document content Copyright (Author) [as stated in document header]
+ Individual document content License (Author) [as stated in document header]
+#+END_SRC
+
+* project variables
+** project name
+
+#+NAME: project_name
+#+BEGIN_SRC org
+sisudoc-ocda-lib
+#+END_SRC
+
+#+BEGIN_SRC org
+SisuDoc Spine, Doc Reform [a part of]
+#+END_SRC
+
+** project description
+
+#+NAME: project_description
+#+BEGIN_SRC org
+part of SiSU spine (Doc Reform): documents, structuring,
+processing, publishing and search
+#+END_SRC
+
+#+BEGIN_SRC org
+Spine, Doc Reform (SiSU), a framework for document structuring, publishing and search
+#+END_SRC
+
+#+BEGIN_SRC org
+documents, structuring, processing, publishing, search
+#+END_SRC
+
+** project start year
+
+#+NAME: project_year_started
+#+BEGIN_SRC org
+2026
+#+END_SRC
+
+** project paths
+
+#+NAME: project_code_paths
+#+BEGIN_SRC org
+dlang/src/* gleam/src/*
+#+END_SRC
+
+#+BEGIN_SRC org
+src/* src/sisudoc/*
+#+END_SRC
+
+* spine version info SET VERSION
+** version in full
+
+#+NAME: spine_project_version
+#+BEGIN_SRC org
+<<spine_project_version_part_major>>.<<spine_project_version_part_minor>>.<<spine_project_version_part_patch>>
+#+END_SRC
+
+** version parts (major, minor, patch)
+
+#+NAME: spine_project_version_major_minor_patch
+#+BEGIN_SRC org
+<<spine_project_version_part_major>>, <<spine_project_version_part_minor>>, <<spine_project_version_part_patch>>
+#+END_SRC
+
+*** major
+
+#+NAME: spine_project_version_part_major
+#+BEGIN_SRC org
+0
+#+END_SRC
+
+*** minor
+
+#+NAME: spine_project_version_part_minor
+#+BEGIN_SRC org
+25
+#+END_SRC
+
+*** patch
+
+#+NAME: spine_project_version_part_patch
+#+BEGIN_SRC org
+2
+#+END_SRC
+
+* year
+
+#+NAME: year
+#+BEGIN_SRC org
+2026
+#+END_SRC
+
+* sisudoc spine git source SET
+** sisudoc-spine src
+
+#+NAME: sisudoc_spine_nix_src
+#+BEGIN_SRC org
+git://git.sisudoc.org/software/sisudoc-spine
+#+END_SRC
+
+#+BEGIN_SRC org
+git+file:///home/ralph/grotto/repo/git.repo/projects/doc-reform/code/software/sisudoc-spine
+#+END_SRC
+
+** dlang overlay src
+
+#+NAME: sisudoc_dlang_nix_overlay_src
+#+BEGIN_SRC org
+git://git.sisudoc.org/package/dlang-nix-overlays-and-flake
+#+END_SRC
+
+#+BEGIN_SRC org
+git+file:///home/ralph/grotto/repo/git.repo/projects/doc-reform/packages/nix-flakes/dlang/dlang-nix-flakes
+#+END_SRC
+
+* ocda ssp version format SET
+** ocda ssp version in full
+
+#+NAME: ocda_ssp_format_version
+#+BEGIN_SRC org
+<<ocda_ssp_format_version_part_major>>.<<ocda_ssp_format_version_part_minor>>
+#+END_SRC
+
+** parts
+*** major
+
+#+NAME: ocda_ssp_format_version_part_major
+#+BEGIN_SRC org
+2
+#+END_SRC
+
+*** minor
+
+#+NAME: ocda_ssp_format_version_part_minor
+#+BEGIN_SRC org
+1
+#+END_SRC
+
+* sqlite collection search db schema version
+
+#+NAME: sqlite_collection_search_db_schema_version
+#+BEGIN_SRC org
+1
+#+END_SRC
+
+* nipkgs source
+
+- github:NixOS/nixpkgs/nixpkgs-unstable
+- https://channels.nixos.org/nixpkgs-unstable/nixexprs.tar.zst
+
+#+NAME: nixpkgs_unstable
+#+BEGIN_SRC org
+https://channels.nixos.org/nixpkgs-unstable/nixexprs.tar.zst
+#+END_SRC
+
+#+BEGIN_SRC org
+github:NixOS/nixpkgs/nixpkgs-unstable
+#+END_SRC