diff options
| author | Ralph Amissah <ralph.amissah@gmail.com> | 2026-09-24 16:33:53 -0400 |
|---|---|---|
| committer | Ralph Amissah <ralph.amissah@gmail.com> | 2026-09-24 16:33:53 -0400 |
| commit | bbe505c8d48e822c012c9820dfe93b83dc61bcab (patch) | |
| tree | 3385f5420959ff20af295c3125a080297555a72b /gleam | |
| parent | dealing with six properties including .is_of_type (diff) | |
--validate, and a second reader, in Gleam
--validate: the specification, executed. A validator written from
ssp-format-specification.md and from nothing else, so that a file can
be held against the document rather than against an implementation.
Each finding names the specification section it comes from, so a
disagreement can be taken back to the text.
It checks the version banner and its major part, the file shape and
the order of the preamble blocks and the nine sections, the two
spaces of a preamble pair, the object declaration and its sixteen
kinds and eight heading levels, that a property is one of the 42 and
in the emitted order, that text lines carry their prefix, the ten
keys of doc_has, and that no line is of a shape the specification
does not define.
Every .ssp of the markup document samples validates clean. Five
classes of deliberate breakage are caught: an unknown major version, a
property that is not one of the 42, a property out of order, a kind
that is not one of the sixteen, and a block that is neither preamble
nor section. Writing it found two rules the specification implied
and did not state (repeated properties are consecutive; no property
follows the text), and both were written into it.
One repository, a reader per language. The readers are not separate
tools that share a name. They are the same specification implemented
more than once, and the interesting thing about them is that they
agree, so they are published as one thing with that claim at the top
of it. The D reader moves to dlang/; the Gleam reader arrives at
gleam/. The language directory is named for the language and not
for the tool: gleam refuses a hyphenated package name, so ssp-reader
cannot be a gleam package and must be ssp_reader.
A .gitignore for the repository, denying everything and whitelisting
what belongs, covering both languages.
Both build under nix, sandboxed and without network. There is no gleam
builder in nixpkgs and gleam build fetches from Hex, so the Gleam build
uses nix-gleam's builder, kept in gleam/nix-gleam/ (MIT), which fetches
each Hex package by the checksum manifest.toml already carries, with no
hash to maintain; the result is an escript wrapped with erlang. The D
reader depends on nothing beyond phobos, so --skip-registry=all is the
whole of what it takes: ldc by default, dmd available as ssp-reader-dmd
(given gcc14, GCC 15 having introduced nullptr in headers dmd's ImportC
does not yet parse).
(assisted by Claude-Code)
Diffstat (limited to 'gleam')
| -rw-r--r-- | gleam/.envrc | 1 | ||||
| -rw-r--r-- | gleam/README.md | 82 | ||||
| -rw-r--r-- | gleam/flake.lock | 27 | ||||
| -rw-r--r-- | gleam/flake.nix | 61 | ||||
| -rw-r--r-- | gleam/gleam.toml | 21 | ||||
| -rw-r--r-- | gleam/manifest.toml | 21 | ||||
| -rw-r--r-- | gleam/nix-gleam/LICENSE | 21 | ||||
| -rw-r--r-- | gleam/nix-gleam/README.md | 32 | ||||
| -rw-r--r-- | gleam/nix-gleam/default.nix | 280 | ||||
| -rw-r--r-- | gleam/src/ssp/parse.gleam | 284 | ||||
| -rw-r--r-- | gleam/src/ssp/types.gleam | 109 | ||||
| -rw-r--r-- | gleam/src/ssp_reader.gleam | 199 | ||||
| -rw-r--r-- | gleam/test/ssp_reader_test.gleam | 142 |
13 files changed, 1280 insertions, 0 deletions
diff --git a/gleam/.envrc b/gleam/.envrc new file mode 100644 index 0000000..3550a30 --- /dev/null +++ b/gleam/.envrc @@ -0,0 +1 @@ +use flake diff --git a/gleam/README.md b/gleam/README.md new file mode 100644 index 0000000..32d4cf7 --- /dev/null +++ b/gleam/README.md @@ -0,0 +1,82 @@ +# ssp-reader, in Gleam + +Reads a `.ssp`, the object-centric document abstraction written as text, +and says what it found and what it did not understand. + +**Written from the specification alone**, as an experiment in whether that +specification suffices: + +- `ssp-format-specification.md`, from the **doc-abstraction-format** + specification +- `ocda-model.md`, from the same + +No reference was made to spine's emitter or to the D reader while writing +it. The constants it carries (the sixteen kinds, the nine sections, the +four preamble blocks, the 42 properties in their emitted order) were taken +from the specification's own tables. + +## Building and running + +```sh +nix build # the reader, as a single-file escript +./result/bin/ssp_reader <file.ssp> +``` + +`nix build` works **sandboxed and without network**. There is no Gleam +builder in nixpkgs and `gleam build` fetches from Hex, so the build uses +nix-gleam's builder, kept in `nix-gleam/` (MIT, see its README), which +fetches each Hex package on its own by the checksum `manifest.toml` already +carries. There is no hash in `flake.nix` to keep up to date. The result +needs only what it carries: run it with an empty environment and it works. + +For development: + +```sh +nix develop # gleam, erlang, rebar3 +gleam run -- <file.ssp> +gleam run -- --outline <file.ssp> +gleam run -- --report-unknown <file.ssp> +gleam run -- --any-version <file.ssp> +``` + +## What it does + +- Reads the version banner and **checks the major part**, refusing a major + it does not know (specification section 2.1). `--any-version` overrides. +- Parses the four preamble blocks, the nine sections, and the object + records: declaration line, properties, text. +- `--outline` prints the headings indented by level. +- `--report-unknown` lists every property it did not use, with counts. + +## Agreement with the D reader + +Both implementations were run over the 36 document reference collection +and **agree on the section counts of all 36**. + +## Tests + +```sh +gleam test +``` + +Ten, against small literals rather than files, since the parser is a pure +function from text to a document. Each names the specification section it +holds the parser to: the version banner, the `% Source:` line, `@source` +being preamble and not a section, the object declaration, text line order, +ancestors as numbers, that a `comment` object does not render, that an +unused property is counted rather than dropped, and that a line of no known +shape is counted. + +## What it does not do + +- The `.ocda.db`, which would want an Erlang sqlite binding. +- HTML, JSONL or plain-text output. +- Packaging as an escript, so it needs `gleam run`. + +## Layout + +``` +src/ssp_reader.gleam the program: arguments, version gate, output +src/ssp/types.gleam the document, the object, the version rule +src/ssp/parse.gleam the parser, a fold over the lines +``` diff --git a/gleam/flake.lock b/gleam/flake.lock new file mode 100644 index 0000000..5f236ab --- /dev/null +++ b/gleam/flake.lock @@ -0,0 +1,27 @@ +{ + "nodes": { + "nixpkgs": { + "locked": { + "lastModified": 1790185690, + "narHash": "sha256-xJ+X4hBtOcAFGBOe5nAMyMUeF9foJBmIOu3NjBqBycU=", + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "4975466d324710c576dc11ad614684e6bd8cad8e", + "type": "github" + }, + "original": { + "owner": "NixOS", + "ref": "nixos-unstable", + "repo": "nixpkgs", + "type": "github" + } + }, + "root": { + "inputs": { + "nixpkgs": "nixpkgs" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/gleam/flake.nix b/gleam/flake.nix new file mode 100644 index 0000000..57c92e3 --- /dev/null +++ b/gleam/flake.nix @@ -0,0 +1,61 @@ +{ + description = "ssp-reader: a reader for the .ssp document abstraction, in Gleam"; + + inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; + + outputs = { self, nixpkgs }: + let + system = "x86_64-linux"; # adjust for aarch64-linux / darwin + pkgs = import nixpkgs { inherit system; }; + + name = "ssp_reader"; + + /* The builder, nix-gleam's, kept here in nix-gleam/ (see the README + there). It fetches each Hex dependency on its own, with the + checksum gleam has already written into manifest.toml, so there is + no hash in this file to keep up to date, and nothing a run of gleam + could make come out differently. + */ + inherit (pkgs.callPackage ./nix-gleam { }) buildGleamApplication; + in { + packages.${system} = { + default = self.packages.${system}.${name}; + + /* The reader as an escript: precompiled Erlang in a single file, + which needs only erlang to run, no gleam and no dependencies. + Verified reproducible: a second export is byte identical. + nix-gleam builds an erlang-shipment by default; the build phase + is replaced, its configure phase having already put the + dependencies in place. + */ + ${name} = buildGleamApplication { + pname = name; + version = "2.0.0"; + src = ./.; + nativeBuildInputs = [ pkgs.makeWrapper ]; + buildPhase = '' + runHook preBuild + export HOME=$TMPDIR + gleam export escript + runHook postBuild + ''; + installPhase = '' + runHook preInstall + mkdir -p $out/bin + install -m755 ${name} $out/bin/${name} + wrapProgram $out/bin/${name} \ + --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.beamPackages.erlang pkgs.coreutils ]} + runHook postInstall + ''; + meta = { + description = "reads a .ssp document abstraction and says what is in it"; + mainProgram = name; + }; + }; + }; + + devShells.${system}.default = pkgs.mkShell { + packages = [ pkgs.gleam pkgs.beamPackages.erlang pkgs.rebar3 ]; + }; + }; +} diff --git a/gleam/gleam.toml b/gleam/gleam.toml new file mode 100644 index 0000000..8a56dbb --- /dev/null +++ b/gleam/gleam.toml @@ -0,0 +1,21 @@ +name = "ssp_reader" +version = "2.0.0" + +# Fill out these fields if you intend to generate HTML documentation or publish +# your project to the Hex package manager. +# +# description = "" +# licences = ["Apache-2.0"] +# repository = { type = "github", user = "", repo = "" } +# links = [{ title = "Website", href = "" }] +# +# For a full reference of all the available options, you can have a look at +# https://gleam.run/writing-gleam/gleam-toml/. + +[dependencies] +gleam_stdlib = ">= 1.0.0 and < 2.0.0" +simplifile = ">= 2.7.0 and < 3.0.0" +argv = ">= 1.1.0 and < 2.0.0" + +[dev_dependencies] +gleeunit = ">= 1.0.0 and < 2.0.0" diff --git a/gleam/manifest.toml b/gleam/manifest.toml new file mode 100644 index 0000000..1f256ca --- /dev/null +++ b/gleam/manifest.toml @@ -0,0 +1,21 @@ +# Do not manually edit this file, it is managed by Gleam. +# +# This file locks the dependency versions used, to make your build +# deterministic and to prevent unexpected versions from being included +# in your application. +# +# You should check this file into your source control repository. + +packages = [ + { name = "argv", version = "1.1.0", build_tools = ["gleam"], requirements = [], otp_app = "argv", source = "hex", outer_checksum = "3277D100448BDB4A29B6D58C0F36F631CBC349E8BDD09766C6309DF202831140" }, + { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, + { name = "gleam_stdlib", version = "1.0.5", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "CEE5B6C076A85B45F60C585F4316C63EC8B7127C119D5738C3958A9C4D50404E" }, + { name = "gleeunit", version = "1.11.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleeunit", source = "hex", outer_checksum = "EC31ABA74256AEA531EDF8169931D775BBB384FED0A8A1BDC4DD9354E3E21826" }, + { name = "simplifile", version = "2.7.0", build_tools = ["gleam"], requirements = ["filepath", "gleam_stdlib"], otp_app = "simplifile", source = "hex", outer_checksum = "A2727627B063E87351934C7F7F008F2D1FDB16F6DE0B8C79F9E46459CFC9C164" }, +] + +[requirements] +argv = { version = ">= 1.1.0 and < 2.0.0" } +gleam_stdlib = { version = ">= 1.0.0 and < 2.0.0" } +gleeunit = { version = ">= 1.0.0 and < 2.0.0" } +simplifile = { version = ">= 2.7.0 and < 3.0.0" } diff --git a/gleam/nix-gleam/LICENSE b/gleam/nix-gleam/LICENSE new file mode 100644 index 0000000..c35ad09 --- /dev/null +++ b/gleam/nix-gleam/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2023 Arnar Gauti Ingason + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/gleam/nix-gleam/README.md b/gleam/nix-gleam/README.md new file mode 100644 index 0000000..39bd1b6 --- /dev/null +++ b/gleam/nix-gleam/README.md @@ -0,0 +1,32 @@ +# nix-gleam, kept here + +`default.nix` is the builder of **nix-gleam**, by Arnar Gauti Ingason, +copied unchanged from + +- https://github.com/arnarg/nix-gleam +- `builder/default.nix` at commit `1f38154` (2026-10-06) + +under the MIT licence, which is in `LICENSE` beside it. + +It is kept here rather than taken as a flake input so that the build does +not depend on a third repository being reachable, and so that what builds +this package is in the package. + +## Why it is used + +There is no Gleam builder in nixpkgs, and a sandboxed build cannot fetch +from Hex. The usual answer, one fixed output derivation that runs +`gleam deps download` and is pinned by a hash of everything it fetched, has +two faults: the hash has to be updated by hand whenever a dependency +changes, and gleam writes `packages.toml` in a different order on each run, +so the same dependencies do not always give the same hash. + +nix-gleam fetches each Hex package on its own with `fetchHex`, using the +checksum gleam has already written for it into `manifest.toml`, and writes +`packages.toml` itself. So there is no hash to maintain, and nothing a run +of gleam could make come out differently. + +## Updating it + +Copy `builder/default.nix` from a newer commit over `default.nix`, update +the commit above, and build. diff --git a/gleam/nix-gleam/default.nix b/gleam/nix-gleam/default.nix new file mode 100644 index 0000000..d074263 --- /dev/null +++ b/gleam/nix-gleam/default.nix @@ -0,0 +1,280 @@ +{ + stdenv, + lib, + git, + xxhash, + fetchHex, + gleam, + beamPackages, + rsync, + nodejs, +}: let + inherit (builtins) fromTOML readFile; +in { + buildGleamApplication = { + src, + nativeBuildInputs ? [], + localPackages ? [], + erlangPackage ? beamPackages.erlang, + rebar3Package ? beamPackages.rebar3, + ... + } @ attrs: let + # gleam.toml contains an application name and version. + gleamToml = lib.importTOML (src + "/gleam.toml"); + + # manifest.toml contains a list of required packages including a sha256 checksum + # that can be used by nix fetchHex fetcher. + manifestToml = lib.importTOML (src + "/manifest.toml"); + + # Specify which target to build for. + buildTarget = attrs.target or gleamToml.target or "erlang"; + + # Generates a packages.toml expected by gleam compiler. + packagesTOML = with lib; + concatStringsSep "\n" ( + ["[packages]"] + ++ (map + (p: "${p.name} = \"${p.version}\"") + manifestToml.packages) + ++ (lib.optional (gitRecordsTOML != "") "") + ++ (lib.optional (gitRecordsTOML != "") gitRecordsTOML) + ); + + # Gleam records a git dependency twice: once in `[packages]`, + # and once in a `[git.<name>]` table with the commit. + # Without the second, Gleam will try to download it. + gitRecordsTOML = with lib; + concatMapStringsSep "\n" ( + p: "[git.${p.name}]\ncommit = \"${p.commit}\"" + ) (filterPackagesBySource "git" manifestToml.packages); + + # Helper function to filter manifest.toml packages + filterPackagesBySource = type: packages: lib.lists.filter (p: p.source == type) packages; + + gitDerivs = + map + (p: { + name = p.name; + derivation = fetchGit { + url = p.repo; + rev = p.commit; + }; + }) + (filterPackagesBySource "git" manifestToml.packages); + + # Fetch all dependencies + depsDerivs = + map + (p: { + name = p.name; + derivation = fetchHex { + inherit (p) version; + pkg = p.name; + sha256 = p.outer_checksum; + }; + }) + (filterPackagesBySource "hex" manifestToml.packages); + + # Find replacement paths for `local` package dependencies + # from `localPackages` list. + localDeps = let + # Build a lookup attrset for local packages. + localDerivs = lib.mergeAttrsList (map ( + p: let + name = (fromTOML (readFile (p + "/gleam.toml"))).name; + in { + "${name}" = p; + } + ) + localPackages); + in + map ( + p: { + inherit (p) name path; + localSrc = + if localDerivs ? "${p.name}" + then localDerivs.${p.name} + else throw "Local dependency \"${p.name}\" not found in `localPackages`."; + # Keep local packages in a writable location during build. + newPath = p.path; + } + ) (filterPackagesBySource "local" manifestToml.packages); + + # Check if elixir is needed in nativeBuildInputs by checking if "mix" is in + # required build_tools. + isElixirProject = with lib; p: any (t: t == "mix") p.build_tools; + needsElixir = with lib; any isElixirProject manifestToml.packages; + + # nativeBuildInputs needed for both targets. + defaultNativeBuildInputs = [gleam beamPackages.hex rsync git xxhash]; + in + # Base common mkDerivation attributes + stdenv.mkDerivation (attrs + // { + pname = attrs.pname or gleamToml.name; + version = attrs.version or gleamToml.version; + + src = lib.cleanSource attrs.src; + + # Here we must copy the dependencies into the right spot and + # create a packages.toml file so the gleam compiler does not + # attempt to pull the dependencies from the internet. + configurePhase = + attrs.configurePhase + or '' + runHook preConfigure + + mkdir -p build/packages + + # Write the packages.toml file + cat <<EOF > build/packages/packages.toml + ${packagesTOML} + EOF + + ${ + # Gleam writes build output into the local dependency's own source + # directory, so it cannot be the read-only store path. + lib.concatStringsSep "\n" ( + lib.forEach localDeps ( + d: '' + mkdir -p "${d.newPath}" + rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r ${d.localSrc}/* "${d.newPath}/" + '' + ) + ) + } + + ${ + lib.concatStringsSep "\n" ( + lib.forEach gitDerivs ( + d: '' + rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r ${d.derivation}/* build/packages/${d.name}/ + '' + ) + ) + } + + # Copy all the dependencies into place + ${lib.concatStringsSep "\n" ( + lib.forEach depsDerivs ( + # gleam outputs files inside the dependency's source directory + # and therefor it needs to have permissive permissions. + d: '' + rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r ${d.derivation}/* build/packages/${d.name}/ + '' + ) + )} + + # To prevent dependency resolution in Gleam 1.15+, local packages + # need to have their fingerprint up-to-date. + ${lib.concatStringsSep "\n" ( + lib.forEach localDeps (d: '' + printf "%u" 0x$(xxhsum -H3 ${d.newPath}/gleam.toml | cut -d' ' -f1 | cut -d '_' -f2) > build/packages/${d.name}.config_fingerprint + '') + )} + + ${ + # Prime local package dependency caches so they do not try to fetch. + # + # Gleam compiles a local dependency inside that dependency's own + # directory, using the package cache found there. + lib.concatStringsSep "\n" ( + lib.forEach localDeps ( + d: '' + mkdir -p "${d.newPath}/build/packages" + cp build/packages/packages.toml "${d.newPath}/build/packages/packages.toml" + rsync --chmod=Du=rwx,Dg=rx,Do=rx,Fu=rw,Fg=r,Fo=r -r build/packages/* "${d.newPath}/build/packages/" + '' + ) + ) + } + + runHook postConfigure + ''; + } + # When the build target is erlang + // lib.optionalAttrs (buildTarget == "erlang") { + nativeBuildInputs = + defaultNativeBuildInputs + ++ [erlangPackage rebar3Package] + ++ (lib.optional needsElixir beamPackages.elixir) + ++ nativeBuildInputs; + + # The gleam compiler has a nice export function for erlang shipment. + buildPhase = + attrs.buildPhase + or '' + runHook preBuild + + export REBAR_CACHE_DIR="$TMP/.rebar-cache" + + gleam export erlang-shipment + + runHook postBuild + ''; + + # Install all built packages into lib and create an entrypoint script + # that starts the application. + installPhase = + attrs.installPhase + or '' + runHook preInstall + + mkdir -p $out/{bin,lib} + + rsync --exclude=entrypoint.sh -r build/erlang-shipment/* $out/lib/ + + cat <<EOF > $out/bin/${gleamToml.name} + #!/usr/bin/env sh + ${erlangPackage}/bin/erl \ + -pa $out/lib/*/ebin \ + -eval "${gleamToml.name}@@main:run(${gleamToml.name})" \ + -noshell \ + -extra "\$@" + EOF + chmod +x $out/bin/${gleamToml.name} + + runHook postInstall + ''; + } + # When the build target is javascript + // lib.optionalAttrs (buildTarget == "javascript") { + nativeBuildInputs = defaultNativeBuildInputs ++ nativeBuildInputs; + + # The gleam compiler doesn't provide an export mechanism for javascript target. + buildPhase = + attrs.buildPhase + or '' + runHook prebuild + + gleam build --target javascript + + runHook postBuild + ''; + + # Install all built packages into lib and create an entrypoint script + # that starts the application. + installPhase = + attrs.installPhase + or '' + runHook preInstall + + mkdir -p $out/{bin,lib} + + rsync --exclude=gleam.lock --exclude=gleam_version -r build/dev/javascript/* $out/lib/ + + cat <<EOF > $out/lib/${gleamToml.name}/main.mjs + import { main } from "./${gleamToml.name}.mjs"; + main(); + EOF + + cat <<EOF > $out/bin/${gleamToml.name} + #!/usr/bin/env sh + ${nodejs}/bin/node $out/lib/${gleamToml.name}/main.mjs "\$@" + EOF + chmod +x $out/bin/${gleamToml.name} + + runHook postInstall + ''; + }); +} diff --git a/gleam/src/ssp/parse.gleam b/gleam/src/ssp/parse.gleam new file mode 100644 index 0000000..9458317 --- /dev/null +++ b/gleam/src/ssp/parse.gleam @@ -0,0 +1,284 @@ +//// Parsing a .ssp. +//// +//// Specification section 3: the format is line oriented, and every line +//// is one of eight shapes told apart by its first character. So the +//// parser is a fold over the lines carrying the state the shapes need. + +import gleam/dict.{type Dict} +import gleam/int +import gleam/list +import gleam/option +import gleam/string +import ssp/types.{type Document, type Object, type Section, Object, Section} + +/// specification section 6.1: the sixteen kinds +pub const kinds = [ + "para", "heading", "toc", "bookindex", "endnote", "code", "bibliography", + "comment", "verse", "glossary", "quote", "poem", "block", "table", "blurb", + "group", +] + +/// specification section 4: the four preamble blocks +pub const preamble_blocks = ["source", "meta", "make", "doc_has"] + +/// specification section 4: the nine sections, in the order they appear +pub const section_names = [ + "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex", + "blurb", "tail", +] + +/// specification section 6.3: the 42 properties, in the order emitted +pub const properties = [ + "identifier", "part", "section", "parent", "last_descendant", "children", + "ancestors", "ancestors_collapsed", "dom_status", "dom_status_collapsed", + "heading_lev_collapsed", "parent_lev", "dummy", "ocn_off", "is_of_type", + "attrib", "meta_lang", "meta_syntax", "sha256", "image", "indent", "bullet", + "lang", "has", "table_cols", "table_widths", "table_aligns", "table_header", + "code_linenumbers", "stow_link", "segment", "anchor", "segment_prev", + "segment_next", "heading_lev_anchor", "segment_epub", "segment_html_is", + "segment_epub_is", "segment_lv4_is", "heading_ancestors_text", "lev4_subtoc", + "anchor_tag", +] + +/// The properties this reader makes use of. The rest are counted and +/// ignored, which is conforming behaviour within a major version +/// (specification section 8.1) so long as it is not done in silence. +const properties_used = [ + "part", "parent", "ancestors", "is_of_type", "identifier", +] + +type State { + State( + document: Document, + /// the block currently open, "" when none + block: String, + /// the section currently open, "" when in the preamble + section: String, + /// objects of the current section, newest first + objects: List(Object), + /// the object being read, if any + current: Result(Object, Nil), + /// sections completed, newest first + done: List(Section), + ) +} + +pub fn parse(content: String) -> Document { + let state = + content + |> string.split(on: "\n") + |> list.fold(from: initial_state(), with: line) + finish(state) +} + +fn initial_state() -> State { + State( + document: types.new_document(), + block: "", + section: "", + objects: [], + current: Error(Nil), + done: [], + ) +} + +fn line(state: State, raw: String) -> State { + case raw { + "" -> commit_object(state) + "% SiSU Document Abstraction v" <> version -> + State( + ..state, + document: types.Document( + ..state.document, + version: string.trim(version), + ), + ) + "% Source: " <> filename -> + State( + ..state, + document: types.Document( + ..state.document, + source_filename: string.trim(filename), + ), + ) + "%" <> _ -> state + "@" <> rest -> open_block(state, rest) + "}" -> close_block(state) + " " <> rest -> preamble_pair(state, rest) + "[" <> rest -> declaration(state, rest) + "." <> rest -> property(state, rest) + "| " <> rest -> text_line(state, rest) + _ -> + State( + ..state, + document: types.Document( + ..state.document, + malformed_lines: state.document.malformed_lines + 1, + ), + ) + } +} + +fn open_block(state: State, rest: String) -> State { + let name = case string.split_once(rest, " {") { + Ok(#(name, _)) -> name + Error(_) -> rest + } + case list.contains(preamble_blocks, name), list.contains(section_names, name) { + True, _ -> State(..state, block: name) + _, True -> State(..state, block: name, section: name, objects: []) + _, _ -> + State( + ..state, + block: name, + document: types.Document( + ..state.document, + unknown_blocks: bump(state.document.unknown_blocks, name), + ), + ) + } +} + +fn close_block(state: State) -> State { + let state = commit_object(state) + case state.section { + "" -> State(..state, block: "") + name -> + State( + ..state, + block: "", + section: "", + objects: [], + done: [Section(name: name, objects: list.reverse(state.objects)), ..state.done], + ) + } +} + +fn preamble_pair(state: State, rest: String) -> State { + case string.split_once(rest, ": ") { + Error(_) -> state + Ok(#(key, value)) -> { + let document = state.document + let document = case state.block { + "source" -> + types.Document(..document, source: dict.insert(document.source, key, value)) + "meta" -> + types.Document(..document, meta: dict.insert(document.meta, key, value)) + "make" -> + types.Document(..document, make: dict.insert(document.make, key, value)) + "doc_has" -> + types.Document( + ..document, + doc_has: dict.insert(document.doc_has, key, value), + ) + _ -> document + } + State(..state, document: document) + } + } +} + +/// specification section 6.1 +/// [<ocn>] <kind> +/// [<ocn>] heading :<level> +/// [<ocn>] heading :<level> <identifier> +fn declaration(state: State, rest: String) -> State { + let state = commit_object(state) + case string.split_once(rest, "] ") { + Error(_) -> state + Ok(#(ocn_text, tail)) -> { + let ocn = case int.parse(ocn_text) { + Ok(n) -> n + Error(_) -> 0 + } + let object = case string.split(tail, on: " ") { + ["heading", level, ..identifier] -> + Object( + ..types.new_object(), + ocn: ocn, + kind: "heading", + level: string.drop_start(level, 1), + identifier: string.join(identifier, " "), + ) + [kind, ..] -> Object(..types.new_object(), ocn: ocn, kind: kind) + [] -> types.new_object() + } + State(..state, current: Ok(object)) + } + } +} + +fn property(state: State, rest: String) -> State { + case state.current, string.split_once(rest, ": ") { + Ok(object), Ok(#(name, value)) -> { + let object = + Object(..object, properties: [#(name, value), ..object.properties]) + let object = case name { + "part" -> Object(..object, part: value) + "is_of_type" -> Object(..object, is_of_type: value) + "identifier" -> Object(..object, identifier: value) + "parent" -> + Object(..object, parent: case int.parse(value) { + Ok(n) -> n + Error(_) -> 0 + }) + "ancestors" -> Object(..object, ancestors: numbers(value)) + _ -> object + } + // a property this reader does not use, whether or not the + // specification names it, is counted rather than passed over: within + // a major version ignoring it is conforming, doing so in silence is + // what stops a reader being a check on the specification + let document = case list.contains(properties_used, name) { + True -> state.document + False -> + types.Document( + ..state.document, + unknown_properties: bump(state.document.unknown_properties, name), + ) + } + State(..state, current: Ok(object), document: document) + } + _, _ -> state + } +} + +fn text_line(state: State, rest: String) -> State { + case state.current { + Ok(object) -> State(..state, current: Ok(Object(..object, text: [rest, ..object.text]))) + Error(_) -> state + } +} + +fn commit_object(state: State) -> State { + case state.current { + Error(_) -> state + Ok(object) -> + State( + ..state, + current: Error(Nil), + objects: [Object(..object, text: list.reverse(object.text), + properties: list.reverse(object.properties)), ..state.objects], + ) + } +} + +fn finish(state: State) -> Document { + let state = close_block(commit_object(state)) + types.Document(..state.document, sections: list.reverse(state.done)) +} + +fn bump(counts: Dict(String, Int), key: String) -> Dict(String, Int) { + dict.upsert(counts, key, fn(existing) { + case existing { + option.Some(n) -> n + 1 + option.None -> 1 + } + }) +} + +fn numbers(value: String) -> List(Int) { + value + |> string.split(on: " ") + |> list.filter_map(int.parse) +} diff --git a/gleam/src/ssp/types.gleam b/gleam/src/ssp/types.gleam new file mode 100644 index 0000000..1e98886 --- /dev/null +++ b/gleam/src/ssp/types.gleam @@ -0,0 +1,109 @@ +//// The .ssp document abstraction, as Gleam types. +//// +//// Written from the doc-abstraction-format specification, +//// ssp-format-specification.md and ocda-model.md, and from nothing else: the exercise is to find out +//// whether the specification suffices to write a reader from. + +import gleam/dict.{type Dict} +import gleam/string + +/// The format major version this reader knows. +/// +/// Specification section 2.1: a reader checks the major part and refuses +/// what it does not know, rather than loading an unfamiliar artefact half +/// populated in silence. +pub const format_major_read = "2" + +/// One object of the abstraction. +/// +/// Model section 2: an object carries a citation number, a kind and a +/// type, its place in the structure, its text and a digest of the text. +pub type Object { + Object( + ocn: Int, + /// section 6.1: one of the sixteen kinds + kind: String, + /// headings only: A B C D 1 2 3 4 + level: String, + identifier: String, + /// section 6.3: how the text behaves, para / block / comment + is_of_type: String, + part: String, + parent: Int, + ancestors: List(Int), + /// every property as read, in the order read, including the ones + /// this reader has no use for + properties: List(#(String, String)), + /// section 6.2, in order, without the "| " prefix + text: List(String), + ) +} + +pub fn new_object() -> Object { + Object( + ocn: 0, + kind: "", + level: "", + identifier: "", + is_of_type: "", + part: "", + parent: 0, + ancestors: [], + properties: [], + text: [], + ) +} + +/// Model section 2.1: the type says how the text behaves, and "comment" +/// is markup commentary that no output carries. +pub fn renders(object: Object) -> Bool { + object.is_of_type != "comment" +} + +/// One section of a document, in the order of section 4. +pub type Section { + Section(name: String, objects: List(Object)) +} + +pub type Document { + Document( + /// as declared on the first line, "" when absent + version: String, + source_filename: String, + /// the four preamble blocks of section 5 + source: Dict(String, String), + meta: Dict(String, String), + make: Dict(String, String), + doc_has: Dict(String, String), + sections: List(Section), + /// what was met and not understood, counted rather than dropped in + /// silence: a reader that cannot say what it ignored cannot show that + /// a specification is complete + unknown_properties: Dict(String, Int), + unknown_blocks: Dict(String, Int), + malformed_lines: Int, + ) +} + +pub fn new_document() -> Document { + Document( + version: "", + source_filename: "", + source: dict.new(), + meta: dict.new(), + make: dict.new(), + doc_has: dict.new(), + sections: [], + unknown_properties: dict.new(), + unknown_blocks: dict.new(), + malformed_lines: 0, + ) +} + +/// The major part of the declared version, "" when none was declared. +pub fn version_major(document: Document) -> String { + case string.split_once(document.version, ".") { + Ok(#(major, _)) -> major + Error(_) -> document.version + } +} diff --git a/gleam/src/ssp_reader.gleam b/gleam/src/ssp_reader.gleam new file mode 100644 index 0000000..d8c4c78 --- /dev/null +++ b/gleam/src/ssp_reader.gleam @@ -0,0 +1,199 @@ +//// ssp-reader, in Gleam. +//// +//// Reads a .ssp, the object-centric document abstraction written as text, +//// and says what it found and what it did not understand. +//// +//// Written from the specification alone: +//// ssp-format-specification.md, of the doc-abstraction-format spec +//// ocda-model.md, of the same + +import argv +import gleam/dict +import gleam/int +import gleam/io +import gleam/list +import gleam/result +import gleam/string +import simplifile +import ssp/parse +import ssp/types.{type Document, type Object} + +pub fn main() -> Nil { + case argv.load().arguments { + [] -> usage() + args -> { + let any_version = list.contains(args, "--any-version") + let outline = list.contains(args, "--outline") + let report = list.contains(args, "--report-unknown") + case list.filter(args, fn(a) { !string.starts_with(a, "--") }) { + [path, ..] -> run(path, any_version, outline, report) + [] -> usage() + } + } + } +} + +fn usage() -> Nil { + io.println("ssp-reader - reads a .ssp and says what is in it") + io.println("") + io.println(" ssp_reader [options] <file.ssp>") + io.println("") + io.println(" --outline the headings, as a tree") + io.println(" --report-unknown what was read and not understood") + io.println(" --any-version read a format major this reader does not know") +} + +fn run(path: String, any_version: Bool, outline: Bool, report: Bool) -> Nil { + case simplifile.read(from: path) { + Error(_) -> io.println("error: cannot read " <> path) + Ok(content) -> { + let document = parse.parse(content) + case version_ok(document, any_version) { + False -> Nil + True -> { + summary(document, path) + case outline { + True -> print_outline(document) + False -> Nil + } + case report { + True -> print_unknown(document) + False -> Nil + } + } + } + } + } +} + +/// Specification section 2.1: check the major part and refuse what is not +/// known, rather than loading an unfamiliar artefact half populated. +fn version_ok(document: Document, any_version: Bool) -> Bool { + let major = types.version_major(document) + case document.version, major == types.format_major_read, any_version { + "", _, _ -> { + io.println("warning: no version banner, reading anyway") + True + } + _, True, _ -> True + _, False, True -> { + io.println( + "warning: format v" + <> document.version + <> ", this reader knows v" + <> types.format_major_read + <> ".x; reading anyway", + ) + True + } + _, False, False -> { + io.println( + "error: format v" + <> document.version + <> ", this reader knows v" + <> types.format_major_read + <> ".x", + ) + io.println( + " a major version differs: a property may have changed meaning", + ) + io.println(" --any-version reads it regardless.") + False + } + } +} + +fn summary(document: Document, path: String) -> Nil { + io.println("Parsed: " <> path) + io.println(" Format: v" <> document.version) + io.println(" Source: " <> document.source_filename) + case dict.get(document.source, "language") { + Ok(language) -> io.println(" Language: " <> language) + Error(_) -> Nil + } + io.println(" Title: " <> meta(document, "title.main")) + io.println(" Author: " <> meta(document, "creator.author")) + let counts = + document.sections + |> list.map(fn(s) { s.name <> "=" <> int.to_string(list.length(s.objects)) }) + |> string.join(" ") + io.println(" Sections: " <> counts) +} + +fn meta(document: Document, key: String) -> String { + document.meta |> dict.get(key) |> result.unwrap("") +} + +/// The headings, indented by level. Model section 4: the eight levels are +/// A to D for the title matter and 1 to 4 for the body. +fn print_outline(document: Document) -> Nil { + io.println("") + document.sections + |> list.each(fn(section) { + section.objects + |> list.filter(fn(o) { o.kind == "heading" && types.renders(o) }) + |> list.each(fn(o) { io.println(heading_line(o)) }) + }) +} + +fn heading_line(object: Object) -> String { + let depth = case object.level { + "A" -> 0 + "B" -> 1 + "C" -> 2 + "D" -> 3 + "1" -> 4 + "2" -> 5 + "3" -> 6 + "4" -> 7 + _ -> 8 + } + string.repeat(" ", depth) + <> object.level + <> "~ [" + <> int.to_string(object.ocn) + <> "] " + <> first_line(object.text) +} + +fn first_line(text: List(String)) -> String { + case text { + [line, ..] -> line + [] -> "" + } +} + +/// A reader that drops what it does not know is conforming; one that +/// cannot say what it dropped cannot show a specification to be complete. +fn print_unknown(document: Document) -> Nil { + io.println("") + case dict.size(document.unknown_properties) { + 0 -> io.println("read and understood: every property was used") + n -> { + io.println("read and not used (" <> int.to_string(n) <> " distinct):") + document.unknown_properties + |> dict.to_list + |> list.sort(fn(a, b) { string.compare(a.0, b.0) }) + |> list.each(fn(pair) { + io.println( + " ." <> pair.0 <> " " <> int.to_string(pair.1), + ) + }) + } + } + case dict.size(document.unknown_blocks) { + 0 -> Nil + _ -> { + io.println("blocks not understood:") + document.unknown_blocks + |> dict.to_list + |> list.each(fn(pair) { + io.println(" @" <> pair.0 <> " " <> int.to_string(pair.1)) + }) + } + } + case document.malformed_lines { + 0 -> Nil + n -> io.println("lines of no known shape: " <> int.to_string(n)) + } +} diff --git a/gleam/test/ssp_reader_test.gleam b/gleam/test/ssp_reader_test.gleam new file mode 100644 index 0000000..86f1cce --- /dev/null +++ b/gleam/test/ssp_reader_test.gleam @@ -0,0 +1,142 @@ +//// Tests for the .ssp parser. +//// +//// The parser is a pure function from text to a document, so the tests +//// are small literals rather than files. Each names the specification +//// section it holds the parser to. + +import gleam/dict +import gleam/list +import gleeunit +import ssp/parse +import ssp/types + +pub fn main() -> Nil { + gleeunit.main() +} + +/// A minimal but well-formed document: the two header lines, the four +/// preamble blocks, one section, one object. +fn sample() -> String { + "% SiSU Document Abstraction v2.0 +% Source: example.sst + +@source { + language: en + digest: ABCD +} + +@meta { + title.main: An Example + creator.author: A Writer +} + +@make { + doc_type: article +} + +@doc_has { + inline_links: 2 +} + +@body { + +[1] heading :A +.part: body +.is_of_type: para +| The Title + +[2] para +.part: body +.parent: 1 +.ancestors: 1 0 0 0 0 0 0 0 +.is_of_type: para +.sha256: BEEF +| Some text. +| A second line. + +[0] comment +.part: comment +.is_of_type: comment +| % a markup comment + +} +" +} + +// specification section 2: the version is declared on the first line +pub fn version_is_read_test() { + let document = parse.parse(sample()) + assert document.version == "2.0" + assert types.version_major(document) == "2" +} + +// specification section 4: the "% Source:" line names the markup +pub fn source_filename_is_read_test() { + assert parse.parse(sample()).source_filename == "example.sst" +} + +// specification section 5.1: the @source block, which is preamble and not +// a section +pub fn source_block_is_preamble_test() { + let document = parse.parse(sample()) + assert dict.get(document.source, "language") == Ok("en") + assert dict.get(document.source, "digest") == Ok("ABCD") + // and it did not become a section of objects + assert list.length(document.sections) == 1 +} + +// specification section 5.2 +pub fn meta_is_read_test() { + let document = parse.parse(sample()) + assert dict.get(document.meta, "title.main") == Ok("An Example") + assert dict.get(document.meta, "creator.author") == Ok("A Writer") +} + +// specification section 6.1: [ocn] heading :level +pub fn heading_declaration_test() { + let document = parse.parse(sample()) + let assert [section] = document.sections + let assert [heading, ..] = section.objects + assert heading.kind == "heading" + assert heading.level == "A" + assert heading.ocn == 1 +} + +// specification section 6.2: text lines, in order, without the prefix +pub fn text_lines_are_in_order_test() { + let document = parse.parse(sample()) + let assert [section] = document.sections + let assert [_, para, ..] = section.objects + assert para.text == ["Some text.", "A second line."] +} + +// model section 4: ancestors is eight slots indexed by level +pub fn ancestors_are_numbers_test() { + let document = parse.parse(sample()) + let assert [section] = document.sections + let assert [_, para, ..] = section.objects + assert para.ancestors == [1, 0, 0, 0, 0, 0, 0, 0] + assert para.parent == 1 +} + +// model section 2.1: is_of_type "comment" is markup commentary, which no +// output carries +pub fn comments_do_not_render_test() { + let document = parse.parse(sample()) + let assert [section] = document.sections + let rendered = list.filter(section.objects, types.renders) + assert list.length(section.objects) == 3 + assert list.length(rendered) == 2 +} + +// a property the reader does not use is counted, not dropped in silence +pub fn unused_properties_are_counted_test() { + let document = parse.parse(sample()) + assert dict.get(document.unknown_properties, "sha256") == Ok(1) +} + +// a line of no shape the specification defines is counted +pub fn malformed_lines_are_counted_test() { + let document = parse.parse("% SiSU Document Abstraction v2.0\n% Source: x.sst\n\nnonsense line\n") + assert document.malformed_lines == 1 +} |
