diff options
| author | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-12 20:24:11 -0400 |
|---|---|---|
| committer | Ralph Amissah <ralph.amissah@gmail.com> | 2026-10-12 20:24:11 -0400 |
| commit | 4e4f2f8c2b2c043077390cb4ccf7ce97ffe4ac6b (patch) | |
| tree | bfafd6fb9cfa975db49fbcbcf7e3b23a4f13e0bc /gleam/src/read | |
| parent | model 2.1: ocn_range, the verses a poem holds (diff) | |
format 2.1: ocda-read, the readers folded in
ocda-read, one reader where there were two. The .ssp reader and the
.ocda.db reader each had their own parser; this one is built on the
library, so every mode reads a .ssp, a .ocda.db and a pod, and says
the same thing of each. Its output modules are the .ssp reader's,
copied (the reader repositories are kept, frozen) and moved onto the
library's model; --languages and --files come from the database
reader.
ocda-read [--tree|--html|--llm-text|--llm-jsonl|--images|
--verify --image-dir=<d>|--languages|--files|
--report-unknown|--validate] <file> [lang]
In D every mode of both readers; in Gleam the modes the Gleam readers
had (the summary, --tree, --languages, --files, --report-unknown),
whose output is the D reader's, byte for byte. In each language it
lives in src/read/ and is not part of the library: dub configurations
keep it out of a dependant's build in D, and Gleam compiles all of
src/ but a dependant need not use it.
Checked over the markup document samples: against the .ssp reader on
a .ssp, byte for byte after the first line in every D mode but the
summary, whose closing hint differs; against the database reader on a
.ocda.db, --files and --languages (its id column left out); the same
output from the .ssp, the .ocda.db and the .sisupod; and D == Gleam.
The database reader's --outline is not reproduced: it ordered
headings within their own section, so that other sections' headings
fell among the body's. --outline here is --tree.
What writing it found in the library: the database loader could not
say what it had not understood. It asked for columns by name and so
never saw the rest, and reported that it had understood everything.
It now counts them on the conditions under which the .ssp writes the
same property, and --report-unknown is the same from every form.
ocda.inventory answers two questions about the file rather than the
document: its languages, in the manifest's order, each with the
markup it was made from and the sha256 of its .ssp, for every form
alike; and what a .ocda.db carries besides the abstraction. In Gleam
the sha256 comes from OTP's crypto through a four line Erlang FFI.
The flake's default is all three D programs in one result: built
separately into the same link, each replaced the one before.
(assisted by Claude-Code)
Diffstat (limited to 'gleam/src/read')
| -rw-r--r-- | gleam/src/read/main.gleam | 459 |
1 files changed, 459 insertions, 0 deletions
diff --git a/gleam/src/read/main.gleam b/gleam/src/read/main.gleam new file mode 100644 index 0000000..307b8d8 --- /dev/null +++ b/gleam/src/read/main.gleam @@ -0,0 +1,459 @@ +//// ocda-read, in Gleam: reads the abstraction, in any of the forms the +//// library reads, and says what is in it. +//// +//// ocda-read [--tree|--outline|--languages|--files|--report-unknown] +//// <file>.ssp | <file>.ocda.db | <file>.zip [lang] +//// +//// The modes the Gleam readers had (the summary, the outline, the unknown +//// report, and the database's languages and files), over the library, so +//// that each reads every form. Their output is the D ocda-read's, byte for +//// byte: two readers in unrelated languages, neither written from the other +//// line by line, agreeing is what says each has read the model and not its +//// own idea of it. A `.sisupod` is the one form the Gleam library cannot +//// open (its zstd frame; see `ocda/load_pod`), and it says so. + +import gleam/dict +import gleam/int +import gleam/io +import gleam/list +import gleam/result +import gleam/string +import ocda/inventory.{type Language} +import ocda/load +import ocda/model.{type Document, type Object} + +/// The format major this reader knows. +const major_read = "2" + +type Options { + Options( + tree: Bool, + languages: Bool, + files: Bool, + report_unknown: Bool, + any_version: Bool, + lang: String, + positional: List(String), + ) +} + +/// Run with the arguments after the program name; returns the exit status. +pub fn run(args: List(String)) -> Int { + case parse(args, Options(False, False, False, False, False, "", [])) { + Error(message) -> { + io.println_error("ocda-read: " <> message <> "; --help lists the options") + 2 + } + Ok(Options(positional: [], ..)) -> { + help() + 0 + } + Ok(options) -> read(options) + } +} + +fn parse(args: List(String), o: Options) -> Result(Options, String) { + case args { + [] -> Ok(Options(..o, positional: list.reverse(o.positional))) + ["--help", ..] | ["-h", ..] -> Ok(Options(..o, positional: [])) + ["--tree", ..rest] | ["--outline", ..rest] -> + parse(rest, Options(..o, tree: True)) + ["--languages", ..rest] -> parse(rest, Options(..o, languages: True)) + ["--files", ..rest] -> parse(rest, Options(..o, files: True)) + ["--report-unknown", ..rest] -> + parse(rest, Options(..o, report_unknown: True)) + ["--any-version", ..rest] -> parse(rest, Options(..o, any_version: True)) + ["--lang", code, ..rest] -> parse(rest, Options(..o, lang: code)) + [flag, ..rest] -> + case string.starts_with(flag, "--lang=") { + True -> parse(rest, Options(..o, lang: string.drop_start(flag, 7))) + False -> + case string.starts_with(flag, "-") { + True -> + case + list.contains( + [ + "--html", "--llm-text", "--llm-jsonl", "--images", "--verify", + "--validate", + ], + flag, + ) + { + True -> + Error(flag <> " is a mode of the D ocda-read, not of this one") + False -> Error("Unrecognized option " <> flag) + } + False -> + parse(rest, Options(..o, positional: [flag, ..o.positional])) + } + } + } +} + +fn help() -> Nil { + io.println("ocda-read - reads the document abstraction, in any of its forms") + io.println("") + io.println( + "Usage: ocda-read [options] <file>.ssp|<file>.ocda.db|<file>.zip [lang]", + ) + io.println("") + io.println(" --tree the headings, as a tree") + io.println(" --outline the same as --tree") + io.println(" --languages the languages the file holds") + io.println(" --files what a .ocda.db carries besides the abstraction") + io.println(" --report-unknown what was read and not understood") + io.println(" --lang=<code> the language to read (default: the first)") + io.println(" --any-version read a format major this reader does not know") +} + +fn read(o: Options) -> Int { + let assert [path, ..more] = o.positional + let lang = case o.lang, more { + "", [given, ..] -> given + l, _ -> l + } + let lang_named = lang != "" + let loaded = { + use held <- result.try(inventory.languages(path)) + let lang = case lang, held { + "", [first, ..] -> first.lang + l, _ -> l + } + use _ <- result.try(case + held != [] && !list.any(held, fn(l) { l.lang == lang }) + { + True -> + Error( + "no language '" + <> lang + <> "' in " + <> path + <> "; it holds: " + <> string.join(list.map(held, fn(l) { l.lang }), " "), + ) + False -> Ok(Nil) + }) + use doc <- result.try(load.load(path, lang, o.any_version)) + Ok(#(held, lang, doc)) + } + case loaded { + Error(message) -> { + io.println_error("Error reading " <> path <> ": " <> message) + 2 + } + Ok(#(held, lang, doc)) -> + case gate(path, doc, o.any_version) { + False -> 3 + True -> { + summary(path, doc, held, lang, lang_named) + case o.languages { + True -> print_languages(held) + False -> Nil + } + case o.files { + True -> print_files(path) + False -> Nil + } + case o.tree { + True -> tree(doc) + False -> Nil + } + case o.report_unknown { + True -> report_unknown(doc) + False -> Nil + } + case o.tree || o.languages || o.files || o.report_unknown { + True -> Nil + False -> + io.println( + "\n(this is the summary: --help lists the modes, each adding its output)", + ) + } + 0 + } + } + } +} + +/// The major part, and refuse what is not known. +fn gate(path: String, doc: Document, any_version: Bool) -> Bool { + case doc.format_version, model.version_major(doc) == major_read, any_version { + "", _, _ -> { + io.println_error("warning: " <> path <> " declares no format version") + True + } + _, True, _ -> True + v, False, True -> { + io.println_error( + "warning: " + <> path + <> " is format v" + <> v + <> ", this reader knows v" + <> major_read + <> ".x; reading anyway", + ) + True + } + v, False, False -> { + io.println_error( + "error: " + <> path + <> " is format v" + <> v + <> ", this reader knows v" + <> major_read + <> ".x", + ) + io.println_error( + " a major version differs: a property may have changed meaning or gone away.", + ) + io.println_error(" --any-version reads it regardless.") + False + } + } +} + +const sections = [ + "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex", + "blurb", "tail", +] + +/// The sections whose images are counted: every one but the tail, as the D +/// reader counts them. +const image_sections = [ + "head", "toc", "body", "endnotes", "glossary", "bibliography", "bookindex", + "blurb", +] + +fn summary( + path: String, + doc: Document, + held: List(Language), + lang: String, + lang_named: Bool, +) -> Nil { + io.println("Read: " <> path) + case doc.format_version { + "" -> Nil + v -> io.println(" Format: v" <> v) + } + case dict.get(doc.source, "language") { + Error(_) -> Nil + Ok(language) -> { + let of = case dict.get(doc.source, "languages") { + Ok(ls) -> " of " <> ls + Error(_) -> "" + } + let digest = case dict.get(doc.source, "digest") { + Ok(d) -> ", markup digest " <> string.slice(d, 0, 16) <> "..." + Error(_) -> "" + } + io.println(" Source: " <> language <> of <> digest) + } + } + case list.length(held) > 1 && !lang_named { + True -> io.println(" (reading " <> lang <> "; --lang=<code> for another)") + False -> Nil + } + io.println(" Title: " <> model.meta(doc, "title.main")) + io.println(" Author: " <> model.meta(doc, "creator.author")) + io.println( + " Sections: " + <> string.join( + list.map(sections, fn(s) { + s <> "=" <> int.to_string(list.length(model.section(doc, s))) + }), + " ", + ), + ) + let records = + list.fold(image_sections, 0, fn(n, s) { + list.fold(model.section(doc, s), n, fn(n, o) { + n + list.length(o.images) + }) + }) + case records { + 0 -> Nil + 1 -> io.println(" Images: 1 record") + n -> io.println(" Images: " <> int.to_string(n) <> " records") + } +} + +fn print_languages(held: List(Language)) -> Nil { + io.println("") + io.println(" lang source ssp digest") + list.each(held, fn(l) { + let digest = case string.length(l.ssp_digest) > 16 { + True -> string.slice(l.ssp_digest, 0, 16) <> "..." + False -> l.ssp_digest + } + io.println( + " " + <> string.pad_end(l.lang, 7, " ") + <> " " + <> string.pad_end(l.source_filename, 31, " ") + <> " " + <> digest, + ) + }) +} + +fn print_files(path: String) -> Nil { + io.println("") + case inventory.is_carrier(path) { + False -> + io.println( + " carried files: only a .ocda.db carries files; this is " + <> load.source_name(load.source_of(path)), + ) + True -> + case inventory.carried(path) { + Error(message) -> io.println_error("error: " <> message) + Ok([]) -> io.println(" carries no files") + Ok(rows) -> { + io.println(" carried files:") + io.println(" role stored n original in the file") + list.each(rows, fn(r) { + io.println( + " " + <> string.pad_end(r.role, 9, " ") + <> " " + <> string.pad_end(r.compression, 7, " ") + <> " " + <> string.pad_end(r.count, 5, " ") + <> " " + <> string.pad_end(r.bytes, 13, " ") + <> " " + <> r.stored, + ) + }) + } + } + } +} + +/// The headings, as a tree: from the document's root heading, down the +/// children each heading lists, as the D reader walks it. +fn tree(doc: Document) -> Nil { + io.println("Document: " <> model.meta(doc, "title.main")) + case model.meta(doc, "title.subtitle") { + "" -> Nil + sub -> io.println(" " <> sub) + } + io.println("Author: " <> model.meta(doc, "creator.author")) + io.println("") + let headings = + model.section(doc, "body") + |> list.filter(fn(o) { o.is_a == "heading" && o.ocn > 0 }) + |> list.map(fn(o) { #(o.ocn, o) }) + |> dict.from_list + case model.section(doc, "head") { + [root, ..] if root.is_a == "heading" -> { + io.println(heading_line(root, 0)) + walk(root, headings, 1) + } + _ -> Nil + } +} + +fn walk(parent: Object, headings: dict.Dict(Int, Object), depth: Int) -> Nil { + list.each(parent.children, fn(ocn) { + case dict.get(headings, ocn) { + Ok(child) -> { + io.println(heading_line(child, depth)) + walk(child, headings, depth + 1) + } + Error(_) -> Nil + } + }) +} + +fn heading_line(o: Object, depth: Int) -> String { + let level = case o.heading_level { + "" -> "?" + l -> l + } + let ocn = case o.ocn > 0 { + True -> " [" <> int.to_string(o.ocn) <> "]" + False -> "" + } + string.repeat(" ", depth) <> level <> "~" <> ocn <> " " <> strip_markup(o.text) +} + +/// The internal markers a heading's text may carry, removed for display, as +/// the D reader removes them. +fn strip_markup(text: String) -> String { + text + |> string.replace("⑆*", "") + |> string.replace("⑆!", "") + |> string.replace("⑆/", "") + |> string.replace("⑆_", "") + |> string.replace("⑆^", "") + |> string.replace("⑆,", "") + |> string.replace("┨", "") + |> string.replace("┣", "") + |> string.replace("┙", " ") + |> string.replace("┘", "\n") +} + +/// Say what was read and not understood. +fn report_unknown(doc: Document) -> Nil { + let u = doc.unknown + case model.any_unknown(u) { + False -> + io.println( + "\nread and understood: every property, block and line was known", + ) + True -> { + io.println("\nread and NOT understood:") + section_of_unknown(u.property, " properties ignored (", fn(name, n) { + " ." + <> string.pad_end(name, 26, " ") + <> " " + <> string.pad_start(int.to_string(n), 8, " ") + }) + section_of_unknown(u.block, " blocks ignored (", fn(name, n) { + " @" + <> string.pad_end(name, 26, " ") + <> " " + <> string.pad_start(int.to_string(n), 8, " ") + }) + case dict.size(u.line) { + 0 -> Nil + _ -> { + io.println(" lines of no known shape, by first character:") + dict.to_list(u.line) + |> list.sort(fn(a, b) { string.compare(a.0, b.0) }) + |> list.each(fn(e) { + io.println( + " " + <> string.pad_end("\"" <> e.0 <> "\"", 27, " ") + <> " " + <> string.pad_start(int.to_string(e.1), 8, " "), + ) + }) + } + } + io.println( + " (ignoring these is conforming within a major version; the count is so that it is not silent)", + ) + } + } +} + +fn section_of_unknown( + counts: dict.Dict(String, Int), + heading: String, + line: fn(String, Int) -> String, +) -> Nil { + case dict.size(counts) { + 0 -> Nil + n -> { + io.println(heading <> int.to_string(n) <> " distinct):") + dict.to_list(counts) + |> list.sort(fn(a, b) { string.compare(a.0, b.0) }) + |> list.each(fn(e) { io.println(line(e.0, e.1)) }) + } + } +} |
