path: root/README

-*- mode: org -*-
#+TITLE:       spine (doc_reform) (project) README
#+DESCRIPTION: README for spine
#+FILETAGS:    :spine:build:tools:
#+AUTHOR:      Ralph Amissah
#+EMAIL:       [[mailto:ralph.amissah@gmail.com][ralph.amissah@gmail.com]]
#+COPYRIGHT:   Copyright (C) 2015 - 2021 Ralph Amissah
#+LANGUAGE:    en
#+STARTUP:     content hideblocks hidestars noindent entitiespretty
#+OPTIONS:     H:3 num:nil toc:t \n:nil @:t ::t |:t ^:nil _:nil -:t f:t *:t <:t
#+PROPERTY:    header-args  :exports code
#+PROPERTY:    header-args+ :noweb yes
#+PROPERTY:    header-args+ :eval no
#+PROPERTY:    header-args+ :results no
#+PROPERTY:    header-args+ :cache no
#+PROPERTY:    header-args+ :padline no

project_name:  Spine, Doc Reform

  description: [
      "documents, structuring, processing, publishing",
      search,
      object numbering,
      static content generator,
      sisu markup
    ]

    author:
      name:    Ralph Amissah
      email:   ralph.amissah@gmail.com

    copyright: "(C) 2015 - 2021 Ralph Amissah, All Rights Reserved."

    license:   "AGPL 3 or later"

    homepage: [
        "https://www.doc_reform.org",
        "https://www.sisudoc.org"
      ]

* Installation, Compilation

SiSU spine is written in the programming language D for which there are 3 compilers:

- dmd
- ldc
- gdc

D projects tend to use dub as project manager
https://code.dlang.org/packages/dub
https://code.dlang.org/packages/dub
https://github.com/dlang/dub/blob/master/source/dub/commandline.d

  dub --compiler=ldc2 -color --config=ldc -b release

  dub --compiler=dmd -color --config=dmd

  dub --compiler=gdc-10 -color --config=gdc -b release

  make ldc

  make dmd

there has been some coalescence around the Meson build system
https://mesonbuild.com/

  meson

  ninja -C build

  meson setup --wipe build && ninja -v -C build

  make meson

dub --force --compiler=ldc2 && sudo cp -v cgi-bin/spine-search /usr/lib/cgi-bin/.

* Configuration

Configuration files are yaml files

The following paths are searched:

  ~/.dr/config_local_site
  ~/path_to_pod_root/.dr/config_local_site

e.g. processing

  ~spineMarkupSamples/pod/*

will search:

  ~spineMarkupSamples/pod/.dr/config_local_site

  ~/.dr/config_local_site

to specify an alternative configuration file to use on the command line (in this
example named "my_config"):

  spine -v --html --config=~spineMarkupSamples/pod/.dr/my_config

here is a sample configuration file:

flag:
  act0:                        "--html"
  act1:                        "--html --epub"
output:
  path:                        "/var/www/html"
default:
  language:                    "en"
  papersize:                   "a4"
  text_wrap:                   "80"
  digest:                      "sha256"
webserv:
  http:                        "http"
  host:                        "localhost"
  data_http:                   "http"
  data_host:                   "localhost"
  data_root_url:               "http://localhost"
  data_root_path:              "/var/www/html"
  data_root_part:              ""
  images_root_part:            "image"
  cgi_search_form_title:       "≅ SiSU Spine search"
  cgi_http:                    "http"
  cgi_host:                    "localhost"
  cgi_bin_url:                 "http://localhost/cgi-bin"
  cgi_bin_subpath:             "/cgi-bin"
  cgi_bin_path:                "/usr/lib/cgi-bin"
  cgi_search_script:           "spine-search"
  cgi_search_script_raw_fn_d:  "spine_search.d"
  cgi_port:                    ""
  cgi_user:                    ""
  cgi_action:                  "http://localhost/cgi-bin/spine-search"
  db_sqlite:                   "spine.search.db"
  db_pg_table:                 ""
  db_pg_user:                  ""

* Commands

for a list of commands from the program type:

  spine -h

at the time of writing this provides the following output:

                  --abstraction document abstraction
                       --assert set optional assertions on
      --cgi-search-form-codegen generates (pre-compiled) d code for search of specified db
   --cgi-sqlite-search-filename =[filename]
                  --concordance file for document
                       --config =/path/to/config/file/including/filename
                         --dark alternative dark theme
                        --debug debug
                       --digest hash digest for each object
                         --epub process epub output
                      --curate extract info on authors & topics from document header metadata
              --curate-authors extract info on authors from document header metadata
               --curate-topics extract info on topics from document header metadata
                     --hide-ocn object cite numbers
                         --html process html output
            --html-link-curate place links back to curate in segmented html
             --html-link-search html embedded search submission
                     --html-seg process html output
                  --html-scroll process html output
                         --lang =[lang code e.g. =en or =en,es]
                        --latex output for pdfs
            --latex-color-links mono or color links for pdfs
                        --light default light theme
                     --manifest process manifest output
                      --ocn-off object cite numbers
                          --odf open document format text (--odt)
                          --odt open document format text
                       --output =/path/to/output/dir specify where to place output
                     --parallel parallelisation
        --parallel-subprocesses nested parallelisation
                          --pdf latex output for pdfs
              --pdf-color-links mono or color links for pdfs
                          --pod spine (doc reform) pod source content bundled
-q                      --quiet output to terminal
           --section-backmatter document backmatter (default)
               --section-biblio document biblio (default)
                --section-blurb document blurb (default)
                 --section-body document body (default)
            --section-bookindex document bookindex (default)
             --section-endnotes document endnotes (default)
             --section-glossary document glossary (default)
                  --section-toc table of contents (default)
                       --serial serial processing
                  --skip-output skip output
                  --show-config show config
                    --show-make show make
                --show-metadata show metadata
                 --show-summary show summary
                       --source document markup source
              --sqlite-discrete process discrete sqlite output
             --sqlite-db-create create db, create tables
               --sqlite-db-drop drop tables & db
           --sqlite-db-recreate create db, create tables
                --sqlite-delete sqlite output
              --sqlite-db-filename =[filename].sql.db
                --sqlite-insert sqlite output
                --sqlite-update sqlite output
                         --text text output
                   --theme-dark alternative dark theme
                  --theme-light default light theme
                          --txt text output
-v                    --verbose output to terminal
                 --very-verbose output to terminal
                       --workon (reserved for some matters under development & testing)
                        --xhtml xhtml output
-h                       --help This help information.

* Examples

** curate

if you have a document collection with documents that have metadata headers a
summary of the collection can be made using the curate command

  spine -v --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --curate ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --html-link-curate --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --html-link-curate --curate ~spineMarkupSamples/pod/*

** sqlite

if configuartion has been set specify just
- the desired output and
- the markup document/pod(s) to process

  spine -v --html ~spineMarkupSamples/markup/pod/sisu-manual

if configuartion has not been set or to overide the set configration specify
- the output path as well as
- the desired output and
- the markup document/pod(s) to process

note: ~webDocRoot should be the path to web doc root, provide a suitable output path.

  spine -v --html --html-link-search --html-link-curate  --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --html-link-curate --epub --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --epub --latex --odt --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

*** create db

if there is no sqlite db you first need to create one, to do so
- the name of the db and
- the root path for document output
must be specified:

  spine -v \
    --sqlite-db-create --sqlite-db-filename="spine.search.db" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod/*

  spine -v --sqlite-db-create --sqlite-db-filename="spine.search.db" --output=`echo ~webDocRoot`

if you have a configration file providing this information that is to be used
for a document collection you can point to the document collection:

  spine -v --sqlite-db-create ~spineMarkupSamples/pod

*** populate db

must specify:
- the name of the db and
- the root path for document output

  spine -v --sqlite-update \
    --sqlite-db-filename="spine.search.db" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod/*

  spine -v --sqlite-update --sqlite-db-filename="spine.search.db" --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

if you have a configration file providing this information that is to be used
for a document collection you can point to the document collection:

  spine -v --sqlite-update ~spineMarkupSamples/pod/*

*** generate a cgi search form in d

  spine -v --cgi-search-form-codegen \
    --output=/var/www/html \
    ~spineMarkupSamples/pod

  spine -v --cgi-search-form-codegen --config=~spineMarkupSamples/pod

  spine -v --cgi-search-form-codegen --config=~spineMarkupSamples/pod/.dr/config_local_site

  spine --cgi-search-form-codegen --output=`echo ~webDocRoot` ~spineMarkupSamples/pod

  spine --cgi-search-form-codegen --cgi-sqlite-search-filename="spine_search" --output=`echo ~webDocRoot`

  spine -v --cgi-search-form-codegen \
    --sqlite-db-filename="spine.search.db" \
    --cgi-sqlite-search-filename="spine-search" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod

**** compile the cgi search form

  cd /var/www/html/cgi # /var/www/html (default document root)

  cd ~webDocRoot/cgi

the directory ~webDocRoot/cgi/src should contain two files
- spine_search.d (or whatever you named it)
- cgi.d (by Adam Rupee)

  dub --force --compiler=ldc2 && sudo cp -v cgi-bin/spine-search /usr/lib/cgi-bin/.

should compile spine-search in ~webDocRoot/cgi/cgi-bin and copy it to the
cgi-bin directory

  spine -v  --sqlite-db-create --sqlite-db-filename="spine.search.db" --cgi-sqlite-search-filename="spine-search" --output=`echo ~webDocRoot`

  spine -v  --sqlite-db-create  ~spineMarkupSamples/pod

  spine -v --html --html-link-search --cgi-sqlite-search-filename="spine-search" --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --cgi-sqlite-search-filename="spine-search" --html-link-curate --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

*** create db & search form

  spine -v \
    --sqlite-db-create --sqlite-db-filename="spine.search.db" \
    --cgi-search-form-codegen --cgi-sqlite-search-filename="spine-search" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod/*

*** html with links to search form

  spine -v --html \
    --html-link-search \
    --output=`echo ~webDocRoot` \
    ~spineMarkupSamples/pod/*