path: root/README.md

project_name:  Spine, Doc Reform

  description: [
      "documents, structuring, processing, publishing",
      search,
      object numbering,
      static content generator,
      sisu markup
    ]

    author:
      name:    Ralph Amissah
      email:   ralph.amissah@gmail.com

    copyright: "(C) 2015 - 2024 Ralph Amissah, All Rights Reserved."

    license:   "AGPL 3 or later"

    homepage: [
        "https://www.sisudoc.org",
        "https://www.doc-reform.org"
      ]

## Installation, Compilation

Development of sisudoc-spine started in 2015 on a Debian linux box.

Development since 2020 has been on a NixOS linux box, my laptop. If you are
fortunate enough to be using the same the build instructions should be presented
on entering the sisudoc-spine directory. It should be little problem building on
other linuxes with the right dependencies. At one time, debconf-18 I was
persuaded to try meson, and for a couple of years maintained a meson build, that
dropped out of use before or on my making the switch to nixos in 2020.

❯❯ D compiler and build manager

SiSU spine is written in the programming language D for which there are 3
compilers: dmd, ldc, gdc
- https://wiki.dlang.org/Compilers

D projects tend to use dub as project manager
- https://code.dlang.org/packages/dub
- https://github.com/dlang/dub/blob/master/source/dub/commandline.d

The default build tools used are dub with ldc2 (dub is also tested)

## make a directory and clone the sisudoc-spine project

  mkdir ~/git.sisudoc
  cd ~/git.sisudoc

  git clone git://git.sisudoc.org/software/sisudoc-spine && \
  git clone git://git.sisudoc.org/software/sisudoc-spine-search-cgi && \
  git clone git://git.sisudoc.org/markup/sisudoc-spine-samples

such a relative directory layout will be assumed in the examples that provided

all work in this installation of and use of sisudoc-spine will take place in the
directory: sisudoc-spine

## build sisudoc-spine

NOTE all actions to build sisudoc-spine are taken within the directory
sisudoc-spine

cd sisudoc-spine

## directly with dub
### ldc2
  # on nix (get dependencies by setting your development environment):
  nix develop ".#dsh-nixpkgs-ldc-dub" --print-build-logs -c zsh

  dub run --compiler=ldmd2 --config=ldmd2 --combined --skip-registry=all
  dub --compiler=ldmd2 --config=ldmd2

  dub run --compiler=ldc2 --config=ldc2 --combined --skip-registry=all
  dub --compiler=ldc2 --config=ldc2

### dmd
  # on nix (get dependencies by setting your development environment):
  nix develop ".#dsh-nixpkgs-dmd-dub" --print-build-logs -c zsh

  dub run --compiler=dmd --config=dmd --combined --skip-registry=all
  dub --compiler=dmd --config=dmd

## with make

### ldc2

  make ldc

### dmd

  make dmd

## with nix on linux / nixos

### ldc2

  nix build ".#spine-nixpkgs-ldc" --print-build-logs

### dmd

  nix build ".#spine-nixpkgs-dmd" --print-build-logs

## the Meson build system was used briefly

On recommendation at debconf-18 meson was used briefly. It has neither been tested nor used since the move to nix.

- https://mesonbuild.com/

  meson
  ninja -C build
  meson setup --wipe build && ninja -v -C build
  make meson

dub --force --compiler=ldc2 && sudo cp -v cgi-bin/spine-search /usr/lib/cgi-bin/.

# Document processing examples

These examples assume the file layout suggested in cloning the git.sisudoc.org
repository, i.e. that the directories sisudoc-spine and sisudoc-spine-samples
are next to each other on a directory tree. Assuming this to be the case, you
may wish to set the following exports with adjustments accoring to your specific
needs for these examples.

# ❯❯ set spine binary location:
export SpineBIN=./result/bin/spine
# ❯❯ nix builds spine binary:
#export SpineBIN=./result/bin/spine
# ❯❯ dub builds spine binary (name depends on build, check):
#export SpineBIN=./bin/spine
#export SpineBIN=./bin/spine-ldc
#export SpineBIN=./bin/spine-dmd
# ❯❯ location of source files:
export SpineDOC=../sisudoc-spine-samples
# ❯❯ location of source files pod:
export SpinePOD=${SpineDOC}/markup/pod
# ❯❯ sisudoc-spine output processing path:
export SpineOUT=./OUTPUT_TEST_sisudocSpine
# ❯❯ sisudoc-spine output processing path (web server e.g.):
#export SpineOUT=/srv/www/spine
export SpineSearchActionLocal='http://localhost/spine_search'
export SpineSearchActionRemote='https://sisudoc.org/spine_search'
# ❯❯ path configured for cgi search form:
export SpineCGIform='spine_search'
# ❯❯ search form db name:
export SpineSQLdb='spine.search.db'
# ❯❯ configuration cgi search form path:
#export SpineCGIbin=/var/www/cgi/cgi-bin
# ❯❯ configuration db path:
#export SpineDBpath=/var/www/sqlite

### html with links to search form

${SpineBIN} -v --epub --html --html-link-curate --curate --output=${SpineOUT} ${SpinePOD}/*

${SpineBIN} -v --source --pod --latex --latex-init --epub --html --html-link-pdf --html-link-curate --html-link-markup --curate --output=${SpineOUT} ${SpinePOD}/*

${SpineBIN} -v --source --pod --latex --latex-init --epub --html --html-link-search --html-link-pdf --html-link-curate --html-link-markup --cgi-sqlite-search-filename="${SpineCGIform}" --cgi-url-action="${SpineSearchActionLocal}" --curate --sqlite-update --sqlite-db-filename="${SpineSQLdb}" --output=${SpineOUT} ${SpinePOD}/*

spine -v --html \
  --html-link-search \
  --output=`echo ~webDocRoot` \
  ${SpinePOD}/*

## curate

if you have a document collection with documents that have metadata headers a
summary of the collection can be made using the curate command

  spine -v --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --curate ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --html-link-curate --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --html-link-curate --curate ~spineMarkupSamples/pod/*

## sqlite

if configuartion has been set specify just
- the desired output and
- the markup document/pod(s) to process

  spine -v --html ~spineMarkupSamples/markup/pod/sisu-manual

if configuration has not been set or to overide the set configuration specify
- the output path as well as
- the desired output and
- the markup document/pod(s) to process

note: ~webDocRoot should be the path to web doc root, provide a suitable output path.

  spine -v --html --html-link-search --html-link-curate  --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --html-link-curate --epub --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --epub --latex --odt --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

### create db

if there is no sqlite db you first need to create one, to do so
- the name of the db and
- the root path for document output
must be specified:

  spine -v \
    --sqlite-db-create --sqlite-db-filename="spine.search.db" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod/*

  spine -v --sqlite-db-create --sqlite-db-filename="spine.search.db" --output=`echo ~webDocRoot`

if you have a configration file providing this information that is to be used
for a document collection you can point to the document collection:

  spine -v --sqlite-db-create ~spineMarkupSamples/pod

### populate db

must specify:
- the name of the db and
- the root path for document output

  spine -v --sqlite-update \
    --sqlite-db-filename="spine.search.db" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod/*

  spine -v --sqlite-update --sqlite-db-filename="spine.search.db" --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

if you have a configration file providing this information that is to be used
for a document collection you can point to the document collection:

  spine -v --sqlite-update ~spineMarkupSamples/pod/*

### generate a cgi search form in d

  spine -v --cgi-search-form-codegen \
    --output=/var/www/html \
    ~spineMarkupSamples/pod
  
  spine -v --cgi-search-form-codegen --config=~spineMarkupSamples/pod
  
  spine -v --cgi-search-form-codegen --config=~spineMarkupSamples/pod/.dr/config_local_site
  
  spine --cgi-search-form-codegen --output=`echo ~webDocRoot` ~spineMarkupSamples/pod
  
  spine --cgi-search-form-codegen --cgi-sqlite-search-filename="spine_search" --output=`echo ~webDocRoot`
  
  spine -v --cgi-search-form-codegen \
    --sqlite-db-filename="spine.search.db" \
    --cgi-sqlite-search-filename="spine-search" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod

#### compile the cgi search form

  cd /var/www/html/cgi # /var/www/html (default document root)

  cd ~webDocRoot/cgi

the directory ~webDocRoot/cgi/src should contain two files
- spine_search.d (or whatever you named it)
- cgi.d (by Adam Rupee)

  dub --force --compiler=ldc2 && sudo cp -v cgi-bin/spine-search /usr/lib/cgi-bin/.

should compile spine-search in ~webDocRoot/cgi/cgi-bin and copy it to the
cgi-bin directory

  spine -v  --sqlite-db-create --sqlite-db-filename="spine.search.db" --cgi-sqlite-search-filename="spine-search" --output=`echo ~webDocRoot`

  spine -v  --sqlite-db-create  ~spineMarkupSamples/pod

  spine -v --html --html-link-search --cgi-sqlite-search-filename="spine-search" --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  spine -v --html --html-link-search --cgi-sqlite-search-filename="spine-search" --html-link-curate --curate --output=`echo ~webDocRoot` ~spineMarkupSamples/pod/*

  ${SpineBIN} -v --source --pod --latex --latex-init --epub --html --html-link-search --html-link-pdf --html-link-curate --html-link-markup --cgi-sqlite-search-filename="${SpineCGIform}" --cgi-url-action="${SpineSearchActionLocal}" --curate --sqlite-update --sqlite-db-filename="${SpineSQLdb}" --output=${SpineOUT} ${SpinePOD}/*


### create db & search form

  spine -v \
    --sqlite-db-create --sqlite-db-filename="spine.search.db" \
    --cgi-search-form-codegen --cgi-sqlite-search-filename="spine-search" \
    --output=/var/www/html \
    ~spineMarkupSamples/pod/*

# Commands

for a list of commands from the program type:

  spine -h

at the time of writing this provides the following output:
                  --abstraction document abstraction
              --allow-downloads allow downloads (includes cgi.d from github)
                       --assert set optional assertions on
                 --cgi-bin-root path to cgi-bin directory
                 --cgi-url-root url to cgi-bin (to find cgi-bin)
               --cgi-url-action url to post to cgi-bin search form
             --cgi-search-title if generating a cgi search form the title to use for it
   --cgi-sqlite-search-filename =[filename] default is spine-search
                  --concordance file for document
                       --curate extract info on authors & topics from document header metadata
               --curate-authors extract info on authors from document header metadata
                --curate-topics extract info on topics from document header metadata
                         --dark alternative dark theme
                       --digest hash digest for each object
                         --epub process epub output
                 --generated-by generated by headers (software version & time)
                     --hide-ocn object cite numbers
                         --html process html output
             --html-link-curate place links back to curate in segmented html
             --html-link-markup provide html link to markup source, shared optionally
                --html-link-pdf provide html link to pdf a4 & letter output
             --html-link-pdf-a4 provide html link to pdf a4 output
         --html-link-pdf-letter provide html link to pdf letter size output
             --html-link-search html embedded search submission
                     --html-seg process html output
                  --html-scroll process html output
                         --lang =[lang code e.g. =en or =en,es]
                        --latex latex output (for pdfs)
            --latex-color-links mono or color links for pdfs
                   --latex-init initialise latex shared files (see latex-header-sty)
             --latex-header-sty latex document header sty files
                        --light default light theme
                     --manifest process manifest output
                      --ocn-off object cite numbers
                          --odf open document format text (--odt)
                          --odt open document format text
                       --output =/path/to/output/dir specify where to place output
                     --parallel parallelisation
        --parallel-subprocesses nested parallelisation
                          --pdf latex output for pdfs
              --pdf-color-links mono or color links for pdfs
                     --pdf-init initialise latex shared files (see latex-header-sty)
                          --pod spine (doc reform) pod source content bundled
-q                      --quiet output to terminal
           --section-backmatter document backmatter (default)
               --section-biblio document biblio (default)
                --section-blurb document blurb (default)
                 --section-body document body (default)
            --section-bookindex document bookindex (default)
             --section-endnotes document endnotes (default)
             --section-glossary document glossary (default)
                  --section-toc table of contents (default)
                       --serial serial processing
                  --skip-output skip output
                  --show-config show config
                  --show-curate show curate
          --show-curate-authors show curate authors
           --show-curate-topics show curate topics
                    --show-epub show epub
                    --show-html show html
                   --show-latex show latex
                    --show-make show make
                --show-manifest show manifest
                --show-metadata show metadata
                     --show-pod show pod
                  --show-sqlite show sqlite
                 --show-summary show summary
                       --source document markup source
                   --set-digest default hash digest type (e.g. sha256)
                --set-papersize default papersize (latex pdf eg. a4 or a5 or b4 or letter)
                 --set-textwrap default textwrap (e.g. 80 (characters)
              --sqlite-discrete process discrete sqlite output
             --sqlite-db-create create db, create tables
               --sqlite-db-drop drop tables & db
           --sqlite-db-filename sqlite db to create, populate & make available for search
               --sqlite-db-path sqlite db path
           --sqlite-db-recreate create db, create tables
                --sqlite-delete sqlite output
                --sqlite-insert sqlite output
                --sqlite-update sqlite output
                     --www-http http or https
                     --www-host web server host (domain) name
            --www-host-doc-root web host host (domain) name with path to doc root
             --www-url-doc-root e.g. http://localhost
                         --text text output
                   --theme-dark alternative dark theme
                  --theme-light default light theme
                          --txt text output
-v                    --verbose output to terminal
                 --very-verbose output to terminal
                       --workon (reserved for some matters under development & testing)
                        --xhtml xhtml output
                       --config =/path/to/config/file/including/filename
                        --debug debug
                 --debug-curate debug curate
         --debug-curate-authors debug curate authors
          --debug-curate-topics debug curate topics
                   --debug-epub debug epub
                --debug-harvest debug harvest
                   --debug-html debug html
                  --debug-latex debug latex
               --debug-manifest debug manifest
               --debug-metadata debug metadata
                    --debug-pod debug pod
                 --debug-sqlite debug sqlite
                 --debug-stages debug stages
-h                       --help This help information.