From e04da71f92bfbb75bb580744136da36f0f4c3810 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Max=20L=C3=BCbke?= <mluebke@gfz-potsdam.de>
Date: Tue, 7 Mar 2023 10:36:31 +0100
Subject: [PATCH] doc: update documentation to latest progress

---
 README.md             | 146 ++++++++++++++++++------------------------
 docs/CMakeLists.txt   |   1 +
 docs/Input_Scripts.md |  95 +++++++++++++++++++++++++++
 docs/Output.md        |  12 ++--
 4 files changed, 165 insertions(+), 89 deletions(-)
 create mode 100644 docs/Input_Scripts.md

diff --git a/README.md b/README.md
index 774fd3f31..224271910 100644
--- a/README.md
+++ b/README.md
@@ -1,10 +1,6 @@
 <!--
     Time-stamp: "Last modified 2023-01-19 12:06:10 delucia"
 -->
-
-**Po**tsdamer **R**eactive **T**ransport
-
-
 # Forked Project
 
 *PORT* is a fork of [POET](https://doi.org/10.5281/zenodo.4757913)
@@ -19,11 +15,20 @@ also applicable for this project.
 POET is a coupled reactive transport simulator implementing a parallel
 architecture and a fast, original MPI-based Distributed Hash Table.
 
+## Parsed code documentiation
+
+A parsed version of POET's documentiation can be found at [Gitlab
+pages](https://sec34.git-pages.gfz-potsdam.de/port).
+
 ## External Libraries
 
 The following external header library is shipped with POET:
 
 - **argh** - https://github.com/adishavit/argh (BSD license)
+- **PhreeqcRM** with patches from GFZ -
+  https://www.usgs.gov/software/phreeqc-version-3 -
+  https://git.gfz-potsdam.de/mluebke/phreeqcrm-gfz
+- **tug** - https://git.gfz-potsdam.de/sec34/tug
 
 ## Installation
 
@@ -35,23 +40,18 @@ To compile POET you need several software to be installed:
 - MPI-Implementation (tested with OpenMPI and MVAPICH)
 - R language and environment
 - CMake 3.9+
-
-If you want to build documentation during compilation, `doxygen`and
-`graphviz` must be provided too.
+- *optional*: `doxygen` with `dot` bindings for documentiation
 
 The following R libraries must then be installed, which will get the
 needed dependencies automatically:
 
-- [devtools](https://www.r-project.org/nosvn/pandoc/devtools.html)
 - [Rcpp](https://cran.r-project.org/web/packages/Rcpp/index.html)
 - [RInside](https://cran.r-project.org/web/packages/RInside/index.html)
-- [RedModRphree](https://git.gfz-potsdam.de/delucia/RedModRphree)
-- [Rmufits](https://git.gfz-potsdam.de/delucia/Rmufits)
 
 ### Compiling source code
 
-The generation of makefiles is done with CMake. If you obtained POET
-from git, you should be able to generate Makefiles by running
+The generation of makefiles is done with CMake. You should be able to generate
+Makefiles by running:
 
 ```sh
 mkdir build && cd build
@@ -62,34 +62,21 @@ This will create the directory `build` and processes the CMake files
 and generate Makefiles from it. You're now able to run `make` to start
 build process.
 
-If POET was obtained from the official SVN repository or the redmine
-at <https://redmine.cs.uni-potsdam.de/projects/poet> the branch or tag
-to be used have to be set via
-
-```sh
-mkdir build && cd build
-cmake -D POET_SET_BRANCH="<BRANCH>" ..
-```
-
-where currently available branches/tags are:
-
-- dev
-
 If everything went well you'll find the executable at
-`build/src/poet`, but it is recommended to install the POET project
+`build/app/poet`, but it is recommended to install the POET project
 structure to a desired `CMAKE_INSTALL_PREFIX` with `make install`.
 
 During the generation of Makefiles, various options can be specified
-via `cmake -D <option>=<value> [...]`. Currently there are the
+via `cmake -D <option>=<value> [...]`. Currently, there are the
 following available options:
 
-- **DHT_Debug**=_boolean_ - toggles the output of detailed statistics
-  about DHT usage (`cmake -D DHT_Debug=ON`). Defaults to _OFF_.
-- **BUILD_DOC**=_boolean_ - toggles the generation of documantiation
-  during compilation process. Defaults to _ON_.
-- _only from svn version:_ **POET_SET_BRANCH**=_string_ - set branch
-  or tag whose code is used
-
+- **POET_DHT_Debug**=_boolean_ - toggles the output of detailed statistics about
+  DHT usage. Defaults to _OFF_.
+- **POET_ENABLE_TESTING**=_boolean_ - enables small set of unit tests (more to
+  come). Defaults to _OFF_.
+- **POET_USE_PRM_BACKEND**=_bollean_ - use the PhreeqcRM parallelization instead
+  of POET's one. Intended for debugging purposes for modellers.
+  
 ### Example: Build from scratch
 
 Assuming that only the C/C++ compiler, MPI libraries, R runtime
@@ -101,9 +88,7 @@ follows:
 $ R
 
 # install R dependencies
-> install.packages(c("devtools", "Rcpp", "RInside"))
-> devtools::install_gitlab("delucia/RedModRphree", host="https://git.gfz-potsdam.de")
-> devtools::install_gitlab("delucia/Rmufits", host="https://git.gfz-potsdam.de")
+> install.packages(c("Rcpp", "RInside"))
 > q(save="no")
 
 # cd into POET project root
@@ -124,60 +109,53 @@ POET we **do not recommend** to install to hierarchies like
 The correspondending directory tree would look like this:
 
 ```sh
-.
-└── poet/
-    ├── bin/
-    │   └── poet
-    ├── data/
-    │   └── SimDol2D.R
-    ├── docs/
-    │    └── html/
-    │       ├── index.html
-    │       └── ...
-    └── R_lib/
-        ├── kin_r_library.R
-        └── parallel_r_library.R
+poet
+├── bin
+│   └── poet
+├── R_lib
+│   └── kin_r_library.R
+└── share
+    └── poet
+        ├── bench
+        │   ├── dolo_diffu_inner_large.R
+        │   ├── dolo_diffu_inner.R
+        │   └── dolo_inner.pqi
+        └── examples
+            ├── dol.pqi
+            ├── phreeqc_kin.dat
+            ├── SimDol1D_diffu.R
+            └── SimDol2D_diffu.R
 ```
 
 The R libraries will be loaded at runtime and the paths are hardcoded
 absolute paths inside `poet.cpp`. So, if you consider to move
 `bin/poet` either change paths of the R source files and recompile
-POET or also move `R_lib/*` according to the binary.
-
-To display the generated html documentation just open
-`docs/html/index.html` with the browser of your choice.
+POET or also move `R_lib/*` relative to the binary.
 
 ## Running
 
-Before POET is ready to run, a working directory must be created. In
-this directory you should find the executable file, the R scripts
-`<POET_ROOT>/R_lib/kin_r_library.R` and
-`<POET_ROOT>/R_lib/parallel_r_library.R` and the simulation
-description e.g. `<POET_ROOT>/data/chem_problems/SimDol2D.R`.
-
 Run POET by `mpirun ./poet <OPTIONS> <SIMFILE> <OUTPUT_DIRECTORY>`
 where:
 
 - **OPTIONS** - runtime parameters (explained below)
-- **SIMFILE** - simulation described as R script (currently supported:
-  `<POET_INSTALL_DIR>/data/SimDol2D.R`)
+- **SIMFILE** - simulation described as R script (e.g.
+  `<POET_INSTALL_DIR>/share/examples/SimDol2D_diffu.R`)
 - **OUTPUT_DIRECTORY** - path, where all output of POET should be stored
 
 ### Runtime options
 
 The following parameters can be set:
 
-| Option                   | Value        | Description                                                    |
-| ------------------------ | ------------ | -------------------------------------------------------------- |
-| **--work-package-size=** | _1..n_       | size of work packages (defaults to _5_)                        |
-| **--ignore-result**      |              | disables store of simulation resuls                            |
-| **--dht**                |              | enabling DHT usage (defaults to _OFF_)                         |
-| **--dht-nolog**          |              | disabling applying of logarithm before rounding                |
-| **--dht-signif=**        | _1..n_       | set rounding to number of significant digits (defaults to _5_) |
-| **--dht-strategy=**      | _0-1_        | change DHT strategy. **NOT IMPLEMENTED YET** (Defaults to _0_) |
-| **--dht-size=**          | _1-n_        | size of DHT per process involved in byte (defaults to _1 GiB_) |
-| **--dht-snaps=**         | _0-2_        | disable or enable storage of DHT snapshots                     |
-| **--dht-file=**          | `<SNAPSHOT>` | initializes DHT with the given snapshot file                   |
+| Option                   | Value        | Description                                                                                                              |
+|--------------------------|--------------|--------------------------------------------------------------------------------------------------------------------------|
+| **--work-package-size=** | _1..n_       | size of work packages (defaults to _5_)                                                                                  |
+| **--ignore-result**      |              | disables store of simulation resuls                                                                                      |
+| **--dht**                |              | enabling DHT usage (defaults to _OFF_)                                                                                   |
+| **--dht-signif=**        | _1..n_       | set rounding to number of significant digits (defaults to _5_) (it is recommended to use `signif_vec` in R input script) |
+| **--dht-strategy=**      | _0-1_        | change DHT strategy. **NOT IMPLEMENTED YET** (Defaults to _0_)                                                           |
+| **--dht-size=**          | _1-n_        | size of DHT per process involved in megabyte (defaults to _1000 MByte_)                                                  |
+| **--dht-snaps=**         | _0-2_        | disable or enable storage of DHT snapshots                                                                               |
+| **--dht-file=**          | `<SNAPSHOT>` | initializes DHT with the given snapshot file                                                                             |
 
 #### Additions to `dht-signif`
 
@@ -197,26 +175,26 @@ Following values can be set:
 ### Example: Running from scratch
 
 We will continue the above example and start a simulation with
-`SimDol2D.R`, which is the only simulation supported at this moment.
-The required flow velocities snapshots are included in the R package
-Rmufits. It's a 2D, 50x50 grid, with 20 time steps. To start the
-simulation with 4 processes `cd` into your previously installed
-POET-dir `<POET_INSTALL_DIR>/bin` and run:
+`SimDol2D_diffu.R`. As transport a simple fixed-coefficient diffusion is used.
+It's a 2D, 100x100 grid, simulating 10 time steps. To start the simulation with
+4 processes `cd` into your previously installed POET-dir
+`<POET_INSTALL_DIR>/bin` and run:
 
 ```sh
-mpirun -n 4 ./poet ../data/SimDol2D.R output
+mpirun -n 4 ./poet ../share/poet/examples/SimDol2D_diffu.R output
 ```
 
 After a finished simulation all data generated by POET will be found
 in the directory `output`.
 
-You might want to use the DHT to cache previously simulated data and
-reuse them in further time-steps. Just append `--dht` to the options
-of POET to activate the usage of the DHT. The resulting call would
-look like this:
+You might want to use the DHT to cache previously simulated data and reuse them
+in further time-steps. Just append `--dht` to the options of POET to activate
+the usage of the DHT. Also, after each iteration a DHT snapshot shall be
+produced. This is done by appending the `--dht-snaps=<value>` option. The
+resulting call would look like this:
 
 ```sh
-mpirun -n 4 ./poet --dht SimDol2D.R output
+mpirun -n 4 ./poet --dht --dht-snaps=2 ../share/poet/examples/SimDol2D_diffu.R output
 ```
 
 ## About the usage of MPI_Wtime()
diff --git a/docs/CMakeLists.txt b/docs/CMakeLists.txt
index 5b8adb797..b839a6c45 100644
--- a/docs/CMakeLists.txt
+++ b/docs/CMakeLists.txt
@@ -13,6 +13,7 @@ if(DOXYGEN_FOUND)
   doxygen_add_docs(doxygen
     ${PROJECT_SOURCE_DIR}/include
     ${PROJECT_SOURCE_DIR}/README.md
+    ${PROJECT_SOURCE_DIR}/docs/Input_Scripts.md
     ${PROJECT_SOURCE_DIR}/docs/Output.md
     COMMENT "Generate html pages")
 endif()
diff --git a/docs/Input_Scripts.md b/docs/Input_Scripts.md
new file mode 100644
index 000000000..305ac554f
--- /dev/null
+++ b/docs/Input_Scripts.md
@@ -0,0 +1,95 @@
+# Input Scripts
+
+In the following the expected schemes of the input scripts is described.
+Therefore, each section of the input script gets its own chapter. All sections
+should return a `list` as results, which are concatenated to one setup list at
+the end of the file. All values must have the same name in order to get parsed
+by POET.
+
+## Grid initialization
+
+| name           | type           | description                                                           |
+|----------------|----------------|-----------------------------------------------------------------------|
+| `n_cells`      | Numeric Vector | Number of cells in each direction                                     |
+| `s_cells`      | Numeric Vector | Spatial resolution of grid in each direction                          |
+| `type`         | String         | Type of initialization, can be set to *scratch*, *phreeqc* or *rds*   |
+| `init_cell`    | Data Frame     | Containing all exactly one value per species to initialize the field. |
+| `props`        | String Vector  | Names of all species                                                  |
+| `database`     | String         | Path to Phreeqc database                                              |
+| `input_script` | String         | Path to Phreeqc initial script                                        |
+
+## Diffusion parameters
+
+| name           | type                 | description                               |
+|----------------|----------------------|-------------------------------------------|
+| `init`         | Named Numeric Vector | Initial state for each diffused species   |
+| `vecinj`       | Data Frame           | Defining all boundary conditions row wise |
+| `vecinj_inner` | List of Triples      | Inner boundaries                          |
+| `vecinj_index` | List of 4 elements   | Ghost nodes boundary conditions           |
+| `alpha`        | Named Numeric Vector | Constant alpha for each species           |
+
+### Remark on boundary conditions
+
+Each boundary condition should be defined in `vecinj` as a data frame, where one
+row holds one boundary condition.
+
+To define inner (constant) boundary conditions, use a list of triples in
+`vecinj_inner`, where each triples is defined by $(i,x,y)$. $i$ is defining the
+boundary condition, referencing to the row in `vecinj`. $x$ and $y$ coordinates
+then defining the position inside the grid. 
+
+Ghost nodes are set by `vecinj_index` which is a list containing boundaries for
+each celestial direction (**important**: named by `N, E, S, W`). Each direction
+is a numeric vector, also representing a row index of the `vecinj` data frame
+for each ghost node, starting at the left-most and upper cell respectively. By
+setting the boundary condition to $0$, the ghost node is set as closed boundary.
+
+#### Example
+
+Suppose you have a `vecinj` data frame defining 2 boundary conditions and a grid
+consisting of $10 \times 10$ grid cells. Grid cell $(1,1)$ should be set to the
+first boundary condition and $(5,6)$ to the second. Also, all boundary
+conditions for the ghost nodes should be closed. Except the southern boundary,
+which should be set to the first boundary condition injection. The following
+setup describes how to setup your initial script, where `n` and `m` are the
+grids cell count for each direction ($n = m = 10$):
+
+```R
+vecinj_inner <- list (
+  l1 = c(1, 1, 1),
+  l2 = c(2, 5, 6)
+)
+
+vecinj_index <- list(
+  "N" = rep(0, n),
+  "E" = rep(0, m),
+  "S" = rep(1, n),
+  "W" = rep(0, m)
+)
+```
+
+## Chemistry parameters
+
+| name           | type   | description                       |
+|----------------|--------|-----------------------------------|
+| `database`     | String | Path to the Phreeqc database      |
+| `input_script` | String | Path the the Phreeqc input script |
+
+## Final setup
+
+| name           | type           | description                                                |
+|----------------|----------------|------------------------------------------------------------|
+| `grid`         | List           | Grid parameter list                                        |
+| `diffusion`    | List           | Diffusion parameter list                                   |
+| `chemistry`    | List           | Chemistry parameter list                                   |
+| `iterations`   | Numeric Value  | Count of iterations                                        |
+| `timesteps`    | Numeric Vector | $\Delta t$ to use for specific iteration                   |
+| `store_result` | Boolean        | Indicates if results should be stored                      |
+| `out_save`     | Numeric Vector | *optional:* At which iteration the states should be stored |
+
+### DHT setup
+
+| name            | type           | description                                                                     |
+|-----------------|----------------|---------------------------------------------------------------------------------|
+| `signif_vector` | Numeric Vector | Indicates significant digits to use for DHT rounding. Order of `props` vector.  |
+| `prop_type`     | String Vector  | Set type of species for rounding, can be left blank or set to *act* or *ignore* |
diff --git a/docs/Output.md b/docs/Output.md
index 05b3fa284..4044c84a6 100644
--- a/docs/Output.md
+++ b/docs/Output.md
@@ -46,11 +46,13 @@ and possible to read out within a R runtime with
 If running parallel there are also measured timings which are subsets of
 *simtime\_​chemistry*.
 
-| Value                      | Description                                        |
-|----------------------------|----------------------------------------------------|
-| simtime\_workers           | time spent in send/​recv loop of master             |
-| simtime\_chemistry\_master | sequential part of master chemistry                |
-| phreeqc                    | measured time of each worker in PHREEQC subroutine |
+| Value                 | Description                                               |
+|-----------------------|-----------------------------------------------------------|
+| chemistry\_loop       | time spent in send/​recv loop of master                    |
+| chemistry\_sequential | sequential part of master chemistry                       |
+| idle\_master          | idling time (waiting for any free worker) of the master   |
+| idle\_worker          | idling time (waiting for work from master) of the workers |
+| phreeqc\_time         | accumulated times for Phreeqc calls of every worker       |
 
 ### DHT usage {#DHT-usage}