Standardize gotap

Author: mmaelicke

README.md

@@ -3,84 +3,76 @@
 [![Docker Image CI](https://github.com/tool-spec/tool_template_python/actions/workflows/docker-image.yml/badge.svg)](https://github.com/tool-spec/tool_template_python/actions/workflows/docker-image.yml)
 [![DOI](https://zenodo.org/badge/558416591.svg)](https://zenodo.org/badge/latestdoi/558416591)
 
-This is the template for a generic containerized Python tool following the [Tool Specification](https://tool-spec.github.io/tool-specs/) for reusable research software using Docker.
+Template repository for building a Python tool that follows the [Tool Specification](https://tool-spec.github.io/tool-specs/) container contract.
 
-This template can be used to generate new Github repositories from it.
+## How `gotap` works here
 
+This template ships with [`gotap`](https://github.com/tool-spec/gotap) inside the image. The default container command is:
 
-## How generic?
+```Dockerfile
+CMD ["gotap", "run", "foobar", "--input-file", "/in/input.json"]
+```
 
-Tools using this template can be run by the [toolbox-runner](https://github.com/tool-spec/tool-runner). 
-That is only convenience, the tools implemented using this template are independent of any framework.
+At build time, `gotap generate` creates `parameters.py` from `src/tool.yml`. At runtime, `run.py` uses the generated bindings to:
 
-The main idea is to implement a common file structure inside container to load inputs and outputs of the 
-tool. The template shares this structures with the [R template](https://github.com/tool-spec/tool_template_r),
-[NodeJS template](https://github.com/tool-spec/tool_template_node) and [Octave template](https://github.com/tool-spec/tool_template_octave), 
-but can be mimiced in any container.
+- validate `/in/input.json`
+- load typed parameters and data paths
+- emit structured run logs
 
-Each container needs at least the following structure:
+## Required file structure
 
-```
+```text
 /
 |- in/
-|  |- parameters.json
+|  |- input.json
 |- out/
 |  |- ...
 |- src/
 |  |- tool.yml
 |  |- run.py
+|  |- parameters.py   (generated at build time)
+|  |- CITATION.cff
 ```
 
-* `parameters.json` are parameters. Whichever framework runs the container, this is how parameters are passed.
-* `tool.yml` is the tool specification. It contains metadata about the scope of the tool, the number of endpoints (functions) and their parameters
-* `run.py` is the tool itself, or a Python script that handles the execution. It has to capture all outputs and either `print` them to console or create files in `/out`
-
-## How to build the image?
-
-You can build the image from within the root of this repo by
-```
-docker build -t tbr_python_tempate .
-```
+- `/in/input.json` contains parameter values and data references
+- `/out/` receives generated files plus `gotap` metadata such as `_metadata.json`
+- `/src/tool.yml` defines the tool metadata and command
+- `/src/run.py` is the tool entrypoint referenced by `tool.yml`
 
-Use any tag you like. If you want to run and manage the container with [toolbox-runner](https://github.com/tool-spec/tool-runner)
-they should be prefixed by `tbr_` to be recognized. 
+## Build and run
 
-Alternatively, the contained `.github/workflows/docker-image.yml` will build the image for you 
-on new releases on Github. You need to change the target repository in the aforementioned yaml.
+Build the image from the template root:
 
-## How to run?
+```bash
+docker build -t tbr_python_template .
+```
 
-This template installs the json2args python package to parse the parameters in the `/in/parameters.json`. This assumes that
-the files are not renamed and not moved and there is actually only one tool in the container. For any other case, the environment variables
-`PARAM_FILE` can be used to specify a new location for the `parameters.json` and `TOOL_RUN` can be used to specify the tool to be executed.
-The `run.py` has to take care of that.
+Run the sample tool with the bundled input and output folders:
 
-To invoke the docker container directly run something similar to:
-```
-docker run --rm -it -v /path/to/local/in:/in -v /path/to/local/out:/out -e TOOL_RUN=foobar tbr_python_template
+```bash
+docker run --rm -it \
+  -v "$(pwd)/in:/in" \
+  -v "$(pwd)/out:/out" \
+  -e TOOL_RUN=foobar \
+  tbr_python_template
 ```
 
-Then, the output will be in your local out and based on your local input folder. Stdout and Stderr are also connected to the host.
+`TOOL_RUN` is only needed when the image contains more than one tool entry. The primary runtime contract is still `/in/input.json` plus `gotap run`.
 
-With the [toolbox runner](https://github.com/tool-spec/tool-runner), this is simplyfied:
+## Customize
 
-```python
-from toolbox_runner import list_tools
-tools = list_tools() # dict with tool names as keys
-
-foobar = tools.get('foobar')  # it has to be present there...
-foobar.run(result_path='./', foo_int=1337, foo_string="Please change me")
-```
-The example above will create a temporary file structure to be mounted into the container and then create a `.tar.gz` on termination of all 
-inputs, outputs, specifications and some metadata, including the image sha256 used to create the output in the current working directory.
+1. Update `src/tool.yml` to describe your real tool.
+2. Add Python or system dependencies in `Dockerfile`.
+3. Implement the tool logic in `src/run.py`.
+4. Rebuild the image so `gotap generate` refreshes `parameters.py`.
 
-## What about real tools, no foobar?
+## Generated bindings and logging
 
-Yeah. 
+The generated `parameters.py` file is not edited by hand. It exposes:
 
-1. change the `tool.yml` to describe your actual tool
-2. add any `pip install` or `apt-get install` needed to the dockerfile
-3. add additional source code to `/src`
-4. change the `run.py` to consume parameters and data from `/in` and useful output in `out`
-5. build, run, rock!
+- `get_parameters()`
+- `get_data()`
+- `get_run_context()`
+- `get_logger()`
 
+The starter code uses `get_logger()` to write structured JSON Lines logs to the file chosen by `gotap`.

src/run.py

@@ -1,29 +1,31 @@
-import logging
 import os
-from datetime import datetime as dt
+import sys
 
-from parameters import get_parameters, get_data
+from parameters import get_data, get_logger, get_parameters
 
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-# parse parameters (generated by gotap at container build time)
 params = get_parameters()
 data = get_data()
+logger = get_logger()
 
-# check if a toolname was set in env
-toolname = os.environ.get('TOOL_RUN', 'foobar').lower()
+toolname = os.environ.get("TOOL_RUN", "foobar").lower()
 
-# switch the tool
-if toolname == 'foobar':
-    # RUN the tool here and create the output in /out
-    logger.info('This toolbox does not include any tool. Did you run the template?\n')
+logger.info("start", "Starting tool run", tool=toolname)
+logger.info(
+    "input-loaded",
+    "Loaded validated parameters and data paths",
+    tool=toolname,
+    parameter_count=len(vars(params)),
+    data_keys=sorted(data.keys()),
+)
 
-    logger.info(vars(params))
+if toolname == "foobar":
+    sys.stderr.write("This toolbox does not include any tool. Did you run the template?\n")
+    sys.stderr.write(f"{vars(params)}\n")
 
     for name, path in data.items():
-        logger.info(f"\n### {name}: %s", path)
+        sys.stderr.write(f"\n### {name}: {path}\n")
 
-# In any other case, it was not clear which tool to run
+    logger.info("finished", "Template run finished successfully", tool=toolname)
 else:
-    raise AttributeError(f"[{dt.now().isocalendar()}] Either no TOOL_RUN environment variable available, or '{toolname}' is not valid.\n")
+    logger.error("error", "Requested tool is not implemented in the template", tool=toolname)
+    raise AttributeError(f"Either no TOOL_RUN environment variable available, or '{toolname}' is not valid.\n")