Skip to content

Commit a4dda1f

Browse files
committed
Standardize gotap
1 parent 7d609c4 commit a4dda1f

2 files changed

Lines changed: 49 additions & 60 deletions

File tree

README.md

Lines changed: 37 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -3,89 +3,67 @@
33
[![Docker Image CI](https://github.com/tool-spec/tool_template_node/actions/workflows/docker-image.yml/badge.svg)](https://github.com/tool-spec/tool_template_node/actions/workflows/docker-image.yml)
44
[![DOI](https://zenodo.org/badge/564802876.svg)](https://doi.org/10.5281/zenodo.7892415)
55

6-
This is the template for a generic containerized NodeJS tool following the [Tool Specification](https://tool-spec.github.io/tool-specs/) for reusable research software using Docker.
6+
Template repository for building a Node.js tool that follows the [Tool Specification](https://tool-spec.github.io/tool-specs/) container contract.
77

8-
This template can be used to generate new Github repositories from it.
8+
## How `gotap` works here
99

10-
Please note that this project has nothing to do with Javascript frontend tools/libs and NodeJS is also not used to
11-
build some kind of webproject backend stack. We want to integrate NodeJS into a generic workflow runner to make tools available,
12-
scientists might be used to from frontend development.
10+
This template installs [`gotap`](https://github.com/tool-spec/gotap) inside the image and uses it as the default runtime shim:
1311

14-
## How generic?
15-
16-
Tools using this template can be run by the [toolbox-runner](https://github.com/tool-spec/tool-runner).
17-
That is only convenience, the tools implemented using this template are independent of any framework.
12+
```Dockerfile
13+
CMD ["gotap", "run", "foobar", "--input-file", "/in/input.json"]
14+
```
1815

19-
The main idea is to implement a common file structure inside container to load inputs and outputs of the
20-
tool. The template shares this structures with the [R template](https://github.com/tool-spec/tool_template_r),
21-
[Python template](https://github.com/tool-spec/tool_template_python)
22-
and [Octave template](https://github.com/tool-spec/tool_template_octave), but can be mimiced in any container.
16+
At build time, `gotap generate` creates `parameters.js` from `src/tool.yml`. At runtime, `run.js` uses the generated bindings to validate `/in/input.json`, load parameters and data paths, and write structured run logs.
2317

24-
Each container needs at least the following structure:
18+
## Required file structure
2519

26-
```
20+
```text
2721
/
2822
|- in/
29-
| |- tool.json
23+
| |- input.json
3024
|- out/
3125
| |- ...
3226
|- src/
3327
| |- tool.yml
34-
| |- run.py
28+
| |- run.js
29+
| |- parameters.js (generated at build time)
30+
| |- CITATION.cff
3531
```
3632

37-
* `tool.json` are parameters. Whichever framework runs the container, this is how parameters are passed.
38-
* `tool.yml` is the tool specification. It contains metadata about the scope of the tool, the number of endpoints (functions) and their parameters
39-
* `run.js` is the tool itself, or a NodeJS script that handles the execution. It has to capture all outputs and either `print` them to console or create files in `/out`
33+
## Build and run
4034

35+
Build the image from the template root:
4136

42-
## How to build the image?
43-
44-
You can build the image from within the root of this repo by
45-
```
46-
docker build -t tbr_node_tempate .
37+
```bash
38+
docker build -t tbr_node_template .
4739
```
4840

49-
Use any tag you like. If you want to run and manage the container with [toolbox-runner](https://github.com/tool-spec/tool-runner)
50-
they should be prefixed by `tbr_` to be recognized.
51-
52-
Alternatively, the contained `.github/workflows/docker-image.yml` will build the image for you
53-
on new releases on Github. You need to change the target repository in the aforementioned yaml and the repository needs a
54-
[personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
55-
in the repository secrets in order to run properly.
41+
Run the sample tool:
5642

57-
## How to run?
58-
59-
This template installs a few packages to parse the parameters in the `/in/tool.json`. This assumes that
60-
the files are not renamed and not moved and there is actually only one tool in the container. For any other case, the environment variables `PARAM_FILE` can be used to specify a new location for the `tool.json` and `TOOL_RUN` can be used to specify the tool to be executed.
61-
The `run.js` has to take care of that.
62-
63-
To invoke the docker container directly run something similar to:
64-
```
65-
docker run --rm -it -v /path/to/local/in:/in -v /path/to/local/out:/out -e TOOL_RUN=foobar tbr_node_template
43+
```bash
44+
docker run --rm -it \
45+
-v "$(pwd)/in:/in" \
46+
-v "$(pwd)/out:/out" \
47+
-e TOOL_RUN=foobar \
48+
tbr_node_template
6649
```
6750

68-
Then, the output will be in your local out and based on your local input folder. Stdout and Stderr are also connected to the host.
51+
`TOOL_RUN` is only needed when the image contains more than one tool entry. The main runtime interface is still `/in/input.json` plus `gotap run`.
6952

70-
With the toolbox runner, this is simplyfied:
53+
## Customize
7154

72-
```python
73-
from toolbox_runner import list_tools
74-
tools = list_tools() # dict with tool names as keys
75-
76-
foobar = tools.get('foobar') # it has to be present there...
77-
foobar.run(result_path='./', foo_int=1337, foo_string="Please change me")
78-
```
79-
The example above will create a temporary file structure to be mounted into the container and then create a `.tar.gz` on termination of all
80-
inputs, outputs, specifications and some metadata, including the image sha256 used to create the output in the current working directory.
55+
1. Update `src/tool.yml` to describe your tool.
56+
2. Add npm or system dependencies in `Dockerfile`.
57+
3. Implement your tool logic in `src/run.js`.
58+
4. Rebuild the image so `gotap generate` refreshes `parameters.js`.
8159

82-
## What about real tools, no foobar?
60+
## Generated bindings and logging
8361

84-
Yeah.
62+
The generated `parameters.js` file is not edited by hand. It exposes:
8563

86-
1. change the `tool.yml` to describe your actual tool
87-
2. add any `npm install` or `apt-get install` needed to the dockerfile
88-
3. add additional source code to `/src`
89-
4. change the `run.js` to consume parameters and data from `/in` and useful output in `out`
90-
5. build, run, rock!
64+
- `getParameters()`
65+
- `getData()`
66+
- `getRunContext()`
67+
- `getLogger()`
9168

69+
The starter runtime uses `getLogger()` for structured JSON Lines logging while keeping the sample parameter echo on standard output.

src/run.js

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,25 @@
1-
import { getParameters, getData } from "./parameters.js";
1+
import { getData, getLogger, getParameters } from "./parameters.js";
22

33
const toolName = process.env.TOOL_RUN || "foobar";
44

55
const params = getParameters();
66
const data = getData();
7+
const logger = getLogger();
8+
9+
logger.info("start", "Starting tool run", { tool: toolName });
10+
logger.info("input-loaded", "Loaded validated parameters and data paths", {
11+
tool: toolName,
12+
parameter_count: Object.keys(params).length,
13+
data_keys: Object.keys(data).sort(),
14+
});
715

816
if (toolName === "foobar") {
917
console.log("You are running the template directly. Please change the foobar function.");
1018
console.log(params);
1119
console.log(data);
20+
logger.info("finished", "Template run finished successfully", { tool: toolName });
1221
} else {
22+
logger.error("error", "Requested tool is not implemented in the template", { tool: toolName });
1323
console.error(`The toolname ${toolName} is not recognized. Did you forget to implement it?`);
24+
process.exit(1);
1425
}

0 commit comments

Comments
 (0)