Add lineage v1beta1 schema#18
Open
jorgee wants to merge 3 commits into
Open
Conversation
Add the JSON Schema for the lineage data model wire format emitted by nf-lineage (LinEncoder), plus tests covering valid documents for each LinSerializable subtype and invalid cases (missing envelope, unknown kind, unsupported version). Wire lineage/v1beta1 into validate.sh and update README.md to list the lineage and module schemas. Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Member
|
Any chance to make it the schema a bit more readable (maybe just asking claude to reformat it)? |
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Improves readability of the lineage schema by reordering each $defs entry so title and description appear before structural keywords. Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Author
I have fixed some issues with arrays split into several lines and put the title and description at the beginning of the definitions. I also tried to compress the |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nf-lineage(LinEncoder) atlineage/v1beta1/schema.json.oneOfover the sixLinSerializablesubtypes (FileOutput,TaskOutput,TaskRun,Workflow,WorkflowOutput,WorkflowRun), each wrapped in the runtime envelope{version, kind, spec}withversionandkindasconstdiscriminators.Checksum,DataPath,Parameter,Workflow) deduplicated via$defs/$ref;OffsetDateTime/Pathmodelled asstring(withdate-timeformat where applicable); modelwithSerializeNulls(true)accommodated via["X", "null"]typing.Tests
lineage/v1beta1/tests/:valid_file_output.json,valid_task_output.json,valid_task_run.json,valid_workflow_run.json. The first three are real.data.jsondocuments from annf-core/rnaseqrun; the workflow_run is trimmed for readability.invalid_missing_envelope.json(bare spec body at root),invalid_wrong_kind.json(unknownkind),invalid_wrong_version.json(unsupportedversion).Wired
lineage/v1beta1intovalidate.sh; updatedREADME.mdto list thelineageandmoduleschemas.Test plan
./validate.shpasses locally — confirmed: all 7 lineage cases behave as expected (4 valid pass, 3 invalid rejected); rest of repo unaffected..data.jsonfiles from a fullnf-core/rnaseqlineage run — all pass.Source of truth
The schema is generated by a Gradle task in
nextflow-io/nextflow(./gradlew :nf-lineage:generateLineageSchema). When the lineage model changes, regenerate there and re-sync this repo.