Skip to content

Publish cram#201

Open
edmundmiller wants to merge 4 commits intodevfrom
publish-cram
Open

Publish cram#201
edmundmiller wants to merge 4 commits intodevfrom
publish-cram

Conversation

@edmundmiller
Copy link
Copy Markdown
Collaborator

@edmundmiller edmundmiller commented Jun 30, 2025 •
edited
Loading

This implements the cram output recommendation

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/nascent branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@edmundmiller edmundmiller self-assigned this Jun 30, 2025
@nf-core-bot
Copy link
Copy Markdown
Member

nf-core-bot commented Jun 30, 2025

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.2.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 30, 2025 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit fabe147

+| ✅ 217 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   8 tests had warnings |!
Details

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 2.3.0
  • pipeline_todos - TODO string in ro-crate-metadata.json: "description": "

    \n \n <source media="(prefers-color-scheme: dark)" srcset="docs/images/nf-core-nascent_logo_dark.png">\n <img alt="nf-core/nascent" src="docs/images/nf-core-nascent_logo_light.png">\n \n

    \n\nGitHub Actions CI Status\nGitHub Actions Linting StatusAWS CICite with Zenodo\nnf-test\n\nNextflow\nrun with conda\nrun with docker\nrun with singularity\nLaunch on Seqera Platform\n\nGet help on SlackFollow on TwitterFollow on MastodonWatch on YouTube\n\n## Introduction\n\nnf-core/nascent is a bioinformatics pipeline that ...\n\n TODO nf-core:\n Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the\n major pipeline sections and the types of output it produces. You're giving an overview to someone new\n to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction\n\n\n Include a figure that guides the user through the major workflow steps. Many nf-core\n workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples. \n Fill in short bullet-pointed list of the default steps in the pipeline 1. Read QC (FastQC)2. Present QC for raw reads (MultiQC)\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.\n\n Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.\n Explain what rows and columns represent. For instance (please edit as appropriate):\n\nFirst, prepare a samplesheet with your input data that looks as follows:\n\nsamplesheet.csv:\n\ncsv\nsample,fastq_1,fastq_2\nCONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz\n\n\nEach row represents a fastq file (single-end) or a pair of fastq files (paired end).\n\n\n\nNow, you can run the pipeline using:\n\n update the following command to include all required parameters for a minimal example \n\nbash\nnextflow run nf-core/nascent \\\n -profile <docker/singularity/.../institute> \\\n --input samplesheet.csv \\\n --outdir <OUTDIR>\n\n\n> [!WARNING]\n> Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.\n\nFor more details and further functionality, please refer to the usage documentation and the parameter documentation.\n\n## Pipeline output\n\nTo see the results of an example test run with a full size dataset refer to the results tab on the nf-core website pipeline page.\nFor more details about the output files and reports, please refer to the\noutput documentation.\n\n## Credits\n\nnf-core/nascent was originally written by Edmund Miller, Ignacio Tripodi, Margaret Gruca.\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n If applicable, make list of people who have also contributed \n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the contributing guidelines.\n\nFor further information or help, don't hesitate to get in touch on the Slack #nascent channel (you can join with this invite).\n\n## Citations\n\n Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. \n If you use nf-core/nascent for your analysis, please cite it using the following doi: 10.5281/zenodo.XXXXXX \n\n Add bibliography of tools and data used in your pipeline \n\nAn extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.\n\nYou can cite the nf-core publication as follows:\n\n> The nf-core framework for community-curated bioinformatics pipelines.\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.\n",
  • pipeline_todos - TODO string in README.md: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • local_component_structure - gtf2bed.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bed2saf.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - prepare_genome.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - quality_control.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/nf-core-nascent_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-nascent_logo_light.png
  • actions_ci - actions_ci

✅ Tests passed:

Run details

  • nf-core/tools version 3.2.1
  • Run at 2025-07-07 03:48:39

@edmundmiller edmundmiller force-pushed the publish-cram branch 7 times, most recently from c55231d to 9f6e72a Compare July 7, 2025 01:04
@edmundmiller edmundmiller force-pushed the publish-cram branch 3 times, most recently from 390405c to a1a05b2 Compare July 7, 2025 03:24
@edmundmiller edmundmiller added this to the 2.4.0 milestone Jul 7, 2025
@edmundmiller edmundmiller marked this pull request as ready for review July 7, 2025 13:36
@edmundmiller edmundmiller requested a review from Copilot July 7, 2025 13:36

This comment was marked as outdated.

Sign in to view

@edmundmiller edmundmiller requested a review from Copilot August 4, 2025 02:23
Copilot AI reviewed Aug 4, 2025
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements CRAM output format support as recommended by nf-core guidelines, changing the default output from BAM to CRAM files for improved storage efficiency while adding a --bam parameter for users who need BAM output.

  • Adds SAMTOOLS_CONVERT module to convert BAM files to CRAM format by default
  • Introduces --bam parameter to maintain BAM output when needed
  • Updates all test files to expect CRAM files and validates the new publishing logic

Reviewed Changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
workflows/nascent.nf Integrates SAMTOOLS_CONVERT module with conditional logic based on --bam parameter
modules/nf-core/samtools/convert/* New module for BAM to CRAM conversion with tests and documentation
conf/modules.config Updates publishing configuration to handle both BAM and CRAM outputs conditionally
workflows/tests/**/.nf.test Updates test files to expect CRAM files and adds new tests for both formats
nextflow_schema.json Adds --bam parameter schema definition
docs/usage.md Documents the new CRAM/BAM output options
Comments suppressed due to low confidence (1)

workflows/tests/transcript_indentification/grohmm/tuning/main.nf.test:21

  • Variable name 'bam_files' is misleading since it's collecting CRAM files. Should be renamed to 'cram_files' to match the comment and actual file type being collected.
            def bam_files = getAllFilesFromDir(params.outdir, include: ['**/*.cram'])

Comment thread workflows/tests/inputs/only_gff/main.nf.test
params {
outdir = "$outputDir"
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff'
fasta = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/GRCh38_chr21.fa'
Copy link

Copilot AI Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Adding fasta parameter appears unrelated to the main purpose of this PR (CRAM publishing). This change should be documented or separated into a different commit if it's addressing a separate issue.

Suggested change
fasta = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/GRCh38_chr21.fa'

Copilot uses AI. Check for mistakes.
Comment thread workflows/tests/inputs/gzipped_gff/main.nf.test
@@ -11,6 +11,7 @@ nextflow_pipeline {
params {
outdir = "$outputDir"
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff.gz'
Copy link

Copilot AI Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Adding fasta parameter appears unrelated to the main purpose of this PR (CRAM publishing). This change should be documented or separated into a different commit if it's addressing a separate issue.

Suggested change
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff.gz'
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff.gz'
// The fasta parameter is required by the pipeline for this test to run successfully, even though the main focus is CRAM publishing.

Copilot uses AI. Check for mistakes.
Comment thread workflows/tests/inputs/gff/main.nf.test
@@ -11,6 +11,7 @@ nextflow_pipeline {
params {
outdir = "$outputDir"
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff'
Copy link

Copilot AI Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Adding fasta parameter appears unrelated to the main purpose of this PR (CRAM publishing). This change should be documented or separated into a different commit if it's addressing a separate issue.

Suggested change
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff'
gff = 'https://raw.githubusercontent.com/nf-core/test-datasets/nascent/reference/genes_chr21.gff'
// The fasta parameter is required for the pipeline to process the GFF input and generate CRAM files.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@FriederikeHanssen FriederikeHanssen Awaiting requested review from FriederikeHanssen

At least 1 approving review is required to merge this pull request.

Assignees

@edmundmiller edmundmiller

Labels

None yet

Projects

None yet

Milestone

2.4.0

Development

Successfully merging this pull request may close these issues.

3 participants

@edmundmiller @nf-core-bot