clip_vg

[jwli-code]

        [2026-04-11T10:51:06+0800] [MainThread] [I] [toil-rt] 2026-04-11 10:51:06.355821: Successfully ran: "bash -c 'set -eo pipefail && clip-vg /share/org/YZWL/yzwl_chenmj/jwli/cactus_pangenome/Bnapan/wk/toilwf-e4b6fe8f995858768c5253a5bdc796d7/0b56/job/tmp70nxeb37/ZS11_A06.vg.gfaffixed -f -e ZS11T2T -d _MINIGRAPH_ -L | vg mod -X 1024 - | vg paths -x - -n -Q ZS11T2T -t 1 | vg clip -d 1 - -P ZS11T2T | vg ids -s -'" in 1602.4175 seconds and 34.9 Gi memory with job-memory 202.6 Gi. Percent utilization: 17.22
        [2026-04-11T10:51:06+0800] [MainThread] [I] [toil-rt] 2026-04-11 10:51:06.356057: Running the command: "vg validate /share/org/YZWL/yzwl_chenmj/jwli/cactus_pangenome/Bnapan/wk/toilwf-e4b6fe8f995858768c5253a5bdc796d7/0b56/job/tmp70nxeb37/ZS11_A06.vg.clip"
        [2026-04-11T10:57:06+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files:
        [2026-04-11T10:57:06+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-export_vg/instance-evmsiz4h/file-89888e62a3904e10a7921b8aad07f95d/out.vg' to path '/share/org/YZWL/yzwl_chenmj/jwli/cactus_pangenome/Bnapan/wk/toilwf-e4b6fe8f995858768c5253a5bdc796d7/0b56/job/tmp70nxeb37/ZS11_A06.vg'
        [2026-04-11T10:57:06+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] LOG-TO-LEADER: Job 'clip_vg' kind-clip_vg/instance-p_dbucqz v4 used 1.01x more CPU than the requested 1 cores. Consider increasing the job's required CPU cores or limiting the number of processes/threads launched.
        [2026-04-11T10:57:06+0800] [MainThread] [C] [toil.worker] Worker crashed with traceback:
        Traceback (most recent call last):
          File "/share/org/YZWL/yzwl_chenmj/micromamba/envs/cactus314/lib/python3.9/site-packages/toil/worker.py", line 595, in workerScript
            job._runner(
          File "/share/org/YZWL/yzwl_chenmj/micromamba/envs/cactus314/lib/python3.9/site-packages/toil/job.py", line 3254, in _runner
            returnValues = self._run(jobGraph=None, fileStore=fileStore)
          File "/share/org/YZWL/yzwl_chenmj/micromamba/envs/cactus314/lib/python3.9/site-packages/toil/job.py", line 3137, in _run
            return self.run(fileStore)
          File "/share/org/YZWL/yzwl_chenmj/micromamba/envs/cactus314/lib/python3.9/site-packages/toil/job.py", line 3452, in run
            rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs)
          File "/share/org/YZWL/yzwl_chenmj/micromamba/envs/cactus314/lib/python3.9/site-packages/cactus/refmap/cactus_graphmap_join.py", line 829, in clip_vg
            cactus_call(parameters=['vg', 'validate', clipped_path])
          File "/share/org/YZWL/yzwl_chenmj/micromamba/envs/cactus314/lib/python3.9/site-packages/cactus/shared/common.py", line 916, in cactus_call
            raise RuntimeError("{}Command {} exited {}: {}".format(sigill_msg, call, process.returncode, out))
        RuntimeError: Command /usr/bin/time -f "CACTUS-LOGGED-MEMORY-IN-KB: %M" vg validate /share/org/YZWL/yzwl_chenmj/jwli/cactus_pangenome/Bnapan/wk/toilwf-e4b6fe8f995858768c5253a5bdc796d7/0b56/job/tmp70nxeb37/ZS11_A06.vg.clip exited 1: stderr=graph invalid: missing edge between 2291th step (1935312:0) and 2292th step (1935312:1) of path Express617#0#R21641#0
        graph invalid: missing edge between 2292th step (1935312:0) and 2291th step (1935312:0) of path Express617#0#R21641#0
        graph: invalid
        Command exited with non-zero status 1
        CACTUS-LOGGED-MEMORY-IN-KB: 17611236


        [2026-04-11T10:57:06+0800] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host c02n0067
<=========
Job 70466526 stderr output:

``` I encountered this problem. May I ask how it should be resolved? Because I have a lot of genomes, I would like to ask how to solve this without having to run from scratch

[glennhickey]

To work around this problem. Use cactus-graphmap-join to rerun the final stage of the workflow, passing the input using --vg <your output dir>/chrom-alignments/*.vg and use --configFile to set gfaffix="0".

To resolve the problem, I think you will need to share your input graph chrom-alignments/ZS11_A06.vg so I can reproduce and share with the gfaffix developers.

[jwli-code]

Thank you for your reply. Will setting the parameter gfaffix="0" affect the results of the generated SV.vcf, or impact the use of genotyping tools like giraffe? My files are somewhat large, and I am trying to rerun a test using a smaller number of genomes to send you the results.

[jwli-code]

Hi,

I am seeking help to apply a configuration change to an ongoing but failed cactus-pangenome run.

Background:
My run failed on April 11th due to a vg validate error in the clip_vg step (log attached). To bypass this, I disabled gfaffix by setting gfaffix="0" in my XML config file.

The Problem:
On April 15th, I attempted to resume the run with the updated config using:
cactus-pangenome ... --configFile ${PWD}/config-nofix.xml --restart

However, the change didn't seem to take effect. The logs from April 15th show the exact same error as April 11th, suggesting the pipeline is still using the original config stored in the jobStore.

Request for Remedy:
This workflow has been running for a very long time, and I am desperate to avoid a complete restart from scratch.

Is there a way to force a restart to recognize the updated configFile?

Alternatively, is there any manual "remedy" or hack (e.g., using toil commands to clear specific job states) that would allow the pipeline to skip the problematic step or use the new config?

I have attached the logs from both dates for comparison. Any advice on how to salvage this run would be greatly appreciated.

Thank you!

[jwli-code]

cactus_pangenome.err.zip

[jwli-code]

To work around this problem. Use cactus-graphmap-join to rerun the final stage of the workflow, passing the input using --vg <your output dir>/chrom-alignments/*.vg and use --configFile to set gfaffix="0".

To resolve the problem, I think you will need to share your input graph chrom-alignments/ZS11_A06.vg so I can reproduce and share with the gfaffix developers.

ok，I have confirmed that setting gfaffix="0" allows the pipeline to bypass the current error and proceed. However, I suspect that my previous attempt to use --configFile with --restart failed because Toil's jobStore cached the old configuration.

Since the genome Express617 is not critical to my downstream analysis, I am considering removing it (or the problematic contig within it) to allow the rest of the workflow to run with gfaffix enabled normally.

I have the following questions:

Is it possible to manually edit or "prune" a specific genome or contig from the already generated .hal and .vg files without breaking the internal consistency of the jobStore?

[glennhickey]

You can only change toil-specifc options when running --restart. anything else you change will have no effect.

You should be able to safely remove a genome from hal with halRemoveGenome.

To remove from a .vg file, you can use

vg paths -x graph.vg -Q Express617 -d | vg clip -d 1 - -P <REFERENCE>

[jwli-code]

Hi there,

I'm encountering an issue with the HAL and VG files generated for A06. I've uploaded the problematic files here: [🔗 paste download link]

Could you please take a look and advise on how to resolve or work around this error? I've also included the relevant error log and environment details below. Thanks in advance for your help! https://drive.google.com/file/d/1KpMEsvMz9RFr-v--RNg7r859iwt8sarO/view?usp=sharing