Gemma Collins
04/29/2024, 12:58 AMnextflow run ausarg/pipesnake -profile test,singularity --outdir /path/to/output
ERROR ~ Error executing process > 'AUSARG_PIPESNAKEPIPESNAKEIQTREE (batch of 4120 fasta files)'
Caused by:
Process AUSARG_PIPESNAKE:PIPESNAKE:IQTREE (batch of 4120 fasta files)
terminated with an error exit status (2)Arthur
04/29/2024, 12:54 PMinput:
tuple val(meta), path(vcf), path(tbi), path(vcf2), path(tbi2), path(vcf3), path(tbi3)
output:
tuple val(meta), path(output_file)
script:
def path1arg = vcf ? '--arg1 ${vcf}' : ''
def path2arg = vcf2 ? '--arg2 ${vcf2}' : ''
def path3arg = vcf3 ? '--arg3 ${vcf3}' : ''
"""
my-tool \\
${path1arg} \\
${path2arg} \\
${path3arg}
"""
At what point is there too many args in a single input channel?Anabella Trigila
04/29/2024, 2:26 PMnextflow run nf-core/phaseimpute -profile test,singularity --outdir test --tools glimpse1,stitch -r dev
ERROR ~ ERROR: Validation of pipeline parameters failed!
-- Check '.nextflow.log' file for details
The following invalid input values have been detected:
* --tools: 'glimpse1,stitch' is not a valid choice (Available choices: glimpse1, glimpse2, quilt, stitch)
We are using nf-validation
to validate params, but I am not sure how to make the available choices not exclusive.
Thank you!Riley Grindle
04/29/2024, 6:05 PM[6e/f772ad] process > NFCORE_ORTHOLOGYMAP:ORTHOLOGYMAP:POST_PROC (Processing ortholog... [100%] 1 of 1, cached: 1 ✔
[- ] process > NFCORE_ORTHOLOGYMAP:ORTHOLOGYMAP:CONSENSUS_VOTE:TBL_2_JSON_1V1 -
[- ] process > NFCORE_ORTHOLOGYMAP:ORTHOLOGYMAP:CONSENSUS_VOTE:TBL_2_JSON_1VM -
[- ] process > NFCORE_ORTHOLOGYMAP:ORTHOLOGYMAP:CUSTOM_DUMPSOFTWAREVERSIONS -
The subworkflow CONSENSUS_VOTE
is listed as a portion of the workflow but is never actually initiated. The workflow block that highlights this disconnect looks much like any other modular connection within the pipeline.
POST_PROC(
ch_out_dir,
PREP_INPUT.out.ortho_l_data,
PREP_INPUT.out.egg.first(),
params.sup_ensembl_data,
params.taxa_db
)
ch_vote = CONSENSUS_VOTE(
POST_PROC.out.ortho_l,
POST_PROC.out.ortho_f,
POST_PROC.out.eggnog.first(),
POST_PROC.out.tree.first()
)
For context, POST_PROC
is a module that outputs a series of files, and CONSENSUS_VOTE
is a subworkflow that performs a set of file manipulations on said files. It's important to note that no error message is displayed and the pipeline is deemed as successful. Any help with what may be causing this would be greatly appreciated!Ramiro Barrantes Reynolds
04/30/2024, 5:41 PMSAMPLE SIGNAL BACKGROUND
replicate1 signal1.fastq.gz background1.fastq.gz
replicate2 signal2.fastq.gz background2.fastq.gz
Now I would need to somehow run a custom program, for example
process NORMALIZE {
...
script:
"""
customProgram.pl \
signalN.sorted.bam \
backgroundN.sorted.bam \
signalN.bed \
signalN.readnum.txt \
backgroundN.readnum.txt \
output.bed
"""
}
I am just not clear what is the best way to process the beds, bams and txt in pairs according to the samplesheet. I was thinking of something like this:
Channel.fromPath( file(params.sample_sheet) )
.splitCsv(header: true, sep: ',')
.map{row ->
def signal = row.SIGNAL
def background = row.BACKGROUND
return [ signal, background ]
}
.set{ samples_pairs }
workflow {
...
PROCESS1(...).set{ch_bam}
PROCESS2(...).set{ch_bed}
PROCESS3(...).set{ch_txt}
NORMALIZE(samples_pairs, ch_bam, ch_bed, ch_txt)
}
I am just not clear how to go through samples_pairs appropriately so as to run normalize on each pairs?Tony Qu
04/30/2024, 7:26 PMWARNING: Could not load nf-core/config profiles: <https://raw.githubusercontent.com/nf-core/configs/master/nfcore_custom.config>
has anyone ran nfcore on a HPC before? If so have you countered issues like this? Thanks so much!Ramiro Barrantes Reynolds
05/01/2024, 12:58 PMworkflow {
...
PROCESS1(...).set{ch_bam}
PROCESS2(...).set{ch_bed}
PROCESS3(...).set{ch_txt}
...
}
I also have a sample sheet which groups some of these files into signal and background and which I put into a Channel factory (thanks @Simon Pearce and @Riley Grindle:)
id indiv type fastq
replicate1_signal rep1 signal signal1.fastq.gz
replicate2_signal rep2 signal signal2.fastq.gz
replicate1_signal rep1 background background1.fastq.gz
replicate2_signal rep2 background background2.fastq.gz
Channel.fromSamplesheet(params.samplesheet)
I have a custom program that would take signal and backgrounds in pairs according to the sample sheet, for example:
customProgram.pl signal1.sorted.bam background1.sorted.bam \
signal1.bed signal1.readnum.txt \
background1.readnum.txt \
output.bed
I just don't know how to specify using the samplesheet to process files in this way in nextflow (especially as the processes might finish in random orders so I likely need to wait for all or some to finish, not sure), I know that this might be a very common use case but I have never done it before and don't see a good example online. I understand that I might need to do a "filter/branch by type, and join by indiv", what does that mean? Any examples anywhere of something like this?arushi shrivastava
05/01/2024, 3:39 PMhengbin gao'
05/02/2024, 5:16 AMNINO PARAMBALOTH LEENUS
05/02/2024, 6:04 AMNINO PARAMBALOTH LEENUS
05/02/2024, 6:11 AMKarla Ruiz
05/02/2024, 9:18 AMERROR ~ Error executing process > 'NFCORE_SMRNASEQ:MIRNA_QUANT:MIRTOP_QUANT'
Caused by:
Failed to create Conda environment
command: conda create --mkdir --yes --quiet --prefix /net/192.168.120.240/home/tigem/k.ruiz/work/conda/env-f6207f68e2d2d68777c6463d18984735 mirtop=0.4.25 bioconda::samtools=1.15.1 conda-base::r-base=4.1.1 conda-base::r-data.table=1.14.2
status : 1
message:
Channels:
- defaults
- bioconda
- conda-base
Platform: linux-64
Collecting package metadata (repodata.json): ...working... failed
UnavailableInvalidChannel: HTTP 404 NOT FOUND for channel conda-base <<https://conda.anaconda.org/conda-base>>
The channel is not accessible or is invalid.
You will need to adjust your conda configuration to proceed.
Use `conda config --show channels` to view your configuration's current state,
and use `conda config --show-sources` to view config file locations.
The command I used was:
nextflow run nf-core/smrnaseq -r 2.3.1 \
-profile conda \
--input 'small_PB.csv' \
--fasta Mus_musculus.GRCm39.dna.toplevel.fa \
--mirtrace_species 'mmu' \
--mirna_gtf mmu.gff3 \
--hairpin hairpin.fa \
--mature mature.fa \
--protocol 'illumina' \
--outdir v23 \
-resume
Is there any suggestion on how to solve this?
I've already tried the following commands but it still appears the same error:
conda clean -i
conda config --remove-key channels
conda config --append channels conda-forge --append channels bioconda --append channels defaults
Thanks!Annick Renevey
05/02/2024, 9:24 AMSvenja Schorlemmer
05/03/2024, 8:21 AMLili Andersson-Li
05/03/2024, 8:32 AMFASTQ
files from being published in the workflow? I’m considering writing a local module to handle this, but I suspect there might be a simpler solution. The module nf-core/krakentools/extractkrakenreads/ is used in this block.
if ( params.extract_kraken2_reads ) {
if ( params.taxid ) {
KRAKENTOOLS_EXTRACTKRAKENREADS(
params.taxid,
ch_input_kraken2.kraken2_result,
ch_input_kraken2.reads,
ch_input_kraken2.kraken2_report
)
ch_versions = ch_versions.mix( KRAKENTOOLS_EXTRACTKRAKENREADS.out.versions )
}
Emilio Garcia
05/03/2024, 8:56 AM'-n 10 -d 0 -c 0.99 -aL 0.9'
Error:
# comparing sequences from 2590343 to 2590359
................---------- new table with 16 representatives
# comparing sequences from 2590359 to 2590374
...............---------- new table with 15 representatives
# comparing sequences from 2590374 to 2590389
100.0%---------- new table with 15 representatives
# comparing sequences from 2590389 to 2590404
..............---------- new table with 14 representatives
# comparing sequences from 2590404 to 2590419
100.0%---------- new table with 15 representatives
# comparing sequences from 2590419 to 2591414
.....................double free or corruption (!prev)
work/28/77bbda806ab22a8cb5d870dbf9c084/.command.sh: line 7: 42 Aborted (core dumped) cd-hit-est -n 10 -d 0 -c 0.99 -aL 0.9 -i vmx.contigs.catalogue.fna.gz -o vmx.fa -M 204800 -T 64
Saeideh Ashouri
05/05/2024, 3:38 PMMirela Balan
05/06/2024, 5:23 AM#!/bin/bash
#PBS -l select=1:ncpus=2:mem=10gb
#PBS -l walltime=23:59:59
#PBS -A "CUBISB"
#PBS -N "scRNAseq"
#---------------------------
export NXF_OPTS="-Xms1G -Xmx10G"
export NXF_OFFLINE="true"
export NXF_SINGULARITY_CACHEDIR="/my_path/resources/nxf_cache/"
export NXF_DISABLE_CHECK_LATEST="true"
nextflow run \
${workdir}"/pipelines/nf-core-scrnaseq_dev/dev" \
-profile singularity,hilbert \
-c ${workdir}"/scripts/nxf_scripts/config_nxf/run.config" \
-params-file ${workdir}"/scripts/nxf_scripts/config_nxf/nf_scRNA-params.json" \
-resume -offline
I have run in this way several nxf pipelines and I never had an issue.
I get this error:
[7f/8c8b05] process > NFCORE_SCRNASEQ:SCRNASEQ:FA... [100%] 8 of 8, cached: 8 ✔
[29/0770a2] process > NFCORE_SCRNASEQ:SCRNASEQ:GT... [100%] 1 of 1, cached: 1 ✔
[8b/8f0e16] process > NFCORE_SCRNASEQ:SCRNASEQ:CE... [100%] 1 of 1, cached: 1 ✔
[6a/0f5e02] process > NFCORE_SCRNASEQ:SCRNASEQ:CE... [100%] 1 of 1, cached: 1 ✔
[- ] process > NFCORE_SCRNASEQ:SCRNASEQ:CE... [ 0%] 0 of 8
[- ] process > NFCORE_SCRNASEQ:SCRNASEQ:EM... -
[- ] process > NFCORE_SCRNASEQ:SCRNASEQ:MT... -
[- ] process > NFCORE_SCRNASEQ:SCRNASEQ:MT... -
[- ] process > NFCORE_SCRNASEQ:SCRNASEQ:MT... -
[- ] process > NFCORE_SCRNASEQ:SCRNASEQ:MU... -
ERROR ~ Error executing process > 'NFCORE_SCRNASEQ:SCRNASEQ:CELLRANGER_ALIGN:CELLRANGER_COUNT (983-2)'
Caused by:
Failed to submit process to grid scheduler for execution
Command executed [/my_path/pipelines/nf-core-scrnaseq_dev/dev/./workflows/../subworkflows/local/../../modules/nf- core/cellranger/count/templates/cellranger_count.py]:
qsub -N nf-NFCORE_SCRNA .command.run
Command exit status:
191
Command output:
qsub: Job rejected by all possible destinations
Work dir:
/my_path/pipelines/runs_nxf/work/1f/3ef0222a35d48457509364d985d802
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
-- Check '.nextflow.log' file for details
If I go to my working folder and I run bash .command.run
then that particular command finishes running.
After googling it, I found that it could be memory-related, so I have decreased the parameters in the header, but the result is always the same.
Could you point me in the right direction?Joon Klaps
05/06/2024, 12:20 PMERROR ~ No such file or directory: <https://viralzone.expasy.org/resources/Virosaurus/2020_4/virosaurus90_vertebrate-20200330.fas.gz>
But I only encounter it in github actions whereas when run locally or in gitpod I’ve never seen it. I’m trying to figure out if nextflow is expecting a local file or if it’s because the hosting website is giving github a time-out. How should I solve this? Thanks in advance!Arthur
05/07/2024, 11:20 AMparams {
ref_a = null
ref_a_idx = null
ref_b = null
ref_b_idx = null
...
ref_y = null
ref_y_idx = null
ref_z = null
ref_z_idx = null
}
When each index is required to be something determinable directly from the reference file e.g. <filename>.tbi
because it creates a great many extra params for the user.
I lean towards having some function e.g. gather_index(file, idx_ext)
which would simply get and check the existence of an index file with the required extension, but is there a reason not to go this route?Bulut Hamali
05/07/2024, 2:51 PMNisha Boora
05/08/2024, 7:50 AMnf-core create
template and nf-core modules, but I'm getting the following error:Oskar Wacker
05/08/2024, 9:13 AMJSONDecodeError
in the nf-core linting of my PR: https://github.com/nf-core/differentialabundance/actions/runs/8999075722/job/24720863161?pr=256
I had this error previously in another PR but it vanished when I fixed some missing square brackets in the Changelog (no idea how those would break the linting though). Does someone know what causes this error and how to solve it?lilin
05/08/2024, 10:44 AMArnelyn Larano
05/08/2024, 11:25 AMmvforster
05/08/2024, 2:03 PMRamiro Barrantes Reynolds
05/08/2024, 3:06 PMRyan Neilson
05/08/2024, 3:30 PMAMMAR SABIR
05/08/2024, 8:07 PMMarie Wong
05/09/2024, 1:41 AM