Otdfbt

Is there a term for someone whose preferred policies are a mix of Left and Right?

Do Veracrypt encrypted volumes have any kind of brute force protection?

Must a CPU have a GPU if the motherboard provides a display port (when there isn't any separate video card)?

Why is it bad to use your whole foot in rock climbing

Optimising matrix generation time

Placement of positioning lights on A320 winglets

Approach sick days in feedback meeting

Harley Davidson clattering noise from engine, backfire and failure to start

Parallelized for loop in Bash

Background for black and white chart

Purpose of cylindrical attachments on Power Transmission towers

Print the phrase "And she said, 'But that's his.'" using only the alphabet

Someone who is granted access to information but not expected to read it

Why can't we feel the Earth's revolution?

Is it ethical to cite a reviewer's papers even if they are rather irrelevant?

Why does this Apple //e drops into system monitor when booting?

How effective would a full set of plate armor be against wild animals found in temperate regions (bears, snakes, wolves)?

Should I worry about having my credit pulled multiple times while car shopping?

Does WiFi affect the quality of images downloaded from the internet?

Opposite of "Concerto Grosso"?

Shouldn't it take more energy to break CO2 compared to CO?

Fastest way from 10 to 1 with everyone in between

Is there a term for when fiction refers to fiction

Is fission/fusion to iron the most efficient way to convert mass to energy?

Autodetect max number of cores and pass as an argument in Nextflow

What is the difference between a Bioinformatics pipeline and workflow?

3

$begingroup$

I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one.

Is there a way in Nextflow to pass the maximum number of available threads as an argument? At the moment the best I can come up with is defining an arbitrarily large number of threads as an argument which will cap out and run with the max number of threads. I would like to know, is there a 'cleaner' or more canonical way of doing this?

At the moment my process looks like this:

process roary 

 publishDir "$params.outdir/roary", mode: 'copy'

 cpus 200

 input:
 file gff from gff.collect()

 output:
 file("*") into roary
 file("pan_genome_reference.fa") into pan_genome
 file("pan_genome_sequences/*") into alignment_files
 file("gene_presence_absence.Rtab") into gene_presence
 set file("*accessory*"),
 file("*.Rtab"),
 file("_*"),
 file("*.txt"),
 file("*csv") into roary_out


 script:
 """
 roary -p $task.cpus -e -n -v -z $gff
 """

Thanks in advance. Also, apologies as I can see why this might not strictly be bioinformatics, but it's where Nextflow has the most traction.

workflow-management roary nextflow pangenome

share|improve this question

edited May 29 at 13:06

Daniel Standage

3,083732

asked May 29 at 10:55

TW93TW93

1748

$endgroup$

• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  $begingroup$
  Workflow management has a firm foothold in bioinformatics and is in my opinion directly on topic on this site.
  $endgroup$
  – Daniel Standage
  May 29 at 13:01
  

  
  
  
  

add a comment | 

3

$begingroup$

I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one.

Is there a way in Nextflow to pass the maximum number of available threads as an argument? At the moment the best I can come up with is defining an arbitrarily large number of threads as an argument which will cap out and run with the max number of threads. I would like to know, is there a 'cleaner' or more canonical way of doing this?

At the moment my process looks like this:

process roary 

 publishDir "$params.outdir/roary", mode: 'copy'

 cpus 200

 input:
 file gff from gff.collect()

 output:
 file("*") into roary
 file("pan_genome_reference.fa") into pan_genome
 file("pan_genome_sequences/*") into alignment_files
 file("gene_presence_absence.Rtab") into gene_presence
 set file("*accessory*"),
 file("*.Rtab"),
 file("_*"),
 file("*.txt"),
 file("*csv") into roary_out


 script:
 """
 roary -p $task.cpus -e -n -v -z $gff
 """

Thanks in advance. Also, apologies as I can see why this might not strictly be bioinformatics, but it's where Nextflow has the most traction.

workflow-management roary nextflow pangenome

share|improve this question

edited May 29 at 13:06

Daniel Standage

3,083732

asked May 29 at 10:55

TW93TW93

1748

$endgroup$

• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  $begingroup$
  Workflow management has a firm foothold in bioinformatics and is in my opinion directly on topic on this site.
  $endgroup$
  – Daniel Standage
  May 29 at 13:01
  

  
  
  
  

add a comment | 

$begingroup$

I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one.

Is there a way in Nextflow to pass the maximum number of available threads as an argument? At the moment the best I can come up with is defining an arbitrarily large number of threads as an argument which will cap out and run with the max number of threads. I would like to know, is there a 'cleaner' or more canonical way of doing this?

At the moment my process looks like this:

process roary 

 publishDir "$params.outdir/roary", mode: 'copy'

 cpus 200

 input:
 file gff from gff.collect()

 output:
 file("*") into roary
 file("pan_genome_reference.fa") into pan_genome
 file("pan_genome_sequences/*") into alignment_files
 file("gene_presence_absence.Rtab") into gene_presence
 set file("*accessory*"),
 file("*.Rtab"),
 file("_*"),
 file("*.txt"),
 file("*csv") into roary_out


 script:
 """
 roary -p $task.cpus -e -n -v -z $gff
 """

Thanks in advance. Also, apologies as I can see why this might not strictly be bioinformatics, but it's where Nextflow has the most traction.

workflow-management roary nextflow pangenome

share|improve this question

edited May 29 at 13:06

Daniel Standage

3,083732

asked May 29 at 10:55

TW93TW93

1748

$endgroup$

process roary 

 publishDir "$params.outdir/roary", mode: 'copy'

 cpus 200

 input:
 file gff from gff.collect()

 output:
 file("*") into roary
 file("pan_genome_reference.fa") into pan_genome
 file("pan_genome_sequences/*") into alignment_files
 file("gene_presence_absence.Rtab") into gene_presence
 set file("*accessory*"),
 file("*.Rtab"),
 file("_*"),
 file("*.txt"),
 file("*csv") into roary_out


 script:
 """
 roary -p $task.cpus -e -n -v -z $gff
 """

share|improve this question

edited May 29 at 13:06

Daniel Standage

3,083732

asked May 29 at 10:55

TW93TW93

1748

share|improve this question

edited May 29 at 13:06

Daniel Standage

3,083732

asked May 29 at 10:55

TW93TW93

1748

edited May 29 at 13:06

Daniel Standage

3,083732

edited May 29 at 13:06

Daniel Standage

3,083732

asked May 29 at 10:55

TW93TW93

1748

asked May 29 at 10:55

TW93TW93

1748

1 Answer
1

active

oldest

votes

4

$begingroup$

I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.

However, you can try to automatize this task by using the nproc command.

nproc gives you the number of processing units available. An idea would be to include something such nproc - 1 as the max number of threads for Roary.

share|improve this answer

edited May 29 at 13:57

terdon♦

5,2412931

answered May 29 at 12:20

platplat

647211

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks, as far I could see there are ways to dynamically allocate resources (nextflow.io/docs/latest/…) but not detect threads, so this is a nice workaround.
  $endgroup$
  – TW93
  May 29 at 14:46
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "676"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fbioinformatics.stackexchange.com%2fquestions%2f8712%2fautodetect-max-number-of-cores-and-pass-as-an-argument-in-nextflow%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

4

$begingroup$

I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.

However, you can try to automatize this task by using the nproc command.

nproc gives you the number of processing units available. An idea would be to include something such nproc - 1 as the max number of threads for Roary.

share|improve this answer

edited May 29 at 13:57

terdon♦

5,2412931

answered May 29 at 12:20

platplat

647211

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks, as far I could see there are ways to dynamically allocate resources (nextflow.io/docs/latest/…) but not detect threads, so this is a nice workaround.
  $endgroup$
  – TW93
  May 29 at 14:46
  

  
  
  
  

add a comment | 

4

$begingroup$

I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.

However, you can try to automatize this task by using the nproc command.

nproc gives you the number of processing units available. An idea would be to include something such nproc - 1 as the max number of threads for Roary.

share|improve this answer

edited May 29 at 13:57

terdon♦

5,2412931

answered May 29 at 12:20

platplat

647211

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks, as far I could see there are ways to dynamically allocate resources (nextflow.io/docs/latest/…) but not detect threads, so this is a nice workaround.
  $endgroup$
  – TW93
  May 29 at 14:46
  

  
  
  
  

add a comment | 

$begingroup$

I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.

However, you can try to automatize this task by using the nproc command.

nproc gives you the number of processing units available. An idea would be to include something such nproc - 1 as the max number of threads for Roary.

share|improve this answer

edited May 29 at 13:57

terdon♦

5,2412931

answered May 29 at 12:20

platplat

647211

$endgroup$

share|improve this answer

edited May 29 at 13:57

terdon♦

5,2412931

answered May 29 at 12:20

platplat

647211

edited May 29 at 13:57

terdon♦

5,2412931

edited May 29 at 13:57

terdon♦

5,2412931

answered May 29 at 12:20

platplat

647211

answered May 29 at 12:20

platplat

647211