6 Tips for Setting Up Your Nextflow Dev Environment

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

犯傻的水桶 · 周家礽：滇虹药业创始人82岁再创业 | 人物· 1 月前 ·

体贴的弓箭 · 如何使用python ...· 3 月前 ·

乖乖的手术刀 · 在windows上安装tensorflow ...· 5 月前 ·

另类的开水瓶 · TCHAR和CHAR类型的互转_tchart ...· 5 月前 ·

一直单身的烈酒 · Zebra EHS Advanced ...· 5 月前 ·

</noscript>  <div class="navbar navbar-inverse navbar-fixed-top" role="navigation"> <div class="container"> <div class="navbar-header"> <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar"/> <span class="icon-bar"/> <span class="icon-bar"/> </button> <a class="navbar-brand" href="/index.html"> <img src="/img/nextflow.svg" title="Nextflow"/> </a> </div> <div class="navbar-collapse collapse"> <ul class="nav navbar-nav"> <li class="show animated flipInX"> <a href="/docs/latest/index.html">Documentation</a> </li> <li class="dropdown show animated flipInX"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Examples <b class="caret"/></a> <ul class="dropdown-menu"> <li><a href="/example1.html">Basic pipeline</a></li> <li><a href="/example2.html">Mixing scripting languages</a></li> <li><a href="/example3.html">BLAST pipeline</a></li> <li><a href="/example4.html">RNA-Seq pipeline</a></li> <li><a href="/example5.html">Machine Learning pipeline</a></li> <li> <a href="https://github.com/nextflow-io/rnaseq-nf" target="_blank"> Simple RNAseq pipeline <i class="fa fa-sm fa-external-link" aria-hidden="true"/> </a> </li> </ul> </li> <li class="show animated flipInX"> <a href="http://training.nextflow.io">Training</a> </li> <li class="dropdown show animated flipInX"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Resources <b class="caret"/></a> <ul class="dropdown-menu"> <li><a href="/blog.html">Blog</a></li> <li><a href="/podcasts.html">Podcast</a></li> <li><a href="/blog/2023/learn-nextflow-in-2023.html">Learn Nextflow</a></li> <li> <a href="http://nextflow-io.github.io/patterns/index.html"> Implementation patterns <i class="fa fa-sm fa-external-link" aria-hidden="true"/> </a> </li> <li><a href="/ambassadors.html">Nextflow Ambassadors</a></li> <li> <a href="https://www.nextflow.io/slack-invite.html"> Slack community chat nf-core pipelines <i class="fa fa-sm fa-external-link" aria-hidden="true"/> </a> </li> <li><a href="/about-us.html">About Nextflow</a></li> </ul> </li> <li class="show animated flipInX"> <a href="https://community.seqera.io/c/nextflow/5" target="_blank"> Forums <i class="fa fa-sm fa-external-link" aria-hidden="true"/> </a> </li> <li class="show animated flipInX"> <a href="https://github.com/nextflow-io/nextflow" title="GitHub Repository"> <i class="fa fa-github hidden-xs" aria-hidden="true"/> <span class="visible-xs">GitHub repository</span> </a> </li> </ul> </div> </div> </div> <div class="wrapper"> <div class="container"> <div class="row"> <div class="col-sm-8"> <div class="blg-summary"> <h2><a href="#">6 Tips for Setting Up Your Nextflow Dev Environment</a></h2> <ul class="text-muted list-inline blg-header"> <li> <i class="fa fa-user"/> Evan Floden </li> <li><i class="fa fa-calendar"/> 4 March 2021</li> </ul> <hr/> <div class="blg-text"> <p><em>This blog follows up the Learning Nextflow in 2020 blog <a href="https://www.nextflow.io/blog/2020/learning-nextflow-in-2020.html">post</a>.</em></p> <p>This guide is designed to walk you through a basic development setup for writing Nextflow pipelines.</p> <h3 id="1-installation">1. Installation</h3> <p>Nextflow runs on any Linux compatible system and MacOS with Java installed. Windows users can rely on the <a href="https://docs.microsoft.com/en-us/windows/wsl/install-win10">Windows Subsystem for Linux</a>. Installing Nextflow is straightforward. You just need to download the <code>nextflow</code> executable. In your terminal type the following commands:</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ curl get.nextflow.io | bash</span></span> <span class="line"><span>$ sudo mv nextflow /usr/local/bin</span></span> <p>The first line uses the curl command to download the nextflow executable, and the second line moves the executable to your PATH. Note <code>/usr/local/bin</code> is the default for MacOS, you might want to choose <code>~/bin</code> or <code>/usr/bin</code> depending on your PATH definition and operating system.</p> <h3 id="2-text-editor-or-ide">2. Text Editor or IDE?</h3> <p>Nextflow pipelines can be written in any plain text editor. I’m personally a bit of a Vim fan, however, the advent of the modern IDE provides a more immersive development experience.</p> <p>My current choice is Visual Studio Code which provides a wealth of add-ons, the most obvious of these being syntax highlighting. With <a href="https://code.visualstudio.com/download">VSCode installed</a>, you can search for the Nextflow extension in the marketplace.</p> <p>Other syntax highlighting has been made available by the community including:</p> <li><a href="https://github.com/Emiller88/nextflow-mode">Emacs</a></li> <h3 id="3-the-nextflow-repl-console">3. The Nextflow REPL console</h3> <p>The Nextflow console is a REPL (read-eval-print loop) environment that allows one to quickly test part of a script or segments of Nextflow code in an interactive manner. This can be particularly useful to quickly evaluate channels and operators behaviour and prototype small snippets that can be included in your pipeline scripts.</p> <p>Start the Nextflow console with the following command:</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ nextflow console</span></span> <p>Use the <code>CTRL+R</code> keyboard shortcut to run (<code>⌘+R</code>on the Mac) and to evaluate your code. You can also evaluate by selecting code and use the <strong>Run selection</strong>.</p> <h3 id="4-containerize-all-the-things">4. Containerize all the things</h3> <p>Containers are a key component of developing scalable and reproducible pipelines. We can build Docker images that contain an OS, all libraries and the software we need for each process. Pipelines are typically developed using Docker containers and tooling as these can then be used on many different container engines such as Singularity and Podman.</p> <p>Once you have <a href="https://docs.docker.com/engine/install/">downloaded and installed Docker</a>, try pull a public docker image:</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ docker pull quay.io/nextflow/rnaseq-nf</span></span> <p>To run a Nextflow pipeline using the latest tag of the image, we can use:</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>nextflow run nextflow-io/rnaseq-nf -with-docker quay.io/nextflow/rnaseq-nf:latest</span></span> <p>To learn more about building Docker containers, see the <a href="https://seqera.io/training/#_manage_dependencies_containers">Seqera Labs tutorial</a> on managing dependencies with containers.</p> <p>Additionally, you can install the VSCode marketplace addon for Docker to manage and interactively run and test the containers and images on your machine. You can even connect to remote registries such as Dockerhub, Quay.io, AWS ECR, Google Cloud and Azure Container registries.</p> <h3 id="5-use-tower-to-monitor-your-pipelines">5. Use Tower to monitor your pipelines</h3> <p>When developing real-world pipelines, it can become inevitable that pipelines will require significant resources. For long-running workflows, monitoring becomes all the more crucial. With <a href="https://tower.nf">Nextflow Tower</a>, we can invoke any Nextflow pipeline execution from the CLI and use the integrated dashboard to follow the workflow run.</p> <p>Sign-in to Tower using your GitHub credentials, obtain your token from the Getting Started page and export them into your terminal, <code>~/.bashrc</code>, or include them in your nextflow.config.</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ export TOWER_ACCESS_TOKEN=my-secret-tower-key</span></span> <p>We can then add the <code>-with-tower</code> child-option to any Nextflow run command. A URL with the monitoring dashboard will appear.</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ nextflow run nextflow-io/rnaseq-nf -with-tower</span></span> <h3 id="6-nf-core-tools">6. nf-core tools</h3> <p><a href="https://nf-co.re/">nf-core</a> is a community effort to collect a curated set of analysis pipelines built using Nextflow. The pipelines continue to come on in leaps and bounds and nf-core tools is a python package for helping with developing nf-core pipelines. It includes options for listing, creating, and even downloading pipelines for offline usage.</p> <p>These tools are particularly useful for developers contributing to the community pipelines on <a href="https://github.com/nf-core/">GitHub</a> with linting and syncing options that keep pipelines up-to-date against nf-core guidelines.</p> <p><code>nf-core tools</code> is a python package that can be installed in your development environment from Bioconda or PyPi.</p> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ conda install nf-core</span></span> <pre class="astro-code light-plus" style="background-color:#FFFFFF;color:#000000; overflow-x: auto;" tabindex="0" data-language="plaintext"><code><span class="line"><span>$ pip install nf-core</span></span>