Release notes for version 22.1.x#
Nextflow Tower 22.1.x brings the following changes:
- Added: Pipeline Secrets feature
- Added: Custom workflows run name
- Added: CSV and TSV reports rendering
- Added: Support for AWS agent and logging
- Added: Support for Moab platform
GetLogsEventsperm to AWS Batch instance role
- Added: Support for local pipelines
- Added: Navigate back button to second level screens
- Added: Validation for SSH hostname and username
- Added: Credentials view page
- Added: ECS pull strategy in user-data template
- Added: Root users environment when
TOWER_ROOT_USERSvariable is provided
- Added: Tower system message
- Added: Config option to disable user private workspace
- Added: Support for JSON formatted logs
- Stability improvements
- Security improvements as for penetration test performed on March 2022
New features and usability improvements#
We're pleased to announce the
22.1 release of Tower. This release brings long-awaited features including a customizable run name for workflows, enhanced secrets handling, better reports, and improved HPC scheduler support.
The release naming convention is changed and it reflects the year and the quarter, so our customers know to expect 4 enterprise releases per year.
Customizable workflow run name#
From this release, Tower allows users to specify a custom name for each workflow run. This name will override the default runName parameter for the Nextflow run name. This feature allows to identify each run at a glance, facilitating the retrieval of a specific run from the list presented in the Runs page from the application.
More than 60% of new workflow runs in tower.nf already make use of the custom run name.
Pipeline secrets - improved#
Starting from 21.12, Tower uses the concept of secrets to store the keys and tokens used by workflow tasks to interact with external systems e.g., a password to connect to an external database or an API token. In version 22.1 we have improved the secrets management by:
- Introducing the possibility for users to create user-level secrets.
- Implementing a selection mechanism so users can dynamically select which secrets will be used by each pipeline launch.
- Enable Tower to capture specific OIDC attributes as secrets.
Full information about secrets is available here
Improved web reports#
Most Nextflow pipelines will generate reports or output files which are useful to inspect at the end of the pipeline execution. Since 21.12, Tower has a Reports feature that allows to directly visualise supported file types or to download them directly via the user interface.
The key improvements in 22.1 - Reports are configurable and are no longer limited to workdir subdirectories. - Tower enables the rendering of CSV and TSV files.
Comprehensive documentation on setup and constraints is available here.
Enhanced HPC support#
Tower 22.1 allows working with HPC MOAB platform, expanding the supported alternatives for HPC computing.
We've also implemented a first version of a much requested feature from HPC users: Launch local repositories. Using this feature, users can instruct Tower to source the pipeline code stored as a Git bare repository and launch the workflows.
- As of version 22.1.x, Nextflow Tower Enterprise will follow a three month release cadence, using the following version number scheme:
YYrepresents the year,
Qrepresents the quarter and
PATCHthe incremental patch number.
- As of version 21.02.x, a license key must be provided to enable the Tower deployment feature. The license key should be specified using the configuration variable
TOWER_LICENSE. If you don't have a license key, contact firstname.lastname@example.org.
- This version now expects the use of HTTPS by default for all browser client connections. If your Tower installation requires the use of unsecured HTTP, set the following environment variable in the infrastructure hosting the Tower application:
- If you're upgrading from a version of Tower prior to
21.04.x, please update your implementation to
21.04.xbefore installing this release.
This Tower version requires a database schema update. Follow these steps to update your DB instance and the Tower installation.
- Make a backup of the Tower database.
Download and update your container versions to:
Redeploy the Tower application:
- Restart the application with
docker-compose restart. This will automatically migrate the database schema.
- Update the cron service with
kubectl apply -f tower-cron.yml. This will automatically migrate the database schema.
- Update the frontend and backend services with
kubectl apply -f tower-srv.yml.
- Run the
/migrate-db.shscript provided in the
backendcontainer. This will migrate the database schema.
- Deploy Tower following your usual procedures.
- Restart the application with
Nextflow launcher image#
If you must host your container images on a private image registry, copy the nf-launcher image to your private registry. Then update your
tower.env with the following environment variable:
If you're using AWS Batch, you will need to configure a custom job-definition and populate the
TOWER_LAUNCH_CONTAINER with the job-definition name instead.
The Tower compute environments to be made available to users must be specified in the Tower configuration.
The following ids options are available:
awsbatch-platform: AWS Batch cloud compute service
gls-platform: Google LifeSciences cloud compute service
azbatch-platform: Azure Batch cloud compute service
lsf-platform: IBM LSF batch scheduler
slurm-platform: Slurm batch scheduler
altair-platform: Altair PBS Pro batch scheduler
univa-platform: (Univa/Sun) GridEngine
moab-platform: Adaptive Computing Moab batch scheduler
k8s-platform: Kubernetes compute platform
eks-platform: AWS EKS compute platform
gke-platform: Google GKE compute platform
Choose one or more of these platform ids specify them via the environment variable
TOWER_ENABLE_PLATFORMS, separating them via a comma.
For a detailed list of all changes, please consult the Nextflow Tower Changelog.
Contact us at email@example.com.