Amazon EKS is a managed Kubernetes cluster that allows the execution of containerized workloads in the AWS cloud at scale.
Tower offers native support for AWS EKS clusters and streamlines the deployment of Nextflow pipelines in such environments.
You need to have an EKS cluster up and running. Make sure you have followed the cluster preparation instructions to create the cluster resources required by Tower. In addition to the generic Kubernetes instructions, you will need to make a few modifications specific to EKS.
Assign service account role to IAM user. You will need to assign the service role with an AWS user that will be used by Tower to access the EKS cluster.
First, use the following command to modify the EKS auth configuration:
Once the editor is open, add the following entry:
1 2 3 4 5
Your user ARN can be retrieved from the AWS IAM console or from the AWS CLI:
The same user needs to be used when specifying the AWS credentials in the configuration of the Tower compute environment for EKS.
The AWS user should have the following IAM policy:
Click to view eks-iam-policy.json
For more details, refer to the AWS documentation.
In a workspace, select Compute Environments and then New Environment.
Enter a descriptive name for this environment, e.g. "Amazon EKS (eu-west-1)".
Select Amazon EKS as the target platform.
Select your AWS credentials or add new credentials by selecting the + button.
Make sure the user has the IAM permissions required to describe and list EKS clusters as explained here.
Select a Region, for example "eu-west-1 - Europe (Ireland)".
Select a Cluster name from the list of available EKS clusters in the selected region.
Specify the Namespace created in the cluster preparation instructions, which is
Specify the Head service account created in the cluster preparation instructions, which is
Specify the Storage claim created in the cluster preparation instructions, which serves as a scratch filesystem for Nextflow pipelines. In each of the provided examples, the storage claim is called
You can use the Environment variables option to specify custom environment variables for the Head job and/or Compute jobs.
Configure any advanced options described below, as needed.
Select Create to finalize the compute environment setup.
Jump to the documentation for Launching Pipelines.
The Storage mount path is the file system path where the Storage claim is mounted (default:
The Work directory is the file system path used as a working directory by Nextflow pipelines. It must be the the storage mount path (default) or a subdirectory of it.
The Compute service account is the service account used by Nextflow to submit tasks (default: the
defaultaccount in the given namespace).
The Pod cleanup policy determines when terminated pods should be deleted.
You can use Custom head pod specs to provide custom options for the Nextflow workflow pod (
affinity, etc). For example:
1 2 3
spec: nodeSelector: disktype: ssd
You can use Custom service pod specs to provide custom options for the compute environment pod. See above for an example.
You can use Head Job CPUs and Head Job Memory to specify the hardware resources allocated for the Nextflow workflow pod.