Google GKE is a managed Kubernetes cluster that allows the execution of containerized workloads in Google Cloud at scale.
Tower offers native support for Google GKE clusters and streamlines the deployment of Nextflow pipelines in such environments.
Refer to the Google Cloud section for instructions on how to set up your Google Cloud account and any other services (e.g. Cloud Storage) that you intend to use.
You need to have a GKE cluster up and running. Make sure you have followed the cluster preparation instructions to create the cluster resources required by Tower. In addition to the generic Kubernetes instructions, you will need to make a few modifications specific to GKE.
Assign service account role to IAM user. You will need to grant the cluster access to the service account used to authenticate the Tower compute environment. This can be done by updating the role binding as shown below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
In the above snippet, replace
<IAM SERVICE ACCOUNT> with the corresponding service account, e.g.
For more details, refer to the Google documentation.
In a workspace, select Compute Environments and then New Environment.
Enter a descriptive name for this environment, e.g. "Google GKE (europe-west1)".
Select Google GKE as the target platform.
Select your Google Cloud credentials or add new credentials by selecting the + button.
Select the Location of your GKE cluster.
Regional and zonal clusters
GKE clusters can be either regional or zonal. For example,
us-west1identifies the United States West-Coast region, which has three zones:
Tower self-completion only shows regions. You should manually edit this field if you are using a zonal GKE cluster.
Select or enter the Cluster name of your GKE cluster.
Specify the Namespace created in the cluster preparation instructions, which is
Specify the Head service account created in the cluster preparation instructions, which is
Specify the Storage claim created in the cluster preparation instructions, which serves as a scratch filesystem for Nextflow pipelines. In each of the provided examples, the storage claim is called
You can use the Environment variables option to specify custom environment variables for the Head job and/or Compute jobs.
Configure any advanced options described below, as needed.
Select Create to finalize the compute environment setup.
Jump to the documentation for Launching Pipelines.
The Storage mount path is the file system path where the Storage claim is mounted (default:
The Work directory is the file system path used as a working directory by Nextflow pipelines. It must be the the storage mount path (default) or a subdirectory of it.
The Compute service account is the service account used by Nextflow to submit tasks (default: the
defaultaccount in the given namespace).
The Pod cleanup policy determines when terminated pods should be deleted.
You can use Custom head pod specs to provide custom options for the Nextflow workflow pod (
affinity, etc). For example:
1 2 3
spec: nodeSelector: disktype: ssd
You can use Custom service pod specs to provide custom options for the compute environment pod. See above for an example.
You can use Head Job CPUs and Head Job Memory to specify the hardware resources allocated for the Nextflow workflow pod.