Reports Overview

Overview

Most Nextflow pipelines will generate reports or output files which are useful to inspect at the end of the pipeline execution. Reports may be in various formats (e.g. HTML, PDF, TXT) and would typically contain quality control (QC) metrics that would be important to assess the integrity of the results. Tower has a Reports feature that allows you to directly visualise supported file types or to download them directly via the user interface (see Limitations). This saves users the time and effort from having to retrieve and visualise output files from their local storage.

Visualizing Reports

Available reports are listed in a Reports tab within the Runs page. Users can select a report from the table and open or download it (see Limitations for supported file types and sizes).

To open a report preview, the file must be smaller than 10MB.

Users can download a report directly from Tower or using the path. Download is not available if a report is larger than 25 MB. Option to download from path is suggested instead.

Providing Reports

To render reports users need to create a Tower config file that defines the paths to a selection of output files published by the pipeline. There are 2 ways users can provide the Tower config file both of which have to be in YAML format:

1. Pipeline repository: If a file called tower.yml exists in the root of the pipeline repository then this will be fetched automatically before the pipeline execution.,
2. Tower UI: Providing the YAML definition within the Advanced options > Tower config file box when: 1.Creating a Pipeline in the Launchpad 2.Amending the Launch settings when launching a Pipeline. Users with Maintain role only.

Any configuration provided in the Tower UI will completely override that which is supplied via the pipeline repository.

Reports implementation

Pipeline Reports need to be specified via YAML syntax

reports:
  <path pattern>:
    display: text to display (required)
    mimeType: file mime type (optional)

Path pattern

Only the published files (using the Nextflow publishDir directive) are possible report candidates files. The path pattern is used to match published files to a report entry. It can be a partial path, a glob expression or just a file name.

Examples of valid path patterns are:

• multiqc.html: This will match all the published files with this name.
• **/multiqc.html: This is a glob expression that matches any subfolder. It is equivalent to the previous expression.
• results/output.txt: This will match all the output.txt files inside any results folder.
• *_output.tsv: This will match any file that ends with “_output.tsv”

When you use * it is important to also use double quotes, otherwise it is not a valid YAML.

Display

Display defines the title that will be shown on the website. If there are multiple files that match the same pattern an automatic suffix will be added. The suffix is the minimum difference between all the matching paths. For example given this report definition:

reports:
  "**/out/sheet.tsv":
    display: "Data sheet"

If you have these two paths /workdir/sample1/out/sheet.tsv and /workdir/sample2/out/sheet.tsv both of them will match the path pattern and their final display name will be Data sheet (sample1) and Data sheet (sample2).

mimeType

By default the mime type is deduced from the file extension, so in general you don’t need to explicitly define it. Optionally, you can define it to force a viewer, for example showing a txt file as a tsv. It is important that it is a valid mime type text, otherwise it will be ignored and the extension will be used instead.

Limitations

The current reports implementation limits the rendering to the following formats: HTML, CSV, tsv, pdf, and txt.

In-page rendering/report preview is restricted to files smaller than 10MB to reduce the UI overload. Larger files need to be downloaded first.

The download is restricted to files smaller than 25 MB to reduce the overload. Larger files need to be downloaded from the path.

Currently, there is a YAML formatting validation in place checking both the tower.yml file inside the repository and the UI configuration box. The validation phase will emit an error message when users try to launch a pipeline with non-compliant YAML definitions.