Overview
The Workflow Description Language has a number of versions that are constantly improving. This guide lists all major versions of the language—the major improvements for each version, common pitfalls when upgrading, and how to get help.
If you're interested in learning more about how versioning works and how you can specify the version of your WDL documents, see the Versioning section within the Language Guide.
TIP
Changes are grouped in terms of their impact. The word "impact" here is a combination of "how important is the change" along with "how many existing WDL documents are likely to be affected by this change". They are subjective determinations made by the upgrade guide curator during the curator process, but they are often helpful in quickly scanning upgrade elements to see which might apply to your situation.
WDL v1.3
WDL v1.3 introduces enumeration types, else if/else conditional clauses, new standard library functions, and enhanced retry capabilities for tasks. All changes are backwards compatible with previous v1.x releases. The full release notes can be found here.
Checklist
Use this checklist to ensure you hit all of the sections.
Moderate impact changes
New enumeration types (link).
New else if and else conditional clauses (link).
New split standard library function (link).
Clarified path existence requirements (link).
Low impact changes
New task.previous variable (link).
New task.max_retries variable (link).
Moderate impact changes
New enumeration types
Enumeration types (enum) were introduced in #695 to define closed sets of named variants with associated values. Enums support explicit and implicit typing, and variant values can be of any WDL type including primitives, compound types, and user-defined types. A new value() standard library function was also added to extract the inner value from an enum variant.
version 1.3
enum PrimaryColor {
Red = "#FF0000"
Green = "#00FF00"
Blue = "#0000FF"
}
workflow example {
PrimaryColor color = PrimaryColor.Red
String hex_code = value(color) # "#FF0000"
}Look for anything that has a closed set of choices—these are good candidates to replace with enumerations.
New else if and else conditional clauses
The else if and else clauses were introduced in #699 to provide more expressive conditional logic within workflows. Previously, to achieve an else, you had to write two if statements with negated conditions.
version 1.3
workflow example {
input {
String mode
}
if (mode == "fast") {
call fast_analysis
} else if (mode == "thorough") {
call thorough_analysis
} else {
call default_analysis
}
}Look for pairs of if statements where the second condition is the negation of the first—these can be simplified using else.
New split standard library function
The split function was introduced in #729 to split a string into an array of substrings based on a delimiter pattern.
Array[String] parts = split("a,b,c", ",") # ["a", "b", "c"]Look for command blocks using awk, cut, or tr to split strings—these can be replaced with the native split function.
Clarified path existence requirements
The behavior of relative paths in File and Directory declarations was clarified in #735. Relative paths are resolved relative to the WDL document's parent directory outside the output section, and relative to the task's execution directory inside the output section. Additionally, File and Directory values must exist at declaration evaluation time—if a path does not exist when the declaration is evaluated, a runtime error will occur. Optional files evaluate to None if the path does not exist.
Review any File or Directory declarations that reference paths which might not exist yet—these should either exist before the workflow runs, be marked as optional (File? or Directory?), or be declared in the output section. Also ensure that relative paths are updated to be relative to the correct location.
Low impact changes
New task.previous variable
The task.previous variable was added in #734, enabling runtime access to the previous attempt's computed requirements. This is useful for implementing retry logic that adjusts resources based on previous failures.
If you have tasks that sometimes fail due to insufficient memory or CPU, consider using task.previous to automatically scale up resources on retry.
New task.max_retries variable
The task.max_retries variable was added in #733 to provide access to the maximum number of retry attempts configured for a task.
If you have retry logic that needs different behavior on the final attempt, you can now use task.max_retries to detect when you're on the last retry.
WDL v1.2
WDL v1.2 introduces a great deal of new syntax and changes to the specification. Notably, changes were backwards compatible with all previous v1.x releases. This sections only covers the high points related to upgrading—the full release notes can be found here.
Checklist
Use this checklist to ensure you hit all of the sections.
Moderate impact changes
New Directory type (link).
Replacement of runtime section with requirements and hints sections (link).
Deprecation of runtime section (link).
New standard library functions (link).
Multi-line strings (link).
Optional input: statement (link).
Low impact changes
Addition of workflow hints (link).
New requirements and hints keys (link).
Metadata sections for structs (link).
Exponentiation operator (link).
Moderate impact changes
New Directory type
The Directory type was introduced in #641 to better semantically indicate the use of a directory. If the intention of any of your arguments is to be used to refer to a directory on the filesystem, you are encouraged to update the parameters to a Directory type.
Deprecation of runtime section
The runtime section, which previously held both requirement constraints and hints to the execution engine, has now been split into the requirements (#540) section and hints (#541) section respectively. You should split out these keys based on the definitions in the specification1, 2.
🗑️ This change deprecates the runtime section, which will be removed in WDL v2.0. That being said, if desired, you can continue to use the runtime section in your WDL documents—it's just not recommended.
New standard library functions
The following are new standard library functions and their definitions. You are encouraged to read through them and replace any custom functionality that would now be duplicated in favor of these functions.
contains_key: whether or not aMaporObjectcontain a specific member (link).values: get the values from aMap(link).find: search for a regular expression in a string (link).matches: whether a string match a regular expression (link).chunk: split an array into sub-arrays (link).join_paths: join two or more paths (link).contains: whether an array contains a specified value (link).
Multi-line strings
Multi-line strings were introduced with the following syntax.
String foo = <<<
my
multi-line
string
>>>Optional input: statement
As noted in #524, the input: statement that precedes call bodies is unnecessary historical boilerplate. This statement is now optional in call bodies. You are encouraged to remove these from your call bodies.
Low impact changes
Addition of workflow hints
Workflows gained a hints section in #543. You are encouraged to go read the supported workflow hints1 and apply them to your workflow if any are relevant.
New requirements and hints keys
- The following keys were added to the task
requirementssection:fpga1. - The following keys were added to the task
hintssection:disks1,gpu2, andfpga2.
You are encouraged to examine these keys are determine if any of these should be specified for your tasks.
Metadata sections for structs
Similarly to tasks and workflows, structs now have meta and parameter_meta sections. You are encouraged to use these fields according to the definition in the specification1.
Exponentiation operator
The exponentiation operator (**) was added in this release. You are encouraged to go and update any manual exponentiation to use this operator instead.
You are encouraged to go read the specification section on this concept (link) and use them where appropriate.
Prior WDL versions
Versions of WDL prior to the ones outlined in this guide did not exist at the time the upgrade guide was created. As such, they are not included in the guide.