Concord is a workflow server. It is the orchestration engine that connects different systems together using scenarios and plugins created by users. More details about the tool can be found here.
Creating Custom Concord Tasks
Tasks, or Plugins, are executable Java code which are called from Concord workflows to perform actions. These actions are usually more complex than what can be accomplished with the basic DSL and Expression Language syntax supported by Concord’s YAML-based project definition.
This post will expand on the basic task documentation as well as point to actual implementation examples in various Concord task source code.
Tasks can be invoked in two main ways: Task Syntax and Expression Syntax.
Task syntax is a YAML definition of the task call and parameters. The task’s
execute method is invoked by the runtime.
Expression syntax calls a
public method of a Task directly.
- Puppet Task — Execute method use with task syntax
- Resource Task — Public methods used with expression syntax
Leverage The Concord SDK
concord-sdk module must be used to implement the
Task interface. Further, the module includes a number of other useful classes which make implementing a Task easier.
MapUtils classes provide methods which simplify wrangling variables from the Process in which the Task is called. Since variables are returned as
Object from a
Context object, these utility classes can be leveraged to avoid the boilerplate of casting the values to appropriate types and checking for
The Concord SDK also includes interfaces which can be used for injecting services to be used within a Task such as
SecretService for retrieving Secret values and
DockerService for running Docker containers during Task execution.
- Docker Task — Uses
ContextUtilsto obtain task parameter values
- Git Task — Uses
SecretServiceto get private key
Default Parameters and Precedence
It is advisable to support global default for some or all task parameters. This allows certain parameters to be defined once and re-used implicitly with each task call. For example, if a flow calls the Puppet task multiple times, it can repeat itself much less by using a global variable to hold some of the task’s parameters.
Process variables can be defined (and redefined) in a number of ways. Understanding the order of variable definition precedence is important when a Task retrieves variable values from a
Context object. From lowest to highest precedence:
- Process default variables defined by the Concord system
- Variables defined in workflow’s
- Variables defined when starting a workflow (e.g.
curl -F arguments.myVar=myVal)
- Variables created with the
— set:call or
context.setVariable('name', 'val')during flow execution
- Task in-parameters and expression parameters
- Jenkins Task — Support default parameters in
- Puppet Task — Supports default parameters in
Returning Data to the Process
When setting Process context variables in a Task, use a
Map to make data access in the Process more intuitive. Don’t set multiple variables. In the Task, a simple
Serializable class can be converted to a
Map with Jackson’s
- Jenkins Task — Builds a result
- Puppet Task — Converts a
PuppetResultobject to a
- MS Teams Task — Converts a
Resultobject to a
By default, Tasks should throw an exception with a helpful message when an unrecoverable error is encountered. The plugin can provide an argument to ignore errors (e.g.
ignoreErrors) which gracefully exits the task and sets a Process context variable with the message.
Extra Logging for Debugging
Default logging for a Task should be limited to exception details and important task progress in order to avoid an overly chatty Process log. For deeper debugging purposes, a Task can provide a
debug option to output more detailed information on the Task’s operations.
- Ansible Task —
verboseparameters for extensive logging
- HTTP Task —
debugparameter outputs request and response data
Using External Artifacts and Executables
Tasks may require an external artifact to perform their actions. Rather than package these files with the Task, use Concord’s
DependencyManager to resolve them at runtime. This cuts down the size of the Task’s compiled binary, which in turn makes Process startup more efficient. The external artifact will not be resolved until the task is called, and will be cache-able for subsequent processes.
- Terraform Task — Downloads and execute Terraform binary artifact
Working with Secrets
- Crypto Task — Exports any secret type in a flow
- Ansible Task — Exports key pairs for ssh connections and file secrets for WinRM connections
- Git Task — Exports key pairs to clone git repositories
Most Task implementations should be as quiet as possible and accept data through parameters; however, there is a use case for tasks to build and/or call a particularly complex dynamic form.
NOTE: Tasks cannot read values from a Form they execute. This is because the form call is executed after the task finishes.
Call Other Tasks (or Let other Tasks Call Your Task)
Tasks can be made even more useful when they are available to use by other Tasks. Most tasks can be called without any extra work to accommodate being called from another task. Just set the scope to
provided in the Task’s
pom.xml and configure the class to be injected at runtime. The big upside to relying on injection is that the Concord runtime will handle further dependency injection that the other Task may require.
There are some downsides to this approach. The other task may not be available as
provided (most aren’t) or the code injected at runtime may be from a very different version which may have unintended behavior due to method refactoring or deprecation. A more reliable way to handle dependency versioning is to include them as transient dependencies (instead of
provided) in the Task’s
pom.xml and instantiate the class directly (instead of relying on injection at runtime). Be aware that if the other task relies on variable injection, it will fail.
The most considerate way to design a plugin for re-use with other plugins is to implement a client layer for the Task. That is, the Task itself calls a client class to handle the actual work, and other Tasks can call the same client class rather than the Task class which may have undocumented
@InjectVariable usage. Further, it will likely be advantageous to provide a utility method or class that handles gathering the required parameters.