Passing Variables

Passing context and variables around steps is a core concept in the process execution pipeline. This is done by templating which dynamically replaces placeholders within inputs with values. This enables the seamless passing of variables between different steps of a process, ensuring that each step has access to the necessary data and context to execute correctly.

Overview

Unmeshed facilitates the dynamic injection of data into inputs by recognizing and replacing placeholders with actual values from various sources. This mechanism supports secure handling of sensitive information such as secrets, flexible configuration through variables, and data flow between different steps within a process.

Syntax for Passing Variables

Variables within inputs are denoted using double curly braces ({{ }}). The syntax allows for optional quotes and supports different data sources. Here's the syntax:

{{ optionalQuote? variablePath optionalEndQuote? }}

Parameter	Description
optionalQuote & optionalEndQuote	An optional single (') or double (") quote to wrap the variable.
variablePath	The path indicating the source of the variable, such as secrets, variables, step references, or process inputs.

Variable Types

Secrets

Syntax:

{{ secrets.secretName }}

Description: References a secret value stored securely. Secrets are typically sensitive information like API keys, passwords, or tokens.

Usage Example:

{{ secrets.databasePassword }}

Example in JSON:

{
  "database": {
    "password": "{{ secrets.databasePassword }}"
  }
}

(Environment) Variables

Syntax:

{{ variables.variableName }}

Description: References a predefined variable within the Unmeshed Cluster. Variables can store configuration values or parameters that are reused anywhere across process definitions or steps. This is the equivalent of an environment variable

Usage Example:

{{ variables.retryCount }}

Example in JSON:

{
  "maxRetries": "{{ variables.retryCount }}"
}

Step References

Syntax:

{{ steps.stepRef.output.key }}

Description: References the output from a previous step in the process. This allows subsequent steps to utilize data generated by earlier steps.

Usage Example:

{{ steps.dataProcessing.output.result }}

Example in JSON:

{
  "processedData": "{{ steps.dataProcessing.output.result }}"
}

Process Inputs

Syntax:

{{ context.input.inputKey }}

Description: References the inputs provided when initiating the process. These inputs can influence the behavior and execution of the process.

Usage Example:

{{ context.input.userId }}

Example in JSON:

{
  "userId": "{{ context.input.userId }}"
}

Passing Variables Between Steps

To pass inputs into steps from other steps, use the appropriate syntax based on the source of the data. Here's how you can integrate different types of variables within your process steps:

Define the output in the Source Step: Ensure that the step generating the data exposes the necessary output. Reference the output in the Target Step: Use the steps syntax to access the output from the source step.

Example Workflow

Data Processing

Output from Source:

{
  "result": "Processed Data"
}

Target Input definition

{
  "result": "{{ steps.dataProcessing.output.result }}"
}

Usage Examples

Example 1: Using Secrets

Scenario: Injecting a database password securely into a configuration.

Template:

{
  "database": {
    "password": "{{ secrets.databasePassword }}"
  }
}

Resolved Output:

{
  "database": {
    "password": "s3cr3tP@ssw0rd"
  }
}

Example 2: Using Variables Scenario: Setting the maximum number of retries for a process.

Input Definition:

{
  "maxRetries": "{{ variables.retryCount }}"
}

Resolved Output:

{
  "maxRetries": 5
}

Example 3: Referencing Step Outputs Scenario: Using the result from a data processing step in a subsequent step.

{
  "processedData": "{{ steps.dataProcessing.output.result }}"
}

Resolved Output:

{
  "processedData": "Processed Data"
}

Example 4: Accessing Process Inputs Scenario: Utilizing user-provided input to customize a process step.

{
  "userId": "{{ context.input.userId }}"
}

Resolved Output:

{
  "userId": "user_12345"
}

Best Practices

• Secure Handling of Secrets: Always use the secrets prefix to handle sensitive information. Avoid exposing secrets in logs or error messages.
• Consistent Naming Conventions: Use clear and consistent names for variables, secrets, and step references to enhance readability and maintainability.
• Error Handling: Implement appropriate error handling to manage scenarios where a referenced variable or secret might be missing or invalid.
• Documentation: Document the purpose and usage of each variable and secret to aid team members in understanding the process flow.
• Avoid Hardcoding Values: Utilize variables and secrets instead of hardcoding values to make the process more flexible and secure.
• Validate Inputs: Ensure that the inputs provided to the process are validated to prevent issues during template resolution.
• Monitor and Log: Keep track of variable resolutions and monitor for any discrepancies or failures in the template resolution process.