A Compound Flow is an advanced flow type designed for complex, multi-stage processing pipelines. Unlike Elemental Flows which perform single, focused tasks, Compound Flows can create sophisticated decision trees, implement custom processing logic, and orchestrate multiple processing stages. 🎯

Compound Flow Attributes

Version ⚡

ComponentDescriptionRequiredExample
versionFlow specification version using semantic versioningYes"0.1.0"

Metadata

ComponentDescriptionRequiredExample
flow_typeType of flow (must be “compound”)Yes"compound"
nameUnique identifier for the flowYes"your-flow-name"
descriptionExplanation of flow’s purposeYes"A brief description of your flow"
authorCreator’s usernameYes"your-username"
tagsKeywords for categorizationNo[tag1, tag2, tag3]
privateAccess control settingYesfalse

Input Configuration

ComponentDescriptionRequiredExample
inputsMap of input parametersYesCollection of input definitions
typeData type of input (currently only string)Yes"string"
descriptionPurpose of the inputYes"Description of input1"
requiredWhether input is mandatoryYestrue or false
exampleSample input valueNo"Example value for input1"

Workflow Configuration

Each Compound Flow must define a workflow consisting of multiple processing stages. There are two types of workflow stages:

Elemental Stage Configuration

ComponentDescriptionRequiredExample
typeMust be “elemental”Yes"elemental"
flow_nameReference to existing Elemental FlowYes"author/flow-name"
inputsMapping to flow inputsYesInput mapping object
depends_onList of dependent stagesNo["stage1", "stage2"]

Custom Stage Configuration

ComponentDescriptionRequiredExample
typeMust be “custom”Yes"custom"
modelLLM configurationYesModel settings object
promptProcessing instructionsYesTemplate with placeholders
inputsStage input parametersYesInput mapping object
datasetRAG dataset configurationNo"author_name/dataset_name"
depends_onList of dependent stagesNo["stage1", "stage2"]

Dependency Configuration

ComponentDescriptionRequiredExample
depends_onList of stages that must complete before executionNo["stage1", "stage2"]

Execution Behavior

  • Stages without depends_on can execute in parallel

  • Stages with depends_on will wait for specified stages to complete

  • Multiple stages can depend on the same previous stage

Output Configuration

ComponentDescriptionRequiredExample
output.valueList of stage outputs to combineYes[${stage1.outputs}, ${stage2.outputs}]

Basic Flow Structure

# Flow specification version
version: "0.1.0"

# Flow metadata and configuration
metadata:
  flow_type: "compound"
  name: "your-flow-name"
  description: "Describe what this compound flow accomplishes"
  author: "your-username"
  tags: [tag1, tag2, tag3]
  private: true

# Primary input definitions
inputs:
  prime_input_1:
    type: string
    description: "What this input is used for"
    required: true
    example: "Example value"
  prime_input_2:
    type: string
    description: "Description of this input"
    required: false
    example: "Example"

# Workflow definition
workflow:
  # Elemental Flow stage - starts immediately
  first_flow:
    type: "elemental"
    flow_name: "john_doe/first_elemental_flow"
    inputs:
      abc: ${inputs.prime_input_1}
      xyz: ${inputs.prime_input_2}

  # Custom processing stage - starts immediately (parallel to first_flow)
  second_flow:
    type: "custom"
    inputs:
      input1: ${inputs.input_name}
    model:
      provider: "provider-name"
      name: "model-name"
    prompt: |
      Your flow's primary instruction or role...
      You can use {input1} placeholders to reference inputs.

  # Waits for both first_flow and second_flow to complete
  third_flow:
    type: "custom"
    depends_on: [first_flow, second_flow]
    inputs:
      input1: ${first_flow.output}
      input2: ${second_flow.output}
    model:
      provider: "provider-name"
      name: "model-name"
    dataset:
      source: "author_name/dataset_name"
    prompt: |
      Your flow's primary instruction or role...
      You can use {input1} and {input2} placeholders to reference inputs.

# Output configuration
output:
  value:
    - ${first_flow.output}
    - ${second_flow.output}
    - ${third_flow.output}

# Flow documentation
readme: |
  This is a detailed explanation of what your flow does.

  ### Workflow Overview:
  1. **First Flow**: Executes `john_doe/first_elemental_flow` with the provided inputs.
  2. **Second Flow**: Runs in parallel with first flow
  3. **Third Flow**: Depends on both previous flows, executes after they complete

  ### Outputs:
  - Combined outputs from all flows are returned in order.

Implementation Structure 🛠️

Compound Flows can either define custom processing logic within their workflow or integrate existing Elemental Flows, providing maximum flexibility for solving complex challenges. The workflow allows for:

  1. Advanced Decision Making with conditional processing paths 🧠

  2. Custom Processing stages with direct model and prompt configurations ⚙️

  3. Integration of existing Elemental Flows for reusability 🔄

  4. Parallel execution of independent stages ⚡

  5. Sequential execution through depends_on configurations đź”—

  6. Flexible input/output mapping between stages 🔀