Pipeline Orchestration

Overview

The Reasoning Engine orchestrates a 7-step analysis pipeline where each step builds upon the results of previous steps. This page explains the overall execution model, data flow patterns, and database architecture.

---

Pipeline Flow

---

Design Principles

1. Sequential Processing

Each step receives enriched state from prior steps, ensuring context builds progressively. Steps cannot run in parallel.

2. Minimal State

Only essential data passes between steps. Full outputs are stored in the database to prevent JSON payload bloat.

3. Database-Backed

All step outputs are persisted to data_WorkflowStepOutputs, enabling:

• Auditability - Full trace of analysis decisions
• Debugging - Inspect intermediate results
• Replay - Re-run from any step

4. Fail-Safe

If a step fails, the workflow continues with degraded data. The validation step flags missing analysis.

5. Separation of Concerns

• Reasoning Engine - Analytical logic (trends, patterns, correlations)
• Outer Workflow - Business logic (normalization, formatting, tone)

---

Data Flow Pattern

---

Step Summary

Step	Purpose	Agent	Key Output
0	Validate input & plan execution	Step Orchestrator	Canonical data, step plan
1	Analyze time-series trends	Temporal	Statistics, trends
2	Detect behavioral patterns	Pattern	Detected patterns
3	Compute metric relationships	Correlation	Correlation pairs
4	Retrieve domain knowledge	Domain Retrieval	Perplexity context or local fallback
5	Generate narrative	Explanation	Human-readable analysis
6	Verify claims & assemble result	Validation	Validated final output

info

See the Steps section for detailed documentation on each step's inputs, outputs, and processing logic.

---

Database Architecture

log_ReasoningEngineTasks

Parent table tracking workflow executions.

Column	Type	Description
TaskId	uniqueidentifier	Primary key
WorkflowRunId	uniqueidentifier	Mastra workflow run ID
Objective	nvarchar(MAX)	Analysis goal
Status	nvarchar(50)	running, completed, failed
StartedAtUtc	datetime2(3)	Workflow start time
CompletedAtUtc	datetime2(3)	Workflow completion time

log_StepReasoningEngineTasks

Child table tracking individual step executions.

Column	Type	Description
StepTaskId	uniqueidentifier	Primary key
TaskId	uniqueidentifier	Foreign key to parent task
StepName	nvarchar(128)	Step identifier
StepOrder	int	Execution order (0-6)
Status	nvarchar(50)	pending, running, completed, skipped, failed
StartedAtUtc	datetime2(3)	Step start time
CompletedAtUtc	datetime2(3)	Step completion time

data_WorkflowStepOutputs

Stores JSON output from each step.

Column	Type	Description
OutputId	uniqueidentifier	Primary key
WorkflowRunId	uniqueidentifier	Workflow identifier
StepName	nvarchar(128)	Step identifier
OutputData	nvarchar(MAX)	JSON output
CreatedAtUtc	datetime2(3)	Timestamp

---

Configuration Options

Enable Step Orchestration

Allow AI to select which analysis steps to run:

{
  enableStepOrchestration: true
}

Provide Pre-computed Plan

Skip AI orchestration by providing explicit step list:

{
  stepPlan: {
    steps: ['temporal', 'correlation', 'explanation', 'validation'],
    reasoning: 'Focus on temporal and correlations only'
  }
}

Output Profile

Specify output formatter (handled by outer workflow):

{
  outputProfile: 'recovered-summary'
}

---

Next Steps

Step 0: Preflight

Input validation and execution planning

Step 1: Temporal

Time-series analysis and trend detection

Step 2: Pattern

Behavioral pattern recognition

Step 3: Correlation

Metric relationship analysis

Step 4: Domain

Domain knowledge retrieval

Step 5: Explanation

Narrative generation

Step 6: Validation

Claim verification and result assembly