Preflight

Overview

The Preflight step is the entry point for the reasoning engine workflow. It performs:

1. Schema validation (deterministic) - validates input structure
2. Step orchestration (AI-driven, optional) - selects which analysis steps to run

File: steps/preflight.step.ts

Design Principles

Validation is Deterministic

Schema validation has no LLM calls. Same input = same validation result.

Orchestration is AI-Driven

Step selection uses an agent to optimize which steps are relevant.

No Normalization

Input must already be in canonical format. Transformation happens in outer workflows.

Agnostic Input

Expects pre-normalized canonical input from outer workflows.

What It Does

Phase 1: Schema Validation (Deterministic)

Check	Description
✅ Validate objective	Ensures objective is present and non-empty
✅ Validate series	Checks structure: id, points[]
✅ Validate datasets	Checks structure: id, values[]
✅ Validate stepPlan	If provided, validates step names
✅ Initialize taskId	Creates UUID for tracking

Phase 2: Step Orchestration (AI-Driven, Optional)

When enableStepOrchestration: true:

• Calls the Step Orchestrator Agent
• Agent analyzes objective and data characteristics
• Returns optimized list of relevant steps
• Skips unnecessary analysis (saves time & cost)

What It Does NOT Do

Responsibility	Where It Happens
❌ Dataset normalization	Outer workflow's Phase 1 (intake adapters)
❌ Format detection	Outer workflow's intake adapters
❌ Raw data parsing	Outer workflow's normalization step

Configuration

Field	Type	Description
enableStepOrchestration	boolean	If true, AI selects which steps to run
stepPlan	object	Pre-computed step plan (bypasses orchestration)

Input Schema

{
  objective: string,         // Required: Analysis goal
  series?: Series[],         // Pre-normalized time-series
  datasets?: Dataset[],      // Pre-normalized datasets  
  hints?: string[],          // Analysis hints
  enableStepOrchestration?: boolean,  // Enable AI step selection
  stepPlan?: {               // OR provide pre-computed plan
    steps: ('temporal' | 'pattern' | 'correlation' | 'domain' | 'explanation' | 'validation')[],
    reasoning?: string
  }
}

Output Schema

{
  taskId: string,           // UUID for tracking
  hints: string[],          // Trimmed/filtered hints
  series: Series[],         // Passed through unchanged
  datasets: Dataset[],      // Passed through unchanged
  stepPlan: {               // AI-selected or default (all steps)
    steps: string[],
    reasoning: string
  },
  executedSteps: [{
    name: 'preflight-intake',
    status: 'run',
    startedAt: string,
    completedAt: string,
    reason: string
  }]
}

Step Orchestration

When enableStepOrchestration is true, preflight calls the Step Orchestrator Agent:

const orchestrationInput = {
  objective: inputData.objective,
  hints: normalizedHints,
  datasetCount: datasets.length,
  seriesCount: series.length,
  dataPoints: series[0]?.points?.length ?? 0,
  hasNotes: !!compactedRows.length,
  seriesIds: series.map(s => s.id),
  datasetIds: datasets.map(d => d.id),
};

const response = await stepOrchestratorAgent.generate([
  { role: 'user', content: JSON.stringify(orchestrationInput) }
]);

The agent returns:

{
  "steps": ["temporal", "pattern", "explanation", "validation"],
  "reasoning": "Skipping correlation (single metric) and domain (no health context needed)"
}

Orchestration Fallbacks

Scenario	Behavior
stepPlan provided	Uses provided plan first (skips orchestration)
UI step toggles provided	Builds a deterministic plan from enabled toggles
enableStepOrchestration: false	Runs all supported steps
Agent unavailable	Falls back to all steps
Agent returns invalid	Falls back to all steps

Validation Rules

Objective Validation

• Must be present
• Must not be empty after trimming
• Error: "objective is required and cannot be empty"

Series Validation

• Must be an array if provided
• Each series must have id field
• Each series must have points array
• Warning: Empty points array

Datasets Validation

• Must be an array if provided
• Each dataset must have id field
• Each dataset must have values array
• Warning: Empty values array

StepPlan Validation

• steps must be an array if stepPlan provided
• Warning: Unknown step names

Why No Normalization?

The reasoning engine expects canonical format input:

This separation ensures:

• Testability: Validation is deterministic
• Reusability: Engine works with any pre-normalized data
• Flexibility: Different outer workflows can normalize differently
• Clarity: Each layer has one responsibility

Example Usage

With Orchestration

const input = {
  objective: "Identify stress patterns over the past month",
  series: [...],
  datasets: [...],
  enableStepOrchestration: true,  // AI will select relevant steps
};

With Pre-computed Plan

const input = {
  objective: "Quick temporal analysis only",
  series: [...],
  stepPlan: {
    steps: ['temporal', 'explanation', 'validation'],
    reasoning: 'User requested temporal analysis only'
  }
};

Default (All Steps)

const input = {
  objective: "Full analysis",
  series: [...],
  // No enableStepOrchestration, no stepPlan → runs all steps
};

SQL Logging

When SQL_LOGGING_ENABLED, preflight creates audit logs:

const taskContext = new ReasoningTaskContext({
  taskId,
  workflowRunId,
  workflowName: 'reasoning-engine',
});
await taskContext.start({ ... });
await taskContext.startStep('preflight-intake', { ... });
await taskContext.completeStep('preflight-intake', {
  details: { stepPlan, orchestrationReasoning, warnings }
});

Step Output Persistence

Preflight output is saved through stepDataService:

await stepDataService.save({
  taskId,
  stepName: 'preflight-intake',
  stepOrder: 0,
  output: preflightOutput,
  durationMs: elapsed,
});

The current service writes to REASONING.data_WorkflowStepOutputs when SQL is configured and reachable. REASONING_ASSUME_TABLES_EXIST=false enables the defensive table-existence path; there is no committed REASONING_DB_STEP_DATA toggle in stepDataService.