Prompt Tuning Workflow

Overview

The Prompt Tuning Workflow is a deterministic 6-step pipeline that improves SDAC agent prompts based on user feedback. It reads feedback, analyzes patterns, generates a revised prompt, applies it with safety gates, logs the result for audit, and can mark consumed feedback as processed.

• Workflow ID: sdac-prompt-tuning
• Endpoint: POST /sdac/workflows/prompt-tuning/start

Input

{
  "targetAgentId": "sdac-coordinator-release",
  "daysBack": 30,
  "minFeedbackCount": 5,
  "triggeredBy": "admin@example.com",
  "selectedFeedbackIds": [101, 102, 103]
}

Field	Type	Default	Description
targetAgentId	string	required	ID of the agent whose prompt to revise
daysBack	number	30	Days of feedback to include (1-90)
minFeedbackCount	number	5	Minimum feedback count required to proceed
triggeredBy	string	required	User or system that triggered the run
applyPrompt	boolean	true	Whether to apply the proposed prompt or return a draft-only suggestion
unprocessedOnly	boolean	false	Whether to only analyze feedback that has not already been processed
markFeedbackProcessed	boolean	false	Whether to mark consumed feedback rows after the run
selectedFeedbackIds	number[]	optional	Explicit FeedbackSK rows to analyze instead of the full feedback window

Pipeline Steps

Step 1: Read Feedback

ID: sdac-prompt-tuning-read-feedback

Queries the feedback database for the target agent and calculates aggregate statistics. If selectedFeedbackIds is provided, the query is constrained to those exact feedback rows and minFeedbackCount should normally match the selected count.

Output includes:

• totalCount, avgRating, ratingDistribution, categoryBreakdown
• Up to 20 recent feedback items with user and response previews
• feedbackInsights -- deterministic, source-backed signals extracted from feedback comments, user requests, and response evidence
• gatedOut: true if feedback count is below minFeedbackCount (short-circuits remaining steps)

Step 2: Read Current Profile

ID: sdac-prompt-tuning-read-profile

Loads the target agent's current system prompt from Core.AgentProfiles.

Output includes:

• currentPrompt -- the active system prompt text
• currentProfileHash -- for change detection
• currentVersionNumber -- for version tracking

Skipped if Step 1 gated out.

Step 3: Analyze and Generate

ID: sdac-prompt-tuning-analyze

Calls the Prompt Tuner Agent with feedback data and current prompt. The agent analyzes feedback patterns and returns a proposed revision.

Output includes:

• promptProposed -- whether the agent recommends changes
• revisedPrompt -- the proposed new prompt (if applicable)
• confidenceScore -- 0.0 to 1.0 confidence in the revision
• rationale -- explanation of changes
• feedbackSummary -- concise summary of patterns found
• changesApplied -- list of specific modifications, expected to cite relevant FeedbackSK values

Skipped if Step 1 gated out.

Step 4: Apply Prompt

ID: sdac-prompt-tuning-apply

Applies the revised prompt to the target agent's profile, subject to 5 safety gates:

Gate	Condition	Result
Insufficient data	gatedOut === true	Skip -- not enough feedback
No proposal	promptProposed !== true	Skip -- LLM chose not to change
Low confidence	confidenceScore < 0.25	Skip -- below minimum threshold
Invalid content	Prompt < 50 chars or matches error patterns	Skip -- likely error message
No change	Revised prompt matches current prompt	Skip -- identical content

If all gates pass, the revised prompt is saved as a new version in Core.AgentProfiles (SCD Type 2 versioning).

Step 5: Log Tuning Run

ID: sdac-prompt-tuning-log

Logs the complete pipeline result to SDAC.log_PromptTuningRuns for audit.

Always executes -- even for gated-out or no-change runs, ensuring a complete audit trail.

Logged fields: target agent, feedback count, date range, current prompt, proposed prompt, rationale, confidence score, status (applied, no-change, or skipped), execution time.

Step 6: Mark Feedback Processed

ID: sdac-prompt-tuning-mark-feedback-processed

Marks the feedback rows consumed by the run when markFeedbackProcessed is true. Draft-only dashboard suggestions normally leave this false so reviewers can generate a candidate prompt without consuming the underlying feedback.

Output

{
  "success": true,
  "promptChanged": true,
  "gatedOut": false,
  "runId": 42,
  "executionTimeMs": 15234,
  "revisedPrompt": "New prompt text...",
  "rationale": "Improved clarity of fringe analysis instructions",
  "confidenceScore": 0.87,
  "changesApplied": ["instruction_clarity_improved", "context_expanded"],
  "feedbackStats": {
    "totalCount": 15,
    "avgRating": 3.8,
    "categoryBreakdown": { "accuracy": 5, "clarity": 8, "relevance": 2 }
  }
}

Admin API

Trigger Prompt Tuning

POST /admin/sdac/tune-prompt
Content-Type: application/json

{
  "targetAgentId": "sdac-coordinator-release",
  "daysBack": 30,
  "minFeedbackCount": 5,
  "triggeredBy": "admin@example.com",
  "selectedFeedbackIds": [101, 102, 103]
}

When the admin dashboard user selects feedback cards manually, it sends selectedFeedbackIds and sets minFeedbackCount to the number of selected rows. If no cards are selected, prompt tuning uses the normal daysBack feedback window.

List Tuning Runs

GET /admin/sdac/tune-prompt/runs

Returns the history of all tuning runs with status, confidence scores, and whether changes were applied.