Understanding QRG Generation Results
Overview
After submitting a request to generate Quick Reference Guides (QRGs) from an Emergency Operation Plan (EOP), you receive a result that describes what happened to every annex in the plan. Some annexes produce QRGs successfully, some are skipped because their content is not suitable, and some may fail due to processing issues.
This page explains how to read and interpret those results so you can validate the output, identify annexes that need attention, and understand what the AI pipeline produced at each stage. Everything you need to diagnose generation outcomes is covered here -- no other pages are required.
How the Generation Pipeline Works
Before diving into results, it helps to understand the stages every annex passes through. Each stage can succeed, skip, or fail independently:
An annex must clear content extraction, classification, group extraction, role mapping, and content generation. A failure at any stage either skips the annex entirely or produces a QRG with missing information (such as no role assignments).
Reading the Request-Level Summary
Every generation request has a top-level summary that tells you the overall outcome:
| Field | What It Tells You |
|---|---|
| Status | Whether the entire request completed, failed, or was cancelled |
| Total annexes processed | How many annexes the system attempted to process |
| Total QRGs generated | How many Quick Reference Guides were successfully produced |
| Duration | How long the entire generation run took |
| Error message | If the request failed at the workflow level, what went wrong |
Start here to get a quick sense of whether the run was successful. If the status is "Completed" and the QRG count matches your expectations, the run went well. If the count is lower than expected, drill into the annex-level details.
Annex Processing Statuses
Each annex in the request receives one of four statuses:
Generated
The annex was successfully processed through all pipeline stages, and one or more QRGs were produced. This is the ideal outcome.
Skipped
The annex was not processed because its content did not meet the minimum requirements for QRG generation. The system detected an issue before sending anything to the AI pipeline.
Failed
The annex entered the AI pipeline but encountered an error during processing. This typically indicates an unexpected issue at one of the pipeline stages (classification, extraction, mapping, or generation).
No Output
The AI processed the annex content but could not produce meaningful results. For example, the classification succeeded but the content generation stage returned empty output. The annex was not skipped (it had valid content) and did not error (no technical failure), but the AI determined it could not generate useful QRG content from the material.
Common Skip Reasons
When an annex is skipped, the result includes a reason explaining why. These are the most common patterns:
| Skip Reason | What It Means | What to Do |
|---|---|---|
| Form has no data | The annex form object exists but contains no fields or data properties | Verify the annex was saved correctly in the EOP system; it may be an empty placeholder |
| Form data is empty (0 fields) | The form structure exists but every field is blank | The annex needs actual content added before it can generate QRGs |
| Content is empty after cleaning | The form fields contained HTML formatting tags but no readable text | Replace formatting-only content with substantive text describing emergency procedures |
| Content too short (under 50 characters) | The extracted text was too brief to support meaningful QRG generation | Expand the annex content -- a few words or a sentence fragment is not enough to generate role-specific guidance |
Pattern to watch for: If many annexes are skipped with "empty" or "too short" reasons, the EOP may contain placeholder or template annexes that have not been filled in yet. This is normal for plans that are still under development.
Understanding Each Pipeline Stage
When an annex is not skipped but produces unexpected results, understanding what happens at each stage helps you pinpoint where things went wrong.
Content extraction
The system reads the annex form data, strips HTML formatting, and extracts the plain text content. If the resulting text is empty or below the minimum length threshold, the annex is skipped before reaching the AI.
What can go wrong: Annexes that contain only images, embedded documents, or rich formatting without underlying text will produce empty content after extraction.
Category classification
The AI reads the extracted content and assigns an emergency category (such as "Fire Safety" or "Lockdown") and a type (FUNCTIONAL or EMERGENCY).
What can go wrong: If the annex content is ambiguous or describes multiple unrelated emergencies, the AI may misclassify it. This affects the QRG name, type label, and the framing of the generated content. If no category can be determined at all, the annex fails at this stage.
Group extraction
The AI identifies all audience groups mentioned in the annex (such as "Teachers", "Administrators", "Bus Drivers").
What can go wrong: If the annex does not mention any specific groups or roles, the extraction returns an empty list. When this happens, the system may still generate a QRG, but it will not be targeted to any specific audience. Alternatively, the AI may extract overly broad groups (like "everyone" or "all staff") that are difficult to map to specific roles.
Role mapping
Extracted group names are matched to the organization's standardized role tags.
What can go wrong: If the extracted group names do not match any available role tags, the mapping returns empty results. The QRG will still be generated, but the role assignment field will be empty, meaning the guide is not automatically delivered to any specific audience. This is the most common cause of "generated but unassigned" QRGs.
Content generation
The AI generates the BEFORE, DURING, and AFTER phase content for each role and scenario.
What can go wrong: If the source content is vague or lacks procedural detail, the generated content may be generic or thin. In rare cases, the generation stage may produce no output despite receiving valid inputs, resulting in a "No Output" status.
Interpreting Partial Success
It is common for a generation run to produce mixed results -- some annexes generate QRGs successfully while others are skipped or fail. This is expected and usually reflects differences in annex content quality rather than a system-wide problem.
How to evaluate partial success:
- Check the ratio. If 15 out of 20 annexes generated QRGs and 5 were skipped, that is a normal outcome for an EOP that includes some placeholder or incomplete annexes.
- Review skip reasons. If all skipped annexes share the same reason (such as "content too short"), the issue is content quality, not a system problem.
- Look for failed annexes. Skipped annexes are expected; failed annexes are not. If multiple annexes failed, something may be wrong with the content format or the pipeline configuration.
- Compare to the EOP structure. Not every annex in an EOP necessarily contains enough information to generate QRGs. Supporting annexes (like contact lists or resource inventories) will be skipped because they are not procedural content.
Understanding the Generated QRG Structure
Each successfully generated QRG contains the following components:
| Component | Description | Example |
|---|---|---|
| Name | A descriptive title derived from the annex and classification | "Fire Safety - Teachers" |
| Type | Either FUNCTIONAL or EMERGENCY | EMERGENCY |
| Functionality | For FUNCTIONAL type: the operational procedure described | "Evacuation Procedures" |
| Hazard | For EMERGENCY type: the specific hazard addressed | "Tornado" |
| Role assignments | The organizational roles this guide targets | Building Administrators, Teachers |
| Content blocks | The BEFORE, DURING, and AFTER phase guidance | Three sections of action-oriented instructions |
FUNCTIONAL vs. EMERGENCY types
- FUNCTIONAL QRGs describe general operational procedures (evacuation, shelter-in-place, lockout). They have a "functionality" label but no "hazard" label.
- EMERGENCY QRGs describe responses to specific hazards (tornado, fire, active shooter). They have a "hazard" label but no "functionality" label.
The type is determined by the Category Classification stage and affects how the QRG is organized and labeled in the system.
Content block structure
Each QRG contains up to three content blocks:
- BEFORE -- Actions to take in advance: planning, preparation, training, supplies, drills.
- DURING -- Immediate response actions: what to do right now when the emergency is happening.
- AFTER -- Recovery actions: reporting, follow-up, assessment, resumption of normal operations.
Each block contains role-specific, action-oriented instructions. The content is generated to be concise and directly usable by the target audience.
Expected Generation Timeline
Processing time depends on the size of the EOP and the number of annexes:
| EOP Size | Approximate Annexes | Expected Duration |
|---|---|---|
| Small | 5--10 annexes | 30 seconds to 2 minutes |
| Medium | 10--25 annexes | 2 to 5 minutes |
| Large | 25--50+ annexes | 5 to 15 minutes |
These are estimates. Actual duration depends on annex content length, the number of roles extracted per annex, and current system load. Annexes with rich, detailed content take longer because the AI processes more text at each stage.
If processing takes significantly longer than expected, it may indicate that some annexes contain unusually large amounts of content. This is not an error -- it simply means the AI is working through more material.
Troubleshooting Guide
No QRGs were generated (zero output)
If the entire request produces zero QRGs, check these causes in order:
- All annexes skipped. Review the skip reasons. If every annex has empty or insufficient content, the EOP needs more substantive text before QRGs can be generated.
- Classification failed across the board. If no annexes could be classified, the content may not describe recognizable emergency procedures. Verify that the annexes contain procedural safety content.
- No groups extracted. If no audience groups were found in any annex, the content may not mention specific roles or audiences. QRGs require role identification to produce targeted guidance.
- Request-level failure. Check the request status and error message. A workflow-level error (as opposed to annex-level errors) indicates a system issue rather than a content issue.
No roles were matched
If QRGs were generated but none have role assignments:
- The extracted group names did not match any of your organization's role tags.
- This can happen when the EOP uses informal language for roles (such as "front office people" instead of "Office Staff") or when the organization's role tag list is incomplete.
- The QRGs are still valid and contain useful content -- they just need manual role assignment.
High skip count
If a large percentage of annexes were skipped:
- Review the EOP to identify which annexes are placeholders or templates.
- Focus on annexes that have substantive procedural content.
- Consider whether the EOP is complete enough for QRG generation, or whether more content needs to be added first.
High error count
If multiple annexes show a "Failed" status rather than "Skipped":
- Check the error details for each failed annex to identify common patterns.
- Errors at the classification stage may indicate content that the AI cannot interpret (for example, content in an unsupported language or heavily corrupted text).
- Errors at the generation stage may indicate content that is too complex or contradictory for the AI to produce coherent guidance.
- If errors affect many annexes, contact your administrator -- there may be a configuration or infrastructure issue.
QRGs generated but content seems incorrect
If the QRGs were produced but the content does not match expectations:
- Wrong category. The classification stage may have misidentified the emergency type. If an annex about "Severe Weather" was classified as "Evacuation," all downstream content will be framed incorrectly.
- Wrong or missing roles. Check the group extraction and role mapping results. If the wrong groups were extracted, the content will be tailored to the wrong audience.
- Generic content. If the source annex content is vague, the generated QRG content will also be vague. More detailed source material produces more specific guidance.
Using Results to Improve Future Runs
QRG generation results are not just a pass/fail report -- they provide actionable feedback for improving your Emergency Operation Plans:
- Skipped annexes highlight content gaps. Each skip reason tells you exactly what the annex is missing.
- Empty role mappings reveal mismatches between how your EOP describes audiences and how your organization defines roles.
- Misclassified annexes suggest that content may need clearer structure or labeling to help the AI understand what type of emergency is being described.
- Thin generated content indicates that the source material lacks the procedural detail needed for rich, actionable QRGs.
By reviewing generation results after each run and addressing the identified issues, subsequent runs will produce more complete and accurate QRGs.