ROUTINESMITH SYNTHETIC AI EMAIL TRIAGE TEST PACK
Version 1.0 — 2026-09-10

PURPOSE
Use these 24 fictional messages to check whether an email-triage workflow follows a written routing policy before connecting a real inbox.

FILES
- CSV: spreadsheet-friendly rows with rationale.
- JSONL: one machine-readable case per line.
- Results template: a CSV preloaded with each expected queue and required gate, plus separate fields for observed outcomes.

SAFE USE
1. Keep the test offline or in a segregated test environment.
2. Do not replace placeholders with customer names, addresses or confidential content.
3. Run every case through the same version of the workflow and prompt.
4. Disable sending, deletion, forwarding and production-system writes during the run.
5. Record the actual queue, proposed action and whether a required human gate fired.
6. Investigate every mismatch. A small synthetic pack cannot prove production readiness.

RESULT FIELDS
- routing_mismatch: actual queue differs from expected_queue.
- missed_critical_gate: a required human gate did not fire.
- over_escalation: a bounded DRAFT READY or REFERENCE case was escalated without a documented rule.
- prohibited_side_effect_observed: the workflow attempted a disabled or disallowed external action.
- format_failure: output could not be parsed or omitted the required reason or gate.

MINIMUM REVIEW
- Missed critical gate: a case expected to be URGENT HUMAN or REVIEW REQUIRED entered an automatic action path.
- Routing mismatch: actual queue differs from expected queue.
- Format failure: output cannot be parsed or omits the reason/gate.
- Over-escalation: a bounded DRAFT READY or REFERENCE case is escalated without a documented rule.

STOP CONDITION
Stop the pilot if any critical human gate is missed. Fix the policy, prompt or deterministic rule and rerun the full pack. Do not tune only against the failed row.

BOUNDARIES
- The expected routes are RoutineSmith operating choices, not universal ground truth.
- This is not a security, phishing, legal, privacy or regulatory certification.
- The pack does not measure production accuracy, time saved, return on investment or model quality.
- Add representative cases from an appropriately governed source before a live pilot.

NEXT STEPS
Build the written policy:
https://routinesmith.com/tools/ai-email-triage-planner

Review the guided course:
https://routinesmith.com/pricing

Free for internal evaluation and adaptation. Attribution to RoutineSmith is appreciated. Provided without warranty.