Scenario YAML

Define what the actor is trying to accomplish — their first message, goals, success/failure criteria, and exit conditions.

A scenario file defines the conversation objective. The actor uses success/failure criteria to automatically determine when to stop.

Schema

scenario_id: string           # Unique identifier for this scenario
first_message: string         # The actor's opening message to the agent
max_turns: number             # Maximum number of back-and-forth turns
 
context_vars:                 # Optional variables available in prompts
  key: value
 
success_criteria:             # List of strings — any match = SUCCESS
  - string
 
failure_criteria:             # List of strings — any match = FAILURE
  - string
 
exit_conditions:              # List of strings — stop without success/failure
  - string

Example — Billing dispute

scenario_id: billing-dispute-001
first_message: >
  Hi, I need help with my invoice from last month.
  I was charged twice for the same order (#ORD-4892).
max_turns: 8
 
context_vars:
  order_id: "ORD-4892"
  amount: "$49.99"
 
success_criteria:
  - refund has been processed
  - will receive a refund within
  - ticket has been created
  - escalated to billing team
 
failure_criteria:
  - cannot help with billing
  - please contact support via email
  - I don't have access to account information
 
exit_conditions:
  - goodbye
  - is there anything else

At the end of each agent turn, KyroActor checks the agent's response against the criteria lists using a case-insensitive substring match. The first criterion that matches terminates the scenario with the corresponding outcome.

ℹInfo

Criteria are checked after each agent message, not each actor message. The actor always gets to respond once before criteria are evaluated.

Example — Password reset flow

scenario_id: password-reset-happy-path
first_message: I forgot my password and can't log in.
max_turns: 5
 
success_criteria:
  - reset email has been sent
  - check your email for a reset link
  - password reset instructions have been sent
 
failure_criteria:
  - unable to reset your password
  - please try again later

Schema

Example — Billing dispute

Criteria evaluation

Example — Password reset flow

On this page