Used for every agent - voice, chat, UI, or multimodal.

Use this pack for every audit, regardless of domain. If the agent performs actions, not just conversation - this pack is mandatory.

These reveal 70% of all UX problems.

These expose clarity, context, consistency, predictability, recoverability, error-handling - everything foundational to agent UX.

When to Use This Pack

summarizing or generating content
multi-step flows
commerce
search and retrieval
recommendations and planning
assistants or copilots
education
children
finance
legal / compliance
any agent embedded in a UI

What This Pack Tests

Clarity (Items 1–7): Entry-point clarity, expectation-setting, understandable outputs
Context & Relevance (Items 14–16): Maintaining constraints, responding appropriately to user intent
Conversation Quality (Items 17–25): Multi-step task management, clarifications, turn-taking
Error Recovery (Items 26–30): Handling mistakes, recovering gracefully
User Control (Items 31–37): Predictability, Undo / cancel / correct behaviors

How This Pack Integrates Into Your Audit

During task testing → Run all 3 Core Tasks because they cover the basics.

You write notes → Each issue = one notes block, even if the issue appeared multiple times.

You map issues → A single issue might map to multiple criteria (Clarity, Context, Predictability, Error Recovery, User Control) - This is expected.

You assign severity → Using the predefined severities for each checklist item.

Checklist → Mark PASSED / VIOLATED / N/A where appropriate.

Category score → Based on overall risk, not the number of failures.

CORE SCENARIOS (Base Usability Tests)

Run these for ANY agent — they expose the majority of real problems.

Scenario 1: Perform a Simple, Obvious Task

Prompt Examples:

“Summarize this text.”
“What’s the weather today?”
“Turn on the living room lights.”
“Rewrite this sentence to be clearer.”
“Find a shirt under £30.”

Look For:

Clear starting point (Item 5 — CRITICAL)
Accurate expectation-setting (Item 6 — MAJOR)
No hidden side effects
Output clarity and usefulness
Predictability (Item 45 — CRITICAL)

Pass Example

Agent responds clearly, stays within scope, and behaves predictably.

Failure Example

Agent starts with unexpected behavior, like overwriting text or switching modes.

Scenario 2: Perform a Multi-Step Task

This is where most agents fail.

Prompt Examples:

“Compare these 3 products and recommend one.”
“Plan a 2-day trip with a budget of £200.”
“Rewrite this paragraph, then turn it into bullet points.”
“Find a recipe and add missing ingredients to my shopping list.”

Look For:

Context retention (Item 14 — MAJOR)
Multi-turn consistency (Items 17–25)
Following constraints (budget, style, preferences)
Clarifying questions when needed
No silent context loss or contradictions

Pass Example

Agent keeps a stable memory of the task and checks understanding clearly.

Failure Example

Agent forgets constraints, contradicts itself, or resets behavior mid-task.

Scenario 3: Recovery or Correction Task

This tests error handling - the #1 UX failure point in agents.

Prompt Examples:

“No, I meant the blue one.”
“Undo that.”
“Change the date to next Friday.”
“That’s not what I meant.”
“Cancel that action.”

Look For:

Ability to recover without restarting (Items 26–30)
Correcting misinterpretations
Clear confirmations
Maintaining original context
User control (Items 31–37)

Pass Example

Agent handles correction smoothly and maintains context.

Failure Example

Agent forces entire task restart after a small correction.

STRESS SCENARIOS

These push the agent harder and expose deeper consistency problems.

Scenario 4: Constraint Stress Test

Prompt Examples:

“Find me a jacket under £80.”
“Actually make it waterproof.”
“Actually make it red.”
“Keep it under £80.”

Look For:

Constraint stacking
Respecting earlier limits
No silent overrides
Context accuracy

Failure Example

Agent violates a constraint (e.g., price jumps).

Scenario 5: Mode Switching Stress Test (Multimodal)

Prompt Examples:

Start in UI/chat → Continue in voice → Return to UI

Look For:

State continuity
No context reset
Same agent behavior across modes

Failure Example

Voice interaction changes the underlying task or resets it.

EDGE CASES (Optional)

These push the agent harder and expose deeper consistency problems.

Scenario 6: Ambiguity Handling

Prompt Examples:

“Book me a table for four.”

Look For:

Clarifying questions
Avoiding unsafe assumptions
Maintaining polite, grounded tone

Failure Example

Agent: “Okay, booked!” without asking date/time.

Scenario 7: Content Overwrite Awareness

Prompt Examples:

“Summarize this paragraph.”

Look For:

Clear warning before overwriting
Preview option
Undo support

Failure Example

Silent overwrite without confirmation.

OPTIONAL VOICE VERSIONS

Prompt Examples:

“Turn on the lights. No, only the kitchen.”
“Cancel that.”
“Change that to tomorrow instead.”

Voice Specific Risks:

Not hearing correction
Overconfidence
No confirmation
Hard to undo

Key Takeaway

Core Usability Tasks reveal whether the agent is:

clear
predictable
consistent
context-aware
safe
recoverable

If an agent fails here, nothing else matters.

01. Core Usability Tasks

You must be logged in to access this course

Foundation

Agent Experience Assessment

Task Scenario Library

Failure Patterns

Real Audit Example

Downloads & Templates