Voice vs Conversational vs Multimodal Agents

Same Failure. Different Stakes.

AI agents behave differently depending on how users interact with them. The same issue can be minor in one modality and critical in another.Before you audit, identify the modality - it changes what you test and how you interpret failures.

Text/Chat Agents

Examples: ChatGPT, Claude, customer service bots, Notion AI, copilotsStrengths: Users can re-read, copy, reference earlier messagesRisks: Verbosity, context loss, overconfidence in wrong answersKey test focus:

Context retention across turns
Conciseness (not wall-of-text)
Correction handling ("No, I meant X")
Clarity of explanations

Voice-Only Agents

Voice agents must be conservative. Overconfidence is far more dangerous without a screen. The same issue that's MAJOR in text may be CRITICAL in voice.

Examples: Alexa (no screen), Siri, phone banking, drive-thru orderingStrengths: Hands-free, natural interactionRisks: No "back button," mishearing, no visual confirmation

Key test focus:

Mishearing recovery
Verbal confirmation before actions
Ability to interrupt (barge-in)
Conservative behavior when uncertain

Multimodal Agents

Examples: Alexa Echo Show, Canva, Figma AI, apps with chat + UIStrengths: Best of both — speak naturally, see detailsRisks: Mode confusion, inconsistent entry points, hidden side effects

Key test focus:

Voice/screen consistency (same information in both)
Context preserved when switching modes
Action visibility (can users see what changed?)
Multiple entry points behave the same

Issue

Forgets context
No undo
Doesn't confirm

Severity Impact

MAJOR in text → CRITICAL in voice
MAJOR in text → CRITICAL in voice
MINOR in text → CRITICAL in voice

Which Packs to Use by Modality?

Always start with Pack 01 (Core Usability) regardless of modality.

Text/Chat Agents: Pack 01, 02, 03Voice Agents: 01, 02, 03 + Pack 05 (Voice)Multimodal Agents: 01, 02, 03 + Pack 05 + Pack 06 (Multimodal)Multi-agent: Add Pack 07 to any of the above

Voice vs Conversational vs Multimodal

You must be logged in to access this course

Foundation

Agent Experience Assessment

Task Scenario Library

Failure Patterns

Real Audit Example

Downloads & Templates