The Audit Plan

Your Audit Blueprint

This lesson walks through the complete Amazon Rufus audit plan. Download it, study it, and use it as a model for your own audits.The plan follows the same structure you'd use for any agent - adapted for Rufus's specific complexity.

How the Plan Was Built

1. Scope the agent. Rufus is a multi-agent e-commerce assistant. We identified four observable capabilities: search/planning, comparison/answering, cart/checkout, and price tracking/auto-buy.

2. Select scenario packs. Based on the agent type, we combined Core Usability, E-Commerce, Multi-Agent, and Long-Context packs.

3. Design tasks. Three tasks designed to trigger multiple agents and expose coordination failures: guided shopping journey, price tracking/auto-buy, and post-purchase handoff.

4. Map criteria. Each task maps to specific criteria from the 66-item checklist, with containment-sensitive items flagged for extra scrutiny.

What to Notice in the Plan

Each task includes:

core flow prompts
what to observe
criteria to evaluate
stress variations
failure patterns to watch for

This structure keeps the audit focused and makes findings easy to document.

The plan also flags safety-critical sections. Task 2 (auto-buy) involves irreversible financial actions - multiple containment-sensitive criteria apply.

Key Principle:

You don't need Amazon's internal architecture diagrams.

Audit from what you can observe. Routing, handoff, safety, and memory failures reveal themselves through realistic user journeys.

This plan is ready to execute. Run the tasks, document behavior, map to criteria, score categories, write findings. A completed audit walkthrough with real findings will be added once the audit is performed.

The three test tasks

Guided Shopping Journey - test Search → compare → recommend → cart
Price Tracking / Auto-Buy - test long-term commitments, safety, consent
Post-Purchase Handoff - test context survival across agent transitions

Downloads:

Amazon Rufus Audit Plan (PDF)

The Audit Plan

You must be logged in to access this course

Foundation

Agent Experience Assessment

Task Scenario Library

Failure Patterns

Real Audit Example

Downloads & Templates