Understanding Severity Levels

Some Failures Can Wait. Others Can't.

Not all problems are equal. A confusing label is not the same as an agent that takes irreversible action without permission.Severity levels help you prioritize: what must be fixed now vs. what can wait.

The Three Severity Levels

CRITICAL: Prevents safe or effective use of the agent

MAJOR: Significant problems, but system still functions

MINOR: Noticeable issues with minimal real impact

CRITICAL: Must Fix

The agent fails at something fundamental. Users cannot complete tasks safely or effectively.Signs:User can't complete core tasksAgent acts without permissionNo way to stop, undo, or recoverTrust is brokenSafety risk existsExamples:Agent takes action before user confirmsNo way to cancel mid-taskAgent exposes sensitive dataUser can't tell what agent will do nextNo escalation to human supportResponse: Fix before launch or next release. Non-negotiable.

MAJOR: Should Fix

The agent works, but something significant is wrong. Users struggle, get confused, or lose confidence.Signs:Friction that slows users downConfusion about what happenedInconsistent behaviorMissing context or memoryPoor error recoveryExamples:Agent forgets earlier contextError messages don't help user recoverResponses too long or too short for contextAgent doesn't explain its limitationsUnclear what agent can/cannot doResponse: Fix in near-term roadmap. Don't ignore.

MINOR: Nice to Fix

Small issues. Users notice but aren't blocked or harmed.Signs:Polish problemsSlight awkwardnessMissed optimizationMinor inconsistencyExamples:Tone slightly off for contextCould be more conciseSuggestions not personalizedMinor UI label unclearResponse: Fix when time allows. Don't derail priorities.

How Severity Affects Findings

When a finding maps to multiple criteria.

Always use the highest. One critical criterion makes the entire finding critical.

Critical issues cap scores. You can't score 5 stars with a critical failure.

Without severity

"We found 12 issues"
Team doesn't know where to start
Everything feels equal

With severity

"We found 2 critical, 6 major, 4 minor"
Team knows exactly what's urgent
Priorities are clear

Severity is Pre-Assigned in the Framework

You don't decide severity per audit. Each of the 66 criteria already has a severity level. When you mark a criterion as violated, its severity applies automatically.

Severity turns a list of problems into a roadmap.

Understanding Severity Levels

You must be logged in to access this course

Foundation

Agent Experience Assessment

Task Scenario Library

Failure Patterns

Real Audit Example

Downloads & Templates