How to Score the 66 Criteria

Observe First. The Spreadsheet Scores Automatically.

You don't score during testing. You observe, take notes, then score after.Scoring is simple: you mark what you observed, the spreadsheet calculates the rest.

How to Mark Criteria

Don't mark YES without evidence. Don't mark NO unless you actually tested for it.For each of the 66 criteria, you mark:

YES: You observed this issue during testing

NO: You tested for this and didn't observe it

N/A: Criterion doesn't apply to this agent

Scoring IS

Marking what you observed
Recording evidence
Noting failures AND successes
Judgment based on evidence

Scoring ISN'T

Grading AI intelligence
Counting bugs
Pass/fail on every item
Arbitrary opinion

Severity is Pre-Assigned

Each criterion already has a severity level. You don't decide - the framework does.

When you mark YES on a criterion, its pre-assigned severity applies automatically.

CRITICAL: Prevents safe or effective use

MAJOR: Significant problems, system still functions

MINOR: Noticeable but minimal impact

How Category Scores are Calculated

Each category gets a star rating (1-5) based on what you marked:

5 Stars: 0 issues OR 1 MINOR only

4 Stars: 2 MINOR, no MAJOR or CRITICAL

3 Stars: 1 MAJOR OR 3+ MINOR, no CRITICAL

2 Stars: 2+ MAJOR OR 1 CRITICAL

1 Star: 2+ CRITICAL OR complete task failure

The Spreadsheet Does the Math

You don't calculate scores manually.Audit tab → Mark YES/NO/N/A on criteriaScoring tab → Auto-calculates stars per categorySummary tab → Shows overall resultsYour job: observe and mark honestly. The spreadsheet handles the rest.

The Flow

Run tasks → take notesGroup notes into issuesMap issues to criteria (mark YES)Spreadsheet calculates category scoresReview and verify scores make senseYou provide the observations. The framework provides the structure.

How to Score the 66 Criteria

You must be logged in to access this course

Foundation

Agent Experience Assessment

Task Scenario Library

Failure Patterns

Real Audit Example

Downloads & Templates