01 AI Agent Foundations: Model → Agent → Tools → UX → Product → User

What you'll learn

What you'll learn:

• Foundation model (the AI brain)
• Agent (the decision-making layer)
• Tools (capabilities the agent can invoke)
• UX layer (how users interact)
• Product constraints (what's allowed)
• User needs (what matters)

Key insight:"Orchestration patterns operate at the agent layer. UX and Product layers constrain agent authority."

Kept from original - excellent foundation.

Model → Agent → Tools → UX → Product → User

(The Core Mental Model)**

Most confusion around AI systems happens because people collapse multiple layers into one.

They say:

“The AI did this.”

But in reality, five different layers were involved.

Understanding those layers — and separating them — is essential for:

• Design

• Auditing

• Safety

• Accountability

• UX clarity

This mental model is how you stop treating AI as “magic” and start treating it as a system

1. The Problem This Model Solves

When something goes wrong, teams often can’t answer:

• Where did the failure occur?

• Was it a model issue?

• An agent decision?

• A tool problem?

• A UX problem?

• A product rule?

Without a shared model, blame floats.

This framework gives you precision.

2. The Five Layers (From Deepest to Shallowest)

Layer 1 — The Model

What it is:
A statistical language model (LLM).

What it does:

• Predicts text

• Classifies inputs

• Generates responses

• Has no goals or memory on its own

What it does NOT do:

• Decide when to act

• Choose tools

• Understand product rules

• Remember user history reliably

The model is not an agent.
It does not know your product exists.

Layer 2 — The Agent

What it is:
The logic layer that uses the model.

What it does:

• Interprets user intent

• Decides what to do next

• Chooses tools

• Applies rules

• Manages state & memory

• Handles retries and fallbacks

This is where:

• Autonomy lives

• Decisions happen

• Coordination begins

Agents are actors.
Models are engines.

Layer 3 — Tools / Skills

What they are:
External capabilities the agent can call.

Examples:

• APIs

• Databases

• Search

• File systems

• Payment systems

• Calendars

• Browsers

Key rule:
The agent decides when and why.
Tools execute how.

Tools do not reason.
They enforce constraints.

Layer 4 — UX Layer

What it is:
The interface between system and human.

Includes:

• Text

• Voice

• Buttons

• Status indicators

• Confirmation steps

• Error messages

• Explanations

This layer decides:

• What is visible

• What is hidden

• When to ask permission

• How confidence is expressed

Most AI failures felt by users are UX failures, not model failures.

Layer 5 — The Product

What it is:
The business and policy wrapper around everything.

Includes:

• Permissions

• Feature availability

• Legal rules

• Pricing

• Risk thresholds

• Escalation policies

• Compliance constraints

The product defines:

• What the AI is allowed to do

• When humans must be involved

• Which failures are acceptable

The product is where accountability lives.

Layer 6 — The User

What they are:
Not passive recipients — but supervisors.

Users:

• Provide goals

• Validate outcomes

• Interrupt processes

• Correct mistakes

• Bear the consequences

A well-designed system treats users as collaborators, not passengers.

3. The Flow (End-to-End)

User
→ interacts via UX
→ which triggers agent logic
→ which uses the model
→ optionally calls tools
→ follows product rules
→ returns results to the user

Every audit, failure analysis, or design decision maps back to this flow.

4. Where Teams Commonly Get This Wrong

“The model made a mistake”

Most often it didn’t.

The agent:

• Chose the wrong tool

• Skipped confirmation

• Failed to recover

• Presented results poorly

“We’ll fix it with a better prompt”

Prompts don’t:

• Enforce permissions

• Provide UX clarity

• Guarantee consistency

• Handle edge cases

“Users will just understand”

They won’t — and they shouldn’t have to.

5. Why This Model Matters for Multi-Agent Systems

In multi-agent systems:

• You still have ONE UX

• ONE product

• ONE user

But now you have:

• Multiple agents

• Shared or competing tools

• Complex coordination

Without this mental model:

• Failures become invisible

• Responsibility becomes unclear

• Users lose trust quickly

This framework lets you:
- Audit each layer separately
- Locate failures precisely
- Design transparency intentionally
- Apply control at the right point

How This Toolkit Uses This Model

Throughout this toolkit:

• Scenario packs test UX + agent behavior

• Failure patterns map to agent/tool issues

• Checklists evaluate UX + control layers

• Architecture patterns operate at the agent layer

• Examples show how product rules constrain agents

This is not theoretical.
It’s operational.