---
theme: seriph
title: "Lesson 05 — What Does the Model Actually See?"
info: "English video course for AI Agents in Depth"
author: Bojie Li
transition: slide-left
mdc: true
lineNumbers: false
monaco: false
aspectRatio: 16/9
canvasWidth: 980
layout: cover
class: cover
---
Build · Chapter 2 · Context Engineering
# What Does the Model Actually See?
Messages, tool calls, and the Agent core loop
Lesson 05 of 42 · 18 minutes · Context: The Ceiling of Agent Capability; API-Level Context Structure
---
Build · Chapter 2 · Context Engineering
# Problems this chapter will solve
Lesson 05
What Does the Model Actually See?
Lesson 06
Why Can One Timestamp Make an Agent Slow?
Lesson 07
Why Do Better Prompts Need Structure, Not More Rules?
Lesson 08
How Can an Agent Know What It Needs to Learn?
Lesson 09
How Can an Agent Stay Oriented in a Long Task?
---
# Why this problem matters
Roles
System, user, assistant, and tool messages have different semantics.
Ordering
A tool result must follow the tool call it answers.
Composition
Every call rebuilds a view from static and dynamic context.
---
# Three ideas to keep in view
Context is a list
Messages—not an abstract cloud of memory
Tool call
An assistant message proposing a structured action
Tool result
A new observation appended to the trajectory
---
# The book's visual model
Message sequence for two tool calls
---
# Single turn vs. Agent loop
Single turn
- One request
- One response
- No environmental feedback
Agent loop
- Repeated requests
- Tool calls and results
- Growing trajectory
The API structure is the runtime state machine.
---
# Context at the API boundary
~~~python
messages = [{"role": "user", "content": task}]
while True:
reply = client.responses.create(messages=messages, tools=tools)
messages.append(reply)
if reply.final: break
messages.append(run_tool(reply.tool_call))
~~~
---
# Test the claim
2-13 min
Run local tool calling and watch messages grow
Observe: Assistant tool call, tool result, and final answer
Demo budget: 3 minutes · one contiguous terminal block
---
class: course-terminal
---
Live demo
# Switching to the terminal
~~~bash
$ uv run python chapter2/local_llm_serving/main.py --backend ollama --mode single --task "What is the weather in Tokyo?"
~~~
Run the command(s), narrate decisions, and point to the observation—not just the output.
---
# What the evidence supports
Finding 1
The model only knows a tool exists because its schema is in context.
Finding 2
Tool results are observations, not hidden side effects.
Finding 3
A malformed message sequence changes the task state the model perceives.
---
layout: center
---
Where the claim stops
# Boundary condition
Framework abstractions are convenient, but debugging requires inspecting raw API messages.
---
layout: center
---
Engineering takeaway
# Design rule
Log the exact message list and tool schemas for every reproducible Agent failure.
---
# Continue the experiment
---
layout: center
class: text-center
---
Pause and apply
# Your turn
Which part of your Agent state exists outside the message list, and how is it reintroduced?
---
layout: center
class: text-center
---
Next · Lesson 06
Rearranging that list can change both latency and cost.
→