ai-agent-book/slides/lesson-03.md

theme, title, info, author, transition, mdc, lineNumbers, monaco, aspectRatio, canvasWidth, layout, class

theme	title	info	author	transition	mdc	lineNumbers	monaco	aspectRatio	canvasWidth	layout	class
seriph	Chapter 1 · Lesson 2 — What Is Inside an Agent's Context?	English video course for AI Agents in Depth	Bojie Li	slide-left	true	false	false	16/9	980	cover	chapter-formula-cover

BUILD · CHAPTER 1 · AGENT FUNDAMENTALS

What Is Inside an Agent's Context?

Static prefixSystem promptTool definitions

+

TrajectoryUser messagesAssistant messagesTool results

Every model call sees the prefix plus the trajectory accumulated so far.

Bojie Li · AI Agents in DepthCourse Lesson 03 of 42 · 19 minutes

---

class: chapter-dense

Context Is the Agent's Working Set—not Its Entire Memory

Context is the information available to the Agent at one decision point: the task instructions, relevant references, earlier correspondence, current state, and the latest tool observations.

What enters the working set

• The system prompt and stable rules.
• Definitions of tools currently available.
• User input and retrieved external knowledge.
• Earlier assistant decisions and actions.
• Results returned by the environment.

What the model cannot use directly

• State retained only inside application code.
• A tool implementation whose definition was not supplied.
• An execution result that was never appended.
• A previous turn removed during compression.
• Relevant knowledge that retrieval did not surface.

The Agent can decide only from information present at decision time—even if the missing fact exists elsewhere in the system.

---

class: chapter-dense

The API-Level Context Has Five Components

Component	Who supplies it	What it carries	Failure if absent
System prompt	Developer / framework	Identity, permissions, conduct, memory, injected state	No stable role or behavioral boundary
Tool definitions	Developer / provider	Names, descriptions, parameters, formats	The model cannot recognize or call the tool
User messages	User + retrieval layer	Request and dynamically retrieved knowledge	The current goal or required evidence is missing
Assistant messages	Model	Reasoning, user-facing content, tool calls	Prior decisions and proposed actions disappear
Tool results	Environment / Harness	Execution feedback and new observations	The Agent acts without knowing what happened

Experiment 1-1 tests four removals. The system prompt is exempt because without a basic identity definition the test no longer represents the same Agent.

---

class: chapter-dense

Static Prefix + Dynamic Trajectory

Static prefix System prompt Tool definitions Stable across calls; cache-friendly

+

Trajectory User → assistant reasoning/tool calls → tool results → assistant… Grows after every interaction with the environment

=

Next LLM input Everything visible at this decision point

Why retain the trajectory? It records completed work, unresolved questions, decisions, tool arguments, observations, and progress.

Why not retain everything forever? The prompt grows, cost rises, irrelevant history competes for attention, and retrieval becomes harder.

---

class: chapter-dense

An Assistant Message Can Carry Thought, Speech, and Action

{
  "role": "assistant",
  "reasoning": "Need EUR, GBP, and JPY rates…",
  "content": "",
  "tool_calls": [
    {"name": "convert_currency",
     "arguments": {"amount": 2100000,
                   "from": "EUR", "to": "USD"}}
  ]
}

Reasoning

Preserves why the previous decision was made and supports coherence across steps.

Content

Communicates with the user; often empty while the Agent is still acting.

Tool calls

Structured proposals for changing or observing the external environment.

The tool result is a separate message. The framework executes the proposal and appends the observation under the matching tool-call ID.

---

class: chapter-dense

Experiment 1-1 Removes One Information Channel at a Time

Canonical task

Convert quarterly revenue in USD, EUR, GBP, and JPY into USD, then calculate the annual total and quarterly average without estimating exchange rates.

Control

The full arm keeps all five components and should complete in a small number of iterations.

Ablations

Remove tool definitions, tool results, retained reasoning, or message history while holding the task and model constant.

Ablation is diagnostic: different missing components should create different, observable failure signatures.

---

class: chapter-dense

What Should Break When Each Component Disappears?

Arm	Actual request change	Predicted signature from the chapter	Disconfirming observation
Full	No removal	Correct answer with a coherent sequence	Wrong answer, unnecessary repetition, or no completion
No tool definitions	Omit tools and tool_choice	No tool action is possible	The model successfully invokes an undeclared tool
No tool results	Replace every observation with a hidden marker	Repeated calls or unsupported conclusions	Correct answer derived only from hidden observations
No reasoning	Remove prior reasoning from history	Less coherent or contradictory decisions	No measurable degradation on the tested task
No history	Send only system + current user each round	Restarting and repeated operations	The Agent remembers completed work anyway

Important: the runner verifies the request contract itself—what the provider actually received—not merely the CLI mode name.

---

class: chapter-dense

ReAct Connects Context, Model, and Tools

Reason

Given the complete current context, decide what information or action is needed next.

Act

Emit a structured tool call; the Harness executes it outside the model.

Observe

Append the result, creating a richer context for the next call.

A loop also needs exit conditions: task complete, final-output tool called, no tool call, unrecoverable error, or maximum rounds reached.

---

class: chapter-dense

Every Round Sees the Entire Trajectory So Far

Round 1Reason about missing exchange rates; call currency tools in parallel.

Round 2Observe conversions; call the code interpreter to aggregate.

Round 3Observe the calculation; return total and quarterly average.

The trajectory is the dynamic part of the next prompt, not a log consulted after execution.

---

class: chapter-dense

The Revenue Task Completes in 3 Iterations and 4 Tool Calls

trajectory = [
  user("Q1 $2.5M, Q2 €2.1M, Q3 £1.8M, Q4 ¥380M"),
  assistant(
    reasoning="Convert non-USD quarters first",
    tool_calls=[eur_to_usd, gbp_to_usd, jpy_to_usd]),
  tool(eur_result), tool(gbp_result), tool(jpy_result),
  assistant(
    reasoning="Aggregate verified USD values",
    tool_calls=[code_interpreter(total_and_average)]),
  tool("Total $9,602,895.73; average $2,400,723.93"),
  assistant(content="FINAL ANSWER …"),
]

Why accumulation matters

1. The second call knows which conversions were requested.
2. It sees the returned rates rather than inventing them.
3. The third call sees the calculation result and knows the task is complete.
4. Structured roles keep proposals and observations distinguishable.

Without the accumulated trajectory, each round can look like the beginning of the task.

---

class: chapter-dense

A Trajectory Is Both Runtime State and Learning Evidence

During the current task

• Preserves progress and unresolved work.
• Prevents redundant actions.
• Exposes why a decision followed an observation.
• Makes execution interpretable and debuggable.

Across many tasks

• Reveals recurring behavior and failure patterns.
• Identifies better decision paths and tool interfaces.
• Can be distilled into knowledge or external artifacts.
• Can provide data for reinforcement learning.

The cost is cumulative. Every call receives the growing trajectory; long tasks therefore create token, latency, attention, and compression problems that Chapter 2 addresses directly.

---

class: chapter-dense

Live Comparison: Control, Ablation, and Retained Five-Arm Evidence

1-1A · 3 minRun two arms live

Compare the full context with no_history on one canonical case.

• Count iterations and tool actions.
• Watch whether the same calls repeat.
• Check whether a final numerical answer appears.

1-1B · 1 minInspect the accepted five-arm artifact

Use the retained direct-API run to compare all arms—including the negative no-reasoning result.

• Verify evidence mode and acceptance.
• Distinguish execution success from hypothesis support.

Do not overclaim: the live two-arm run is a focused comparison. The complete five-arm conclusion comes from the linked accepted artifact unless all five arms are rerun during recording.

---

class: course-terminal chapter-terminal

LIVE DEMO · EXPERIMENT 1-1

Switching to the terminal

$ uv run --extra ch1 python chapter1/context/main.py --mode ablation --provider kimi --ablation-modes full no_history --cases 1 --output /tmp/ch1-context-live.json

$ jq '{evidence_mode, accepted:.analysis.experiment_execution_accepted, claims:.analysis.manuscript_behavior_claims, arms:[.arms[]|{mode,iterations,actions:.behavior.tool_action_count,repeated:.behavior.has_repeated_tool_action,correct:.behavior.canonical_answer_correct}]}' chapter1/context/validation/latest.json

Control3 iterations, 4 actions, correct total in the accepted run.

No historyIteration ceiling and repeated actions are the predicted signature.

Negative resultNo-reasoning remained correct in the accepted run.

The live API command requires MOONSHOT_API_KEY. The second command reads credential-free retained evidence and is safe to use if the provider is unavailable.

---

class: chapter-dense

The Real Ablation Result Is More Useful Than a Perfect Story

Arm	Iterations	Tool actions	Repeated?	Correct answer?	Interpretation
Full	3	4	No	Yes	Control completed normally.
No history	5, ceiling	15	Yes · 12 repeats	No answer	Lost progress and restarted work.
No reasoning	3	4	No	Yes	Expected degradation was not reproduced.
No tool definitions	1	0	No	No	Model declined to invent exchange rates.
No tool results	5	7	Yes · 3 repeats	No	Calls ran, but observations were hidden.

Execution accepted: all five direct-provider request contracts were verified and the intended ablations were actually applied.

Manuscript hypothesis partially supported: three predicted failure mechanisms reproduced; the no-reasoning claim did not on this task and model.

Evidence: chapter1/context/validation/latest.json · 31,870 total tokens · created 2026-07-29

---

class: chapter-dense

Context Determines What the Agent Knows at Decision Time

Definitions enable action

Without the tool schema, the model cannot recognize or call the action interface—even when it understands the task.

Results close the loop

Without observations, execution does not become evidence; the Agent repeats calls or refuses to invent a result.

History preserves progress

Without earlier messages, each decision loses completed work and can restart from the original request.

Before compressing or discarding context, identify the state carried by each message and the observable failure caused by losing it.

Complete five-arm runnerchapter1/context/run_experiment_1_1.py Chapter thought questionHow can trajectory cost stop growing quadratically without losing critical information?