---
theme: seriph
title: "Lesson 06 — Why Can One Timestamp Make an Agent Slow?"
info: "English video course for AI Agents in Depth"
author: Bojie Li
transition: slide-left
mdc: true
lineNumbers: false
monaco: false
aspectRatio: 16/9
canvasWidth: 980
layout: cover
class: cover
---
Build · Chapter 2 · Context Engineering
# Why Can One Timestamp Make an Agent Slow?
Chat templates, attention, KV Cache, and stable prefixes
Lesson 06 of 42 · 19 minutes · KV Cache-Friendly Context Design; Chat Template; Prompt Cache
---
layout: center
class: text-center
---
The central question
Why can a harmless dynamic line near the top of the prompt invalidate most cached computation?
---
# Why this problem matters
Token stream
Message objects become one ordered sequence.
Prefix reuse
Matching early tokens reuse previous attention work.
Architecture
Dynamic content placement becomes a systems decision.
---
# Three ideas to keep in view
KV Cache
Reuses keys and values within inference
Prompt Cache
Reuses a stable prefix across API requests
Stable prefix
Instructions and tools that do not change turn to turn
---
# The book's visual model
KV Cache prefix reuse
---
# Cache-friendly vs. Cache-breaking
Cache-friendly
- Stable system prompt
- Stable tool order
- Dynamic state appended late
Cache-breaking
- Timestamp near the front
- Randomized tool order
- Reformatted history
One early mismatch invalidates everything that follows.
---
# Move changing state after the stable prefix
~~~python
static = [system_prompt, stable_tool_schemas]
trajectory = load_messages(session_id)
status = make_dynamic_status(now, progress)
messages = static + trajectory + [status]
~~~
---
# Test the claim
2-32 min
Compare context-management cache reports
Observe: Prefix hits, recomputation, repeated work, and estimated cost
2-22 min
Generate a small attention view
Observe: A token's weighted access to earlier tokens
Demo budget: 4 minutes · one contiguous terminal block
---
class: course-terminal
---
Live demo
# Switching to the terminal
~~~bash
$ uv run python chapter2/kv-cache/main.py --report
$ uv run python chapter2/attention_visualization/attention_cli.py --prompt "Explain attention in one sentence." --output attention.png
~~~
Run the command(s), narrate decisions, and point to the observation—not just the output.
---
# What the evidence supports
Finding 1
The API's message abstraction hides an ordered token prefix.
Finding 2
Cache efficiency depends on exact prefix stability.
Finding 3
Correct context management can improve quality and latency together.
---
layout: center
---
Where the claim stops
# Boundary condition
Cache-friendly does not mean never editing context; it means making edits deliberate and localized.
---
layout: center
---
Engineering takeaway
# Design rule
Place stable, frequently reused information first and dynamic information as late as its semantics allow.
---
# Continue the experiment
---
layout: center
class: text-center
---
Pause and apply
# Your turn
Which dynamic values in your system prompt silently destroy prefix reuse?
---
layout: center
class: text-center
---
Next · Lesson 07
Even a perfectly cached prompt can fail if its instructions are poorly organized.
→
