---
theme: seriph
title: "Lesson 12 — Why Is One Retrieval Index Never Enough?"
info: "English video course for AI Agents in Depth"
author: Bojie Li
transition: slide-left
mdc: true
lineNumbers: false
monaco: false
aspectRatio: 16/9
canvasWidth: 980
layout: cover
class: cover
---
Build · Chapter 3 · Memory and Knowledge
# Why Is One Retrieval Index Never Enough?
Hybrid search, reranking, multimodality, and structured knowledge
Lesson 12 of 42 · 18 minutes · Hybrid Retrieval; Multimodal Extraction; Structured Indexing; Filesystem Paradigm
---
# Why this problem matters
Candidate fusion
Merge dense and sparse result sets.
Reranking
Use a stronger model only on a small candidate pool.
Knowledge shape
Trees, graphs, files, images, and tables preserve different structure.
---
# Three ideas to keep in view
Hybrid retrieval
Broad recall from multiple retrievers
Neural reranker
More precise ordering at higher per-item cost
Structured index
Represent hierarchy or relationships explicitly
---
# The book's visual model
Hybrid retrieval and reranking pipeline
---
# Flat chunks vs. Structured knowledge
Flat chunks
- Simple ingestion
- Local passage questions
- Weak global structure
Structured knowledge
- Hierarchies and graphs
- Multi-hop questions
- More governance cost
Choose an index for the questions—not for fashion.
---
# Fuse ranks before reranking
~~~python
dense = dense_index.search(query, k=20)
sparse = bm25.search(query, k=20)
pool = reciprocal_rank_fusion(dense, sparse)
answer_context = reranker.top(query, pool, k=5)
~~~
---
# Test the claim
3-63 min
Expose every retrieval stage
Observe: Dense candidates, sparse candidates, fusion, reranking, and final rank
3-82 min
Compare RAPTOR and GraphRAG
Observe: Questions favored by hierarchical summaries versus relationship graphs
Demo budget: 5 minutes · one contiguous terminal block
---
class: course-terminal
---
Live demo
# Switching to the terminal
~~~bash
$ uv run python chapter3/retrieval-pipeline/evaluate.py --query "XR-7003"
$ uv run python chapter3/structured-index/main.py demo
~~~
Run the command(s), narrate decisions, and point to the observation—not just the output.
---
# What the evidence supports
Finding 1
Hybrid retrieval improves recall because its component failures differ.
Finding 2
Reranking spends expensive reasoning on a small, diverse pool.
Finding 3
Structured indexes help only when queries need their encoded structure.
---
# Boundary → design rule
More stages increase latency, operational cost, and the number of components that can drift.
Add a retrieval stage only when an evaluation identifies the failure it corrects.
---
# Continue the experiment
---
layout: center
class: text-center
---
Pause and apply
# Your turn
Which query type would reveal that your flat index has lost document structure?
---
layout: center
class: text-center
---
Next · Lesson 13
Let the Agent decide whether another retrieval step is necessary.
→
