ai-agent-book 精选快照(<2MB 代码与文档,来自 github.com/bojieli/ai-agent-book)
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s

This commit is contained in:
2026-08-20 13:12:50 +00:00
commit b119135836
10275 changed files with 3284984 additions and 0 deletions
@@ -0,0 +1,271 @@
# Agentic RAG for User Memory / 面向用户记忆的 Agentic RAG
> Companion material for *AI Agents in Depth*, Chapter 3 — agentic multi-hop retrieval over conversation memory with offline demo and optional pipeline backend.
> 配套《深入理解 AI Agent》第 3 章——对话记忆上的 Agentic 多跳检索;含离线演示与可选检索流水线。
← [Chapter 3 index / 返回第 3 章目录](../README.md)
---
## English
### Canonical live campaign
`python campaign.py` launches the shared controlled Experiment 3-9/3-11
campaign over all 60 authoritative YAML cases. Experiment 3-9 uses fixed-round
windows and a live `search_user_memory` ReAct trajectory; raw retrieved chunks,
agent-generated searches, layer scores, and independent judge receipts are
retained. Canonical evidence is `validation/latest.json`.
### Learning objectives
1. Chunk long conversations for indexing
2. Integrate external retrieval pipelines (hybrid search)
3. Agentic RAG with tool-calling and ReAct
4. Evaluate memory with automatic LLM scoring
5. Optimize retrieval for conversation queries
6. Integrate evaluation frameworks across projects
### Architecture
```
User Memory Test Cases (60 cases, 3 layers)
→ Conversation Chunker (~20-round segments + overlap + enrichment)
→ External Retrieval Pipeline (port 4242) or local BM25
Dense + Sparse hybrid
→ Agentic RAG Agent (ReAct; search_memory / get_conversation_context / get_full_conversation)
→ LLM Evaluation (reward 01, pass/fail, reasoning)
```
### Key concepts
1. **Conversation chunking** — ~20 rounds, searchable, contextual, efficient
2. **Hybrid retrieval** (optional pipeline) — dense + BM25 + fusion; scalable
3. **Agentic RAG** — Reason → Act → Observe → iterate
4. **LLM evaluation** — integrates user-memory-evaluation style scoring (≥0.6 pass)
5. **Contextual enrichment** — metadata, neighbors, tags
### Prerequisites
- Python 3.12 with the root `ch3` extra
- **Port 4242 pipeline is OPTIONAL.** Default `retrieval_backend="auto"`: use pipeline if reachable, else **built-in local BM25** (offline).
- API keys only for LLM modes (`batch` / `interactive` / `demo`).
- **`--mode offline-demo` needs NO API key and NO port 4242.**
### Installation
```bash
# From the repository root: use the shared Chapter 3 environment
uv sync --locked --python 3.12 --extra ch3
# Activate it before changing directories:
# macOS/Linux:
source .venv/bin/activate
# Windows PowerShell: .venv\Scripts\Activate.ps1
# Windows cmd: .venv\Scripts\activate.bat
# pip fallback when uv is not installed:
# python -m pip install -e ".[ch3]"
cd chapter3/agentic-rag-for-user-memory
# Single-project compatibility path, still supported during migration:
# python -m pip install -r requirements.txt
cp env.example .env
# Edit API keys
```
### Retrieval backend
| value | behavior |
|-------|----------|
| `auto` | default — pipeline if up, else local BM25 |
| `local` | always offline BM25 |
| `pipeline` | always port 4242 |
Optional pipeline:
```bash
cd ../retrieval-pipeline
python api_server.py # http://localhost:4242
```
### Running
```bash
# Offline multi-hop vs naive recall (no API, no 4242)
python main.py --mode offline-demo
python offline_demo.py
python offline_demo.py --output results/offline_demo.json
python test_pipeline.py
python main.py
python main.py --mode demo
python main.py --mode batch --category layer1 --backend local
```
CLI: `--mode {interactive,batch,demo,offline-demo}`, `--category`, `--test-id`, `--query`, `--provider`, `--model`, `--index-mode {dense,sparse,hybrid}`, `--backend {auto,local,pipeline}`, `--top-k`, `--rounds-per-chunk`, `--store-path`, `--test-cases-dir`, `--output`, `--config`. See `python main.py --help` (Chinese).
### Offline demo results (reproducible)
On `layer2_01_multiple_vehicles` (Honda + Tesla across sessions), real BM25:
| metric | naive single-query | agentic multi-hop |
|--------|:------------------:|:-----------------:|
| retrieval queries issued | 1 | 5 |
| memory chunks retrieved | 3 | 5 |
| decisive-evidence recall | **50%** | **100%** |
| can fully disambiguate & answer | no | yes |
Naive is dominated by “schedule service” keywords and misses Honda confirmation (`FS-447291`). Agentic discovers the second vehicle, issues focused follow-ups, recovers evidence. Numbers from actual retrieval, not hard-coded.
### Interactive options
Load / view test cases; configure chunking/index/agent; evaluate single or by category; generate reports.
### Example code
```python
from config import Config
from evaluator import UserMemoryEvaluator
config = Config.from_env()
evaluator = UserMemoryEvaluator(config)
test_cases = evaluator.load_test_cases(category="layer1")
result = evaluator.evaluate_test_case("layer1_01_bank_account")
report = evaluator.generate_report("results/evaluation_report.txt")
```
### Config highlights
```python
config.chunking.rounds_per_chunk = 20
config.chunking.overlap_rounds = 2
config.index.mode = "hybrid"
config.index.enable_contextual = True
config.agent.max_search_results = 5
config.evaluation.max_iterations = 10
```
### Test layers
- **L1** simple retrieval — “What is my checking account number?”
- **L2** multi-conversation — “Which vehicle needs service first?”
- **L3** complex reasoning — “What urgent issues before my trip?”
### Components
`chunker.py`, `indexer.py`, `tools.py` (`search_memory`, `get_conversation_context`, `get_full_conversation` — full content), `agent.py` (ReAct), `evaluator.py`.
### Metrics / troubleshooting
Success rate, LLM reward, iterations, tool calls, latency, index time.
**Top-k:** pipeline uses `top_k` (candidates) and `rerank_top_k` (final).
**LLM eval missing:** need evaluator API + criteria.
**Pipeline down:** not fatal with `--backend auto`; force offline with `--backend local`.
### Related
`user-memory`, `user-memory-evaluation`, `agentic-rag`, `contextual-retrieval` (chapter3 paths).
### License
Educational curriculum materials.
---
## 中文
### 学习目标
1. 长对话分块索引
2. 对接外部混合检索流水线
3. 工具调用 + ReAct 的 Agentic RAG
4. LLM 自动打分评测记忆
5. 面向对话查询的检索优化
6. 跨项目评估框架集成
### 架构
用户记忆用例 → 对话分块(约 20 轮 + 重叠 + 上下文增强)→ 外部流水线(4242)或本地 BM25 → Agentic AgentReAct 记忆工具)→ LLM 评估。
### 关键概念
分块、混合检索、Agentic ReAct、自动 LLM 评测、上下文增强——与 English 节一致。
### 前置条件
Python 3.12 与根目录 `ch3` extra。**4242 流水线可选**;默认 `auto` 回退本地 BM25。仅 LLM 模式需 API Key。**`offline-demo` 无需 Key 与 4242。**
### 安装与后端
```bash
# 在仓库根目录使用统一的第 3 章环境
uv sync --locked --python 3.12 --extra ch3
# 切换目录前先激活环境:
# macOS/Linux
source .venv/bin/activate
# Windows PowerShell.venv\Scripts\Activate.ps1
# Windows cmd.venv\Scripts\activate.bat
# 未安装 uv 时可用 pip 兜底:
# python -m pip install -e ".[ch3]"
cd chapter3/agentic-rag-for-user-memory
# 迁移期间仍支持单项目兼容路径:
# python -m pip install -r requirements.txt
cp env.example .env
# 可选
cd ../retrieval-pipeline && python api_server.py
```
| 值 | 行为 |
|----|------|
| `auto` | 默认可达则用流水线,否则本地 BM25 |
| `local` | 始终离线 BM25 |
| `pipeline` | 始终 4242 |
### 运行
```bash
python main.py --mode offline-demo
python offline_demo.py
python offline_demo.py --output results/offline_demo.json
python test_pipeline.py
python main.py
python main.py --mode demo
python main.py --mode batch --category layer1 --backend local
```
CLI 标志见 English 节;`python main.py --help` 含中文说明。
### 离线演示结果
`layer2_01_multiple_vehicles` 上 naive 证据召回 **50%**、agentic **100%**(见 English 表)。
### 配置、用例层级、组件
`config.py` 分块/索引/Agent 参数;L1/L2/L3 用例;`chunker` / `indexer` / `tools` / `agent` / `evaluator`
### 故障排查
Top-k 需同时设 `top_k``rerank_top_k`;流水线不可达时用 `--backend auto/local`LLM 评测需有效 Key 与 `evaluation_criteria`
### 相关与许可
见同章 `user-memory``user-memory-evaluation``agentic-rag``contextual-retrieval`。教学用途。
---
## Notes / 说明
### OpenRouter 通用回退 / Universal OpenRouter fallback
If primary keys are absent and `OPENROUTER_API_KEY` is set, chat LLM routes through OpenRouter with automatic model mapping. See `env.example`.