ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）

chapter3/log-sanitization/README.md

@@ -0,0 +1,425 @@
+# Log Sanitization / 日志脱敏
+
+> Companion material for *AI Agents in Depth*, Chapter 3 — intelligent log sanitization that redacts secrets and PII while preserving debug value.  
+> 配套《深入理解 AI Agent》第 3 章——在保留调试信息的同时检测并脱敏日志中的敏感数据。
+
+← [Chapter 3 index / 返回第 3 章目录](../README.md)
+
+---
+
+## English
+
+### Overview
+
+Demonstrates detecting and sanitizing sensitive data in Agent logs and tool outputs. Two complementary engines:
+
+1. **Offline rule engine (`regex`, default)** — pure regex + validators (Luhn, ID checksum). **No Ollama, no network, no external framework.** Deterministic and fast; first line of defense before logs hit disk. Covers both **secrets** (API keys, cloud tokens, private keys, connection-string passwords) common in Agent scenarios and traditional **PII** (ID cards, phones, credit cards, emails, etc.).
+2. **Local LLM engine (`llm`)** — Ollama with a small local model (default `qwen3:0.6b`) for semantic Level 3 PII. Echoes the chapter point that small models can handle structured tasks, and also shows limits (e.g. descriptive prefixes instead of raw strings → replacement failure).
+
+> Quick demo (offline, no extra deps): `python main.py --demo` — before/after samples and category summary.
+
+### Categories covered by the offline rule engine
+
+`regex_sanitizer.py` processes by priority (higher wins on overlap); each becomes a labeled placeholder:
+
+| Category | Placeholder | Notes |
+| --- | --- | --- |
+| Private key / cert | `[REDACTED_PRIVATE_KEY]` | PEM private key blocks |
+| JWT | `[REDACTED_JWT]` | `eyJ...` three-part tokens |
+| URL credentials | `[REDACTED_URL_CRED]` | `scheme://user:PASSWORD@host` |
+| AWS access key | `[REDACTED_AWS_KEY]` | `AKIA...` |
+| GitHub / Slack / Google / OpenAI keys | `[REDACTED_*_TOKEN]` / `[REDACTED_API_KEY]` | `ghp_`, `xoxb-`, `AIza`, `sk-` |
+| Bearer token | `[REDACTED_BEARER_TOKEN]` | `Authorization: Bearer ...` |
+| Password / secret assignments | `[REDACTED_SECRET]` | `password=...`, `token: ...`, etc. |
+| Email | `[REDACTED_EMAIL]` | |
+| Credit card | `[REDACTED_CREDIT_CARD]` | Luhn-validated to cut false positives |
+| IBAN | `[REDACTED_IBAN]` | |
+| US SSN | `[REDACTED_SSN]` | |
+| National ID | `[REDACTED_ID_CARD]` | Mainland China 18-digit with checksum |
+| Phone | `[REDACTED_PHONE]` | Mainland China |
+| IP address | `[REDACTED_IP]` | IPv4 |
+
+### Level 3 PII categories (LLM engine)
+
+Highly sensitive items in the privacy architecture, including: SSN, credit cards, bank accounts, medical record numbers, diagnoses/treatment, prescriptions, driver’s license, passport, financial PINs, tax IDs, health insurance IDs, biometric data.
+
+### Features
+
+- **Offline rule engine:** regex + Luhn/ID checksum; keys/secrets + PII; no model/network
+- **Local LLM:** Ollama + small model (default `qwen3:0.6b`) for privacy-preserving PII detection
+- **Internal reasoning:** model thinking via `<think>` tags
+- **Streaming:** real-time thinking and detection progress
+- **Performance metrics:** TTFT, token counts, speeds
+- **Batch processing:** user-memory-evaluation Layer 3 cases
+- **Detailed metrics:** prefill / output time and tok/s for both phases
+
+### Installation
+
+#### 1. Install Ollama (LLM path only)
+
+> **OpenRouter fallback:** Default is local Ollama. If Ollama is unavailable and `OPENROUTER_API_KEY` is set, the Agent falls back to OpenRouter (default hosted model `openai/gpt-5.6-luna`). To force fallback: `export OLLAMA_HOST=http://127.0.0.1:1`.
+
+**macOS:**
+```bash
+brew install ollama
+ollama serve  # separate terminal
+```
+
+**Linux:**
+```bash
+curl -fsSL https://ollama.com/install.sh | sh
+systemctl start ollama
+```
+
+**Windows:** Download from [ollama.com](https://ollama.com/download/windows)
+
+> Ollama steps apply only to `--mode llm` or the LLM batch eval path. Offline rule engine (`--demo`, `--input`) needs only the Python stdlib.
+
+#### 2. Pull model
+```bash
+ollama pull qwen3:0.6b
+```
+
+~500MB disk; you may use `qwen3:1.7b` or `qwen3:4b` for higher accuracy.
+
+#### 3. Python deps
+```bash
+# From the repository root: use the shared Chapter 3 environment
+uv sync --locked --python 3.12 --extra ch3
+
+# Activate it before changing directories:
+# macOS/Linux:
+source .venv/bin/activate
+# Windows PowerShell: .\.venv\Scripts\Activate.ps1
+# Windows cmd: .venv\Scripts\activate.bat
+
+# pip fallback when uv is not installed:
+# python -m pip install -e ".[ch3]"
+
+cd chapter3/log-sanitization
+
+# Single-project compatibility path, still supported during migration:
+# python -m pip install -r requirements.txt
+```
+
+### Usage
+
+Full flags: `python main.py --help` (Chinese help text).
+
+#### Offline rule demo (recommended, no Ollama)
+```bash
+python main.py --demo
+```
+
+#### Sanitize a log file (offline)
+```bash
+python main.py --input app.log                 # writes app.log.sanitized
+python main.py --input app.log -o cleaned.log  # custom output
+```
+
+Rule engine alone on built-in samples:
+```bash
+python regex_sanitizer.py
+```
+
+#### Offline validation
+
+```bash
+# From the repository root; include dev tools for pytest.
+uv sync --locked --python 3.12 --extra ch3 --extra dev
+source .venv/bin/activate
+# Windows PowerShell: .\.venv\Scripts\Activate.ps1
+
+cd chapter3/log-sanitization
+python -m pytest tests
+python main.py --demo
+python regex_sanitizer.py
+```
+
+`tests/` contains offline regressions for sanitizer rules and LLM-output parsing. `test_loader.py` remains at the project root intentionally: despite its name, it is a support module imported by `main.py`, not a pytest file. The loader debug helper lives at `tests/manual/loader_debug.py`.
+
+#### Local LLM engine
+```bash
+python main.py --demo --mode llm
+python main.py --input app.log --mode llm --model qwen3:1.7b
+```
+
+#### Process all Layer 3 test cases (LLM batch path)
+
+Uses LLM only; needs Ollama and the chapter3 user-memory-evaluation framework:
+```bash
+python main.py
+```
+
+#### Specific test / limit
+```bash
+python main.py --test-id layer3_13_emergency_medical_cascade
+python main.py --limit 3
+```
+
+#### Model choice
+```bash
+python main.py --demo --mode llm --model qwen3:4b   # default qwen3:0.6b
+```
+
+### Output structure
+
+Under `output/`:
+
+```
+output/
+├── <test_id>_sanitized.txt    # Sanitized conversation text
+├── <test_id>_summary.json     # Summary of PII found and replaced
+├── performance_metrics.json   # Detailed performance metrics
+└── performance_summary.json   # Aggregated performance statistics
+```
+
+### Performance metrics
+
+**Timing:** Prefill (TTFT), Output Time, Total Time (ms)  
+**Tokens:** Input / Output counts; Prefill / Output speed (tok/s)  
+**Sanitization:** PII items found; replacements with `[REDACTED]`
+
+### Architecture
+
+1. **regex_sanitizer.py** — offline rule sanitizer  
+2. **samples.py** — offline demo samples  
+3. **config.py** — Ollama model and PII categories  
+4. **test_loader.py** — loads user-memory-evaluation cases (support module, not pytest)
+5. **agent.py** — LLM sanitization via Ollama  
+6. **metrics.py** — metrics collection  
+7. **main.py** — entry / orchestration  
+8. **tests/** — offline pytest regressions plus manual loader debug helper
+
+### How it works (LLM path)
+
+1. Load conversations from user-memory-evaluation  
+2. Send each to local Qwen3 with a Level 3 PII detection prompt  
+3. Replace detected values with `[REDACTED]`  
+4. Collect performance metrics  
+5. Write sanitized logs and summaries under `output/`  
+
+### Privacy
+
+- Default local Ollama path sends no data to external APIs  
+- OpenRouter fallback (if used) does leave the machine  
+- Sanitized logs use placeholders; handle any logged original PII securely  
+
+### Troubleshooting
+
+**Ollama not found:** install and `ollama serve`  
+**Model not found:** `ollama pull qwen3:0.6b`  
+**Evaluation framework not found:** expect `../user-memory-evaluation/` (or chapter3 path used by the loader)
+
+---
+
+## 中文
+
+### 概述
+
+演示如何从 Agent 的日志与工具输出中检测并脱敏敏感信息。提供**两种互补的脱敏引擎**：
+
+1. **离线规则引擎（regex，默认）** —— 纯正则表达式 + 校验算法（Luhn、身份证校验码），**无需 Ollama、无需网络、无需外部框架**，结果确定、速度快，适合作为日志落盘前的第一道防线。同时覆盖 Agent 场景中最常泄露的**密钥类**敏感信息（API Key、云厂商令牌、私钥、连接串口令）与传统 **PII**（身份证、手机号、信用卡、邮箱等）。
+2. **本地 LLM 引擎（llm）** —— 通过 Ollama 调用本地小模型（默认 `qwen3:0.6b`）语义识别 Level 3 PII。呼应本章「小模型也能胜任结构化任务」的论点，同时也暴露小模型的局限（例如可能返回带描述前缀的值而非原始字符串，导致回填失败）。
+
+> 想快速看效果，直接运行 `python main.py --demo`（离线，无需任何依赖）即可看到多个代表性样本的 before/after 对比与脱敏类别汇总。
+
+### 离线规则引擎覆盖的敏感信息类别
+
+`regex_sanitizer.py` 按优先级处理以下类别（重叠时高优先级规则胜出），每类替换为带标签的占位符：
+
+| 类别 | 占位符 | 说明 |
+| --- | --- | --- |
+| 私钥 / 证书 | `[REDACTED_PRIVATE_KEY]` | PEM 私钥块 |
+| JWT | `[REDACTED_JWT]` | `eyJ...` 三段式令牌 |
+| 连接串凭据 | `[REDACTED_URL_CRED]` | `scheme://user:PASSWORD@host` |
+| AWS 访问密钥 | `[REDACTED_AWS_KEY]` | `AKIA...` |
+| GitHub / Slack / Google / OpenAI 密钥 | `[REDACTED_*_TOKEN]` / `[REDACTED_API_KEY]` | `ghp_`、`xoxb-`、`AIza`、`sk-` |
+| Bearer 令牌 | `[REDACTED_BEARER_TOKEN]` | `Authorization: Bearer ...` |
+| 口令 / 密钥赋值 | `[REDACTED_SECRET]` | `password=...`、`token: ...` 等 |
+| 邮箱 | `[REDACTED_EMAIL]` | |
+| 信用卡号 | `[REDACTED_CREDIT_CARD]` | 通过 Luhn 校验，降低误报 |
+| IBAN | `[REDACTED_IBAN]` | 国际银行账号 |
+| 美国社保号 | `[REDACTED_SSN]` | |
+| 身份证号 | `[REDACTED_ID_CARD]` | 中国大陆 18 位，含校验码验证 |
+| 手机号 | `[REDACTED_PHONE]` | 中国大陆 |
+| IP 地址 | `[REDACTED_IP]` | IPv4 |
+
+### Level 3 PII 类别（LLM 引擎）
+
+隐私架构中的高敏感信息，包括：社保号、信用卡、银行账号、病历号、诊断与治疗信息、处方、驾照、护照、金融 PIN、税号、医保 ID、生物特征数据等。
+
+### 功能
+
+- **离线规则引擎**：正则 + Luhn/身份证校验；覆盖密钥/机密与 PII；无需模型与网络
+- **本地 LLM**：Ollama + 小模型（默认 `qwen3:0.6b`）做隐私友好的 PII 检测
+- **内部推理**：通过 `<think>` 展示模型思考过程
+- **流式输出**：实时显示思考与检测进度
+- **性能指标**：TTFT、token 数、处理速度
+- **批量处理**：user-memory-evaluation 框架的 Layer 3 用例
+- **详细指标**：prefill / 输出时间与两阶段 tok/s
+
+### 安装
+
+#### 1. 安装 Ollama（仅 LLM 路径需要）
+
+> **通用回退（OpenRouter）**：本实验默认用本地 Ollama 小模型。若 Ollama 不可用（未运行 / 不可达）且设置了 `OPENROUTER_API_KEY`，Agent 会自动改走 OpenRouter（默认托管模型 `openai/gpt-5.6-luna`）。想强制走回退做验证，可把 Ollama 指到一个不可达端口：`export OLLAMA_HOST=http://127.0.0.1:1`。
+
+**macOS:**
+```bash
+brew install ollama
+ollama serve  # 另开终端
+```
+
+**Linux:**
+```bash
+curl -fsSL https://ollama.com/install.sh | sh
+systemctl start ollama
+```
+
+**Windows:** 从 [ollama.com](https://ollama.com/download/windows) 下载
+
+> 说明：以下 Ollama 相关步骤仅在使用 `--mode llm`（本地 LLM 引擎）或运行 LLM 批量评测路径时才需要。离线规则引擎（`--demo`、`--input`）只依赖 Python 标准库，无需安装 Ollama。
+
+#### 2. 拉取模型
+```bash
+ollama pull qwen3:0.6b
+```
+
+0.6B 模型约需 500MB 磁盘；可按需换用 `qwen3:1.7b`、`qwen3:4b` 提升准确率。
+
+#### 3. 安装 Python 依赖
+```bash
+# 在仓库根目录使用统一的第 3 章环境
+uv sync --locked --python 3.12 --extra ch3
+
+# 切换目录前先激活环境：
+# macOS/Linux：
+source .venv/bin/activate
+# Windows PowerShell：.\.venv\Scripts\Activate.ps1
+# Windows cmd：.venv\Scripts\activate.bat
+
+# 未安装 uv 时可用 pip 兜底：
+# python -m pip install -e ".[ch3]"
+
+cd chapter3/log-sanitization
+
+# 迁移期间仍支持单项目兼容路径：
+# python -m pip install -r requirements.txt
+```
+
+### 用法
+
+完整参数说明见 `python main.py --help`（中文）。
+
+#### 离线规则演示（推荐，无需 Ollama）
+```bash
+python main.py --demo
+```
+
+#### 脱敏任意日志文件（离线）
+```bash
+python main.py --input app.log                 # 结果写到 app.log.sanitized
+python main.py --input app.log -o cleaned.log  # 指定输出文件
+```
+
+也可以直接运行规则引擎模块，仅对内置样本做演示：
+```bash
+python regex_sanitizer.py
+```
+
+#### 离线验证
+
+```bash
+# 从仓库根目录开始；pytest 需要 dev 依赖。
+uv sync --locked --python 3.12 --extra ch3 --extra dev
+source .venv/bin/activate
+# Windows PowerShell: .\.venv\Scripts\Activate.ps1
+
+cd chapter3/log-sanitization
+python -m pytest tests
+python main.py --demo
+python regex_sanitizer.py
+```
+
+`tests/` 包含规则脱敏与 LLM 输出解析的离线回归测试。`test_loader.py` 刻意保留在项目根目录：它虽然以 `test_` 开头，但实际是 `main.py` 导入的用例加载支持模块，不是 pytest 文件。加载器调试助手位于 `tests/manual/loader_debug.py`。
+
+#### 使用本地 LLM 引擎
+```bash
+python main.py --demo --mode llm
+python main.py --input app.log --mode llm --model qwen3:1.7b
+```
+
+#### 处理全部 Layer 3 测试用例（LLM 批量评测路径）
+
+该路径固定使用 LLM，需要 Ollama 与 chapter3 评测框架：
+```bash
+python main.py
+```
+
+#### 指定用例 / 限制数量
+```bash
+python main.py --test-id layer3_13_emergency_medical_cascade
+python main.py --limit 3
+```
+
+#### 选择模型
+```bash
+python main.py --demo --mode llm --model qwen3:4b   # 默认 qwen3:0.6b
+```
+
+### 输出结构
+
+脱敏日志与指标保存在 `output/` 目录：
+
+```
+output/
+├── <test_id>_sanitized.txt    # 脱敏后的对话文本
+├── <test_id>_summary.json     # 发现与替换的 PII 摘要
+├── performance_metrics.json   # 详细性能指标
+└── performance_summary.json   # 聚合性能统计
+```
+
+### 性能指标
+
+**时间：** Prefill（TTFT）、输出时间、总时间（毫秒）  
+**Token：** 输入/输出数量；Prefill/输出速度（tok/s）  
+**脱敏：** 发现的 PII 条数；替换为 `[REDACTED]` 的次数
+
+### 架构
+
+1. **regex_sanitizer.py**：离线规则脱敏（正则 + Luhn/身份证校验）  
+2. **samples.py**：离线演示用的代表性 Agent 日志样本  
+3. **config.py**：Ollama 模型与 PII 类别配置  
+4. **test_loader.py**：从 user-memory-evaluation 加载用例（支持模块，不是 pytest）
+5. **agent.py**：基于 Ollama 的 LLM 脱敏逻辑  
+6. **metrics.py**：性能指标采集与报告  
+7. **main.py**：入口与编排  
+8. **tests/**：离线 pytest 回归测试与手动加载器调试助手
+
+### 工作原理（LLM 路径）
+
+1. 从 user-memory-evaluation 加载对话历史  
+2. 将每段对话送入本地 Qwen3，用专用提示检测 Level 3 PII  
+3. 将检出值替换为 `[REDACTED]`  
+4. 采集性能指标  
+5. 将脱敏日志与性能摘要写入 `output/`  
+
+### 隐私考量
+
+- 默认本地 Ollama 路径不向外部 API 发送数据  
+- 若走 OpenRouter 回退则会离开本机  
+- 脱敏日志使用占位符；任何原始 PII 日志都应妥善保管  
+
+### 故障排除
+
+**找不到 Ollama：** 安装并运行 `ollama serve`  
+**找不到模型：** `ollama pull qwen3:0.6b`  
+**找不到评测框架：** 确认 loader 所期望的 `../user-memory-evaluation/`（或 chapter3 路径）存在
+
+---
+
+## Notes / 说明
+
+- Prefer `--demo` first; Ollama is optional for the rule path.  
+- 建议先跑 `--demo`；规则路径无需 Ollama。

chapter3/log-sanitization/agent.py

@@ -0,0 +1,409 @@
+"""
+Log Sanitization Agent using Local Ollama LLM
+"""
+
+import os
+import time
+import re
+import json
+from typing import List, Tuple, Dict, Optional
+from pathlib import Path
+import ollama
+
+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except ImportError:
+    pass
+
+from config import (
+    OLLAMA_MODEL, 
+    OLLAMA_TEMPERATURE,
+    SYSTEM_PROMPT, 
+    USER_PROMPT_TEMPLATE,
+    PII_DETECTION_SCHEMA,
+    OUTPUT_DIR
+)
+from metrics import PerformanceMetrics, MetricsCollector
+
+
+def _value_appears_in_text(value: str, text: str) -> bool:
+    """Return True if *value* appears as a substring of *text* (case-insensitive)."""
+    return value.lower() in text.lower()
+
+
+class LogSanitizationAgent:
+    """Agent for sanitizing logs using local Qwen3 0.6B model via Ollama"""
+    
+    def __init__(self, model: str = OLLAMA_MODEL):
+        """Initialize the sanitization agent.
+
+        Primary backend is the local Ollama model. If Ollama is unavailable
+        (not running / not reachable) and OPENROUTER_API_KEY is set, the agent
+        falls back to OpenRouter (default hosted model: openai/gpt-5.6-luna),
+        so the experiment still runs without a local model.
+        """
+        self.model = model
+        self.backend = "ollama"
+        self.metrics_collector = MetricsCollector(OUTPUT_DIR)
+
+        # Try the local Ollama backend first.
+        try:
+            self.client = ollama.Client()
+            models = self.client.list()
+            # models is a dict with 'models' key containing a list
+            if isinstance(models, dict) and 'models' in models:
+                available_models = [m.get('name', '') for m in models['models']]
+            else:
+                # If it's a direct list (older API versions)
+                available_models = [m.get('name', '') for m in models] if isinstance(models, list) else []
+
+            if not any(self.model in m for m in available_models):
+                print(f"⚠️  Model {self.model} not found. Pulling it now...")
+                self.client.pull(self.model)
+                print(f"✅ Model {self.model} pulled successfully")
+            else:
+                print(f"✅ Using model: {self.model}")
+
+        except Exception as e:
+            # Universal fallback: route through OpenRouter when Ollama is down.
+            openrouter_key = os.getenv("OPENROUTER_API_KEY")
+            if openrouter_key:
+                from openai import OpenAI
+
+                from agentbook.providers import resolve_backend
+                # 这里的回退条件是“本地 Ollama 连不上”，而非缺少凭证，
+                # 因此由本实验判定后再向注册表要一个 OpenRouter backend。
+                # 本地小模型（qwen3:0.6b 等）在 OpenRouter 上未必可用，
+                # substitute_unknown 让注册表替换成可用的默认模型。
+                backend = resolve_backend(
+                    "openrouter", model=self.model, api_key=openrouter_key
+                )
+                self.backend = "openrouter"
+                self.client = OpenAI(api_key=backend.api_key,
+                                     base_url=backend.base_url)
+                self.model = backend.model
+                print(f"⚠️  Ollama unavailable ({e}); "
+                      f"falling back to OpenRouter model: {self.model}")
+            else:
+                print(f"❌ Failed to connect to Ollama: {e}")
+                print("Please ensure Ollama is running: ollama serve, "
+                      "or set OPENROUTER_API_KEY as a fallback")
+                raise
+
+    def _chat_stream(self, messages):
+        """Yield content chunks from the active backend (Ollama or OpenRouter)."""
+        if self.backend == "ollama":
+            stream = self.client.chat(
+                model=self.model,
+                messages=messages,
+                stream=True,
+                format=PII_DETECTION_SCHEMA,  # Use structured output format
+                options={
+                    "temperature": OLLAMA_TEMPERATURE,
+                    "num_predict": 1000,
+                }
+            )
+            for chunk in stream:
+                yield chunk.get('message', {}).get('content', '')
+        else:
+            # 用与 Ollama 相同的 JSON Schema 强约束输出结构 (pii_items 数组),
+            # 避免模型自行发明字段名. strict 模式要求 additionalProperties=false.
+            strict_schema = dict(PII_DETECTION_SCHEMA)
+            strict_schema["additionalProperties"] = False
+            stream = self.client.chat.completions.create(
+                model=self.model,
+                messages=messages,
+                stream=True,
+                temperature=OLLAMA_TEMPERATURE,
+                response_format={
+                    "type": "json_schema",
+                    "json_schema": {
+                        "name": "pii_detection",
+                        "strict": True,
+                        "schema": strict_schema,
+                    },
+                },
+                max_tokens=1000,
+            )
+            for chunk in stream:
+                if not chunk.choices:
+                    continue
+                yield chunk.choices[0].delta.content or ""
+    
+    def count_tokens(self, text: str) -> int:
+        """Estimate token count (rough approximation)"""
+        # Rough estimate: 1 token ≈ 4 characters for English text
+        # For more accurate counting, we'd need the actual tokenizer
+        return len(text) // 4
+    
+    def detect_pii(self, conversation_text: str) -> Tuple[List[str], Dict]:
+        """
+        Detect Level 3 PII in conversation text using local LLM
+        
+        Args:
+            conversation_text: Text to analyze
+        
+        Returns:
+            - List of detected PII values
+            - Performance metrics dictionary
+        """
+        # Prepare the prompt
+        user_prompt = USER_PROMPT_TEMPLATE.format(conversation_text=conversation_text)
+        
+        # Count input tokens
+        input_tokens = self.count_tokens(SYSTEM_PROMPT + user_prompt)
+        
+        # Measure prefill time (time to first token)
+        start_time = time.perf_counter()
+        
+        # Create messages for Ollama
+        messages = [
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user", "content": user_prompt}
+        ]
+        
+        # Track first token time
+        first_token_time = None
+        output_tokens_count = 0
+        full_response = ""
+        
+        try:
+            # Use structured output with JSON schema (backend-agnostic stream)
+            print("\n   🧠 Analyzing (JSON): \033[90m", end="", flush=True)  # Gray color for JSON
+
+            for content in self._chat_stream(messages):
+                if first_token_time is None and content:
+                    first_token_time = time.perf_counter()
+
+                full_response += content
+                output_tokens_count += len(content) // 4  # Rough token estimate
+
+                # Stream the actual content
+                if content:
+                    print(content, end="", flush=True)
+
+            print("\033[0m")  # Reset color and new line
+            
+            end_time = time.perf_counter()
+            
+        except Exception as e:
+            print(f"\n❌ Error during PII detection: {e}")
+            return [], {}
+        
+        # Calculate performance metrics
+        prefill_time_ms = (first_token_time - start_time) * 1000 if first_token_time else 0
+        total_time_ms = (end_time - start_time) * 1000
+        output_time_ms = total_time_ms - prefill_time_ms
+        
+        prefill_speed = input_tokens / (prefill_time_ms / 1000) if prefill_time_ms > 0 else 0
+        output_speed = output_tokens_count / (output_time_ms / 1000) if output_time_ms > 0 else 0
+        
+        # Parse JSON response
+        pii_values = []
+        accepted_items = []
+        
+        try:
+            response_json = json.loads(full_response)
+            if not isinstance(response_json, dict):
+                return [], {}
+            
+            raw_items = response_json.get('pii_items')
+            if isinstance(raw_items, list):
+                for item in raw_items:
+                    if isinstance(item, dict):
+                        value = item.get('value')
+                        if value and isinstance(value, str) and _value_appears_in_text(value, conversation_text):
+                            pii_values.append(value)
+                            accepted_items.append(item)
+                    elif isinstance(item, str) and item and _value_appears_in_text(item, conversation_text):
+                        pii_values.append(item)
+                        accepted_items.append({"value": item})
+            else:
+                legacy_values = response_json.get('pii_values')
+                if isinstance(legacy_values, list):
+                    for pii in legacy_values:
+                        if pii and isinstance(pii, str):
+                            cleaned = pii.strip().strip('-').strip()
+                            if cleaned:
+                                pii_values.append(cleaned)
+
+        except json.JSONDecodeError as e:
+            print(f"\n   ⚠️  Failed to parse JSON response: {e}")
+            # Fallback to simple line splitting if JSON parsing fails
+            pii_values = [line.strip() for line in full_response.split('\n') if line.strip()]
+            accepted_items = []
+
+        metrics = {
+            'input_tokens': input_tokens,
+            'output_tokens': output_tokens_count,
+            'prefill_time_ms': prefill_time_ms,
+            'output_time_ms': output_time_ms,
+            'total_time_ms': total_time_ms,
+            'prefill_speed_tps': prefill_speed,
+            'output_speed_tps': output_speed,
+            'pii_items_found': len(pii_values),
+            'pii_items': accepted_items
+        }
+        
+        return pii_values, metrics
+    
+    def sanitize_text(self, text: str, pii_values: List[str]) -> Tuple[str, int]:
+        """
+        Replace PII values with [REDACTED] in the text
+        
+        Returns:
+            - Sanitized text
+            - Number of replacements made
+        """
+        sanitized = text
+        replacements = 0
+        
+        for pii_value in pii_values:
+            # Escape special regex characters in PII value
+            escaped_value = re.escape(pii_value)
+            # Count occurrences before replacement
+            occurrences = len(re.findall(escaped_value, sanitized, re.IGNORECASE))
+            # Replace all occurrences
+            sanitized = re.sub(escaped_value, '[REDACTED]', sanitized, flags=re.IGNORECASE)
+            replacements += occurrences
+        
+        return sanitized, replacements
+    
+    def sanitize_conversation(
+        self, 
+        conversation: Dict,
+        test_id: str = "unknown"
+    ) -> Dict:
+        """
+        Sanitize a single conversation and collect metrics
+        
+        Returns:
+            Dictionary with sanitized conversation and metrics
+        """
+        # Format conversation text
+        conv_text = self.format_conversation(conversation)
+        conv_id = conversation.get('conversation_id', 'unknown')
+        
+        print(f"🔍 Processing conversation: {conv_id}")
+        
+        # Detect PII
+        pii_values, perf_metrics = self.detect_pii(conv_text)
+        accepted_items = perf_metrics.get('pii_items', [])
+        
+        if pii_values:
+            print(f"   ✅ Found {len(pii_values)} PII items:")
+            for pii in pii_values:
+                print(f"      - {pii}")
+        else:
+            print("   ⚠️  No PII items detected")
+        
+        # Sanitize the text
+        sanitized_text, replacements = self.sanitize_text(conv_text, pii_values)
+        
+        # Create performance metric. detect_pii() returns an empty metrics dict
+        # when the LLM backend fails (e.g. Ollama not running) — fall back to
+        # zeros so one failed conversation doesn't crash the whole batch.
+        metric = PerformanceMetrics(
+            test_id=test_id,
+            conversation_id=conv_id,
+            input_text_length=len(conv_text),
+            input_tokens=perf_metrics.get('input_tokens', 0),
+            prefill_time_ms=perf_metrics.get('prefill_time_ms', 0),
+            output_time_ms=perf_metrics.get('output_time_ms', 0),
+            total_time_ms=perf_metrics.get('total_time_ms', 0),
+            output_tokens=perf_metrics.get('output_tokens', 0),
+            prefill_speed_tps=perf_metrics.get('prefill_speed_tps', 0),
+            output_speed_tps=perf_metrics.get('output_speed_tps', 0),
+            pii_items_found=perf_metrics.get('pii_items_found', 0),
+            replacements_made=replacements,
+            sanitized_text_length=len(sanitized_text)
+        )
+        
+        self.metrics_collector.add_metric(metric)
+        
+        return {
+            'conversation_id': conv_id,
+            'original_length': len(conv_text),
+            'sanitized_length': len(sanitized_text),
+            'pii_found': pii_values,
+            'replacements_made': replacements,
+            'sanitized_text': sanitized_text,
+            'pii_items': accepted_items,
+            'metrics': metric.to_dict()
+        }
+    
+    def format_conversation(self, conversation: Dict) -> str:
+        """Format conversation dictionary into text"""
+        lines = []
+        lines.append(f"Conversation ID: {conversation.get('conversation_id', 'unknown')}")
+        lines.append(f"Timestamp: {conversation.get('timestamp', 'unknown')}")
+        lines.append("-" * 50)
+        
+        messages = conversation.get('messages', [])
+        for msg in messages:
+            role = msg.get('role', 'unknown').upper()
+            content = msg.get('content', '')
+            lines.append(f"{role}: {content}")
+            lines.append("")  # Empty line between messages
+        
+        return "\n".join(lines)
+    
+    def save_sanitized_log(self, test_id: str, results: List[Dict]):
+        """Save sanitized logs to output directory"""
+        output_file = OUTPUT_DIR / f"{test_id}_sanitized.txt"
+        summary_file = OUTPUT_DIR / f"{test_id}_summary.json"
+        
+        # Save sanitized text
+        with open(output_file, 'w') as f:
+            for result in results:
+                f.write(f"\n{'='*60}\n")
+                f.write(f"Conversation: {result['conversation_id']}\n")
+                f.write(f"{'='*60}\n")
+                f.write(result['sanitized_text'])
+                f.write("\n")
+        
+        # Save summary
+        summary = {
+            'test_id': test_id,
+            'total_conversations': len(results),
+            'total_pii_found': sum(len(r['pii_found']) for r in results),
+            'total_replacements': sum(r['replacements_made'] for r in results),
+            'conversations': [
+                {
+                    'conversation_id': r['conversation_id'],
+                    'pii_count': len(r['pii_found']),
+                    'replacements': r['replacements_made']
+                }
+                for r in results
+            ]
+        }
+        
+        with open(summary_file, 'w') as f:
+            json.dump(summary, f, indent=2)
+        
+        print(f"✅ Sanitized log saved to: {output_file}")
+        print(f"✅ Summary saved to: {summary_file}")
+    
+    def process_test_case(self, test_id: str, conversations: List[Dict]) -> List[Dict]:
+        """Process all conversations in a test case"""
+        results = []
+        
+        print(f"\n{'='*60}")
+        print(f"Processing Test Case: {test_id}")
+        print(f"Total Conversations: {len(conversations)}")
+        print(f"{'='*60}")
+        
+        for i, conv in enumerate(conversations, 1):
+            print(f"\n[{i}/{len(conversations)}] ", end="")
+            result = self.sanitize_conversation(conv, test_id)
+            results.append(result)
+        
+        # Save results
+        self.save_sanitized_log(test_id, results)
+        
+        # Save metrics
+        self.metrics_collector.save_metrics()
+        self.metrics_collector.print_summary()
+        
+        return results

chapter3/log-sanitization/campaign.py

@@ -0,0 +1,269 @@
+#!/usr/bin/env python3
+"""Real local-model benchmark for Experiment 3-3.
+
+Compares the existing regex engine, local Qwen3 through Ollama, and a hybrid
+regex-first/LLM-second path on labeled logs.  Gold values are intentionally
+fictional.  Metrics are exact-span precision/recall, residual leakage, utility,
+and latency; the local model emits type/location/confidence JSON.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from collections import defaultdict
+from pathlib import Path
+from typing import Any, Dict, List, Tuple
+
+import ollama
+
+HERE = Path(__file__).resolve().parent
+sys.path.insert(0, str(HERE.parent))
+from experiment_utils import jsonable, write_campaign_evidence
+
+import regex_sanitizer
+
+
+CASES = [
+    {"id": "structured_cn", "text": "客户身份证号11010519491231002X，手机号13800138000。", "gold": [("cn_id_card", "11010519491231002X"), ("cn_phone", "13800138000")]},
+    {"id": "structured_us", "text": "SSN 123-45-6789; Visa 4111 1111 1111 1111; ticket 123-45-6780 is not an SSN.", "gold": [("us_ssn", "123-45-6789"), ("credit_card", "4111 1111 1111 1111")]},
+    {"id": "credentials", "text": "postgres://alice:S3cr3t-Pass@db.internal:5432/prod failed; retry token=not-a-real-token-but-secret", "gold": [("password", "S3cr3t-Pass"), ("secret", "not-a-real-token-but-secret")]},
+    {"id": "natural_password", "text": "用户说：我网银的登录口令是‘Blue moon over river 77’，请帮我记住。", "gold": [("password", "Blue moon over river 77")]},
+    {"id": "address", "text": "Please ship replacements to apartment 3B, 1847 Maple Street, Portland, Oregon 97205 after Tuesday.", "gold": [("address", "apartment 3B, 1847 Maple Street, Portland, Oregon 97205")]},
+    {"id": "medical", "text": "Chart note: patient Maria Chen, MRN-789456, was diagnosed with stage II breast cancer and starts doxorubicin Friday.", "gold": [("medical_record", "MRN-789456"), ("medical_diagnosis", "stage II breast cancer"), ("treatment", "doxorubicin")]},
+    {"id": "passport", "text": "Traveler passport P987654321 expires 2027-04-09; booking code P987654320 is unrelated.", "gold": [("passport", "P987654321")]},
+    {"id": "financial", "text": "Wire account 4429853327 routing 123006800. Product SKU 4429853328 should remain visible.", "gold": [("bank_account", "4429853327"), ("routing_number", "123006800")]},
+    {"id": "email_bearer", "text": "Authorization: Bearer fakeBearerToken1234567890; notify alice.wang@example.com.", "gold": [("bearer_token", "fakeBearerToken1234567890"), ("email", "alice.wang@example.com")]},
+    {"id": "negative_numbers", "text": "Metrics: latency 13800138000 ns, build 4111111111111112 vectors, model XR-7003. None identify a person.", "gold": []},
+    {"id": "negative_language", "text": "The password field must contain at least 12 characters. Never log a user's real password.", "gold": []},
+    {"id": "mixed_context", "text": "Support transcript: 'call me at 13912345678'. Agent replied that order 13912345679 ships tomorrow and wrote password: hunter2xyz.", "gold": [("cn_phone", "13912345678"), ("password", "hunter2xyz")]},
+]
+
+SCHEMA = {
+    "type": "object",
+    "properties": {
+        "findings": {
+            "type": "array",
+            "items": {
+                "type": "object",
+                "properties": {
+                    "type": {"type": "string"},
+                    "value": {"type": "string"},
+                    "start": {"type": "integer"},
+                    "end": {"type": "integer"},
+                    "confidence": {"type": "number"},
+                },
+                "required": ["type", "value", "start", "end", "confidence"],
+            },
+        }
+    },
+    "required": ["findings"],
+}
+
+SYSTEM = """You detect sensitive information in logs. Return exact substrings only.
+Types include cn_id_card, cn_phone, us_ssn, credit_card, password, secret,
+address, medical_record, medical_diagnosis, treatment, passport, bank_account,
+routing_number, bearer_token, email, private_key, and api_key. Distinguish a
+real disclosed value from a field name, instruction, product code, metric, or
+explicit decoy. start/end are zero-based Python slice offsets and confidence is
+0..1. Return JSON matching the schema; never redact or paraphrase the value."""
+
+
+ALIASES = {
+    "url_credential": "password",
+    "secret_assignment": "secret",
+    "us_ssn": "us_ssn",
+    "credit_card": "credit_card",
+    "cn_id_card": "cn_id_card",
+    "cn_phone": "cn_phone",
+    "bearer_token": "bearer_token",
+    "email": "email",
+    "api_key": "api_key",
+    "private_key": "private_key",
+}
+
+
+def normalize_findings(text: str, findings: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+    out = []
+    occupied = set()
+    for finding in findings or []:
+        value = str(finding.get("value") or "").strip(" \t\n\r'\"“”‘’")
+        if not value:
+            continue
+        start = int(finding.get("start", -1))
+        end = int(finding.get("end", -1))
+        if start < 0 or end <= start or text[start:end] != value:
+            start = text.find(value)
+            end = start + len(value) if start >= 0 else -1
+        key = (start, end, value)
+        if start < 0 or key in occupied:
+            continue
+        occupied.add(key)
+        out.append({
+            "type": str(finding.get("type") or "unknown").lower(),
+            "value": value,
+            "start": start,
+            "end": end,
+            "confidence": float(finding.get("confidence", 0.5)),
+        })
+    return sorted(out, key=lambda x: (x["start"], x["end"]))
+
+
+def regex_findings(text: str) -> Tuple[List[Dict[str, Any]], float]:
+    start = time.perf_counter()
+    _, raw = regex_sanitizer.sanitize(text)
+    elapsed = (time.perf_counter() - start) * 1000
+    findings = [{
+        "type": ALIASES.get(x["category"], x["category"]),
+        "value": x["value"], "start": x["start"], "end": x["end"], "confidence": 1.0,
+    } for x in raw]
+    return findings, elapsed
+
+
+def llm_findings(client: ollama.Client, model: str, text: str, purpose: str, receipts: List[Dict[str, Any]]) -> Tuple[List[Dict[str, Any]], float]:
+    request = {
+        "model": model,
+        "messages": [{"role": "system", "content": SYSTEM}, {"role": "user", "content": text}],
+        "format": SCHEMA,
+        "options": {"temperature": 0, "seed": 37, "num_predict": 1200},
+        "stream": False,
+    }
+    started = time.perf_counter()
+    response = client.chat(**request)
+    latency = (time.perf_counter() - started) * 1000
+    raw_text = response["message"]["content"]
+    parsed = json.loads(raw_text)
+    findings = normalize_findings(text, parsed.get("findings") or [])
+    receipts.append({
+        "purpose": purpose,
+        "provider": "ollama-local",
+        "endpoint": "http://127.0.0.1:11434",
+        "model": model,
+        "request": request,
+        "response": jsonable(response),
+        "latency_ms": round(latency, 3),
+        "usage": {
+            "prompt_tokens": response.get("prompt_eval_count"),
+            "completion_tokens": response.get("eval_count"),
+        },
+    })
+    return findings, latency
+
+
+def redact(text: str, findings: List[Dict[str, Any]]) -> str:
+    accepted = []
+    for finding in sorted(findings, key=lambda x: (-float(x.get("confidence", 0)), x["start"])):
+        if any(not (finding["end"] <= x["start"] or finding["start"] >= x["end"]) for x in accepted):
+            continue
+        accepted.append(finding)
+    result = text
+    for finding in sorted(accepted, key=lambda x: x["start"], reverse=True):
+        result = result[:finding["start"]] + f"[REDACTED_{finding['type'].upper()}]" + result[finding["end"]:]
+    return result
+
+
+def evaluate_case(case: Dict[str, Any], findings: List[Dict[str, Any]]) -> Dict[str, Any]:
+    gold = {(t, v) for t, v in case["gold"]}
+    predicted = {(f["type"], f["value"]) for f in findings}
+    # Value equality is decisive; type aliases are also audited separately.
+    gold_values = {v for _, v in gold}
+    pred_values = {v for _, v in predicted}
+    tp_values = gold_values & pred_values
+    redacted = redact(case["text"], findings)
+    false_redacted_chars = sum(len(f["value"]) for f in findings if f["value"] not in gold_values)
+    non_sensitive_chars = max(1, len(case["text"]) - sum(len(v) for v in gold_values))
+    return {
+        "findings": findings,
+        "precision": len(tp_values) / len(pred_values) if pred_values else (1.0 if not gold_values else 0.0),
+        "recall": len(tp_values) / len(gold_values) if gold_values else 1.0,
+        "typed_exact": len(gold & predicted) / len(gold) if gold else (1.0 if not predicted else 0.0),
+        "residual_leaks": [value for value in gold_values if value in redacted],
+        "utility": max(0.0, 1.0 - false_redacted_chars / non_sensitive_chars),
+        "sanitized_text": redacted,
+    }
+
+
+def aggregate(rows: List[Dict[str, Any]]) -> Dict[str, Any]:
+    n = len(rows)
+    return {
+        "cases": n,
+        "mean_precision": sum(r["precision"] for r in rows) / n,
+        "mean_recall": sum(r["recall"] for r in rows) / n,
+        "mean_typed_exact": sum(r["typed_exact"] for r in rows) / n,
+        "residual_leaks": sum(len(r["residual_leaks"]) for r in rows),
+        "mean_utility": sum(r["utility"] for r in rows) / n,
+        "mean_latency_ms": sum(r["latency_ms"] for r in rows) / n,
+    }
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Experiment 3-3 real Ollama sanitization benchmark")
+    parser.add_argument("--model", default="qwen3:0.6b")
+    parser.add_argument("--limit", type=int, default=len(CASES))
+    args = parser.parse_args()
+    cases = CASES[: args.limit]
+    client = ollama.Client()
+    model_info = jsonable(client.show(args.model))
+    receipts: List[Dict[str, Any]] = []
+    results: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
+
+    for index, case in enumerate(cases, start=1):
+        regex_hits, regex_ms = regex_findings(case["text"])
+        llm_hits, llm_ms = llm_findings(client, args.model, case["text"], f"llm:{case['id']}", receipts)
+
+        regex_redacted = redact(case["text"], regex_hits)
+        remaining_hits, hybrid_llm_ms = llm_findings(
+            client, args.model, regex_redacted, f"hybrid-after-regex:{case['id']}", receipts
+        )
+        # Map LLM values from the regex-redacted text back into original text.
+        mapped = []
+        for finding in remaining_hits:
+            start = case["text"].find(finding["value"])
+            if start >= 0:
+                mapped.append({**finding, "start": start, "end": start + len(finding["value"])})
+        hybrid_hits = normalize_findings(case["text"], regex_hits + mapped)
+
+        for strategy, hits, latency in (
+            ("regex", regex_hits, regex_ms),
+            ("llm", llm_hits, llm_ms),
+            ("hybrid", hybrid_hits, regex_ms + hybrid_llm_ms),
+        ):
+            row = evaluate_case(case, hits)
+            row.update({"case_id": case["id"], "strategy": strategy, "latency_ms": round(latency, 3)})
+            results[strategy].append(row)
+        print(f"[{index}/{len(cases)}] {case['id']} complete")
+
+    summaries = {strategy: aggregate(rows) for strategy, rows in results.items()}
+    full = len(cases) == len(CASES) and len(receipts) == len(CASES) * 2
+    evidence = {
+        "status": "passed" if full else "partial",
+        "configuration": {
+            "backend": "ollama-local",
+            "endpoint": "http://127.0.0.1:11434",
+            "model": args.model,
+            "model_info": model_info,
+            "seed": 37,
+            "schema": SCHEMA,
+        },
+        "acceptance": {
+            "local_model": True,
+            "qwen3_model": args.model.startswith("qwen3:"),
+            "structured_type_location_confidence": True,
+            "structured_semistructured_natural_language_cases": len(cases) >= len(CASES),
+            "regex_llm_hybrid_compared": set(results) == {"regex", "llm", "hybrid"},
+            "leakage_utility_latency_measured": True,
+            "passed": full,
+        },
+        "summary": summaries,
+        "dataset": cases,
+        "results": dict(results),
+    }
+    manifest = write_campaign_evidence(HERE, "3-3", evidence, receipts)
+    print(json.dumps(manifest["summary"], ensure_ascii=False, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

chapter3/log-sanitization/config.py

@@ -0,0 +1,84 @@
+"""
+Configuration for Log Sanitization with Local LLM
+"""
+
+import os
+from pathlib import Path
+
+# Ollama Configuration
+# 默认使用 0.6B 超小模型，呼应本章“小模型也能胜任结构化任务”的论点，
+# 且可在 CPU / 消费级设备上运行；可用 --model 覆盖为 qwen3:1.7b、qwen3:4b 等。
+OLLAMA_MODEL = "qwen3:0.6b"
+OLLAMA_TEMPERATURE = 0.1  # Low temperature for consistent detection
+
+# Paths
+PROJECT_ROOT = Path(__file__).parent
+OUTPUT_DIR = PROJECT_ROOT / "output"
+OUTPUT_DIR.mkdir(exist_ok=True)
+
+# Performance Metrics Configuration
+METRICS_FILE = OUTPUT_DIR / "performance_metrics.json"
+
+# Evaluation Framework Path
+# The user-memory-evaluation framework lives in chapter3, not chapter2.
+# PROJECT_ROOT = chapter3/log-sanitization, so go up two levels to the repo root.
+EVAL_FRAMEWORK_PATH = PROJECT_ROOT.parent.parent / "chapter3" / "user-memory-evaluation"
+
+# System Prompt for PII Detection
+SYSTEM_PROMPT = """You are a privacy protection agent that detects Level 3 PII.
+
+Level 3 PII includes:
+- Social Security Numbers (SSN) - format: XXX-XX-XXXX or XXXXXXXXX
+- Credit Card Numbers - format: XXXX XXXX XXXX XXXX or 16 digits
+- Credit Card Expiry Date and CVV
+- Bank Account Numbers
+- Full Residential Addresses
+- Medical Record Numbers
+- Medical Diagnoses and Treatment Details
+- Prescription Information
+- Driver's License Numbers
+- Passport Numbers
+- Financial PINs
+- Tax ID Numbers
+- Health Insurance IDs
+- Biometric Data
+- Usernames for Financial Accounts
+- Passwords
+
+Analyze the conversation and return JSON with a pii_items array. Each item must include:
+- type: the PII category label
+- value: the exact sensitive substring copied verbatim from the input text
+
+Do not include labels or explanations inside value. NEVER use placeholders."""
+
+USER_PROMPT_TEMPLATE = """Analyze the following conversation for Level 3 PII:
+
+{conversation_text}"""
+
+# JSON Schema for structured output
+PII_DETECTION_SCHEMA = {
+    "type": "object",
+    "properties": {
+        "pii_items": {
+            "type": "array",
+            "items": {
+                "type": "object",
+                "properties": {
+                    "type": {
+                        "type": "string",
+                        "description": "PII category label, e.g. ssn, credit_card_number, email"
+                    },
+                    "value": {
+                        "type": "string",
+                        "description": "Exact sensitive substring copied verbatim from the input text"
+                    }
+                },
+                "required": ["type", "value"],
+                "additionalProperties": False
+            },
+            "description": "Array of structured PII items. The value field must be copied verbatim from the input text."
+        }
+    },
+    "required": ["pii_items"],
+    "additionalProperties": False
+}

chapter3/log-sanitization/env.example

@@ -0,0 +1,9 @@
+# 本实验主用本地 Ollama 小模型（qwen3:0.6b），无需任何云端 API key。
+# 只需先启动 Ollama：ollama serve
+
+# 通用回退：当 Ollama 不可用（未运行/不可达）且设置了 OPENROUTER_API_KEY 时，
+# 自动改走 OpenRouter（默认托管模型 openai/gpt-5.6-luna）。
+# OPENROUTER_API_KEY=your-openrouter-api-key
+
+# 可选：强制走 OpenRouter 回退（把 Ollama 指到一个不可达端口即可）
+# OLLAMA_HOST=http://127.0.0.1:1

chapter3/log-sanitization/main.py

@@ -0,0 +1,300 @@
+#!/usr/bin/env python3
+"""
+Main script for Log Sanitization using Local LLM
+"""
+
+import argparse
+import sys
+from collections import Counter
+from pathlib import Path
+from typing import Optional
+
+from config import OUTPUT_DIR, OLLAMA_MODEL
+import regex_sanitizer
+from samples import SAMPLES
+
+
+def main(test_id: Optional[str] = None, limit: Optional[int] = None,
+         model: str = OLLAMA_MODEL):
+    """
+    Main function to run log sanitization
+    
+    Args:
+        test_id: Specific test case ID to process (optional)
+        limit: Maximum number of test cases to process (optional)
+    """
+    print("🚀 Starting Log Sanitization with Local LLM")
+    print("=" * 60)
+
+    # Initialize components
+    try:
+        from agent import LogSanitizationAgent
+        from test_loader import TestCaseLoader
+
+        print("📦 Loading test cases from user-memory-evaluation...")
+        loader = TestCaseLoader()
+
+        print(f"🤖 Initializing Ollama agent (model: {model})...")
+        agent = LogSanitizationAgent(model=model)
+
+    except Exception as e:
+        print(f"❌ Initialization failed: {e}")
+        return 1
+    
+    # Get test cases to process
+    if test_id:
+        # Process specific test case
+        print(f"\n📋 Processing specific test case: {test_id}")
+        conversations = loader.get_test_case_conversations(test_id)
+        
+        if not conversations:
+            print(f"❌ Test case {test_id} not found or has no conversations")
+            return 1
+        
+        agent.process_test_case(test_id, conversations)
+        
+    else:
+        # Process Layer 3 test cases (most complex, likely to have PII)
+        print("\n📋 Getting Layer 3 test cases...")
+        test_cases = loader.get_layer3_test_cases()
+        
+        if not test_cases:
+            print("❌ No Layer 3 test cases found")
+            return 1
+        
+        print(f"Found {len(test_cases)} Layer 3 test cases")
+        
+        # Apply limit if specified
+        if limit:
+            test_cases = test_cases[:limit]
+            print(f"Processing first {limit} test cases")
+        
+        # Process each test case
+        for i, tc in enumerate(test_cases, 1):
+            print(f"\n[{i}/{len(test_cases)}] Test Case: {tc['test_id']}")
+            print(f"   Title: {tc['title']}")
+            print(f"   Conversations: {tc['num_conversations']}")
+            
+            # Get conversation histories
+            conversations = loader.get_test_case_conversations(tc['test_id'])
+            
+            if conversations:
+                agent.process_test_case(tc['test_id'], conversations)
+            else:
+                print(f"   ⚠️  No conversations found for {tc['test_id']}")
+    
+    print("\n" + "=" * 60)
+    print("✅ Log Sanitization Complete!")
+    print(f"📁 Results saved to: {OUTPUT_DIR}")
+    
+    return 0
+
+
+def demo_regex_mode():
+    """离线规则脱敏演示：对多个代表性样本展示 before/after 与类别汇总"""
+    print("🎯 离线规则脱敏演示 (regex 模式，无需 Ollama)")
+    print("=" * 60)
+    print(f"共 {len(SAMPLES)} 个代表性样本，覆盖密钥 / 令牌 / 私钥 / PII 等类别\n")
+
+    total = Counter()
+    total_hits = 0
+    for name, text in SAMPLES:
+        redacted, findings = regex_sanitizer.sanitize(text)
+        regex_sanitizer.print_report(name, text, redacted, findings)
+        total.update(regex_sanitizer.summarize(findings))
+        total_hits += len(findings)
+
+    print(f"\n{'=' * 64}")
+    print("脱敏类别汇总 (across all samples)")
+    print("=" * 64)
+    for category, count in total.most_common():
+        label = regex_sanitizer.CATEGORY_LABELS.get(category, category)
+        print(f"   {label:<16} {count} 处")
+    print(f"\n   合计脱敏 {total_hits} 处敏感信息，覆盖 {len(total)} 个类别")
+    return 0
+
+
+def sanitize_file(input_path: str, output_path: Optional[str] = None,
+                  mode: str = "regex", model: str = OLLAMA_MODEL):
+    """对任意日志文件执行脱敏，结果写入输出文件"""
+    in_file = Path(input_path)
+    if not in_file.exists():
+        print(f"❌ 输入文件不存在: {input_path}")
+        return 1
+
+    text = in_file.read_text(encoding="utf-8", errors="replace")
+    out_file = Path(output_path) if output_path else in_file.with_suffix(in_file.suffix + ".sanitized")
+
+    if mode == "regex":
+        print(f"🔍 使用离线规则引擎脱敏: {input_path}")
+        redacted, findings = regex_sanitizer.sanitize(text)
+        counts = regex_sanitizer.summarize(findings)
+    else:
+        print(f"🔍 使用本地 LLM ({model}) 脱敏: {input_path}")
+        try:
+            from agent import LogSanitizationAgent
+        except Exception as e:
+            print(f"❌ 加载 LLM 引擎失败: {e}")
+            return 1
+        agent = LogSanitizationAgent(model=model)
+        pii_values, _ = agent.detect_pii(text)
+        redacted, _ = agent.sanitize_text(text, pii_values)
+        counts = Counter({"pii": len(pii_values)})
+        findings = pii_values
+
+    out_file.write_text(redacted, encoding="utf-8")
+
+    print(f"\n✅ 已写入脱敏结果: {out_file}")
+    print(f"   共脱敏 {sum(counts.values())} 处敏感信息")
+    for category, count in counts.most_common():
+        label = regex_sanitizer.CATEGORY_LABELS.get(category, category)
+        print(f"   - {label}: {count} 处")
+    return 0
+
+
+def demo_mode(model: str = OLLAMA_MODEL):
+    """Run a quick demo with sample PII-containing text (本地 LLM 模式)"""
+    print("🎯 Running Demo Mode (LLM)")
+    print("=" * 60)
+
+    # Create a sample conversation with Level 3 PII
+    sample_conversation = {
+        'conversation_id': 'demo_001',
+        'timestamp': '2024-01-01 10:00:00',
+        'messages': [
+            {
+                'role': 'user',
+                'content': 'I need to update my information. My SSN is 123-45-6789.'
+            },
+            {
+                'role': 'assistant',
+                'content': 'I can help you update your information. Can you confirm your credit card?'
+            },
+            {
+                'role': 'user',
+                'content': 'Yes, it\'s 4532 1234 5678 9012. Also, my medical record number is MRN-789456.'
+            },
+            {
+                'role': 'assistant',
+                'content': 'Thank you. I\'ve noted your SSN ending in 6789 and card ending in 9012.'
+            },
+            {
+                'role': 'user',
+                'content': 'Great. My driver\'s license is DL-123456789 and passport is P987654321.'
+            }
+        ]
+    }
+    
+    try:
+        from agent import LogSanitizationAgent
+        agent = LogSanitizationAgent(model=model)
+        print("\n📝 Sample conversation created with Level 3 PII")
+        print("🔍 Detecting and sanitizing PII...\n")
+        
+        result = agent.sanitize_conversation(sample_conversation, 'demo')
+        
+        print("\n" + "=" * 60)
+        print("DEMO RESULTS")
+        print("=" * 60)
+        print(f"PII Items Found: {len(result['pii_found'])}")
+        for pii in result['pii_found']:
+            print(f"  - {pii}")
+        
+        print(f"\nReplacements Made: {result['replacements_made']}")
+        print("\n--- SANITIZED TEXT ---")
+        print(result['sanitized_text'])
+        
+        # Save demo results
+        agent.save_sanitized_log('demo', [result])
+        agent.metrics_collector.save_metrics()
+        agent.metrics_collector.print_summary()
+        
+    except Exception as e:
+        print(f"❌ Demo failed: {e}")
+        return 1
+    
+    return 0
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(
+        description="日志脱敏实验：从 Agent 日志 / 工具输出中检测并脱敏敏感信息"
+                    "（API 密钥、令牌、私钥、信用卡、身份证、手机号、邮箱等）。",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""\
+两种脱敏引擎：
+  regex  离线规则引擎（默认），基于正则 + 校验算法，无需 Ollama，结果确定、速度快
+  llm    本地 LLM 引擎，通过 Ollama 调用小模型（默认 qwen3:0.6b）语义识别 Level 3 PII
+
+常用示例：
+  python main.py --demo                     # 离线跑内置样本，展示 before/after 与脱敏汇总
+  python main.py --demo --mode llm          # 用本地 LLM 跑演示样本
+  python main.py --input app.log            # 离线脱敏一个日志文件
+  python main.py --input app.log -o out.log # 指定输出文件
+  python main.py --input app.log --mode llm # 用本地 LLM 脱敏文件
+  python main.py                            # (LLM) 批量处理 chapter3 评测框架中的 Layer 3 用例
+  python main.py --test-id layer3_01_travel_coordination
+  python main.py --limit 3 --model qwen3:1.7b
+""",
+    )
+
+    parser.add_argument(
+        '--mode',
+        choices=['regex', 'llm'],
+        default='regex',
+        help='脱敏引擎：regex=离线规则(默认)，llm=本地 Ollama 模型。'
+             '（注意：不带 --demo/--input 的批量评测路径始终使用 LLM）'
+    )
+
+    parser.add_argument(
+        '-i', '--input',
+        type=str,
+        metavar='FILE',
+        help='待脱敏的日志文件路径（配合 --mode 选择引擎）'
+    )
+
+    parser.add_argument(
+        '-o', '--output',
+        type=str,
+        metavar='FILE',
+        help='脱敏结果输出文件路径（仅 --input 模式生效，默认写到 <输入>.sanitized）'
+    )
+
+    parser.add_argument(
+        '--model',
+        type=str,
+        default=OLLAMA_MODEL,
+        help=f'Ollama 模型名（默认 {OLLAMA_MODEL}），仅 llm 模式生效'
+    )
+
+    parser.add_argument(
+        '--test-id',
+        type=str,
+        help='仅处理指定 ID 的评测用例（LLM 批量路径）'
+    )
+
+    parser.add_argument(
+        '--limit',
+        type=int,
+        help='最多处理多少个评测用例（LLM 批量路径）'
+    )
+
+    parser.add_argument(
+        '--demo',
+        action='store_true',
+        help='运行演示：默认离线规则引擎跑内置代表性样本；加 --mode llm 则用本地 LLM'
+    )
+
+    args = parser.parse_args()
+
+    if args.input:
+        exit_code = sanitize_file(args.input, args.output, mode=args.mode, model=args.model)
+    elif args.demo:
+        if args.mode == 'llm':
+            exit_code = demo_mode(model=args.model)
+        else:
+            exit_code = demo_regex_mode()
+    else:
+        exit_code = main(test_id=args.test_id, limit=args.limit, model=args.model)
+
+    sys.exit(exit_code)

chapter3/log-sanitization/metrics.py

@@ -0,0 +1,175 @@
+"""
+Performance Metrics Module for Log Sanitization
+"""
+
+import time
+import json
+from typing import Dict, List, Optional
+from pathlib import Path
+from dataclasses import dataclass, asdict
+from datetime import datetime
+
+@dataclass
+class PerformanceMetrics:
+    """Store performance metrics for a single sanitization operation"""
+    test_id: str
+    conversation_id: str
+    input_text_length: int
+    input_tokens: int
+    
+    # Timing metrics
+    prefill_time_ms: float  # Time to First Token (TTFT)
+    output_time_ms: float
+    total_time_ms: float
+    
+    # Token metrics
+    output_tokens: int
+    prefill_speed_tps: float  # tokens per second
+    output_speed_tps: float
+    
+    # Sanitization results
+    pii_items_found: int
+    replacements_made: int
+    sanitized_text_length: int
+    
+    # Timestamps
+    timestamp: str = ""
+    
+    def __post_init__(self):
+        if not self.timestamp:
+            self.timestamp = datetime.now().isoformat()
+    
+    def to_dict(self) -> Dict:
+        """Convert to dictionary for JSON serialization"""
+        return asdict(self)
+
+
+class MetricsCollector:
+    """Collect and aggregate performance metrics"""
+    
+    def __init__(self, output_dir: Path):
+        self.output_dir = output_dir
+        self.metrics_file = output_dir / "performance_metrics.json"
+        self.summary_file = output_dir / "performance_summary.json"
+        self.metrics: List[PerformanceMetrics] = []
+    
+    def add_metric(self, metric: PerformanceMetrics):
+        """Add a new metric to the collection"""
+        self.metrics.append(metric)
+    
+    def calculate_summary(self) -> Dict:
+        """Calculate summary statistics across all metrics"""
+        if not self.metrics:
+            return {"error": "No metrics collected"}
+        
+        # Collect all values for each metric
+        prefill_times = [m.prefill_time_ms for m in self.metrics]
+        output_times = [m.output_time_ms for m in self.metrics]
+        total_times = [m.total_time_ms for m in self.metrics]
+        
+        input_tokens = [m.input_tokens for m in self.metrics]
+        output_tokens = [m.output_tokens for m in self.metrics]
+        
+        prefill_speeds = [m.prefill_speed_tps for m in self.metrics]
+        output_speeds = [m.output_speed_tps for m in self.metrics]
+        
+        pii_counts = [m.pii_items_found for m in self.metrics]
+        replacements = [m.replacements_made for m in self.metrics]
+        
+        def calculate_stats(values: List[float]) -> Dict:
+            """Calculate min, max, mean, median for a list of values"""
+            if not values:
+                return {"min": 0, "max": 0, "mean": 0, "median": 0}
+            
+            sorted_values = sorted(values)
+            n = len(sorted_values)
+            
+            return {
+                "min": min(values),
+                "max": max(values),
+                "mean": sum(values) / n,
+                "median": sorted_values[n // 2] if n % 2 == 1 else 
+                         (sorted_values[n // 2 - 1] + sorted_values[n // 2]) / 2
+            }
+        
+        summary = {
+            "total_conversations": len(self.metrics),
+            "timestamp": datetime.now().isoformat(),
+            
+            "timing_metrics": {
+                "prefill_time_ms": calculate_stats(prefill_times),
+                "output_time_ms": calculate_stats(output_times),
+                "total_time_ms": calculate_stats(total_times)
+            },
+            
+            "token_metrics": {
+                "input_tokens": calculate_stats(input_tokens),
+                "output_tokens": calculate_stats(output_tokens),
+                "total_input_tokens": sum(input_tokens),
+                "total_output_tokens": sum(output_tokens)
+            },
+            
+            "speed_metrics": {
+                "prefill_speed_tps": calculate_stats(prefill_speeds),
+                "output_speed_tps": calculate_stats(output_speeds)
+            },
+            
+            "sanitization_metrics": {
+                "pii_items_found": calculate_stats(pii_counts),
+                "replacements_made": calculate_stats(replacements),
+                "total_pii_found": sum(pii_counts),
+                "total_replacements": sum(replacements)
+            }
+        }
+        
+        return summary
+    
+    def save_metrics(self):
+        """Save all metrics and summary to files"""
+        # Save detailed metrics
+        metrics_data = [m.to_dict() for m in self.metrics]
+        with open(self.metrics_file, 'w') as f:
+            json.dump(metrics_data, f, indent=2)
+        
+        # Save summary
+        summary = self.calculate_summary()
+        with open(self.summary_file, 'w') as f:
+            json.dump(summary, f, indent=2)
+        
+        print(f"✅ Metrics saved to {self.metrics_file}")
+        print(f"✅ Summary saved to {self.summary_file}")
+    
+    def print_summary(self):
+        """Print a human-readable summary of metrics"""
+        summary = self.calculate_summary()
+        
+        print("\n" + "=" * 60)
+        print("PERFORMANCE SUMMARY")
+        print("=" * 60)
+        
+        print(f"\n📊 Total Conversations Processed: {summary['total_conversations']}")
+        
+        print("\n⏱️  Timing Metrics (milliseconds):")
+        timing = summary['timing_metrics']
+        print(f"   Prefill (TTFT): {timing['prefill_time_ms']['mean']:.2f} ms (median: {timing['prefill_time_ms']['median']:.2f})")
+        print(f"   Output Time:    {timing['output_time_ms']['mean']:.2f} ms (median: {timing['output_time_ms']['median']:.2f})")
+        print(f"   Total Time:     {timing['total_time_ms']['mean']:.2f} ms (median: {timing['total_time_ms']['median']:.2f})")
+        
+        print("\n📝 Token Metrics:")
+        tokens = summary['token_metrics']
+        print(f"   Average Input Tokens:  {tokens['input_tokens']['mean']:.1f}")
+        print(f"   Average Output Tokens: {tokens['output_tokens']['mean']:.1f}")
+        print(f"   Total Tokens Processed: {tokens['total_input_tokens'] + tokens['total_output_tokens']}")
+        
+        print("\n⚡ Speed Metrics (tokens/second):")
+        speed = summary['speed_metrics']
+        print(f"   Prefill Speed: {speed['prefill_speed_tps']['mean']:.1f} tok/s")
+        print(f"   Output Speed:  {speed['output_speed_tps']['mean']:.1f} tok/s")
+        
+        print("\n🔒 Sanitization Results:")
+        sanitization = summary['sanitization_metrics']
+        print(f"   Total PII Items Found:     {sanitization['total_pii_found']}")
+        print(f"   Total Replacements Made:   {sanitization['total_replacements']}")
+        print(f"   Average PII per Conversation: {sanitization['pii_items_found']['mean']:.1f}")
+        
+        print("\n" + "=" * 60)

chapter3/log-sanitization/regex_sanitizer.py

@@ -0,0 +1,279 @@
+"""
+基于规则（正则）的离线日志脱敏引擎
+
+与 agent.py 中依赖本地 LLM 的方案互补：本模块不需要任何模型或网络，
+纯靠正则表达式 + 校验算法（Luhn、身份证校验码）识别日志 / 工具输出中的
+敏感信息，速度快、结果确定，适合作为 Agent 日志落盘前的第一道防线。
+
+覆盖的敏感信息类别（按匹配优先级从高到低）：
+  - 私钥 / 证书（PEM 块）
+  - JWT
+  - 云厂商与第三方密钥（AWS AKIA、GitHub、Slack、Google、OpenAI 风格 sk-）
+  - HTTP Authorization: Bearer / Basic 令牌
+  - 配置中的口令 / 密钥赋值（password=..., token: ... 等）
+  - 邮箱地址
+  - 信用卡号（Luhn 校验）
+  - IBAN 国际银行账号
+  - 美国社会安全号（SSN）
+  - 中国大陆身份证号（校验码验证）
+  - 中国大陆手机号
+  - IPv4 地址
+
+每一类都会被替换为带类别标签的占位符（如 [REDACTED_API_KEY]），
+既隐去了原值，又保留了“这里原本是什么”的可读性，方便排障。
+"""
+
+import re
+from collections import Counter
+from typing import Dict, List, Tuple
+
+
+def _luhn_ok(number: str) -> bool:
+    """Luhn 校验，用于降低信用卡号的误报率"""
+    digits = [int(c) for c in number if c.isdigit()]
+    if not 13 <= len(digits) <= 19:
+        return False
+    checksum = 0
+    parity = len(digits) % 2
+    for i, d in enumerate(digits):
+        if i % 2 == parity:
+            d *= 2
+            if d > 9:
+                d -= 9
+        checksum += d
+    return checksum % 10 == 0
+
+
+def _cn_id_ok(value: str) -> bool:
+    """中国大陆二代身份证号（18 位）校验码验证"""
+    s = value.upper()
+    if len(s) != 18 or not s[:17].isdigit():
+        return False
+    weights = [7, 9, 10, 5, 8, 4, 2, 1, 6, 3, 7, 9, 10, 5, 8, 4, 2]
+    check_codes = "10X98765432"
+    total = sum(int(s[i]) * weights[i] for i in range(17))
+    return check_codes[total % 11] == s[17]
+
+
+# 每条规则：(类别, 占位符, 编译后的正则, 用于取值的分组号, 可选校验函数)
+# 分组号为 0 表示整段命中都要脱敏；为 N 表示只脱敏第 N 个捕获组（保留键名等上下文）。
+_RULES = [
+    (
+        "private_key", "[REDACTED_PRIVATE_KEY]",
+        # Truncated PEM (BEGIN without END) must still redact through EOF.
+        re.compile(
+            r"-----BEGIN (?:RSA |EC |DSA |OPENSSH |PGP )?PRIVATE KEY-----"
+            r"[\s\S]*?(?:-----END (?:RSA |EC |DSA |OPENSSH |PGP )?PRIVATE KEY-----|(?=\Z))"
+        ),
+        0, None,
+    ),
+    (
+        "jwt", "[REDACTED_JWT]",
+        re.compile(r"eyJ[A-Za-z0-9_-]{10,}\.eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}"),
+        0, None,
+    ),
+    (
+        # 连接串中的口令，如 postgres://user:PASSWORD@host:5432/db
+        "url_credential", "[REDACTED_URL_CRED]",
+        re.compile(r"://[^\s:/@]*:([^\s@]+)@"),
+        1, None,
+    ),
+    (
+        "aws_access_key", "[REDACTED_AWS_KEY]",
+        re.compile(r"\bAKIA[0-9A-Z]{16}\b"),
+        0, None,
+    ),
+    (
+        "github_token", "[REDACTED_GITHUB_TOKEN]",
+        re.compile(r"\b(?:gh[pousr]_[A-Za-z0-9]{36,}|github_pat_[A-Za-z0-9_]{20,})\b"),
+        0, None,
+    ),
+    (
+        "slack_token", "[REDACTED_SLACK_TOKEN]",
+        re.compile(r"\bxox[baprs]-[A-Za-z0-9-]{10,}\b"),
+        0, None,
+    ),
+    (
+        "google_api_key", "[REDACTED_GOOGLE_API_KEY]",
+        re.compile(r"\bAIza[0-9A-Za-z_-]{35}\b"),
+        0, None,
+    ),
+    (
+        "api_key", "[REDACTED_API_KEY]",
+        re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b"),
+        0, None,
+    ),
+    (
+        "bearer_token", "[REDACTED_BEARER_TOKEN]",
+        re.compile(r"(?i)\bBearer\s+([A-Za-z0-9._~+/=-]{10,})"),
+        1, None,
+    ),
+    (
+        "basic_auth", "[REDACTED_BASIC_AUTH]",
+        # Require Authorization: so English "Basic knowledge …" is not redacted.
+        re.compile(r"(?i)\bAuthorization\s*:\s*Basic\s+([A-Za-z0-9+/=]{4,})"),
+        1, None,
+    ),
+    (
+        "secret_assignment", "[REDACTED_SECRET]",
+        re.compile(
+            r"(?i)(?:password|passwd|pwd|secret|token|api[_-]?key|"
+            r"access[_-]?key|auth|credential)[\"']?\s*[=:]\s*"
+            r"(?:\"([^\"]{4,})\"|'([^']{4,})'|([^\s\"',}]{4,}))"
+        ),
+        (1, 2, 3), None,
+    ),
+    (
+        "email", "[REDACTED_EMAIL]",
+        re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"),
+        0, None,
+    ),
+    (
+        "credit_card", "[REDACTED_CREDIT_CARD]",
+        re.compile(r"\b(?:\d[ -]?){13,19}\b"),
+        0, _luhn_ok,
+    ),
+    (
+        "iban", "[REDACTED_IBAN]",
+        re.compile(r"\b[A-Z]{2}\d{2}[A-Z0-9]{11,30}\b"),
+        0, None,
+    ),
+    (
+        "us_ssn", "[REDACTED_SSN]",
+        re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
+        0, None,
+    ),
+    (
+        "cn_id_card", "[REDACTED_ID_CARD]",
+        re.compile(r"\b\d{17}[\dXx]\b"),
+        0, _cn_id_ok,
+    ),
+    (
+        "cn_phone", "[REDACTED_PHONE]",
+        re.compile(r"(?<!\d)1[3-9]\d{9}(?!\d)"),
+        0, None,
+    ),
+    (
+        "ip_address", "[REDACTED_IP]",
+        re.compile(r"\b(?:(?:25[0-5]|2[0-4]\d|1?\d?\d)\.){3}(?:25[0-5]|2[0-4]\d|1?\d?\d)\b"),
+        0, None,
+    ),
+]
+
+# 人类可读的类别中文名，用于打印汇总
+CATEGORY_LABELS = {
+    "private_key": "私钥 / 证书",
+    "jwt": "JWT 令牌",
+    "url_credential": "连接串凭据",
+    "aws_access_key": "AWS 访问密钥",
+    "github_token": "GitHub 令牌",
+    "slack_token": "Slack 令牌",
+    "google_api_key": "Google API Key",
+    "api_key": "API Key (sk-)",
+    "bearer_token": "Bearer 令牌",
+    "basic_auth": "Basic 认证",
+    "secret_assignment": "口令 / 密钥赋值",
+    "email": "邮箱地址",
+    "credit_card": "信用卡号",
+    "iban": "IBAN 银行账号",
+    "us_ssn": "美国社保号(SSN)",
+    "cn_id_card": "身份证号",
+    "cn_phone": "手机号",
+    "ip_address": "IP 地址",
+}
+
+
+def sanitize(text: str) -> Tuple[str, List[Dict]]:
+    """
+    对文本执行离线规则脱敏。
+
+    Returns:
+        - 脱敏后的文本
+        - 命中列表，每项为 {category, value, placeholder, start, end}
+    """
+    candidates: List[Dict] = []
+    for priority, (category, placeholder, pattern, group, validator) in enumerate(_RULES):
+        groups = group if isinstance(group, tuple) else (group,)
+        for m in pattern.finditer(text):
+            start = end = -1
+            for g in groups:
+                start, end = m.span(g)
+                if start >= 0:
+                    break
+            if start < 0:  # 该捕获组未参与本次匹配
+                continue
+            value = text[start:end]
+            if validator and not validator(value):
+                continue
+            candidates.append({
+                "category": category,
+                "placeholder": placeholder,
+                "value": value,
+                "start": start,
+                "end": end,
+                "priority": priority,
+            })
+
+    # 处理重叠：优先级高（数字小）的规则胜出，避免同一段被重复/错误脱敏
+    candidates.sort(key=lambda c: (c["priority"], c["start"]))
+    accepted: List[Dict] = []
+    for c in candidates:
+        if any(not (c["end"] <= a["start"] or c["start"] >= a["end"]) for a in accepted):
+            continue
+        accepted.append(c)
+
+    # 按位置顺序重建脱敏文本
+    accepted.sort(key=lambda c: c["start"])
+    parts: List[str] = []
+    last = 0
+    for c in accepted:
+        parts.append(text[last:c["start"]])
+        parts.append(c["placeholder"])
+        last = c["end"]
+    parts.append(text[last:])
+
+    findings = [
+        {k: c[k] for k in ("category", "value", "placeholder", "start", "end")}
+        for c in accepted
+    ]
+    return "".join(parts), findings
+
+
+def summarize(findings: List[Dict]) -> Counter:
+    """统计各类别命中次数"""
+    return Counter(f["category"] for f in findings)
+
+
+def print_report(name: str, original: str, redacted: str, findings: List[Dict]) -> None:
+    """打印单条样本的 before/after 与命中明细"""
+    print(f"\n{'=' * 64}")
+    print(f"样本: {name}  （命中 {len(findings)} 处敏感信息）")
+    print("=" * 64)
+    print("--- 脱敏前 (BEFORE) ---")
+    print(original.rstrip())
+    print("\n--- 脱敏后 (AFTER) ---")
+    print(redacted.rstrip())
+    if findings:
+        print("\n--- 命中明细 ---")
+        for f in findings:
+            label = CATEGORY_LABELS.get(f["category"], f["category"])
+            print(f"   [{label}] {f['value']}  ->  {f['placeholder']}")
+
+
+if __name__ == "__main__":
+    # 直接运行本模块时，对内置样本做一次快速演示
+    from samples import SAMPLES
+
+    total = Counter()
+    for name, text in SAMPLES:
+        redacted, findings = sanitize(text)
+        print_report(name, text, redacted, findings)
+        total.update(summarize(findings))
+
+    print(f"\n{'=' * 64}")
+    print("脱敏类别汇总")
+    print("=" * 64)
+    for category, count in total.most_common():
+        label = CATEGORY_LABELS.get(category, category)
+        print(f"   {label:<16} {count} 处")
+    print(f"\n   合计脱敏 {sum(total.values())} 处敏感信息")

chapter3/log-sanitization/requirements.txt

@@ -0,0 +1,9 @@
+# Core dependencies
+
+# Shared provider resolver from the repository root. Run this requirements file
+# from the experiment directory, as shown in the README.
+-e ../..
+
+ollama>=0.3.0
+pyyaml>=6.0
+python-dotenv>=1.0.0

chapter3/log-sanitization/samples.py

@@ -0,0 +1,65 @@
+"""
+用于日志脱敏演示的代表性样本
+
+这些样本模拟真实 Agent 运行时最容易泄露敏感信息的几种场景：
+工具调用的 HTTP 请求/响应、客服对话、数据库连接报错、CI/Git 日志、
+以及配置转储。它们混合了密钥类（API Key、令牌、私钥）与 PII 类
+（身份证、手机号、信用卡、邮箱）敏感信息，便于展示脱敏的覆盖面。
+
+注意：以下所有密钥、卡号、证件号均为虚构，仅用于演示，不对应任何真实账户。
+"""
+
+OPENAI_STYLE_SAMPLE_KEY = "sk" + "-proj-ABCD1234efgh5678IJKL9012mnop3456qrst"
+GOOGLE_SAMPLE_KEY = "AI" + "zaSyD-EXAMPLEfakeKEY1234567890abcdef12"
+AWS_SAMPLE_KEY = "AK" + "IAIOSFODNN7EXAMPLE"
+GITHUB_SAMPLE_TOKEN = "gh" + "p_16C7e42F292c6912E7710c838347Ae178B4a99"
+SLACK_SAMPLE_TOKEN = "xo" + "xb-PLACEHOLDERfaketoken000000notarealslacktoken"
+JWT_SAMPLE_TOKEN = (
+    "ey" + "JhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"
+    ".eyJzdWIiOiIxMjM0NTY3ODkwIn0"
+    ".SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c"
+)
+PEM_BEGIN = "-----BEGIN " + "RSA PRIVATE KEY-----"
+PEM_END = "-----END " + "RSA PRIVATE KEY-----"
+
+SAMPLES = [
+    (
+        "工具调用日志 (HTTP 请求/响应)",
+        f"""[2024-05-12 09:14:22] TOOL_CALL http_request
+  url: https://api.example.com/v1/users/8842
+  headers: {{"Authorization": "Bearer {OPENAI_STYLE_SAMPLE_KEY}", "X-Api-Key": "{GOOGLE_SAMPLE_KEY}"}}
+  response: {{"user_id": 8842, "email": "alice.wang@example.com", "phone": "13912345678"}}""",
+    ),
+    (
+        "客服对话 (PII 泄露)",
+        """USER: 你好，我要办理报销，我的身份证号是 11010119900307721X，手机号 13800138000。
+ASSISTANT: 好的，请再提供一下银行卡号以便核对。
+USER: 卡号是 4111 1111 1111 1111，另外我的美国社保号是 123-45-6789。
+ASSISTANT: 收到，我这就为您登记。""",
+    ),
+    (
+        "数据库连接报错 (凭据泄露)",
+        f"""[ERROR] db.connect failed after 3 retries
+  dsn: postgres://admin:S3cr3t_P4ssw0rd@db.internal:5432/prod
+  fallback_config: {{"db_password": "hunter2xyz", "aws_access_key_id": "{AWS_SAMPLE_KEY}"}}
+  host_ip: 192.168.10.24""",
+    ),
+    (
+        "CI / Git 日志 (令牌泄露)",
+        f"""Cloning into 'service-repo'...
+  remote: using deploy token {GITHUB_SAMPLE_TOKEN}
+  Slack notify webhook token: {SLACK_SAMPLE_TOKEN}
+  session jwt={JWT_SAMPLE_TOKEN}""",
+    ),
+    (
+        "配置转储 (私钥泄露)",
+        f"""[DEBUG] dumping runtime config
+  service_account_key: |
+{PEM_BEGIN}
+MIIEpAIBAAKCAQEA7QwZbq3vX9kLmN0pQrStUvWxYz1234567890abcdefghijkl
+mnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0987654321zyxwvutsrqponm
+QIDAQAB
+{PEM_END}
+  admin_contact: ops-team@example.com""",
+    ),
+]

chapter3/log-sanitization/test_loader.py

@@ -0,0 +1,173 @@
+"""
+Test Case Loader for User Memory Evaluation Framework
+"""
+
+import sys
+import json
+import subprocess
+from pathlib import Path
+from typing import Dict, List, Optional, Any
+from config import EVAL_FRAMEWORK_PATH
+
+class TestCaseLoader:
+    """Load test cases from user-memory-evaluation framework"""
+    
+    def __init__(self):
+        self.eval_framework_path = EVAL_FRAMEWORK_PATH
+        if not self.eval_framework_path.exists():
+            raise ValueError(f"Evaluation framework not found at {self.eval_framework_path}")
+    
+    def get_all_test_cases(self) -> List[Dict[str, Any]]:
+        """Get all available test cases"""
+        script = """
+import sys
+import json
+from pathlib import Path
+import io
+
+# Suppress rich console output
+import rich.console
+rich.console.Console = lambda *args, **kwargs: type('FakeConsole', (), {
+    'print': lambda self, *a, **k: None,
+    '__getattr__': lambda self, name: lambda *a, **k: None
+})()
+
+# Redirect output
+old_stdout = sys.stdout
+sys.stdout = io.StringIO()
+
+try:
+    from framework import UserMemoryEvaluationFramework
+    
+    framework = UserMemoryEvaluationFramework()
+    test_cases = []
+    
+    for tc in framework.list_test_cases():
+        test_cases.append({
+            'test_id': tc.test_id,
+            'category': tc.category,
+            'title': tc.title,
+            'description': tc.description,
+            'num_conversations': len(tc.conversation_histories),
+            'user_question': tc.user_question
+        })
+    
+    # Restore stdout for JSON output
+    sys.stdout = old_stdout
+    print(json.dumps(test_cases))
+    
+except Exception as e:
+    sys.stdout = old_stdout
+    print(json.dumps([]))
+"""
+        
+        result = subprocess.run(
+            [sys.executable, "-c", script],
+            cwd=self.eval_framework_path,
+            capture_output=True,
+            text=True
+        )
+        
+        if result.returncode != 0:
+            print(f"Error getting test cases: {result.stderr}")
+            return []
+        
+        try:
+            return json.loads(result.stdout)
+        except json.JSONDecodeError:
+            print(f"Error parsing test cases JSON")
+            return []
+    
+    def get_layer3_test_cases(self) -> List[Dict[str, Any]]:
+        """Get only Layer 3 test cases (most complex)"""
+        all_cases = self.get_all_test_cases()
+        return [tc for tc in all_cases if tc['category'] == 'layer3']
+    
+    def get_test_case_conversations(self, test_id: str) -> List[Dict[str, Any]]:
+        """Get detailed conversation histories for a specific test case"""
+        script = f"""
+import sys
+import json
+from pathlib import Path
+import io
+
+# Redirect stdout to suppress any print statements from framework
+old_stdout = sys.stdout
+sys.stdout = io.StringIO()
+
+try:
+    from framework import UserMemoryEvaluationFramework
+    
+    framework = UserMemoryEvaluationFramework()
+    tc = framework.get_test_case("{test_id}")
+    
+    # Restore stdout for our JSON output
+    sys.stdout = old_stdout
+    
+    if not tc:
+        print(json.dumps([]))
+    else:
+        conversations = []
+        for conv in tc.conversation_histories:
+            conv_data = {{
+                'conversation_id': conv.conversation_id,
+                'timestamp': conv.timestamp,
+                'messages': []
+            }}
+            
+            for msg in conv.messages:
+                msg_data = {{
+                    'role': msg.role.value,
+                    'content': msg.content
+                }}
+                # Add metadata if it exists
+                if hasattr(msg, 'metadata'):
+                    msg_data['metadata'] = msg.metadata
+                conv_data['messages'].append(msg_data)
+            
+            conversations.append(conv_data)
+        
+        print(json.dumps(conversations))
+        
+except Exception as e:
+    import traceback
+    sys.stdout = old_stdout
+    sys.stderr.write(traceback.format_exc())
+    print(json.dumps([]))
+"""
+        
+        result = subprocess.run(
+            [sys.executable, "-c", script],
+            cwd=self.eval_framework_path,
+            capture_output=True,
+            text=True
+        )
+        
+        if result.returncode != 0:
+            print(f"Error getting conversation histories: {result.stderr}")
+            return []
+        
+        try:
+            return json.loads(result.stdout)
+        except json.JSONDecodeError as e:
+            print(f"Error parsing conversation histories JSON: {e}")
+            if result.stdout:
+                print(f"stdout (first 500 chars): {result.stdout[:500]}")
+            if result.stderr:
+                print(f"stderr (first 500 chars): {result.stderr[:500]}")
+            return []
+    
+    def format_conversation_text(self, conversation: Dict[str, Any]) -> str:
+        """Format a conversation into readable text"""
+        lines = []
+        lines.append(f"Conversation ID: {conversation['conversation_id']}")
+        lines.append(f"Timestamp: {conversation['timestamp']}")
+        lines.append("-" * 50)
+        
+        for msg in conversation['messages']:
+            role = msg['role'].upper()
+            content = msg['content']
+            lines.append(f"{role}: {content}")
+            lines.append("")  # Empty line between messages
+        
+        return "\n".join(lines)

chapter3/log-sanitization/tests/conftest.py

@@ -0,0 +1,22 @@
+"""Shared bootstrap for log-sanitization regression tests."""
+
+import sys
+from pathlib import Path
+from types import ModuleType, SimpleNamespace
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[1]
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+try:
+    import ollama  # noqa: F401
+except ImportError:
+    ollama_stub = ModuleType("ollama")
+    ollama_stub.Client = object
+    sys.modules["ollama"] = ollama_stub
+
+try:
+    import dotenv  # noqa: F401
+except ImportError:
+    sys.modules["dotenv"] = SimpleNamespace(load_dotenv=lambda *args, **kwargs: None)

chapter3/log-sanitization/tests/manual/loader_debug.py

@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+"""Debug script to test loading conversations"""
+
+import sys
+from pathlib import Path
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+from test_loader import TestCaseLoader
+
+def main():
+    loader = TestCaseLoader()
+    
+    # Get all test cases
+    print("Getting all test cases...")
+    all_cases = loader.get_all_test_cases()
+    print(f"Found {len(all_cases)} test cases")
+    
+    # Get Layer 3 test cases
+    layer3_cases = loader.get_layer3_test_cases()
+    print(f"Found {len(layer3_cases)} Layer 3 test cases")
+    
+    if layer3_cases:
+        # Try to load the first one
+        first_case = layer3_cases[0]
+        print(f"\nTrying to load: {first_case['test_id']}")
+        
+        conversations = loader.get_test_case_conversations(first_case['test_id'])
+        
+        if conversations:
+            print(f"Successfully loaded {len(conversations)} conversations")
+            # Print first conversation snippet
+            if conversations[0]['messages']:
+                print(f"First message: {conversations[0]['messages'][0]['content'][:100]}...")
+        else:
+            print("Failed to load conversations")
+
+if __name__ == "__main__":
+    main()

chapter3/log-sanitization/tests/test_authorization_basic.py

@@ -0,0 +1,40 @@
+"""Authorization: Basic credentials must be redacted like Bearer tokens."""
+from regex_sanitizer import sanitize
+
+
+def test_authorization_basic_redacted():
+    cred = "dXNlcjpwYXNzd29yZA=="
+    text, hits = sanitize(f"Authorization: Basic {cred}")
+    assert cred not in text
+    assert "[REDACTED_BASIC_AUTH]" in text
+    assert any(h["category"] == "basic_auth" for h in hits)
+
+
+def test_authorization_basic_case_insensitive():
+    cred = "YWRtaW46c2VjcmV0"
+    text, hits = sanitize(f"authorization: basic {cred}")
+    assert cred not in text
+    assert "[REDACTED_BASIC_AUTH]" in text
+
+
+def test_bearer_still_redacted():
+    token = "aaaaaaaaaaaaaaaaaaaa"
+    text, hits = sanitize(f"Authorization: Bearer {token}")
+    assert token not in text
+    assert "[REDACTED_BEARER_TOKEN]" in text
+    assert any(h["category"] == "bearer_token" for h in hits)
+
+
+def test_english_basic_prose_not_redacted():
+    prose = "Basic knowledge of Python is required."
+    text, hits = sanitize(prose)
+    assert text == prose
+    assert not any(h["category"] == "basic_auth" for h in hits)
+
+
+def test_www_authenticate_basic_realm_not_treated_as_credential():
+    # Challenge header names the scheme; it does not carry the password blob.
+    line = 'WWW-Authenticate: Basic realm="api"'
+    text, hits = sanitize(line)
+    assert text == line
+    assert not any(h["category"] == "basic_auth" for h in hits)

chapter3/log-sanitization/tests/test_github_pat.py

@@ -0,0 +1,18 @@
+"""Regression: fine-grained github_pat_* tokens must be redacted."""
+from regex_sanitizer import sanitize
+
+
+def test_github_pat_fine_grained_redacted():
+    token = "github_pat_" + "A" * 20 + "_" + "B" * 40
+    text, hits = sanitize(f"Authorization: {token}")
+    assert token not in text
+    assert "[REDACTED_GITHUB_TOKEN]" in text
+    assert any(h["category"] == "github_token" for h in hits)
+
+
+def test_classic_github_token_still_redacted():
+    token = "gh" + "p_" + "x" * 36
+    text, hits = sanitize(token)
+    assert token not in text
+    assert "[REDACTED_GITHUB_TOKEN]" in text
+    assert any(h["category"] == "github_token" for h in hits)

chapter3/log-sanitization/tests/test_null_pii_values.py

@@ -0,0 +1,33 @@
+"""detect_pii must treat JSON null pii_values like an empty list."""
+import json
+from unittest.mock import patch
+
+from agent import LogSanitizationAgent
+
+
+def test_null_pii_values_returns_empty_list():
+    agent = object.__new__(LogSanitizationAgent)
+    agent.count_tokens = lambda text: len(text) // 4
+    agent._chat_stream = lambda messages: iter(
+        [json.dumps({"pii_values": None})]
+    )
+
+    with patch("builtins.print"):
+        pii_values, metrics = agent.detect_pii("Alice phone 555-0100")
+
+    assert pii_values == []
+    assert metrics["pii_items_found"] == 0
+
+
+def test_list_pii_values_still_cleaned():
+    agent = object.__new__(LogSanitizationAgent)
+    agent.count_tokens = lambda text: len(text) // 4
+    agent._chat_stream = lambda messages: iter(
+        [json.dumps({"pii_values": ["  Alice  ", "-", "Bob"]})]
+    )
+
+    with patch("builtins.print"):
+        pii_values, metrics = agent.detect_pii("text")
+
+    assert pii_values == ["Alice", "Bob"]
+    assert metrics["pii_items_found"] == 2

chapter3/log-sanitization/tests/test_quoted_secret_spaces.py

@@ -0,0 +1,23 @@
+"""Regression: quoted secret assignments must redact the full value, including spaces."""
+from regex_sanitizer import sanitize
+
+
+def test_double_quoted_password_with_spaces():
+    text, hits = sanitize('password="hunter 2 with spaces"')
+    assert "hunter" not in text
+    assert "spaces" not in text
+    assert "[REDACTED_SECRET]" in text
+    assert any(h["category"] == "secret_assignment" for h in hits)
+
+
+def test_single_quoted_password_with_spaces():
+    text, hits = sanitize("api_key='test-api-key with spaces'")
+    assert "test-api-key" not in text
+    assert "ghi" not in text
+    assert "[REDACTED_SECRET]" in text
+
+
+def test_unquoted_secret_still_redacted():
+    text, hits = sanitize("password=hunter2xyz")
+    assert "hunter2xyz" not in text
+    assert "[REDACTED_SECRET]" in text

chapter3/log-sanitization/tests/test_sanitize_conversation.py

@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+"""Regression tests for sanitize_conversation() in agent.py.
+
+Bug: detect_pii() catches any backend exception and returns ([], {}), but
+sanitize_conversation then subscripted the empty metrics dict
+(perf_metrics['input_tokens']) -> KeyError that killed the whole batch.
+Fixed with .get(..., 0) defaults so a dead backend degrades gracefully.
+"""
+
+import pytest
+
+from agent import LogSanitizationAgent
+from metrics import MetricsCollector
+
+
+def _make_agent(client, tmp_path):
+    """Build an agent without __init__ (which requires a live Ollama)."""
+    ag = LogSanitizationAgent.__new__(LogSanitizationAgent)
+    ag.model = "qwen3:0.6b"
+    ag.backend = "ollama"
+    ag.metrics_collector = MetricsCollector(tmp_path)
+    ag.client = client
+    return ag
+
+
+CONV = {
+    "conversation_id": "demo_001",
+    "messages": [{"role": "user", "content": "My SSN is 123-45-6789."}],
+}
+
+
+class _DeadClient:
+    def chat(self, **kwargs):
+        raise ConnectionError("[test] Ollama server is not running")
+
+
+class _FakeClient:
+    """Mimics ollama.Client.chat(stream=True) chunk shape."""
+
+    def __init__(self, payload: str):
+        self.payload = payload
+
+    def chat(self, **kwargs):
+        return [{"message": {"content": self.payload}}]
+
+
+def test_dead_backend_returns_result_with_zero_metrics(tmp_path):
+    ag = _make_agent(_DeadClient(), tmp_path)
+    result = ag.sanitize_conversation(CONV, "t1")  # must not raise
+    assert result["pii_found"] == []
+    assert result["replacements_made"] == 0
+    assert result["metrics"]["input_tokens"] == 0
+    assert result["metrics"]["pii_items_found"] == 0
+
+
+def test_working_backend_still_detects_pii(tmp_path):
+    ag = _make_agent(_FakeClient('{"pii_values": ["123-45-6789"]}'), tmp_path)
+    result = ag.sanitize_conversation(CONV, "t1")
+    assert result["pii_found"] == ["123-45-6789"]
+    assert result["replacements_made"] == 1
+    assert "[REDACTED]" in result["sanitized_text"]
+    assert result["metrics"]["pii_items_found"] == 1
+
+
+def test_working_backend_with_structured_pii_items(tmp_path):
+    conv = {
+        "conversation_id": "demo_002",
+        "messages": [
+            {
+                "role": "user",
+                "content": "My SSN is 000-00-0000 and my card is 0000-0000-0000-0000.",
+            }
+        ],
+    }
+    payload = (
+        '{"pii_items": ['
+        '{"type": "social_security_number", "value": "000-00-0000"}, '
+        '{"type": "credit_card_number", "value": "0000-0000-0000-0000"}'
+        ']}'
+    )
+    ag = _make_agent(_FakeClient(payload), tmp_path)
+    result = ag.sanitize_conversation(conv, "t2")
+    assert result["pii_found"] == ["000-00-0000", "0000-0000-0000-0000"]
+    assert result["replacements_made"] == 2
+    assert result["sanitized_text"].count("[REDACTED]") == 2
+    assert result["metrics"]["pii_items_found"] == 2
+
+
+def test_structured_pii_items_preserve_original_value(tmp_path):
+    conv = {
+        "conversation_id": "demo_003",
+        "messages": [
+            {
+                "role": "user",
+                "content": "The secret is -abc- and the password is   p4ss  .",
+            }
+        ],
+    }
+    payload = (
+        '{"pii_items": ['
+        '{"type": "secret", "value": "-abc-"}, '
+        '{"type": "password", "value": "  p4ss  "}'
+        ']}'
+    )
+    ag = _make_agent(_FakeClient(payload), tmp_path)
+    result = ag.sanitize_conversation(conv, "t3")
+    assert result["pii_found"] == ["-abc-", "  p4ss  "]
+    assert result["replacements_made"] == 2
+    assert result["sanitized_text"].count("[REDACTED]") == 2
+
+
+def test_pii_items_metric_excludes_rejected_and_malformed_items(tmp_path):
+    conv = {
+        "conversation_id": "demo_004",
+        "messages": [
+            {
+                "role": "user",
+                "content": "My email is alice@example.com.",
+            }
+        ],
+    }
+    payload = (
+        '{"pii_items": ['
+        '{"type": "email", "value": "alice@example.com"}, '
+        '{"type": "ssn", "value": "999-99-9999"}, '
+        '{"type": "unknown", "value": ""}, '
+        '{"type": "broken"}, '
+        '"just a string"'
+        ']}'
+    )
+    ag = _make_agent(_FakeClient(payload), tmp_path)
+    result = ag.sanitize_conversation(conv, "t4")
+    assert result["pii_found"] == ["alice@example.com"]
+    assert result["replacements_made"] == 1
+    assert result["metrics"]["pii_items_found"] == 1
+    items = result["pii_items"]
+    assert len(items) == 1
+    assert items[0]["type"] == "email"
+    assert items[0]["value"] == "alice@example.com"

chapter3/log-sanitization/tests/test_truncated_pem.py

@@ -0,0 +1,36 @@
+"""Truncated PEM (BEGIN without END) must be redacted, not leaked."""
+
+from regex_sanitizer import sanitize
+
+PEM_HEADER = "-----BEGIN " + "RSA PRIVATE KEY-----\n"
+PEM_FOOTER = "-----END " + "RSA PRIVATE KEY-----"
+
+
+def test_truncated_rsa_pem_without_end_redacted():
+    blob = (
+        PEM_HEADER +
+        "MIIEowIBAAKCAQEA0Z3VS5JJcds3xfn/ygWyF6PZGFw7\n"
+        "ygWyF6PZGFw7morekeymaterialHERE"
+    )
+    text, hits = sanitize(f"key dump:\n{blob}\n")
+    assert "MIIEowIBAAKCAQEA" not in text
+    assert "[REDACTED_PRIVATE_KEY]" in text
+    assert any(h["category"] == "private_key" for h in hits)
+
+
+def test_complete_pem_still_redacted():
+    blob = (
+        PEM_HEADER +
+        "MIIEowIBAAKCAQEA0Z3VS5JJcds3xfn/ygWyF6PZGFw7\n" +
+        PEM_FOOTER
+    )
+    text, hits = sanitize(blob)
+    assert "MIIEowIBAAKCAQEA" not in text
+    assert text.strip() == "[REDACTED_PRIVATE_KEY]"
+    assert any(h["category"] == "private_key" for h in hits)
+
+
+def test_non_key_text_unchanged():
+    text, hits = sanitize("no secrets here, only BEGIN of a story")
+    assert text == "no secrets here, only BEGIN of a story"
+    assert hits == []

chapter3/log-sanitization/tests/test_url_credential_colon_slash.py

@@ -0,0 +1,23 @@
+"""Regression: URL passwords containing ':' or '/' must be fully redacted."""
+from regex_sanitizer import sanitize
+
+
+def test_password_with_slash_redacted():
+    text, hits = sanitize("DATABASE_URL=postgres://alice:a/b@db.example:5432/app")
+    assert "a/b" not in text
+    assert "[REDACTED_URL_CRED]" in text
+    assert any(h["category"] == "url_credential" for h in hits)
+
+
+def test_password_with_colon_redacted():
+    text, hits = sanitize("redis://default:foo:bar@10.0.0.1:6379/0")
+    assert "foo:bar" not in text
+    assert "[REDACTED_URL_CRED]" in text
+    assert any(h["category"] == "url_credential" for h in hits)
+
+
+def test_simple_password_still_redacted():
+    text, hits = sanitize("postgres://alice:secret@db.example:5432/app")
+    assert "secret" not in text
+    assert "[REDACTED_URL_CRED]" in text
+    assert any(h["category"] == "url_credential" for h in hits)

chapter3/log-sanitization/tests/test_url_credential_empty_user.py

@@ -0,0 +1,22 @@
+"""Regression: URL credentials with an empty username must be redacted."""
+from regex_sanitizer import sanitize
+
+
+def test_redis_empty_user_password_redacted():
+    text, hits = sanitize("redis://:secretpass@10.0.0.1:6379/0")
+    assert "secretpass" not in text
+    assert "[REDACTED_URL_CRED]" in text
+    assert any(h["category"] == "url_credential" for h in hits)
+
+
+def test_postgres_empty_user_password_redacted():
+    text, hits = sanitize("DATABASE_URL=postgres://:hunter2@localhost:5432/db")
+    assert "hunter2" not in text
+    assert "[REDACTED_URL_CRED]" in text
+    assert any(h["category"] == "url_credential" for h in hits)
+
+
+def test_named_user_password_still_redacted():
+    text, hits = sanitize("redis://default:secretpass@10.0.0.1:6379/0")
+    assert "secretpass" not in text
+    assert "[REDACTED_URL_CRED]" in text

chapter3/log-sanitization/validation/latest.json

@@ -0,0 +1,52 @@
+{
+  "schema_version": "chapter3-evidence-v1",
+  "experiment": "3-3",
+  "run_id": "20260729T185603Z-3_3-7f84f416",
+  "created_at": "2026-07-29T18:56:03.514546+00:00",
+  "status": "passed",
+  "run_dir": "/Users/boj/book/ai-agent-book/chapter3/log-sanitization/validation/runs/20260729T185603Z-3_3-7f84f416",
+  "artifacts": {
+    "evidence.json": "9a07a5b4e99b7f8468170a3c4593be2f7b4a703e2ac6fd28a9d3390c49c9f7f7",
+    "receipts.json": "3faf1e5e59ed7679d5f26ee23e3c61081566e7391af5ad2ab9d9a927afcf4e86",
+    "manifest.json": "c8bca3e33bce0ca06ab9f6d695cc8b5481ac71861b41f35940d50c34007702b9"
+  },
+  "inputs": [],
+  "summary": {
+    "regex": {
+      "cases": 12,
+      "mean_precision": 0.4166666666666666,
+      "mean_recall": 0.5,
+      "mean_typed_exact": 0.4166666666666667,
+      "residual_leaks": 9,
+      "mean_utility": 0.954197478750736,
+      "mean_latency_ms": 0.09875
+    },
+    "llm": {
+      "cases": 12,
+      "mean_precision": 0.5138888888888888,
+      "mean_recall": 0.6944444444444445,
+      "mean_typed_exact": 0.3333333333333333,
+      "residual_leaks": 3,
+      "mean_utility": 0.7548189868403488,
+      "mean_latency_ms": 3010.9979999999996
+    },
+    "hybrid": {
+      "cases": 12,
+      "mean_precision": 0.4861111111111111,
+      "mean_recall": 0.7777777777777777,
+      "mean_typed_exact": 0.5,
+      "residual_leaks": 2,
+      "mean_utility": 0.8744827477131053,
+      "mean_latency_ms": 2951.5082500000003
+    }
+  },
+  "acceptance": {
+    "local_model": true,
+    "qwen3_model": true,
+    "structured_type_location_confidence": true,
+    "structured_semistructured_natural_language_cases": true,
+    "regex_llm_hybrid_compared": true,
+    "leakage_utility_latency_measured": true,
+    "passed": true
+  }
+}

chapter3/log-sanitization/validation/runs/20260729T181925Z-3_3-cd097048/manifest.json

@@ -0,0 +1,51 @@
+{
+  "schema_version": "chapter3-evidence-v1",
+  "experiment": "3-3",
+  "run_id": "20260729T181925Z-3_3-cd097048",
+  "created_at": "2026-07-29T18:19:25.261714+00:00",
+  "status": "partial",
+  "run_dir": "/Users/boj/book/ai-agent-book/chapter3/log-sanitization/validation/runs/20260729T181925Z-3_3-cd097048",
+  "artifacts": {
+    "evidence.json": "7eea7d8235ca381b2cc0354b857968bc08c735443d843f641d95d491fd8e203c",
+    "receipts.json": "6ea59a988a86a659fbb83032d8f7c6df5ffb1120e70e86538251b78cd2281c56"
+  },
+  "inputs": [],
+  "summary": {
+    "regex": {
+      "cases": 1,
+      "mean_precision": 1.0,
+      "mean_recall": 0.5,
+      "mean_typed_exact": 0.5,
+      "residual_leaks": 1,
+      "mean_utility": 1.0,
+      "mean_latency_ms": 0.037
+    },
+    "llm": {
+      "cases": 1,
+      "mean_precision": 1.0,
+      "mean_recall": 1.0,
+      "mean_typed_exact": 0.0,
+      "residual_leaks": 0,
+      "mean_utility": 1.0,
+      "mean_latency_ms": 3959.616
+    },
+    "hybrid": {
+      "cases": 1,
+      "mean_precision": 1.0,
+      "mean_recall": 1.0,
+      "mean_typed_exact": 1.0,
+      "residual_leaks": 0,
+      "mean_utility": 1.0,
+      "mean_latency_ms": 4287.079
+    }
+  },
+  "acceptance": {
+    "local_model": true,
+    "qwen3_model": true,
+    "structured_type_location_confidence": true,
+    "structured_semistructured_natural_language_cases": false,
+    "regex_llm_hybrid_compared": true,
+    "leakage_utility_latency_measured": true,
+    "passed": false
+  }
+}

chapter3/log-sanitization/validation/runs/20260729T181925Z-3_3-cd097048/receipts.json

@@ -0,0 +1,176 @@
+[
+  {
+    "purpose": "llm:structured_cn",
+    "provider": "ollama-local",
+    "endpoint": "http://127.0.0.1:11434",
+    "model": "qwen3:0.6b",
+    "request": {
+      "model": "qwen3:0.6b",
+      "messages": [
+        {
+          "role": "system",
+          "content": "You detect sensitive information in logs. Return exact substrings only.\nTypes include cn_id_card, cn_phone, us_ssn, credit_card, password, secret,\naddress, medical_record, medical_diagnosis, treatment, passport, bank_account,\nrouting_number, bearer_token, email, private_key, and api_key. Distinguish a\nreal disclosed value from a field name, instruction, product code, metric, or\nexplicit decoy. start/end are zero-based Python slice offsets and confidence is\n0..1. Return JSON matching the schema; never redact or paraphrase the value."
+        },
+        {
+          "role": "user",
+          "content": "客户身份证号11010519491231002X，手机号13800138000。"
+        }
+      ],
+      "format": {
+        "type": "object",
+        "properties": {
+          "findings": {
+            "type": "array",
+            "items": {
+              "type": "object",
+              "properties": {
+                "type": {
+                  "type": "string"
+                },
+                "value": {
+                  "type": "string"
+                },
+                "start": {
+                  "type": "integer"
+                },
+                "end": {
+                  "type": "integer"
+                },
+                "confidence": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "type",
+                "value",
+                "start",
+                "end",
+                "confidence"
+              ]
+            }
+          }
+        },
+        "required": [
+          "findings"
+        ]
+      },
+      "options": {
+        "temperature": 0,
+        "seed": 37,
+        "num_predict": 1200
+      },
+      "stream": false
+    },
+    "response": {
+      "model": "qwen3:0.6b",
+      "created_at": "2026-07-29T18:19:20.944483Z",
+      "done": true,
+      "done_reason": "stop",
+      "total_duration": 3957502042,
+      "load_duration": 57011625,
+      "prompt_eval_count": 445,
+      "prompt_eval_duration": 63544667,
+      "eval_count": 76,
+      "eval_duration": 335304839,
+      "message": {
+        "role": "assistant",
+        "content": "{\"findings\":[{\"type\":\"secret\",\"value\":\"11010519491231002X\",\"start\":0,\"end\":18,\"confidence\":1},{\"type\":\"secret\",\"value\":\"13800138000\",\"start\":0,\"end\":11,\"confidence\":1}]}",
+        "thinking": "好的，我现在需要处理用户提供的客户信息，并检查是否有敏感数据。用户给出的客户身份证号是11010519491231002X，手机号是13800138000。首先，我要确认这些数据是否符合敏感信息的定义。\n\n根据用户提供的类型，敏感信息包括身份证号、手机号等。身份证号有18位，而手机号是11位，但用户提供的都是18位和11位，这可能是一个错误，因为通常身份证号是18位，而手机号通常是11位。不过用户可能输入时有误，或者需要确认是否正确。同样，手机号是13800138000，共11位，这可能是一个错误，但用户可能希望保留。\n\n接下来，我需要检查是否有其他可能的字段或类型被提及，但用户的信息中只有这两个部分。因此，正确的做法是返回这两个部分的值，但需要确保它们是敏感信息。例如，身份证号和手机号都是敏感信息，因此应该直接返回它们的值，而无需额外处理。\n\n最后，确保返回的JSON格式正确，没有其他字段或错误信息，并且只包含敏感数据。这样用户可以清楚知道信息是否被泄露。\n",
+        "images": null,
+        "tool_calls": null
+      }
+    },
+    "latency_ms": 3959.616,
+    "usage": {
+      "prompt_tokens": 445,
+      "completion_tokens": 76
+    }
+  },
+  {
+    "purpose": "hybrid-after-regex:structured_cn",
+    "provider": "ollama-local",
+    "endpoint": "http://127.0.0.1:11434",
+    "model": "qwen3:0.6b",
+    "request": {
+      "model": "qwen3:0.6b",
+      "messages": [
+        {
+          "role": "system",
+          "content": "You detect sensitive information in logs. Return exact substrings only.\nTypes include cn_id_card, cn_phone, us_ssn, credit_card, password, secret,\naddress, medical_record, medical_diagnosis, treatment, passport, bank_account,\nrouting_number, bearer_token, email, private_key, and api_key. Distinguish a\nreal disclosed value from a field name, instruction, product code, metric, or\nexplicit decoy. start/end are zero-based Python slice offsets and confidence is\n0..1. Return JSON matching the schema; never redact or paraphrase the value."
+        },
+        {
+          "role": "user",
+          "content": "客户身份证号11010519491231002X，手机号[REDACTED_CN_PHONE]。"
+        }
+      ],
+      "format": {
+        "type": "object",
+        "properties": {
+          "findings": {
+            "type": "array",
+            "items": {
+              "type": "object",
+              "properties": {
+                "type": {
+                  "type": "string"
+                },
+                "value": {
+                  "type": "string"
+                },
+                "start": {
+                  "type": "integer"
+                },
+                "end": {
+                  "type": "integer"
+                },
+                "confidence": {
+                  "type": "number"
+                }
+              },
+              "required": [
+                "type",
+                "value",
+                "start",
+                "end",
+                "confidence"
+              ]
+            }
+          }
+        },
+        "required": [
+          "findings"
+        ]
+      },
+      "options": {
+        "temperature": 0,
+        "seed": 37,
+        "num_predict": 1200
+      },
+      "stream": false
+    },
+    "response": {
+      "model": "qwen3:0.6b",
+      "created_at": "2026-07-29T18:19:25.232685Z",
+      "done": true,
+      "done_reason": "stop",
+      "total_duration": 4285037500,
+      "load_duration": 58636292,
+      "prompt_eval_count": 565,
+      "prompt_eval_duration": 80994959,
+      "eval_count": 75,
+      "eval_duration": 335223796,
+      "message": {
+        "role": "assistant",
+        "content": "{\"findings\":[{\"type\":\"cn_id_card\",\"value\":\"11010519491231002X\",\"start\":0,\"end\":18,\"confidence\":1},{\"type\":\"cn_phone\",\"value\":\"[REDACTED_CN_PHONE]\",\"start\":0,\"end\":0,\"confidence\":1}]}",
+        "thinking": "好的，我现在需要处理用户提供的敏感信息，并按照要求返回精确的子字符串。用户给出的客户身份证号是11010519491231002X，手机号是[REDACTED_CN_PHONE]。首先，我要确认用户的需求，他们希望返回的值是身份证号和手机号中的敏感字段，但不包括其他信息。\n\n首先，身份证号的长度是18位，而用户提供的身份证号是11010519491231002X，这明显超过了标准的18位，所以可能用户输入时有误，或者存在其他问题。不过根据用户的要求，只要提取出身份证号和手机号中的敏感字段，不管是否正确，都要返回。因此，我需要检查身份证号中的敏感字段，比如可能的身份证号字段，但用户提供的身份证号是11010519491231002X，其中包含的字符是11010519491231002X，这可能是一个身份证号的前几位，但需要确认是否符合标准。不过根据用户的要求，只要提取出这些字段，不管是否正确，所以直接返回身份证号中的敏感部分，即11010519491231002X，手机号中的[REDACTED_CN_PHONE]，但需要确保没有其他字段被包含。\n\n接下来，检查手机号中的[REDACTED_CN_PHONE]，这里需要确认是否正确，但用户已经标记为[REDACTED_CN_PHONE]，所以直接返回这个部分。同时，确保不包含其他字段，比如地址、医疗记录等，只保留身份证号和手机号中的敏感信息。因此，最终的输出应该是包含身份证号和手机号的字段，但不包含其他信息。\n",
+        "images": null,
+        "tool_calls": null
+      }
+    },
+    "latency_ms": 4287.042,
+    "usage": {
+      "prompt_tokens": 565,
+      "completion_tokens": 75
+    }
+  }
+]

chapter3/log-sanitization/validation/runs/20260729T185603Z-3_3-7f84f416/manifest.json

@@ -0,0 +1,51 @@
+{
+  "schema_version": "chapter3-evidence-v1",
+  "experiment": "3-3",
+  "run_id": "20260729T185603Z-3_3-7f84f416",
+  "created_at": "2026-07-29T18:56:03.514546+00:00",
+  "status": "passed",
+  "run_dir": "/Users/boj/book/ai-agent-book/chapter3/log-sanitization/validation/runs/20260729T185603Z-3_3-7f84f416",
+  "artifacts": {
+    "evidence.json": "9a07a5b4e99b7f8468170a3c4593be2f7b4a703e2ac6fd28a9d3390c49c9f7f7",
+    "receipts.json": "3faf1e5e59ed7679d5f26ee23e3c61081566e7391af5ad2ab9d9a927afcf4e86"
+  },
+  "inputs": [],
+  "summary": {
+    "regex": {
+      "cases": 12,
+      "mean_precision": 0.4166666666666666,
+      "mean_recall": 0.5,
+      "mean_typed_exact": 0.4166666666666667,
+      "residual_leaks": 9,
+      "mean_utility": 0.954197478750736,
+      "mean_latency_ms": 0.09875
+    },
+    "llm": {
+      "cases": 12,
+      "mean_precision": 0.5138888888888888,
+      "mean_recall": 0.6944444444444445,
+      "mean_typed_exact": 0.3333333333333333,
+      "residual_leaks": 3,
+      "mean_utility": 0.7548189868403488,
+      "mean_latency_ms": 3010.9979999999996
+    },
+    "hybrid": {
+      "cases": 12,
+      "mean_precision": 0.4861111111111111,
+      "mean_recall": 0.7777777777777777,
+      "mean_typed_exact": 0.5,
+      "residual_leaks": 2,
+      "mean_utility": 0.8744827477131053,
+      "mean_latency_ms": 2951.5082500000003
+    }
+  },
+  "acceptance": {
+    "local_model": true,
+    "qwen3_model": true,
+    "structured_type_location_confidence": true,
+    "structured_semistructured_natural_language_cases": true,
+    "regex_llm_hybrid_compared": true,
+    "leakage_utility_latency_measured": true,
+    "passed": true
+  }
+}