ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）

chapter6/end-to-end-speech/.gitignore

@@ -0,0 +1,4 @@
+.venv/
+__pycache__/
+*.pyc
+validation/latest.json

chapter6/end-to-end-speech/README.md

@@ -0,0 +1,83 @@
+# 实验 6-5：本地运行 MiniCPM-o 4.5 端到端全模态语音
+
+运行器、验证器与 canonical 证据目录均使用实验 6-5 的统一标识 `exp6-5-*`。
+
+本实验属于正文的“**范式二 · 端到端全模态模型（Omni）**”，不属于后文的“边想边说”方案。它用同一个开放权重模型 MiniCPM-o 4.5 比较两条路径：
+
+- **端到端路径**：WAV 直接进入模型的音频编码器与隐空间，模型直接回答；
+- **自级联路径**：同一模型先把 WAV 转成纯文字，再只依据文字回答，主动丢弃语速等副语言信息。
+
+另加一条 audio-to-audio 检查，确认模型不仅能听，还能在本地生成 24kHz 语音。实验关闭 `enable_thinking`，因此结果不能用来声称复现 Step-Audio R1 的 MPS、Speak-First、Think-First 或“边想边说”。
+
+## 实验设计
+
+`fixtures/cases.json` 固定四条小型合成语音：两道只依赖语义的口述算术题，以及文字完全相同、语速分别为快和慢的两条副语言题。每条都运行端到端与自级联两臂。小样本只用于验证机制与本地可运行性，不是模型排行榜；合成音也不能替代真人、多口音与噪声数据集。
+
+| 维度 | 端到端臂 | 自级联臂 |
+| --- | --- | --- |
+| 输入 | 单声道 WAV（读取时重采样为模型要求的 16kHz） | MiniCPM-o 生成的纯文字转录 |
+| 回答模型 | MiniCPM-o 4.5 | 同一个 MiniCPM-o 4.5 |
+| 是否保留语速 | 是 | 否（转录提示明确只保留说出的文字） |
+| 采样 | 关闭 | 关闭 |
+| 思考模式 | 关闭 | 关闭 |
+
+固定模型为 `openbmb/MiniCPM-o-4_5@1f761131fa83f5ed3cd6f2f22b225c4501d154fa`。官方实现由 SigLip2、Whisper-medium、CosyVoice2 与 Qwen3-8B 组成，总计约 9B 参数；本实验只初始化音频与 TTS 分支，不初始化视觉分支。
+
+## 安装
+
+上游明确测试 Python 3.10、`transformers==4.51.0`、PyTorch 2.3–2.8。该组合与仓库共享环境里的其他实验可能冲突，因此这里有意使用独立虚拟环境：
+
+```bash
+cd chapter6/end-to-end-speech
+uv venv .venv --python 3.10
+uv pip install --python .venv/bin/python -r requirements.txt
+source .venv/bin/activate
+
+hf download openbmb/MiniCPM-o-4_5 \
+  --revision 1f761131fa83f5ed3cd6f2f22b225c4501d154fa
+```
+
+需要 Linux、NVIDIA CUDA GPU 和约 21GB 可用显存。只有扩展到视频输入/输出时才需要 FFmpeg；本次 WAV→文本/语音 campaign 不调用 FFmpeg。模型权重与上游 Python 自定义代码会被下载到 Hugging Face cache，运行前应按自己的供应链策略审查并固定 revision。
+
+## 运行
+
+```bash
+python demo.py \
+  --local-files-only \
+  --evidence validation/runs/exp6-5-minicpmo45-20260801-v1/evidence.json \
+  --output-dir validation/runs/exp6-5-minicpmo45-20260801-v1/outputs
+
+python validate_evidence.py \
+  validation/runs/exp6-5-minicpmo45-20260801-v1/evidence.json
+```
+
+`demo.py` 一次加载模型，随后保存每条输入的 SHA-256、两臂原始回复、模型自产转录、分阶段延迟、模型 revision、软件版本、GPU 信息，以及语音输出的 SHA-256/采样率/时长。验收只要求真实本地路径完整且证据闭环，**不要求假设必须为正**。
+
+没有 GPU 时可运行离线单元测试，但不能据此宣称完成真实实验：
+
+```bash
+python -m pytest -q tests
+python demo.py --help
+```
+
+合成输入可用 `python prepare_fixtures.py` 重建（需要 `espeak`）；正式证据以仓库中 WAV 的 hash 为准。
+
+## 本地结果
+
+2026-08-01 的[本地 canonical run](validation/runs/exp6-5-minicpmo45-20260801-v1/evidence.json)已通过[全部 11 项验收](validation/runs/exp6-5-minicpmo45-20260801-v1/acceptance.json)。硬件是单张 96GB RTX PRO 6000 Blackwell，PyTorch 2.8.0+cu128、Transformers 4.51.0、BF16/SDPA；模型加载 6.154 秒，峰值分配显存 20.269GiB。
+
+| 任务 | 端到端 | 自级联 |
+| --- | ---: | ---: |
+| 语义算术（2 条） | 1/2 | 2/2 |
+| 副语言语速（2 条） | 2/2 | 1/2 |
+| 合计 | 3/4 | 3/4 |
+
+总分相同但错误互补。端到端在第一题把 “twelve boxes” 感知成 8，算出 47；自级联先正确转录出 12，再算出 79。相反，快/慢两条音频在自级联中都被压成完全相同的 `Please send the report before lunch.`，于是它把 fast 样本也猜成 slow；端到端保留了速度信息，两条都正确。
+
+加载完成后的平均整次调用为端到端 0.686 秒、自级联 0.551 秒。由于端到端固定先跑、回复长度不同且只有四条，这不是可推广的延迟排名。audio-to-audio 臂另生成了[11.56 秒、24kHz 单声道 WAV](validation/runs/exp6-5-minicpmo45-20260801-v1/outputs/spoken-math-boxes-response.wav)，但它继承了第一题的感知错误。这是有价值的负结果：路径真实跑通不等于答案正确。
+
+---
+
+## English
+
+Experiment 6-5 belongs to Paradigm 2, end-to-end omni models. Historical canonical paths retain the `exp6-5-*` identifier. It runs the pinned MiniCPM-o 4.5 checkpoint locally and compares native audio-to-answer inference against a self-cascade that first flattens the same audio to text. A separate audio-output arm retains a real 24kHz waveform. Thinking is deliberately disabled; this experiment makes no MPS or “thinking while speaking” claim.

chapter6/end-to-end-speech/demo.py

@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+"""Run the local MiniCPM-o 4.5 direct-vs-self-cascade campaign."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import platform
+import re
+import subprocess
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+from speech_model import MODEL_ID, MODEL_REVISION, MiniCPMOClient, sha256_file
+
+
+HERE = Path(__file__).resolve().parent
+DEFAULT_CASES = HERE / "fixtures" / "cases.json"
+DEFAULT_EVIDENCE = HERE / "validation" / "latest.json"
+IMPLEMENTATION_FILES = (
+    "demo.py",
+    "speech_model.py",
+    "validate_evidence.py",
+    "requirements.txt",
+    "fixtures/cases.json",
+)
+
+
+def matches_expected(response: str, aliases: list[str]) -> bool:
+    normalized = response.casefold()
+    return any(re.search(rf"(?<!\w){re.escape(alias.casefold())}(?!\w)", normalized) for alias in aliases)
+
+
+def load_cases(path: Path) -> list[dict[str, Any]]:
+    cases = json.loads(path.read_text(encoding="utf-8"))
+    required = {"id", "audio", "category", "instruction", "expected_aliases"}
+    if not cases or any(required - set(case) for case in cases):
+        raise ValueError(f"Malformed case manifest: {path}")
+    return cases
+
+
+def git_metadata() -> dict[str, Any]:
+    try:
+        root = subprocess.check_output(
+            ["git", "rev-parse", "--show-toplevel"], cwd=HERE, text=True
+        ).strip()
+        commit = subprocess.check_output(
+            ["git", "rev-parse", "HEAD"], cwd=root, text=True
+        ).strip()
+        dirty = bool(subprocess.check_output(
+            ["git", "status", "--porcelain"], cwd=root, text=True
+        ).strip())
+        return {"commit": commit, "workspace_dirty_at_run": dirty}
+    except (OSError, subprocess.CalledProcessError):
+        return {"commit": None, "workspace_dirty_at_run": None}
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description="Experiment 6-5: local MiniCPM-o 4.5 omni speech evaluation"
+    )
+    parser.add_argument("--model", default=MODEL_ID)
+    parser.add_argument("--revision", default=MODEL_REVISION)
+    parser.add_argument("--cases", type=Path, default=DEFAULT_CASES)
+    parser.add_argument("--evidence", type=Path, default=DEFAULT_EVIDENCE)
+    parser.add_argument("--output-dir", type=Path)
+    parser.add_argument("--max-new-tokens", type=int, default=256)
+    parser.add_argument("--local-files-only", action="store_true")
+    parser.add_argument("--skip-speech-output", action="store_true")
+    args = parser.parse_args()
+
+    cases = load_cases(args.cases)
+    output_dir = args.output_dir or args.evidence.parent / "outputs"
+    client = MiniCPMOClient(
+        args.model,
+        args.revision,
+        enable_tts=not args.skip_speech_output,
+        local_files_only=args.local_files_only,
+    )
+    client.load()
+
+    results = []
+    for case in cases:
+        audio_path = (args.cases.parent / case["audio"]).resolve()
+        direct = client.infer_audio(
+            audio_path, case["instruction"], max_new_tokens=args.max_new_tokens
+        )
+        cascade = client.self_cascade(
+            audio_path, case["instruction"], max_new_tokens=args.max_new_tokens
+        )
+        direct_pass = matches_expected(direct.response, case["expected_aliases"])
+        cascade_pass = matches_expected(cascade.response, case["expected_aliases"])
+        print(
+            f"[{case['id']}] direct={direct.response!r} ({direct.latency_seconds:.3f}s, pass={direct_pass})"
+        )
+        print(
+            f"[{case['id']}] cascade={cascade.response!r} ({cascade.latency_seconds:.3f}s, pass={cascade_pass})"
+        )
+        results.append({
+            **case,
+            "audio_path": str(audio_path.relative_to(HERE)),
+            "audio_sha256": sha256_file(audio_path),
+            "direct": {**direct.to_dict(), "passed": direct_pass},
+            "self_cascade": {**cascade.to_dict(), "passed": cascade_pass},
+        })
+
+    speech_output = None
+    if not args.skip_speech_output:
+        speech_case = cases[0]
+        audio_path = (args.cases.parent / speech_case["audio"]).resolve()
+        output_audio_path = (output_dir / f"{speech_case['id']}-response.wav").resolve()
+        speech_output = client.infer_audio(
+            audio_path,
+            "Listen to the question and answer aloud in one short sentence.",
+            max_new_tokens=args.max_new_tokens,
+            output_audio_path=output_audio_path,
+        ).to_dict()
+        speech_output["output_audio"]["path"] = str(output_audio_path.relative_to(HERE))
+        print(f"[speech-output] {speech_output['response']!r}")
+
+    direct_passes = sum(result["direct"]["passed"] for result in results)
+    cascade_passes = sum(result["self_cascade"]["passed"] for result in results)
+    evidence = {
+        "schema_version": 1,
+        "experiment": "6-5",
+        "title": "MiniCPM-o 4.5 local end-to-end omni speech",
+        "timestamp_utc": datetime.now(timezone.utc).isoformat(),
+        "model": client.runtime_metadata(),
+        "host": {"platform": platform.platform(), **git_metadata()},
+        "protocol": {
+            "case_count": len(results),
+            "direct_arm": "audio -> MiniCPM-o latent processing -> text",
+            "self_cascade_arm": "audio -> MiniCPM-o transcript -> MiniCPM-o text reasoning",
+            "speech_arm": "audio -> MiniCPM-o latent processing -> text + generated waveform",
+            "enable_thinking": False,
+            "sampling": False,
+            "max_new_tokens": args.max_new_tokens,
+        },
+        "cases": results,
+        "aggregate": {
+            "direct_correct": direct_passes,
+            "self_cascade_correct": cascade_passes,
+            "total_cases": len(results),
+            "direct_accuracy": direct_passes / len(results),
+            "self_cascade_accuracy": cascade_passes / len(results),
+            "direct_mean_latency_seconds": sum(r["direct"]["latency_seconds"] for r in results) / len(results),
+            "self_cascade_mean_latency_seconds": sum(r["self_cascade"]["latency_seconds"] for r in results) / len(results),
+        },
+        "speech_output": speech_output,
+        "implementation_sha256": {
+            name: sha256_file(HERE / name) for name in IMPLEMENTATION_FILES
+        },
+        "external_api_calls": 0,
+    }
+    args.evidence.parent.mkdir(parents=True, exist_ok=True)
+    args.evidence.write_text(
+        json.dumps(evidence, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
+    )
+    print(f"Evidence: {args.evidence}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

chapter6/end-to-end-speech/fixtures/cases.json

@@ -0,0 +1,30 @@
+[
+  {
+    "id": "spoken-math-boxes",
+    "audio": "boxes-79.wav",
+    "category": "semantic_reasoning",
+    "instruction": "Listen to the spoken math problem and reply with the final numeric answer.",
+    "expected_aliases": ["79", "seventy-nine", "seventy nine"]
+  },
+  {
+    "id": "spoken-math-tickets",
+    "audio": "tickets-42.wav",
+    "category": "semantic_reasoning",
+    "instruction": "Listen to the spoken math problem and reply with the final numeric answer.",
+    "expected_aliases": ["42", "forty-two", "forty two"]
+  },
+  {
+    "id": "speaking-rate-fast",
+    "audio": "pace-fast.wav",
+    "category": "paralinguistic",
+    "instruction": "Classify the speaker's pace as fast or slow. Reply with exactly one word: fast or slow.",
+    "expected_aliases": ["fast"]
+  },
+  {
+    "id": "speaking-rate-slow",
+    "audio": "pace-slow.wav",
+    "category": "paralinguistic",
+    "instruction": "Classify the speaker's pace as fast or slow. Reply with exactly one word: fast or slow.",
+    "expected_aliases": ["slow"]
+  }
+]

chapter6/end-to-end-speech/prepare_fixtures.py

@@ -0,0 +1,38 @@
+#!/usr/bin/env python3
+"""Regenerate the small synthetic WAV fixtures used by Experiment 6-5."""
+
+from __future__ import annotations
+
+import shutil
+import subprocess
+from pathlib import Path
+
+
+HERE = Path(__file__).resolve().parent
+FIXTURES = HERE / "fixtures"
+
+UTTERANCES = {
+    "boxes-79.wav": (145, 50, "A shop has twelve boxes with eight pencils in each box. It gives away seventeen pencils. How many pencils remain?"),
+    "tickets-42.wav": (145, 50, "A theater sold eighteen tickets in the morning and twenty four tickets in the afternoon. How many tickets did it sell altogether?"),
+    "pace-fast.wav": (260, 60, "Please send the report before lunch."),
+    "pace-slow.wav": (85, 40, "Please send the report before lunch."),
+}
+
+
+def main() -> int:
+    espeak = shutil.which("espeak")
+    if not espeak:
+        raise SystemExit("espeak is required to regenerate fixtures (Ubuntu: apt install espeak)")
+    FIXTURES.mkdir(parents=True, exist_ok=True)
+    for filename, (speed, pitch, text) in UTTERANCES.items():
+        output = FIXTURES / filename
+        subprocess.run(
+            [espeak, "-v", "en-us", "-s", str(speed), "-p", str(pitch), "-w", str(output), text],
+            check=True,
+        )
+        print(output)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

chapter6/end-to-end-speech/requirements.txt

@@ -0,0 +1,9 @@
+# MiniCPM-o 4.5's official, isolated local-inference stack (Python 3.10).
+transformers==4.51.0
+accelerate>=1.0
+torch>=2.3.0,<=2.8.0
+torchaudio<=2.8.0
+minicpmo-utils[all]>=1.0.5
+# minicpmo-utils currently resolves librosa 0.9, which imports pkg_resources.
+setuptools<81
+pytest>=7.4

chapter6/end-to-end-speech/speech_model.py

@@ -0,0 +1,214 @@
+"""Native local MiniCPM-o 4.5 inference for Experiment 6-5."""
+
+from __future__ import annotations
+
+import hashlib
+import time
+from dataclasses import asdict, dataclass
+from pathlib import Path
+from typing import Any
+
+
+MODEL_ID = "openbmb/MiniCPM-o-4_5"
+MODEL_REVISION = "1f761131fa83f5ed3cd6f2f22b225c4501d154fa"
+
+
+@dataclass
+class InferenceResult:
+    mode: str
+    response: str
+    latency_seconds: float
+    transcript: str | None = None
+    stage_latencies: dict[str, float] | None = None
+    output_audio: dict[str, Any] | None = None
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+def sha256_file(path: str | Path) -> str:
+    digest = hashlib.sha256()
+    with Path(path).open("rb") as handle:
+        for block in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def audio_metadata(path: str | Path) -> dict[str, Any]:
+    import soundfile as sf
+
+    path = Path(path)
+    info = sf.info(path)
+    return {
+        "path": str(path),
+        "sha256": sha256_file(path),
+        "sample_rate_hz": info.samplerate,
+        "frames": info.frames,
+        "duration_seconds": round(info.duration, 6),
+        "channels": info.channels,
+        "format": info.format,
+    }
+
+
+class MiniCPMOClient:
+    """One-GPU Transformers client for the official MiniCPM-o 4.5 checkpoint."""
+
+    def __init__(
+        self,
+        model_id: str = MODEL_ID,
+        revision: str = MODEL_REVISION,
+        *,
+        device: str = "cuda",
+        enable_tts: bool = True,
+        local_files_only: bool = False,
+    ) -> None:
+        self.model_id = model_id
+        self.revision = revision
+        self.device = device
+        self.enable_tts = enable_tts
+        self.local_files_only = local_files_only
+        self.model = None
+        self.load_seconds: float | None = None
+
+    def load(self) -> None:
+        import torch
+        from transformers import AutoModel
+
+        if self.device == "cuda" and not torch.cuda.is_available():
+            raise RuntimeError("MiniCPM-o 4.5 local precision run requires an NVIDIA CUDA GPU")
+        started = time.perf_counter()
+        self.model = AutoModel.from_pretrained(
+            self.model_id,
+            revision=self.revision,
+            trust_remote_code=True,
+            attn_implementation="sdpa",
+            torch_dtype=torch.bfloat16,
+            init_vision=False,
+            init_audio=True,
+            init_tts=self.enable_tts,
+            local_files_only=self.local_files_only,
+        )
+        self.model.eval().to(self.device)
+        if self.enable_tts:
+            self.model.init_tts()
+        self.load_seconds = time.perf_counter() - started
+
+    def _require_model(self):
+        if self.model is None:
+            raise RuntimeError("Call load() before inference")
+        return self.model
+
+    @staticmethod
+    def load_audio(path: str | Path):
+        import librosa
+
+        audio, _ = librosa.load(path, sr=16000, mono=True)
+        return audio
+
+    def infer_audio(
+        self,
+        audio_path: str | Path,
+        instruction: str,
+        *,
+        max_new_tokens: int = 256,
+        output_audio_path: str | Path | None = None,
+    ) -> InferenceResult:
+        model = self._require_model()
+        audio = self.load_audio(audio_path)
+        generate_audio = output_audio_path is not None
+        if generate_audio and not self.enable_tts:
+            raise RuntimeError("The client was loaded with enable_tts=False")
+        if output_audio_path is not None:
+            output_audio_path = Path(output_audio_path)
+            output_audio_path.parent.mkdir(parents=True, exist_ok=True)
+        started = time.perf_counter()
+        response = model.chat(
+            msgs=[{"role": "user", "content": [instruction, audio]}],
+            do_sample=False,
+            max_new_tokens=max_new_tokens,
+            use_tts_template=generate_audio,
+            enable_thinking=False,
+            generate_audio=generate_audio,
+            output_audio_path=str(output_audio_path) if output_audio_path else None,
+        )
+        latency = time.perf_counter() - started
+        return InferenceResult(
+            mode="direct-audio-to-speech" if generate_audio else "direct-audio-to-text",
+            response=response,
+            latency_seconds=latency,
+            output_audio=audio_metadata(output_audio_path) if output_audio_path else None,
+        )
+
+    def infer_text(
+        self, prompt: str, *, max_new_tokens: int = 256
+    ) -> InferenceResult:
+        model = self._require_model()
+        started = time.perf_counter()
+        response = model.chat(
+            msgs=[{"role": "user", "content": [prompt]}],
+            do_sample=False,
+            max_new_tokens=max_new_tokens,
+            use_tts_template=False,
+            enable_thinking=False,
+            generate_audio=False,
+        )
+        return InferenceResult(
+            mode="text-only",
+            response=response,
+            latency_seconds=time.perf_counter() - started,
+        )
+
+    def transcribe(self, audio_path: str | Path) -> InferenceResult:
+        return self.infer_audio(
+            audio_path,
+            "Please transcribe only the words spoken in this audio. Do not describe tone, pace, or background sound.",
+            max_new_tokens=256,
+        )
+
+    def self_cascade(
+        self, audio_path: str | Path, instruction: str, *, max_new_tokens: int = 256
+    ) -> InferenceResult:
+        transcription = self.transcribe(audio_path)
+        reasoning = self.infer_text(
+            f"{instruction}\n\nUse only this transcript as evidence:\n{transcription.response}",
+            max_new_tokens=max_new_tokens,
+        )
+        return InferenceResult(
+            mode="self-cascade-audio-to-transcript-to-text",
+            response=reasoning.response,
+            latency_seconds=transcription.latency_seconds + reasoning.latency_seconds,
+            transcript=transcription.response,
+            stage_latencies={
+                "transcription_seconds": transcription.latency_seconds,
+                "reasoning_seconds": reasoning.latency_seconds,
+            },
+        )
+
+    def runtime_metadata(self) -> dict[str, Any]:
+        import torch
+        import transformers
+
+        gpu = None
+        if torch.cuda.is_available():
+            properties = torch.cuda.get_device_properties(0)
+            gpu = {
+                "name": properties.name,
+                "total_memory_gib": round(properties.total_memory / 2**30, 3),
+                "peak_allocated_gib": round(torch.cuda.max_memory_allocated() / 2**30, 3),
+            }
+        return {
+            "model_id": self.model_id,
+            "model_revision": self.revision,
+            "device": self.device,
+            "torch_version": torch.__version__,
+            "transformers_version": transformers.__version__,
+            "cuda_version": torch.version.cuda,
+            "cuda_available": torch.cuda.is_available(),
+            "gpu": gpu,
+            "load_seconds": self.load_seconds,
+            "precision": "bfloat16",
+            "attention": "sdpa",
+            "init_vision": False,
+            "init_audio": True,
+            "init_tts": self.enable_tts,
+        }

chapter6/end-to-end-speech/tests/conftest.py

@@ -0,0 +1,7 @@
+import sys
+from pathlib import Path
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[1]
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))

chapter6/end-to-end-speech/tests/test_demo.py

@@ -0,0 +1,16 @@
+import pytest
+
+from demo import load_cases, matches_expected
+
+
+def test_expected_match_uses_word_boundaries():
+    assert matches_expected("The answer is 79.", ["79"])
+    assert matches_expected("FAST", ["fast"])
+    assert not matches_expected("The answer is 179.", ["79"])
+
+
+def test_case_manifest_validation(tmp_path):
+    empty = tmp_path / "cases.json"
+    empty.write_text("[]", encoding="utf-8")
+    with pytest.raises(ValueError, match="Malformed"):
+        load_cases(empty)

chapter6/end-to-end-speech/tests/test_speech_model.py

@@ -0,0 +1,25 @@
+from pathlib import Path
+
+import pytest
+
+from speech_model import MiniCPMOClient, sha256_file
+
+
+def test_sha256_file(tmp_path):
+    path = tmp_path / "sample"
+    path.write_bytes(b"abc")
+    assert sha256_file(path) == "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
+
+
+def test_inference_requires_loaded_model():
+    client = MiniCPMOClient(enable_tts=False)
+    with pytest.raises(RuntimeError, match=r"Call load\(\)"):
+        client.infer_text("hello")
+
+
+def test_speech_output_requires_tts(monkeypatch, tmp_path):
+    client = MiniCPMOClient(enable_tts=False)
+    client.model = object()
+    monkeypatch.setattr(client, "load_audio", lambda _: [0.0])
+    with pytest.raises(RuntimeError, match="enable_tts=False"):
+        client.infer_audio(tmp_path / "input.wav", "answer", output_audio_path=tmp_path / "out.wav")

chapter6/end-to-end-speech/tests/test_validate_evidence.py

@@ -0,0 +1,53 @@
+import json
+import wave
+from pathlib import Path
+
+from speech_model import MODEL_ID, MODEL_REVISION, sha256_file
+from validate_evidence import validate
+
+
+def test_complete_evidence_passes(tmp_path):
+    wav = tmp_path / "answer.wav"
+    with wave.open(str(wav), "wb") as handle:
+        handle.setnchannels(1)
+        handle.setsampwidth(2)
+        handle.setframerate(24000)
+        handle.writeframes(b"\x00\x00" * 2400)
+    arm = {"response": "ok", "transcript": "spoken words"}
+    evidence = {
+        "experiment": "6-5",
+        "model": {
+            "model_id": MODEL_ID,
+            "model_revision": MODEL_REVISION,
+            "cuda_available": True,
+            "init_audio": True,
+        },
+        "protocol": {"enable_thinking": False},
+        "cases": [
+            {"direct": {"response": "ok"}, "self_cascade": arm}
+            for _ in range(4)
+        ],
+        "speech_output": {
+            "output_audio": {
+                "path": str(wav),
+                "sha256": sha256_file(wav),
+                "sample_rate_hz": 24000,
+                "duration_seconds": 0.1,
+            }
+        },
+        "implementation_sha256": {
+            "requirements.txt": sha256_file(
+                Path(__file__).resolve().parents[1] / "requirements.txt"
+            )
+        },
+        "external_api_calls": 0,
+    }
+    path = tmp_path / "evidence.json"
+    path.write_text(json.dumps(evidence), encoding="utf-8")
+    assert validate(path)["passed"] is True
+
+
+def test_missing_direct_response_fails(tmp_path):
+    path = tmp_path / "evidence.json"
+    path.write_text(json.dumps({"experiment": "6-5", "cases": []}), encoding="utf-8")
+    assert validate(path)["passed"] is False

chapter6/end-to-end-speech/validate_evidence.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+"""Fail-closed validation for a retained Experiment 6-5 local run."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from speech_model import MODEL_ID, MODEL_REVISION, sha256_file
+
+
+HERE = Path(__file__).resolve().parent
+
+
+def validate(evidence_path: Path) -> dict:
+    evidence = json.loads(evidence_path.read_text(encoding="utf-8"))
+    implementation_hashes = evidence.get("implementation_sha256", {})
+    checks = {
+        "experiment_is_6_5": evidence.get("experiment") == "6-5",
+        "exact_model": evidence.get("model", {}).get("model_id") == MODEL_ID,
+        "pinned_revision": evidence.get("model", {}).get("model_revision") == MODEL_REVISION,
+        "cuda_used": evidence.get("model", {}).get("cuda_available") is True,
+        "native_audio_enabled": evidence.get("model", {}).get("init_audio") is True,
+        "thinking_not_claimed": evidence.get("protocol", {}).get("enable_thinking") is False,
+        "four_cases": len(evidence.get("cases", [])) == 4,
+        "both_arms_complete": all(
+            (case.get("direct") or {}).get("response")
+            and (case.get("self_cascade") or {}).get("response")
+            and (case.get("self_cascade") or {}).get("transcript")
+            for case in evidence.get("cases", [])
+        ),
+        "speech_waveform_recorded": False,
+        "implementation_hashes_match": bool(implementation_hashes) and all(
+            (HERE / name).is_file() and sha256_file(HERE / name) == digest
+            for name, digest in implementation_hashes.items()
+        ),
+        "no_external_api": evidence.get("external_api_calls") == 0,
+    }
+    audio = (evidence.get("speech_output") or {}).get("output_audio") or {}
+    if audio.get("path") and audio.get("sha256"):
+        path = Path(audio["path"])
+        if not path.is_absolute():
+            path = (HERE / path).resolve()
+        checks["speech_waveform_recorded"] = (
+            path.is_file()
+            and sha256_file(path) == audio["sha256"]
+            and audio.get("sample_rate_hz") == 24000
+            and audio.get("duration_seconds", 0) > 0
+        )
+    return {"passed": all(checks.values()), "checks": checks}
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("evidence", type=Path)
+    parser.add_argument("--output", type=Path)
+    args = parser.parse_args()
+    result = validate(args.evidence)
+    output = args.output or args.evidence.with_name("acceptance.json")
+    output.write_text(json.dumps(result, indent=2) + "\n", encoding="utf-8")
+    print(json.dumps(result, indent=2))
+    return 0 if result["passed"] else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

chapter6/end-to-end-speech/validation/runs/exp6-5-minicpmo45-20260801-v1/acceptance.json

@@ -0,0 +1,16 @@
+{
+  "passed": true,
+  "checks": {
+    "experiment_is_6_5": true,
+    "exact_model": true,
+    "pinned_revision": true,
+    "cuda_used": true,
+    "native_audio_enabled": true,
+    "thinking_not_claimed": true,
+    "four_cases": true,
+    "both_arms_complete": true,
+    "speech_waveform_recorded": true,
+    "implementation_hashes_match": true,
+    "no_external_api": true
+  }
+}

chapter6/end-to-end-speech/validation/runs/exp6-5-minicpmo45-20260801-v1/evidence.json

@@ -0,0 +1,207 @@
+{
+  "schema_version": 1,
+  "experiment": "6-5",
+  "title": "MiniCPM-o 4.5 local end-to-end omni speech",
+  "timestamp_utc": "2026-08-01T04:30:23.044472+00:00",
+  "model": {
+    "model_id": "openbmb/MiniCPM-o-4_5",
+    "model_revision": "1f761131fa83f5ed3cd6f2f22b225c4501d154fa",
+    "device": "cuda",
+    "torch_version": "2.8.0+cu128",
+    "transformers_version": "4.51.0",
+    "cuda_version": "12.8",
+    "cuda_available": true,
+    "gpu": {
+      "name": "NVIDIA RTX PRO 6000 Blackwell Workstation Edition",
+      "total_memory_gib": 94.97,
+      "peak_allocated_gib": 20.269
+    },
+    "load_seconds": 6.154250729829073,
+    "precision": "bfloat16",
+    "attention": "sdpa",
+    "init_vision": false,
+    "init_audio": true,
+    "init_tts": true
+  },
+  "host": {
+    "platform": "Linux-6.8.0-111-generic-x86_64-with-glibc2.35",
+    "commit": "d6a31e8a93c1fed8901d6ea59e1ba80d6101f97a",
+    "workspace_dirty_at_run": true
+  },
+  "protocol": {
+    "case_count": 4,
+    "direct_arm": "audio -> MiniCPM-o latent processing -> text",
+    "self_cascade_arm": "audio -> MiniCPM-o transcript -> MiniCPM-o text reasoning",
+    "speech_arm": "audio -> MiniCPM-o latent processing -> text + generated waveform",
+    "enable_thinking": false,
+    "sampling": false,
+    "max_new_tokens": 256
+  },
+  "cases": [
+    {
+      "id": "spoken-math-boxes",
+      "audio": "boxes-79.wav",
+      "category": "semantic_reasoning",
+      "instruction": "Listen to the spoken math problem and reply with the final numeric answer.",
+      "expected_aliases": [
+        "79",
+        "seventy-nine",
+        "seventy nine"
+      ],
+      "audio_path": "fixtures/boxes-79.wav",
+      "audio_sha256": "ca55172355f1d49a7e769ef379b4daeadea15c0b58a04c1987b4293381ad42e8",
+      "direct": {
+        "mode": "direct-audio-to-text",
+        "response": "Find the total number of pencils at first by multiplying the number of boxes by the number of pencils in each box: 8 boxes * 8 pencils/box = 64 pencils\nThen subtract the number of pencils given away from the total number of pencils to find the number of pencils that remain: 64 pencils - 17 pencils = 47 pencils\nThe answer is 47",
+        "latency_seconds": 1.911103354766965,
+        "transcript": null,
+        "stage_latencies": null,
+        "output_audio": null,
+        "passed": false
+      },
+      "self_cascade": {
+        "mode": "self-cascade-audio-to-transcript-to-text",
+        "response": "First, calculate the total number of pencils: 12 boxes * 8 pencils/box = 96 pencils. Then subtract the pencils given away: 96 - 17 = 79 pencils remain.\n\n79",
+        "latency_seconds": 1.1656107371672988,
+        "transcript": "A shop has twelve boxes with eight pencils in each box. It gave away seventeen pencils. How many pencils remain?",
+        "stage_latencies": {
+          "transcription_seconds": 0.4240540452301502,
+          "reasoning_seconds": 0.7415566919371486
+        },
+        "output_audio": null,
+        "passed": true
+      }
+    },
+    {
+      "id": "spoken-math-tickets",
+      "audio": "tickets-42.wav",
+      "category": "semantic_reasoning",
+      "instruction": "Listen to the spoken math problem and reply with the final numeric answer.",
+      "expected_aliases": [
+        "42",
+        "forty-two",
+        "forty two"
+      ],
+      "audio_path": "fixtures/tickets-42.wav",
+      "audio_sha256": "66c8aa161fc046c93b97f56c7b9b69ea25f63b54f243ea153f95fb4b2d815b49",
+      "direct": {
+        "mode": "direct-audio-to-text",
+        "response": "The theater sold 18 tickets in the morning and 24 tickets in the afternoon. So it sold 18 + 24 = 42 tickets altogether. The answer is 42.",
+        "latency_seconds": 0.6811522357165813,
+        "transcript": null,
+        "stage_latencies": null,
+        "output_audio": null,
+        "passed": true
+      },
+      "self_cascade": {
+        "mode": "self-cascade-audio-to-transcript-to-text",
+        "response": "18 + 24 = 42\nAnswer: 42",
+        "latency_seconds": 0.6580845378339291,
+        "transcript": "The theater sold eighteen tickets in the morning and twenty four tickets in the afternoon. How many tickets did it sell altogether?",
+        "stage_latencies": {
+          "transcription_seconds": 0.40988718904554844,
+          "reasoning_seconds": 0.24819734878838062
+        },
+        "output_audio": null,
+        "passed": true
+      }
+    },
+    {
+      "id": "speaking-rate-fast",
+      "audio": "pace-fast.wav",
+      "category": "paralinguistic",
+      "instruction": "Classify the speaker's pace as fast or slow. Reply with exactly one word: fast or slow.",
+      "expected_aliases": [
+        "fast"
+      ],
+      "audio_path": "fixtures/pace-fast.wav",
+      "audio_sha256": "048cb477c0814e5e193df1425bee1fb0ce75df1cbc59c8e0a8f2db046f65b303",
+      "direct": {
+        "mode": "direct-audio-to-text",
+        "response": "fast",
+        "latency_seconds": 0.07435999717563391,
+        "transcript": null,
+        "stage_latencies": null,
+        "output_audio": null,
+        "passed": true
+      },
+      "self_cascade": {
+        "mode": "self-cascade-audio-to-transcript-to-text",
+        "response": "slow",
+        "latency_seconds": 0.19370830059051514,
+        "transcript": "Please send the report before lunch.",
+        "stage_latencies": {
+          "transcription_seconds": 0.15708006545901299,
+          "reasoning_seconds": 0.03662823513150215
+        },
+        "output_audio": null,
+        "passed": false
+      }
+    },
+    {
+      "id": "speaking-rate-slow",
+      "audio": "pace-slow.wav",
+      "category": "paralinguistic",
+      "instruction": "Classify the speaker's pace as fast or slow. Reply with exactly one word: fast or slow.",
+      "expected_aliases": [
+        "slow"
+      ],
+      "audio_path": "fixtures/pace-slow.wav",
+      "audio_sha256": "1a851ddc5d765b95f89d3a7cdabe137328b4ab7bc50dd06cc52c9c37bfecddd2",
+      "direct": {
+        "mode": "direct-audio-to-text",
+        "response": "slow",
+        "latency_seconds": 0.07847040891647339,
+        "transcript": null,
+        "stage_latencies": null,
+        "output_audio": null,
+        "passed": true
+      },
+      "self_cascade": {
+        "mode": "self-cascade-audio-to-transcript-to-text",
+        "response": "slow",
+        "latency_seconds": 0.1868541669100523,
+        "transcript": "Please send the report before lunch.",
+        "stage_latencies": {
+          "transcription_seconds": 0.15028280764818192,
+          "reasoning_seconds": 0.036571359261870384
+        },
+        "output_audio": null,
+        "passed": true
+      }
+    }
+  ],
+  "aggregate": {
+    "direct_correct": 3,
+    "self_cascade_correct": 3,
+    "total_cases": 4,
+    "direct_accuracy": 0.75,
+    "self_cascade_accuracy": 0.75,
+    "direct_mean_latency_seconds": 0.6862714991439134,
+    "self_cascade_mean_latency_seconds": 0.5510644356254488
+  },
+  "speech_output": {
+    "mode": "direct-audio-to-speech",
+    "response": "The shop had 8 boxes x 8 pencils = 64 pencils in total. After giving away 17 pencils, there are 64 - 17 = 47 pencils remaining.",
+    "latency_seconds": 2.639366910792887,
+    "transcript": null,
+    "stage_latencies": null,
+    "output_audio": {
+      "path": "validation/runs/exp6-5-minicpmo45-20260801-v1/outputs/spoken-math-boxes-response.wav",
+      "sha256": "dfaf64d06cc46c8d52a11f393ad067b1c48e950971ba28851e5dd0180639f111",
+      "sample_rate_hz": 24000,
+      "frames": 277440,
+      "duration_seconds": 11.56,
+      "channels": 1,
+      "format": "WAV"
+    }
+  },
+  "implementation_sha256": {
+    "demo.py": "1193c6ded825c13343723d77f29b444b7fb8a9b8c78c0d43c125aa19b5674e5f",
+    "speech_model.py": "da6cd20900594166b06b8027a7fb6fe21e4715d3912e744b518df3f74449cd8f",
+    "validate_evidence.py": "820e36f229c162f75882296c827709303948118b264441dbd4218dc73346ae31",
+    "requirements.txt": "c3700db0ac3068f91879ccad6423cb5d94c7c2f508b183e0d2de8c82fac3d757",
+    "fixtures/cases.json": "f568c3bfb8b8866831bab2c0bc1940e2aacb9aa7d37282fe50c8b12a17fdb191"
+  },
+  "external_api_calls": 0
+}