ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）

chapter10/autonomous-phone-registration/.gitignore

@@ -0,0 +1,4 @@
+.env
+artifacts/
+__pycache__/
+.pytest_cache/

chapter10/autonomous-phone-registration/README.md

@@ -0,0 +1,111 @@
+# Experiment 10-3 · Autonomous phone/browser orchestration
+
+This is the autonomous arm of Chapter Experiment 10-3. Its retained validation
+artifacts and validators use the current `10-3` identifier; the fixed-topology
+comparison is kept in [`talkact-reproduction`](../talkact-reproduction/) under
+the same chapter experiment number.
+
+This companion implements the autonomous arm of the merged experiment. A real Playwright Computer Use Agent opens an arbitrary registration URL and inspects the rendered form. A real LLM sees the page observation, known user context, and an optional `initiate_phone_call_agent(purpose, required_info)` tool. With `tool_choice=auto`, the model—not a Python field-count rule—decides whether to spawn a Phone Agent.
+
+The default transport is a private local WebRTC call (`--phone-transport webrtc`). It opens a participant page, negotiates an offer/answer pair, and carries agent and participant audio on two RTP tracks. Agent prompts also cross a data channel as non-sensitive captions; answers never use that channel. The remote peer records the participant track ephemerally for ASR, then discards both media and transcript. No E.164 number, PSTN provider, tunnel, or public webhook is required. The old Twilio and direct-microphone transports remain optional.
+
+## Exact concurrency and failure behavior
+
+- Phone and Computer Agents run as independent `asyncio` tasks with separate loops.
+- Each valid spoken value immediately emits `info_collected`; the Phone Agent asks the next question without awaiting `field_filled`.
+- The Computer Agent fills the actual page concurrently. `timing_evidence.overlap_checks` proves whether “ask next” preceded the prior fill completion.
+- HTML types, patterns, options, and format hints become `FieldSpec` validators. Invalid speech emits `format_invalid`, gives precise feedback, and is re-asked up to three times.
+- Page/selector errors are returned as `fill_error`; submission is blocked when any error remains.
+- Any unexpected Phone/Computer exception cancels the still-running peer, closes the
+  call and all media tracks, and then lets the top-level `finally` close the browser.
+  Cleanup is idempotent on normal and exceptional exits.
+- `--submit` is opt-in so a demonstration cannot accidentally create an account.
+- The decision and every message timestamp are written to JSON. Spoken personal values are redacted from console and disk traces.
+
+## Setup and local WebRTC call
+
+```bash
+cd chapter10/autonomous-phone-registration
+pip install -r requirements.txt
+playwright install chromium
+cp env.example .env
+
+python demo.py --confirm-consent --url 'https://your-site.example/register'
+```
+
+The command opens both the target form and a local participant call page. Speak after
+each question, then click **Finish answer**. Localhost is a browser secure context, so
+microphone access works without a certificate. The program refuses to open any live
+audio path unless `--confirm-consent` is present; the focused suite verifies that the
+refusal occurs before constructing a browser or media channel.
+
+Speech provider selection is independent of WebRTC. `WEBRTC_SPEECH_PROVIDER=auto`
+prefers local `say`/`espeak` TTS plus Gemini ASR when those are configured, otherwise
+it uses OpenAI TTS/ASR. `local-whisper` keeps both stages local and requires
+`openai-whisper` plus a cached/downloadable checkpoint:
+
+```bash
+WEBRTC_SPEECH_PROVIDER=local-whisper \
+WHISPER_PYTHON=/path/to/python-with-whisper \
+WHISPER_MODEL=tiny \
+python demo.py --confirm-consent --url 'https://your-site.example/register'
+```
+
+`--submit` remains an explicit opt-in. Without it, the agents fill and validate the
+form but do not create an account. The full acceptance runner submits only to its own
+localhost endpoint.
+
+Optional legacy transports:
+
+```bash
+python demo.py --confirm-consent --phone-transport local --url 'https://demoqa.com/automation-practice-form'
+python demo.py --confirm-consent --phone-transport twilio --url 'https://demoqa.com/automation-practice-form'
+```
+
+## Tests and full acceptance
+
+```bash
+pytest -q
+
+# Real LLM + Playwright + WebRTC/RTP + TTS/ASR + localhost submission.
+# Values are safe synthetic data; they still cross the audio media path and ASR.
+WEBRTC_SPEECH_PROVIDER=local-whisper \
+WHISPER_PYTHON=/path/to/python-with-whisper \
+python run_acceptance.py
+
+# Recompute every retained hash and prove raw ARK request/response consistency.
+python validate_acceptance.py \
+  validation/runs/exp10-3-webrtc-raw-20260731-v4
+```
+
+The formal 2026-07-31 run is committed at
+[`validation/runs/exp10-3-webrtc-raw-20260731-v4/`](validation/runs/exp10-3-webrtc-raw-20260731-v4/).
+A real ARK response (ID and usage retained) autonomously selected six required fields.
+The call completed one offer, one answer, seven media recordings, 9 TTS turns and 7
+local Whisper turns. Both RTP directions carried packets and bytes. A deliberately
+invalid spoken email caused `format_invalid` and a second question; all five adjacent
+ask/fill intervals overlapped; exactly one redacted six-field submission reached the
+localhost endpoint. All 9 acceptance gates pass. The manifest binds the runtime and
+artifacts with SHA-256 hashes, and the secret/value scan is empty. In addition to the
+normalized decision, this run retains the credential-free raw ARK request and raw
+response. They preserve the literal `tool_choice: "auto"`, tool schema, tool-call
+arguments, response ID, model, usage and measured latency. The standalone validator
+recomputes source, input and artifact hashes, independently normalizes those raw
+arguments against the observed form, and requires exact equality with `decision.json`.
+Its 8/8 retained-evidence checks pass; tamper tests cover the raw response, normalized
+decision, manifest and an unexpected unbound artifact.
+
+This run uses a safe synthesized participant so it is automated and reproducible. It
+proves the real media, ASR, orchestration, validation, privacy, and submission paths;
+it is not a human usability study or a test of TURN/NAT traversal. A human call uses
+the same WebRTC path with `--webrtc-answers-json` omitted.
+
+---
+
+## 中文说明
+
+本项目实现实验 10-3 的自主模式：Playwright Computer Use Agent 先访问真实注册页并读取表单；真实 LLM 在 `tool_choice=auto` 下自主决定是否调用 `initiate_phone_call_agent(purpose, required_info)`，代码没有用“字段数大于 N”代替模型决策。固定拓扑的并发基线见同章的 TalkAct 复现记录；两条路径的项目入口、验证器和保留产物均统一使用当前编号 10-3。
+
+默认路径现在是本机浏览器 WebRTC 通话，不需要手机号、PSTN 服务商、公开 webhook 或隧道。页面会完成真实 offer/answer，并用双向 RTP 音轨传输 Agent 语音和用户麦克风；回答只从远端音轨的临时录音进入 ASR，不会通过文本通道旁路，也不会保留原始音频或 transcript。Phone Agent 每拿到一个有效值就立即发给 Computer Agent，然后直接问下一项，不等待网页填写完成；格式错误会反馈并重问，页面错误会阻止提交，`--submit` 仍须显式授权。
+
+正式 raw-v4 验收以安全合成参与者跑通真实 ARK 自主工具调用、Playwright、WebRTC/RTP、本机 TTS、真实本机 Whisper ASR、格式重问、问填并行和一次 localhost 表单提交：9/9 行为门禁通过。除规范化 decision 外，证据还保留不含凭据的 ARK 原始请求和响应，包含字面量 `tool_choice: "auto"`、工具参数、response ID、model、usage 与实测延迟。独立 validator 会重算源码、输入和产物 hash，并把原始工具参数独立规范化后与 `decision.json` 精确比较；8/8 溯源检查及 raw receipt、decision、manifest、未绑定额外产物四类篡改测试均通过。日志只保留 `<redacted>`，不保留参与者值、音频或 transcript。这证明完整技术链路，不等同于真人可用性或跨 NAT/TURN 测试；省略 `--webrtc-answers-json` 即进入同一媒体路径的真人麦克风模式。

chapter10/autonomous-phone-registration/browser.py

@@ -0,0 +1,161 @@
+"""Real Playwright Computer Use surface for registration forms."""
+
+from __future__ import annotations
+
+import asyncio
+from typing import List, Optional
+
+from models import FieldSpec
+
+
+class RecoverableFillError(RuntimeError):
+    """A page-specific field failure that may be reported without aborting the call."""
+
+
+class RegistrationBrowser:
+    """Owns a real Chromium browser/context/page and exposes form operations.
+
+    The selector assigned during discovery is generated from the element itself and
+    remains internal. Values are never included in screenshots or trace files.
+    """
+
+    def __init__(self, url: str, *, headless: bool = False, submit: bool = False):
+        self.url = url
+        self.headless = headless
+        self.submit_enabled = submit
+        self._playwright = None
+        self.browser = None
+        self.context = None
+        self.page = None
+        self.closed = False
+
+    async def open(self) -> None:
+        from playwright.async_api import async_playwright
+
+        self._playwright = await async_playwright().start()
+        self.browser = await self._playwright.chromium.launch(headless=self.headless)
+        self.context = await self.browser.new_context()
+        self.page = await self.context.new_page()
+        await self.page.goto(self.url, wait_until="domcontentloaded", timeout=60_000)
+
+    async def discover_fields(self) -> List[FieldSpec]:
+        if self.page is None:
+            raise RuntimeError("browser is not open")
+        raw = await self.page.locator(
+            "input:not([type=hidden]):not([disabled]), select:not([disabled]), textarea:not([disabled])"
+        ).evaluate_all(
+            """els => els.map((el, i) => {
+              const id = el.id || '';
+              const explicit = id ? document.querySelector(`label[for="${CSS.escape(id)}"]`) : null;
+              const wrapping = el.closest('label');
+              const aria = el.getAttribute('aria-label') || el.getAttribute('aria-labelledby') || '';
+              const label = (explicit?.innerText || wrapping?.innerText || aria || el.placeholder || el.name || id || `field_${i}`).trim();
+              const selector = ((el.type === 'radio' || el.type === 'checkbox') && el.name) ?
+                `input[name="${CSS.escape(el.name)}"]` : id ? `#${CSS.escape(id)}` :
+                (el.name ? `${el.tagName.toLowerCase()}[name="${CSS.escape(el.name)}"]` :
+                 `${el.tagName.toLowerCase()}:nth-of-type(${i + 1})`);
+              return {
+                name: el.name || id || `field_${i}`,
+                label,
+                input_type: el.tagName === 'SELECT' ? 'select' : (el.type || el.tagName.toLowerCase()),
+                required: !!el.required || el.getAttribute('aria-required') === 'true',
+                selector,
+                format_hint: el.title || el.placeholder || '',
+                pattern: el.pattern || '',
+                options: el.tagName === 'SELECT' ? [...el.options].map(o => o.text.trim()).filter(Boolean) :
+                  ((el.type === 'radio' || el.type === 'checkbox') ? [el.value, label].filter(Boolean) : [])
+              };
+            })"""
+        )
+        # Radio buttons with the same name are one logical field.
+        fields: List[FieldSpec] = []
+        seen: set[str] = set()
+        for item in raw:
+            spec = FieldSpec.from_dict(item)
+            if spec.name in seen:
+                existing = next(f for f in fields if f.name == spec.name)
+                existing.options = list(dict.fromkeys(existing.options + spec.options))
+                continue
+            seen.add(spec.name)
+            fields.append(spec)
+        return fields
+
+    @property
+    async def title(self) -> str:
+        return await self.page.title() if self.page else ""
+
+    async def fill(self, field: FieldSpec, value: str) -> None:
+        if self.page is None:
+            raise RuntimeError("browser is not open")
+        from playwright.async_api import Error as PlaywrightError
+
+        try:
+            locator = self.page.locator(field.selector).first
+            await locator.scroll_into_view_if_needed()
+            kind = field.input_type.lower()
+            if kind == "select":
+                try:
+                    await locator.select_option(label=value)
+                except PlaywrightError:
+                    await locator.select_option(value=value)
+            elif kind in {"checkbox", "radio"}:
+                group = self.page.locator(field.selector)
+                wanted = value.strip().casefold()
+                chosen = None
+                for i in range(await group.count()):
+                    item = group.nth(i)
+                    raw_value = (await item.get_attribute("value") or "").strip()
+                    item_id = await item.get_attribute("id")
+                    label = ""
+                    if item_id:
+                        label_node = self.page.locator(f'label[for="{item_id}"]').first
+                        if await label_node.count():
+                            label = (await label_node.inner_text()).strip()
+                    if wanted in {raw_value.casefold(), label.casefold()}:
+                        chosen = item
+                        break
+                if chosen is None:
+                    raise RecoverableFillError(
+                        f"{field.name} has no matching page option"
+                    )
+                await chosen.check()
+            else:
+                await locator.fill(value)
+        except RecoverableFillError:
+            raise
+        except PlaywrightError as exc:
+            # Do not include the value or raw Playwright text: either may contain
+            # user-supplied form data that must not enter logs or traces.
+            raise RecoverableFillError(
+                f"browser could not fill {field.name}: {type(exc).__name__}"
+            ) from exc
+
+    async def submit(self) -> bool:
+        if not self.submit_enabled:
+            return False
+        if self.page is None:
+            raise RuntimeError("browser is not open")
+        button = self.page.locator(
+            'button[type="submit"], input[type="submit"], button:has-text("注册"), button:has-text("Register")'
+        ).first
+        if await button.count() == 0:
+            raise RuntimeError("页面没有可识别的提交按钮")
+        await button.click()
+        await asyncio.sleep(1)
+        return True
+
+    async def close(self) -> None:
+        if self.context:
+            await self.context.close()
+        if self.browser:
+            await self.browser.close()
+        if self._playwright:
+            await self._playwright.stop()
+        self.closed = True
+
+    async def __aenter__(self):
+        await self.open()
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        await self.close()

chapter10/autonomous-phone-registration/bus.py

@@ -0,0 +1,75 @@
+"""Asynchronous, timestamped point-to-point bus for the two live Agents."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import time
+from collections import defaultdict
+from pathlib import Path
+from typing import DefaultDict, List, Optional
+
+from models import AgentMessage
+
+
+class MessageBus:
+    def __init__(self, trace_path: Optional[str] = None):
+        self.started = time.monotonic()
+        self._sequence = 0
+        self._queues: DefaultDict[str, asyncio.Queue[AgentMessage]] = defaultdict(asyncio.Queue)
+        self.history: List[AgentMessage] = []
+        self.trace_path = Path(trace_path) if trace_path else None
+
+    async def send(
+        self,
+        sender: str,
+        recipient: str,
+        type: str,
+        *,
+        sensitive_keys: tuple[str, ...] = (),
+        **payload,
+    ) -> AgentMessage:
+        self._sequence += 1
+        message = AgentMessage(
+            sender=sender,
+            recipient=recipient,
+            type=type,
+            payload=payload,
+            sequence=self._sequence,
+            monotonic_seconds=round(time.monotonic() - self.started, 6),
+            wall_time=time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+        )
+        self.history.append(message)
+        await self._queues[recipient].put(message)
+        printable = {k: ("<redacted>" if k in sensitive_keys else v) for k, v in payload.items()}
+        # Keep the redaction policy beside the in-memory envelope. The receiver gets
+        # the value, while console/disk traces never retain spoken personal data.
+        setattr(message, "_sensitive_keys", sensitive_keys)
+        print(
+            f"[t={message.monotonic_seconds:8.3f}s #{message.sequence:03d}] "
+            f"{sender} -> {recipient} | {type} | "
+            f"{json.dumps(printable, ensure_ascii=False)}"
+        )
+        self.flush()
+        return message
+
+    async def receive(self, recipient: str, timeout: Optional[float] = None) -> AgentMessage:
+        get = self._queues[recipient].get()
+        return await asyncio.wait_for(get, timeout) if timeout else await get
+
+    def flush(self) -> None:
+        if not self.trace_path:
+            return
+        self.trace_path.parent.mkdir(parents=True, exist_ok=True)
+        rows = []
+        for message in self.history:
+            row = message.to_dict()
+            keys = getattr(message, "_sensitive_keys", ())
+            row["payload"] = {
+                k: ("<redacted>" if k in keys else v) for k, v in row["payload"].items()
+            }
+            rows.append(row)
+        self.trace_path.write_text(
+            json.dumps(rows, ensure_ascii=False, indent=2),
+            encoding="utf-8",
+        )

chapter10/autonomous-phone-registration/decision.py

@@ -0,0 +1,249 @@
+"""LLM decision point that may autonomously call ``initiate_phone_call_agent``."""
+
+from __future__ import annotations
+
+import json
+import os
+import time
+from pathlib import Path
+
+from models import DecisionRecord, FieldSpec
+
+TOOL_NAME = "initiate_phone_call_agent"
+
+
+def _clients_and_models():
+    from openai import AsyncOpenAI
+
+    model = os.getenv("OPENAI_MODEL", "gpt-4.1-mini")
+    candidates = []
+    if os.getenv("ARK_API_KEY"):
+        candidates.append(
+            (
+                AsyncOpenAI(
+                    api_key=os.environ["ARK_API_KEY"],
+                    base_url="https://ark.cn-beijing.volces.com/api/v3",
+                ),
+                os.getenv("ARK_MODEL", "doubao-seed-1-6-250615"),
+                "Volcengine ARK",
+            )
+        )
+    if os.getenv("MOONSHOT_API_KEY"):
+        candidates.append(
+            (
+                AsyncOpenAI(
+                    api_key=os.environ["MOONSHOT_API_KEY"],
+                    base_url="https://api.moonshot.cn/v1",
+                ),
+                os.getenv("MOONSHOT_MODEL", "kimi-k3"),
+                "Moonshot",
+            )
+        )
+    if os.getenv("OPENAI_API_KEY"):
+        candidates.append(
+            (
+                AsyncOpenAI(
+                    api_key=os.environ["OPENAI_API_KEY"],
+                    base_url=os.getenv("OPENAI_BASE_URL") or None,
+                ),
+                model,
+                "OpenAI",
+            )
+        )
+    if os.getenv("OPENROUTER_API_KEY"):
+        routed = model if "/" in model else f"openai/{model}"
+        candidates.append(
+            (
+                AsyncOpenAI(
+                    api_key=os.environ["OPENROUTER_API_KEY"],
+                    base_url="https://openrouter.ai/api/v1",
+                ),
+                routed,
+                "OpenRouter",
+            )
+        )
+    if not candidates:
+        raise RuntimeError(
+            "需要 MOONSHOT_API_KEY、ARK_API_KEY、OPENAI_API_KEY 或 OPENROUTER_API_KEY"
+        )
+    return candidates
+
+
+async def decide_orchestration(
+    *,
+    page_url: str,
+    page_title: str,
+    fields: list[FieldSpec],
+    known_values: dict[str, str],
+    elapsed: float,
+    raw_request_path: str | None = None,
+    raw_response_path: str | None = None,
+) -> DecisionRecord:
+    """Let the Computer Use Agent choose whether to initiate a Phone Agent.
+
+    There is intentionally no Python ``if len(fields)`` decision. The model sees the
+    browser observation, available context, and an optional tool; ``tool_choice=auto``
+    is the experiment's autonomy boundary.
+    """
+
+    if bool(raw_request_path) != bool(raw_response_path):
+        raise ValueError("raw decision request and response paths must be provided together")
+    clients = _clients_and_models()
+    visible_fields = [
+        {
+            "name": f.name,
+            "label": f.label,
+            "type": f.input_type,
+            "required": f.required,
+            "format_hint": f.format_hint,
+            "options": f.options,
+        }
+        for f in fields
+    ]
+    tools = [
+        {
+            "type": "function",
+            "function": {
+                "name": TOOL_NAME,
+                "description": (
+                    "Start a live Phone Agent when a user must provide many missing pieces of "
+                    "structured information conversationally. The Phone Agent asks, confirms, "
+                    "validates, and streams each collected field back to the browser Agent."
+                ),
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "purpose": {"type": "string"},
+                        "required_info": {
+                            "type": "array",
+                            "items": {
+                                "type": "object",
+                                "properties": {
+                                    "name": {"type": "string"},
+                                    "label": {"type": "string"},
+                                    "format_hint": {"type": "string"},
+                                },
+                                "required": ["name", "label"],
+                                "additionalProperties": False,
+                            },
+                        },
+                    },
+                    "required": ["purpose", "required_info"],
+                    "additionalProperties": False,
+                },
+            },
+        }
+    ]
+    kwargs = {
+        "messages": [
+            {
+                "role": "system",
+                "content": (
+                    "You are a Computer Use Agent completing a registration request. Inspect "
+                    "the real page observation and the information already in context. When you "
+                    "need to collect a large amount of structured information and it can be done "
+                    "step by step through conversation, consider calling the Phone Agent tool. "
+                    "Do not call it for one or two simple missing values. Never invent user data. "
+                    "Give only a short decision summary; do not reveal private chain-of-thought."
+                ),
+            },
+            {
+                "role": "user",
+                "content": json.dumps(
+                    {
+                        "request": "帮我在这个网站上完成注册",
+                        "page_url": page_url,
+                        "page_title": page_title,
+                        "form_fields": visible_fields,
+                        "known_context_fields": sorted(known_values),
+                    },
+                    ensure_ascii=False,
+                ),
+            },
+        ],
+        "tools": tools,
+        "tool_choice": "auto",
+    }
+    last_error = None
+    for client, model, provider in clients:
+        request_started = time.monotonic()
+        try:
+            response = await client.chat.completions.create(model=model, **kwargs)
+            provider_latency_seconds = round(time.monotonic() - request_started, 6)
+            break
+        except Exception as exc:  # noqa: BLE001 - try the next configured provider
+            last_error = exc
+            print(f"[自主决策] {provider} 调用失败，尝试下一已配置文本端点：{type(exc).__name__}")
+    else:
+        raise RuntimeError("所有已配置的文本模型端点均调用失败") from last_error
+    if raw_request_path and raw_response_path:
+        request_receipt = {
+            "schema_version": 1,
+            "experiment": "10-3",
+            "provider": provider,
+            "endpoint": str(client.base_url).rstrip("/"),
+            "credential_fields_retained": [],
+            "request": {"model": model, **kwargs},
+        }
+        response_receipt = {
+            "schema_version": 1,
+            "experiment": "10-3",
+            "provider": provider,
+            "latency_seconds": provider_latency_seconds,
+            "response": response.model_dump(mode="json"),
+        }
+        for path_value, receipt in (
+            (raw_request_path, request_receipt),
+            (raw_response_path, response_receipt),
+        ):
+            path = Path(path_value)
+            path.parent.mkdir(parents=True, exist_ok=True)
+            path.write_text(
+                json.dumps(receipt, ensure_ascii=False, indent=2),
+                encoding="utf-8",
+            )
+    message = response.choices[0].message
+    call = next((c for c in (message.tool_calls or []) if c.function.name == TOOL_NAME), None)
+    purpose = ""
+    requested: list[FieldSpec] = []
+    if call:
+        args = json.loads(call.function.arguments)
+        purpose = str(args.get("purpose", ""))
+        by_name = {f.name: f for f in fields}
+        by_label = {f.label.casefold(): f for f in fields}
+        for item in args.get("required_info", []):
+            candidate = by_name.get(str(item.get("name", ""))) or by_label.get(
+                str(item.get("label", "")).casefold()
+            )
+            if candidate and candidate.name not in known_values and candidate not in requested:
+                requested.append(candidate)
+
+    return DecisionRecord(
+        page_url=page_url,
+        page_title=page_title,
+        known_fields=sorted(known_values),
+        discovered_fields=fields,
+        tool_called=TOOL_NAME if call else None,
+        purpose=purpose,
+        required_info=requested,
+        rationale_summary=(
+            message.content or "模型通过工具调用决定启动 Phone Agent"
+            if call
+            else "模型决定继续当前流程"
+        ).strip(),
+        model=model,
+        monotonic_seconds=round(time.monotonic() - elapsed, 6),
+        provider=provider,
+        provider_response_id=getattr(response, "id", None),
+        provider_usage={
+            key: int(value)
+            for key, value in {
+                "prompt_tokens": getattr(getattr(response, "usage", None), "prompt_tokens", None),
+                "completion_tokens": getattr(
+                    getattr(response, "usage", None), "completion_tokens", None
+                ),
+                "total_tokens": getattr(getattr(response, "usage", None), "total_tokens", None),
+            }.items()
+            if value is not None
+        },
+    )

chapter10/autonomous-phone-registration/demo.py

@@ -0,0 +1,398 @@
+#!/usr/bin/env python3
+"""Experiment 10-3: autonomously spawn Phone Agent during real browser use."""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import time
+from pathlib import Path
+
+try:
+    from dotenv import load_dotenv
+except ImportError:
+    load_dotenv = None
+else:
+    load_dotenv()
+
+from browser import RegistrationBrowser
+from bus import MessageBus
+from decision import decide_orchestration
+from orchestration import (
+    extraction_receipts,
+    initiate_phone_call_agent,
+    reset_extraction_receipts,
+    run_parallel,
+    timing_evidence,
+)
+from voice import LiveMicrophoneChannel, ScriptedPhoneChannel
+
+DEFAULT_URL = "https://demoqa.com/automation-practice-form"
+
+
+def parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(
+        description="实验 10-3：Computer Use Agent 自主决定并启动实时 Phone Agent",
+    )
+    p.add_argument("--url", default=DEFAULT_URL, help="真实注册/资料表单 URL")
+    p.add_argument("--known-json", default="{}", help="已在上下文中的字段 JSON（键为表单 name/id）")
+    p.add_argument("--headless", action="store_true", help="无界面运行真实 Chromium")
+    p.add_argument(
+        "--submit", action="store_true", help="明确允许最终点击提交；默认只填不提交，避免副作用"
+    )
+    p.add_argument(
+        "--phone-transport",
+        choices=["webrtc", "local", "twilio"],
+        default="webrtc",
+        help="webrtc=本机浏览器通话（默认）；local=本机麦克风；twilio=可选旧 PSTN 路径",
+    )
+    p.add_argument(
+        "--confirm-consent",
+        action="store_true",
+        help="确认参与者已授权本次实验电话/麦克风采集；所有真人语音路径均要求",
+    )
+    p.add_argument("--trace", default="artifacts/message_timeline.json", help="脱敏消息时序输出")
+    p.add_argument("--decision-trace", default="artifacts/decision.json", help="Agent 决策记录输出")
+    p.add_argument(
+        "--raw-decision-request",
+        default=None,
+        help="写入不含凭据的原始编排请求（必须与 --raw-decision-response 同时使用）",
+    )
+    p.add_argument(
+        "--raw-decision-response",
+        default=None,
+        help="写入不含凭据的原始编排响应与延迟（必须与 --raw-decision-request 同时使用）",
+    )
+    p.add_argument(
+        "--acceptance-report",
+        default="artifacts/acceptance_report.json",
+        help="写入机器可读验收门禁",
+    )
+    p.add_argument(
+        "--webrtc-headless",
+        action="store_true",
+        help="无界面运行 WebRTC 参与者（仅用于安全自动验收）",
+    )
+    p.add_argument(
+        "--webrtc-port", type=int, default=0, help="WebRTC 本地通话页端口；0 表示自动选择空闲端口"
+    )
+    p.add_argument(
+        "--webrtc-answers-json",
+        default=None,
+        help=(
+            "安全自动验收：字段名映射到一个回答或回答数组；回答会先合成语音，"
+            "经过真实 WebRTC RTP 音轨，再由 ASR 转录，不会直接注入 Agent"
+        ),
+    )
+    p.add_argument(
+        "--scripted-json",
+        default=None,
+        help="仅用于自动化补充验证：字段名到回答的 JSON；省略则使用真实麦克风 ASR/TTS",
+    )
+    return p
+
+
+def _webrtc_answer_plan(raw: str, fields) -> list[str]:
+    configured = json.loads(raw)
+    if not isinstance(configured, dict):
+        raise SystemExit("--webrtc-answers-json 必须是 JSON object")
+    answers: list[str] = []
+    for field in fields:
+        value = configured.get(field.name, configured.get(field.label))
+        if value is None:
+            raise SystemExit(f"--webrtc-answers-json 缺少字段 {field.name}")
+        if isinstance(value, list):
+            answers.extend(str(item) for item in value)
+        else:
+            answers.append(str(value))
+    return answers
+
+
+def _rtp_is_bidirectional(receipt: dict) -> bool:
+    flowing = {
+        (item.get("side"), item.get("type"))
+        for item in receipt.get("audio_rtp", [])
+        if int(item.get("packets", 0)) > 0 and int(item.get("bytes", 0)) > 0
+    }
+    return {
+        ("agent", "outbound-rtp"),
+        ("agent", "inbound-rtp"),
+        ("participant", "outbound-rtp"),
+        ("participant", "inbound-rtp"),
+    }.issubset(flowing)
+
+
+async def main(args: argparse.Namespace) -> int:
+    known = json.loads(args.known_json)
+    if not isinstance(known, dict):
+        raise SystemExit("--known-json 必须是 JSON object")
+    if args.scripted_json and args.webrtc_answers_json:
+        raise SystemExit("--scripted-json 与 --webrtc-answers-json 不能同时使用")
+    if not args.scripted_json and not args.confirm_consent:
+        raise SystemExit("拒绝电话/音频采集：所有真人语音路径必须显式传入 --confirm-consent")
+    reset_extraction_receipts()
+    started = time.monotonic()
+    bus = MessageBus(args.trace)
+    browser = RegistrationBrowser(args.url, headless=args.headless, submit=args.submit)
+    channel = None
+    try:
+        await browser.open()
+        fields = await browser.discover_fields()
+        title = await browser.title
+        print(f"[Computer Agent] 已打开真实页面：{title} ({args.url})")
+        print(
+            f"[Computer Agent] 发现 {len(fields)} 个可填写字段，其中 {sum(f.required for f in fields)} 个必填"
+        )
+        decision = await decide_orchestration(
+            page_url=args.url,
+            page_title=title,
+            fields=fields,
+            known_values={str(k): str(v) for k, v in known.items()},
+            elapsed=started,
+            raw_request_path=args.raw_decision_request,
+            raw_response_path=args.raw_decision_response,
+        )
+        decision_path = Path(args.decision_trace)
+        decision_path.parent.mkdir(parents=True, exist_ok=True)
+        decision_path.write_text(
+            json.dumps(decision.to_dict(), ensure_ascii=False, indent=2), encoding="utf-8"
+        )
+        print(
+            f"[自主决策] tool_called={decision.tool_called}; summary={decision.rationale_summary}"
+        )
+        if decision.tool_called != "initiate_phone_call_agent":
+            print("Computer Agent 自主判断无需启动 Phone Agent；流程保持在当前 Agent。")
+            report_path = Path(args.acceptance_report)
+            report_path.parent.mkdir(parents=True, exist_ok=True)
+            report_path.write_text(
+                json.dumps(
+                    {
+                        "schema_version": 1,
+                        "experiment": "10-3",
+                        "overall_status": "not_applicable",
+                        "reason": "computer_agent_did_not_spawn_phone_agent",
+                        "decision": decision.to_dict(),
+                    },
+                    ensure_ascii=False,
+                    indent=2,
+                ),
+                encoding="utf-8",
+            )
+            return 2
+
+        if args.scripted_json:
+            scripted = json.loads(args.scripted_json)
+            answers = [str(scripted.get(f.name, "")) for f in decision.required_info]
+            channel = ScriptedPhoneChannel(answers)
+            print("[验证模式] 使用 scripted channel；它只验证编排，不替代实时语音验收。")
+        elif args.phone_transport == "webrtc":
+            from webrtc_channel import WebRTCPhoneChannel
+
+            answer_plan = (
+                _webrtc_answer_plan(args.webrtc_answers_json, decision.required_info)
+                if args.webrtc_answers_json
+                else None
+            )
+            channel = WebRTCPhoneChannel(
+                headless=args.webrtc_headless,
+                port=args.webrtc_port,
+                synthetic_answers=answer_plan,
+            )
+            await channel.start()
+        elif args.phone_transport == "twilio":
+            from twilio_channel import TwilioPhoneChannel
+
+            channel = TwilioPhoneChannel()
+            await channel.start()
+        else:
+            channel = LiveMicrophoneChannel()
+
+        spawned = initiate_phone_call_agent(
+            decision=decision,
+            bus=bus,
+            channel=channel,
+            browser=browser,
+            known_values={str(k): str(v) for k, v in known.items()},
+        )
+        result = await run_parallel(spawned, bus)
+        evidence = timing_evidence(bus)
+        await browser.close()
+        transport = "scripted" if args.scripted_json else args.phone_transport
+        overlap_checks = evidence["overlap_checks"]
+        fill_pass = (
+            not result["errors"]
+            and browser.closed
+            and set(result["filled"]) >= {field.name for field in decision.required_info}
+        )
+        autonomy_pass = bool(
+            decision.tool_called == "initiate_phone_call_agent"
+            and decision.provider
+            and decision.provider_response_id
+        )
+        expected_overlap_count = len(decision.required_info) - 1
+        concurrency_pass = bool(
+            len(decision.required_info) >= 2
+            and evidence["expected_overlap_count"] == expected_overlap_count
+            and len(overlap_checks) == expected_overlap_count
+            and all(item["next_question_before_fill_completed"] for item in overlap_checks)
+        )
+        webrtc_receipt = (
+            channel.acceptance_receipt()
+            if transport == "webrtc" and hasattr(channel, "acceptance_receipt")
+            else None
+        )
+        webrtc_pass = bool(
+            webrtc_receipt
+            and webrtc_receipt["offers"] == 1
+            and webrtc_receipt["answers"] == 1
+            and webrtc_receipt["media_recordings"] >= len(decision.required_info)
+            and webrtc_receipt["status"] == "completed"
+            and _rtp_is_bidirectional(webrtc_receipt)
+        )
+        local_audio_pass = bool(
+            transport == "local"
+            and any("tts_seconds" in item for item in getattr(channel, "latencies", []))
+            and any("asr_seconds" in item for item in getattr(channel, "latencies", []))
+        )
+        submission_pass = bool(args.submit and result["submitted"])
+        repeated_questions = [
+            message
+            for message in bus.history
+            if message.type == "question_asked" and int(message.payload.get("attempt", 1)) > 1
+        ]
+        invalid_events = [message for message in bus.history if message.type == "format_invalid"]
+        reask_pass = bool(invalid_events and repeated_questions)
+        persisted_trace = (
+            Path(args.trace).read_text(encoding="utf-8") if Path(args.trace).exists() else ""
+        )
+        trace_rows = json.loads(persisted_trace or "[]")
+        collected_rows = [row for row in trace_rows if row.get("type") == "info_collected"]
+        privacy_pass = bool(
+            collected_rows
+            and all(row.get("payload", {}).get("value") == "<redacted>" for row in collected_rows)
+            and (
+                not webrtc_receipt
+                or (
+                    webrtc_receipt.get("raw_audio_retained") is False
+                    and webrtc_receipt.get("transcripts_retained") is False
+                )
+            )
+        )
+        live_audio_pass = bool(
+            (webrtc_pass or local_audio_pass)
+            and getattr(channel, "asr_count", 0) >= len(decision.required_info)
+            and getattr(channel, "tts_prompt_count", 0) >= len(decision.required_info) + 2
+        )
+        overall_pass = bool(
+            fill_pass
+            and autonomy_pass
+            and concurrency_pass
+            and webrtc_pass
+            and live_audio_pass
+            and reask_pass
+            and privacy_pass
+            and (submission_pass if args.submit else True)
+        )
+        report = {
+            "schema_version": 2,
+            "experiment": "10-3",
+            "generated_at": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+            "transport": transport,
+            "synthetic_values_used": bool(
+                transport == "scripted" or (webrtc_receipt or {}).get("synthetic_participant")
+            ),
+            "decision_provider": decision.provider,
+            "decision_model": decision.model,
+            "page_url": args.url,
+            "fields_discovered": len(decision.discovered_fields),
+            "required_fields": [field.name for field in decision.required_info],
+            "result": result,
+            "timing_evidence": evidence,
+            "webrtc_receipt": webrtc_receipt,
+            "provider_receipts": {
+                "decision": {
+                    "provider": decision.provider,
+                    "model": decision.model,
+                    "response_id": decision.provider_response_id,
+                    "usage": decision.provider_usage,
+                },
+                "field_extractions": extraction_receipts(),
+                "speech": getattr(channel, "provider_receipts", []),
+            },
+            "gates": {
+                "real_playwright_page_and_fill": {"status": "pass" if fill_pass else "fail"},
+                "autonomous_real_llm_tool_call": {"status": "pass" if autonomy_pass else "fail"},
+                "ask_one_fill_one_concurrency": {"status": "pass" if concurrency_pass else "fail"},
+                "validation_feedback_and_reask": {"status": "pass" if reask_pass else "fail"},
+                "privacy_redaction_and_ephemeral_audio": {
+                    "status": "pass" if privacy_pass else "fail"
+                },
+                "browser_resource_cleanup": {"status": "pass" if browser.closed else "fail"},
+                "real_form_submission": {
+                    "status": "pass"
+                    if submission_pass
+                    else "not_run"
+                    if not args.submit
+                    else "fail",
+                    "reason": None
+                    if submission_pass
+                    else "requires explicit --submit authorization"
+                    if not args.submit
+                    else "submit was authorized but did not complete",
+                },
+                "real_webrtc_session": {
+                    "status": "pass"
+                    if webrtc_pass
+                    else "not_run"
+                    if transport != "webrtc"
+                    else "fail",
+                    "reason": None
+                    if webrtc_pass
+                    else "requires a connected offer/answer and bidirectional RTP audio",
+                },
+                "bidirectional_webrtc_audio_and_real_asr_tts": {
+                    "status": "pass"
+                    if live_audio_pass
+                    else "not_run"
+                    if transport == "scripted"
+                    else "fail",
+                    "reason": None
+                    if live_audio_pass
+                    else "audio media or provider operations did not complete",
+                },
+            },
+            "overall_status": "pass" if overall_pass else "incomplete",
+        }
+        report_path = Path(args.acceptance_report)
+        report_path.parent.mkdir(parents=True, exist_ok=True)
+        report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
+        print(
+            json.dumps(
+                {
+                    "result": result,
+                    "timing_evidence": evidence,
+                    "acceptance_report": str(report_path),
+                },
+                ensure_ascii=False,
+                indent=2,
+            )
+        )
+        return 0 if not result["errors"] else 1
+    finally:
+        if (
+            channel is not None
+            and hasattr(channel, "close")
+            and not getattr(channel, "closed", False)
+        ):
+            try:
+                await channel.close()
+            except Exception as exc:  # noqa: BLE001 - best-effort cleanup must continue
+                print(f"[资源清理] phone channel close failed: {type(exc).__name__}: {exc}")
+        if not browser.closed:
+            await browser.close()
+        print(f"[资源清理] browser/context/page closed={browser.closed}")
+
+
+if __name__ == "__main__":
+    raise SystemExit(asyncio.run(main(parser().parse_args())))

chapter10/autonomous-phone-registration/env.example

@@ -0,0 +1,44 @@
+# LLM orchestration/extraction plus direct ASR/TTS.
+OPENAI_API_KEY=
+OPENAI_MODEL=gpt-4.1-mini
+OPENAI_ASR_MODEL=whisper-1
+OPENAI_TTS_MODEL=tts-1
+
+# Text-model alternatives. The demo tries configured endpoints in order; ASR/TTS
+# still require a direct audio provider.
+# MOONSHOT_API_KEY=
+# MOONSHOT_MODEL=kimi-k3
+# ARK_API_KEY=
+# ARK_MODEL=doubao-seed-1-6-250615
+
+# Optional OpenAI-compatible endpoint for text decisions only. Direct OpenAI is
+# still required for microphone ASR/TTS.
+# OPENAI_BASE_URL=
+# OPENROUTER_API_KEY=
+
+# Tune only if the microphone clips speech or waits too long at sentence end.
+VOICE_SAMPLE_RATE=16000
+VOICE_SILENCE_SECONDS=0.9
+VOICE_RMS_THRESHOLD=0.012
+AUDIO_PLAYER=afplay
+
+# WebRTC audio provider. "auto" prefers local say/espeak + Gemini ASR when
+# available, otherwise OpenAI TTS/ASR. Explicit choices: openai, gemini-system,
+# local-whisper. For local-whisper, install openai-whisper or point to an existing
+# environment; the audio and transcript are both ephemeral.
+WEBRTC_SPEECH_PROVIDER=auto
+GEMINI_API_KEY=
+GEMINI_ASR_MODEL=gemini-2.5-flash
+WHISPER_PYTHON=
+WHISPER_MODEL=tiny
+
+# Optional legacy Twilio transport. TWILIO_WEBHOOK_BASE_URL must point (via an
+# HTTPS tunnel/reverse proxy) to TWILIO_LOCAL_PORT; WebRTC needs none of these.
+TWILIO_ACCOUNT_SID=
+TWILIO_AUTH_TOKEN=
+TWILIO_FROM_NUMBER=
+PHONE_USER_NUMBER=
+TWILIO_WEBHOOK_BASE_URL=
+TWILIO_LOCAL_PORT=8765
+TWILIO_LANGUAGE=zh-CN
+TWILIO_VOICE=Google.zh-CN-Standard-A

chapter10/autonomous-phone-registration/models.py

@@ -0,0 +1,96 @@
+"""Shared contracts for Experiment 10-3.
+
+The contracts are deliberately serialisable: every Computer/Phone Agent exchange is
+also written to the timing trace, so a run can prove what was decided and when.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import asdict, dataclass, field
+from datetime import datetime, timezone
+from typing import Any, Dict, List, Optional
+
+
+@dataclass
+class FieldSpec:
+    name: str
+    label: str
+    input_type: str = "text"
+    required: bool = True
+    selector: str = ""
+    format_hint: str = ""
+    pattern: str = ""
+    options: List[str] = field(default_factory=list)
+
+    @classmethod
+    def from_dict(cls, value: Dict[str, Any]) -> "FieldSpec":
+        allowed = {f.name for f in cls.__dataclass_fields__.values()}
+        return cls(**{k: v for k, v in value.items() if k in allowed})
+
+    def validate(self, value: Optional[str]) -> tuple[bool, str]:
+        val = (str(value) if value is not None else "").strip()
+        if self.required and not val:
+            return False, "该项为必填项，不能留空"
+        if not val:
+            return True, ""
+        if self.options and val not in self.options:
+            lowered = {o.casefold(): o for o in self.options}
+            if val.casefold() not in lowered:
+                return False, f"请选择以下选项之一：{', '.join(self.options)}"
+        if self.pattern:
+            try:
+                if re.fullmatch(self.pattern, val) is None:
+                    return False, self.format_hint or f"格式应匹配 {self.pattern}"
+            except re.error:
+                # A malformed pattern from a web page must not crash the call.
+                pass
+        kind = self.input_type.lower()
+        if kind == "email" and re.fullmatch(r"[^@\s]+@[^@\s]+\.[^@\s]+", val) is None:
+            return False, self.format_hint or "请输入有效邮箱，例如 name@example.com"
+        if kind in {"tel", "phone"} and re.fullmatch(r"[+()\d][+()\d .-]{5,24}", val) is None:
+            return False, self.format_hint or "请输入包含区号的有效电话号码"
+        if kind == "date" and re.fullmatch(r"\d{4}-\d{2}-\d{2}", val) is None:
+            return False, self.format_hint or "日期格式应为 YYYY-MM-DD"
+        if kind == "number":
+            try:
+                float(val)
+            except ValueError:
+                return False, self.format_hint or "请输入数字"
+        return True, ""
+
+
+@dataclass
+class AgentMessage:
+    sender: str
+    recipient: str
+    type: str
+    payload: Dict[str, Any]
+    sequence: int = 0
+    monotonic_seconds: float = 0.0
+    wall_time: str = ""
+
+    def to_dict(self) -> Dict[str, Any]:
+        return asdict(self)
+
+
+@dataclass
+class DecisionRecord:
+    page_url: str
+    page_title: str
+    known_fields: List[str]
+    discovered_fields: List[FieldSpec]
+    tool_called: Optional[str]
+    purpose: str
+    required_info: List[FieldSpec]
+    rationale_summary: str
+    model: str
+    monotonic_seconds: float
+    provider: str = ""
+    provider_response_id: Optional[str] = None
+    provider_usage: Dict[str, int] = field(default_factory=dict)
+    wall_time: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
+
+    def to_dict(self) -> Dict[str, Any]:
+        data = asdict(self)
+        return data

chapter10/autonomous-phone-registration/orchestration.py

@@ -0,0 +1,461 @@
+"""Phone and Computer Agents plus the autonomous tool dispatcher."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+from dataclasses import dataclass
+from typing import Dict, List, Optional
+
+from browser import RecoverableFillError, RegistrationBrowser
+from bus import MessageBus
+from models import DecisionRecord, FieldSpec
+from voice import PhoneChannel
+
+
+_EXTRACTION_RECEIPTS: List[Dict[str, object]] = []
+
+
+def reset_extraction_receipts() -> None:
+    _EXTRACTION_RECEIPTS.clear()
+
+
+def extraction_receipts() -> List[Dict[str, object]]:
+    """Return value-free provider metadata for experiment provenance."""
+    return [dict(item) for item in _EXTRACTION_RECEIPTS]
+
+
+async def _extract_value(field: FieldSpec, utterance: str) -> str:
+    """Use the Phone Agent's LLM to turn a natural spoken answer into one value."""
+    from openai import AsyncOpenAI
+
+    clients = []
+    if os.getenv("ARK_API_KEY"):
+        clients.append((AsyncOpenAI(
+            api_key=os.environ["ARK_API_KEY"],
+            base_url="https://ark.cn-beijing.volces.com/api/v3",
+        ), os.getenv("ARK_MODEL", "doubao-seed-1-6-250615"), "Volcengine ARK"))
+    if os.getenv("MOONSHOT_API_KEY"):
+        clients.append((AsyncOpenAI(
+            api_key=os.environ["MOONSHOT_API_KEY"],
+            base_url="https://api.moonshot.cn/v1",
+        ), os.getenv("MOONSHOT_MODEL", "kimi-k3"), "Moonshot"))
+    if os.getenv("OPENAI_API_KEY"):
+        clients.append((AsyncOpenAI(
+            api_key=os.environ["OPENAI_API_KEY"],
+            base_url=os.getenv("OPENAI_BASE_URL") or None,
+        ), os.getenv("OPENAI_MODEL", "gpt-4.1-mini"), "OpenAI"))
+    if os.getenv("OPENROUTER_API_KEY"):
+        client = AsyncOpenAI(
+            api_key=os.environ["OPENROUTER_API_KEY"],
+            base_url="https://openrouter.ai/api/v1",
+        )
+        raw_model = os.getenv("OPENAI_MODEL", "gpt-4.1-mini")
+        clients.append((client, raw_model if "/" in raw_model else f"openai/{raw_model}", "OpenRouter"))
+    if not clients:
+        raise RuntimeError("Phone Agent 的语义抽取需要任一已支持文本模型 API Key")
+
+    kwargs = dict(messages=[
+            {
+                "role": "system",
+                "content": (
+                    "Extract only the value the user supplied for the requested form field. "
+                    "Never infer a missing value. Preserve identifiers exactly, while normalizing "
+                    "explicitly spoken email words such as 'at' and 'dot' to symbols and spoken "
+                    "number words to digits when the field requires them. Return exactly "
+                    "one JSON object with the schema {\"value\": \"the extracted value\"}."
+                ),
+            },
+            {
+                "role": "user",
+                "content": json.dumps(
+                    {
+                        "field": field.label,
+                        "type": field.input_type,
+                        "format_hint": field.format_hint,
+                        "options": field.options,
+                        "spoken_answer": utterance,
+                    },
+                    ensure_ascii=False,
+                ),
+            },
+        ],
+        response_format={"type": "json_object"},
+    )
+    last_error = None
+    for client, model, provider in clients:
+        try:
+            model_kwargs = dict(kwargs)
+            if "kimi-k3" in model:
+                model_kwargs["temperature"] = 1
+                model_kwargs["max_tokens"] = 2048
+            response = await client.chat.completions.create(model=model, **model_kwargs)
+            if not (response.choices[0].message.content or "").strip():
+                raise ValueError("模型返回空 content")
+            break
+        except Exception as exc:
+            last_error = exc
+            print(f"  [Phone Agent] {provider} 抽取失败，尝试下一端点：{type(exc).__name__}")
+    else:
+        raise RuntimeError("所有已配置的 Phone Agent 文本端点均失败") from last_error
+    data = json.loads(response.choices[0].message.content or "{}")
+    usage = getattr(response, "usage", None)
+    _EXTRACTION_RECEIPTS.append({
+        "operation": "field_value_extraction",
+        "provider": provider,
+        "model": model,
+        "response_id": getattr(response, "id", None),
+        "usage": {
+            key: int(value)
+            for key, value in {
+                "prompt_tokens": getattr(usage, "prompt_tokens", None),
+                "completion_tokens": getattr(usage, "completion_tokens", None),
+                "total_tokens": getattr(usage, "total_tokens", None),
+            }.items()
+            if value is not None
+        },
+        "transcript_or_value_retained": False,
+    })
+    return str(data.get("value", "")).strip()
+
+
+class PhoneAgent:
+    def __init__(
+        self,
+        bus: MessageBus,
+        channel: PhoneChannel,
+        purpose: str,
+        required_info: List[FieldSpec],
+        *,
+        max_retries: int = 3,
+    ):
+        self.bus = bus
+        self.channel = channel
+        self.purpose = purpose
+        self.required_info = required_info
+        self.max_retries = max_retries
+        self.browser_feedback: List[Dict[str, str]] = []
+        self.form_ready = asyncio.Event()
+
+    async def _receive_computer_feedback(self):
+        """Independent inbound loop: Computer -> Phone is not a write-only channel."""
+        while True:
+            message = await self.bus.receive("phone_agent")
+            if message.type == "fill_error":
+                self.browser_feedback.append({
+                    "field": str(message.payload.get("field", "")),
+                    "error": str(message.payload.get("error", "")),
+                })
+            elif message.type == "form_ready":
+                self.form_ready.set()
+                return
+
+    async def run(self) -> None:
+        """Run the dialogue and always release its inbound loop/transport."""
+        self._feedback_task = None
+        try:
+            await self._run_dialogue()
+        finally:
+            if self._feedback_task is not None:
+                self._feedback_task.cancel()
+                await asyncio.gather(self._feedback_task, return_exceptions=True)
+            if hasattr(self.channel, "close") and not getattr(self.channel, "closed", False):
+                await self.channel.close()
+
+    async def _run_dialogue(self) -> None:
+        feedback_task = asyncio.create_task(
+            self._receive_computer_feedback(), name="phone-inbound-computer-messages"
+        )
+        self._feedback_task = feedback_task
+        await self.bus.send(
+            "phone_agent", "computer_agent", "call_started",
+            purpose=self.purpose,
+            fields=[f.name for f in self.required_info],
+        )
+        await self.channel.say(f"您好，我正在{self.purpose}。我会逐项询问并核对格式。")
+
+        for field in self.required_info:
+            accepted = False
+            feedback = ""
+            for attempt in range(1, self.max_retries + 1):
+                question = f"请问您的{field.label}是什么？"
+                if field.format_hint:
+                    question += f" 格式要求：{field.format_hint}。"
+                if feedback:
+                    question = f"刚才的回答无法通过校验：{feedback}。{question}"
+                await self.bus.send(
+                    "phone_agent", "computer_agent", "question_asked",
+                    field=field.name,
+                    attempt=attempt,
+                )
+                await self.channel.say(question)
+                try:
+                    utterance = await self.channel.listen()
+                    # An omitted optional answer is a deliberate skip, not a value
+                    # to write into a stateful page widget (some date controls react
+                    # destructively to programmatic empty-string fills).
+                    value = "" if not field.required and not utterance.strip() else await _extract_value(field, utterance)
+                except Exception as exc:
+                    await self.bus.send(
+                        "phone_agent", "computer_agent", "call_failed",
+                        field=field.name, reason=f"语音/抽取失败：{type(exc).__name__}",
+                    )
+                    if hasattr(self.channel, "close"):
+                        await self.channel.close()
+                    feedback_task.cancel()
+                    await asyncio.gather(feedback_task, return_exceptions=True)
+                    return
+                valid, feedback = field.validate(value)
+                if not valid:
+                    await self.bus.send(
+                        "phone_agent", "computer_agent", "format_invalid",
+                        field=field.name,
+                        attempt=attempt,
+                        reason=feedback,
+                    )
+                    continue
+
+                if not value and not field.required:
+                    await self.bus.send(
+                        "phone_agent", "computer_agent", "info_skipped",
+                        field=field.name, attempt=attempt, reason="optional_blank",
+                    )
+                    accepted = True
+                    break
+
+                await self.bus.send(
+                    "phone_agent",
+                    "computer_agent",
+                    "info_collected",
+                    sensitive_keys=("value",),
+                    field=field.name,
+                    value=value,
+                    attempt=attempt,
+                )
+                # Deliberately do not await a browser acknowledgement: the next
+                # question starts while Computer Agent locates/fills this field.
+                accepted = True
+                break
+            if not accepted:
+                await self.bus.send(
+                    "phone_agent", "computer_agent", "call_failed",
+                    field=field.name,
+                    reason="超过格式重问次数",
+                )
+                await self.channel.say("抱歉，这一项多次未通过格式校验，本次注册已安全暂停。")
+                if hasattr(self.channel, "close"):
+                    await self.channel.close()
+                feedback_task.cancel()
+                await asyncio.gather(feedback_task, return_exceptions=True)
+                return
+
+        await self.bus.send("phone_agent", "computer_agent", "task_completed")
+        # Ask/fill stayed fully concurrent field-by-field; only the final goodbye
+        # waits for Computer Agent's aggregate result so browser errors can flow back.
+        try:
+            await asyncio.wait_for(self.form_ready.wait(), timeout=60)
+        except asyncio.TimeoutError:
+            self.browser_feedback.append({"field": "form", "error": "电脑端最终确认超时"})
+        if self.browser_feedback:
+            await self.channel.say("信息已收集，但电脑端填写遇到问题，表单已暂停提交，请稍后查看错误报告。")
+        else:
+            await self.channel.say("所需信息已经收集并填写完成，电脑端已完成最后确认。")
+        if hasattr(self.channel, "close"):
+            await self.channel.close()
+        feedback_task.cancel()
+        await asyncio.gather(feedback_task, return_exceptions=True)
+
+
+class ComputerAgent:
+    def __init__(
+        self,
+        bus: MessageBus,
+        browser: RegistrationBrowser,
+        field_specs: List[FieldSpec],
+        known_values: Dict[str, str],
+    ):
+        self.bus = bus
+        self.browser = browser
+        self.fields = {f.name: f for f in field_specs}
+        self.known_values = known_values
+        self.filled: List[str] = []
+        self.errors: List[Dict[str, str]] = []
+        self.submitted = False
+
+    async def _fill(self, name: str, value: str) -> None:
+        field = self.fields.get(name)
+        if not field:
+            raise KeyError(f"页面中不存在字段 {name}")
+        await self.browser.fill(field, value)
+        self.filled.append(name)
+        await self.bus.send("computer_agent", "phone_agent", "field_filled", field=name)
+
+    async def _report_fill_error(self, name: str, exc: RecoverableFillError) -> None:
+        """Record one browser failure and forward the shared error envelope."""
+        error = {"field": name, "error": str(exc)}
+        self.errors.append(error)
+        await self.bus.send(
+            "computer_agent",
+            "phone_agent",
+            "fill_error",
+            sensitive_keys=("error",),
+            **error,
+        )
+
+    async def run(self) -> Dict[str, object]:
+        for name, value in self.known_values.items():
+            if name in self.fields:
+                try:
+                    await self._fill(name, value)
+                except RecoverableFillError as exc:
+                    # Mirror the in-dialogue fill path below: surface the failure
+                    # to the phone agent (via browser_feedback) so it doesn't tell
+                    # the user registration succeeded when a known field failed.
+                    await self._report_fill_error(name, exc)
+
+        completed = False
+        while not completed:
+            # This idle cap must exceed the phone side's worst-case per-question
+            # latency (TTS + the channel's own listen window + value extraction).
+            # The default WebRTC human listen allows a start timer plus an answer
+            # timer (~240s total), so a 120s cap here aborts a live call while the
+            # user is still legitimately answering. run_parallel cancels this task
+            # the moment the phone task completes or errors, so a larger cap only
+            # relaxes the false-abort case.
+            message = await self.bus.receive("computer_agent", timeout=600)
+            if message.type == "info_collected":
+                name = message.payload.get("field")
+                value = message.payload.get("value")
+                if not isinstance(name, str) or not name:
+                    raise ValueError("info_collected requires a non-empty field")
+                if not isinstance(value, str):
+                    raise ValueError("info_collected requires a string value")
+                try:
+                    await self._fill(name, value)
+                except RecoverableFillError as exc:
+                    await self._report_fill_error(name, exc)
+            elif message.type == "call_failed":
+                self.errors.append({
+                    "field": message.payload.get("field", ""),
+                    "error": message.payload.get("reason", "Phone Agent failed"),
+                })
+                completed = True
+            elif message.type == "task_completed":
+                completed = True
+            elif message.type == "info_skipped":
+                # Optional blank values require no browser operation. The explicit
+                # envelope keeps the two Agents' timelines auditable.
+                continue
+
+        if not self.errors:
+            self.submitted = await self.browser.submit()
+        await self.bus.send(
+            "computer_agent", "phone_agent", "form_ready",
+            errors=len(self.errors), submitted=self.submitted,
+        )
+        await self.bus.send(
+            "computer_agent", "manager", "registration_finished",
+            filled=self.filled,
+            submitted=self.submitted,
+            errors=self.errors,
+        )
+        return {
+            "filled": self.filled,
+            "submitted": self.submitted,
+            "errors": self.errors,
+        }
+
+
+@dataclass
+class SpawnedAgents:
+    phone: PhoneAgent
+    computer: ComputerAgent
+
+
+def initiate_phone_call_agent(
+    *,
+    decision: DecisionRecord,
+    bus: MessageBus,
+    channel: PhoneChannel,
+    browser: RegistrationBrowser,
+    known_values: Dict[str, str],
+) -> SpawnedAgents:
+    """Tool dispatcher invoked only after the model emits the matching tool call."""
+    if decision.tool_called != "initiate_phone_call_agent":
+        raise RuntimeError("模型未调用 initiate_phone_call_agent，不能预先创建 Phone Agent")
+    if not decision.required_info:
+        raise RuntimeError("Phone Agent 工具调用没有任何可映射的页面字段")
+    return SpawnedAgents(
+        phone=PhoneAgent(bus, channel, decision.purpose, decision.required_info),
+        computer=ComputerAgent(bus, browser, decision.discovered_fields, known_values),
+    )
+
+
+async def run_parallel(agents: SpawnedAgents, bus: MessageBus) -> Dict[str, object]:
+    phone_task = asyncio.create_task(agents.phone.run(), name="phone-agent-react-loop")
+    computer_task = asyncio.create_task(agents.computer.run(), name="computer-agent-react-loop")
+    tasks = (phone_task, computer_task)
+    try:
+        done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_EXCEPTION)
+        failure = next(
+            (task.exception() for task in done if not task.cancelled() and task.exception() is not None),
+            None,
+        )
+        if failure is not None:
+            for task in pending:
+                task.cancel()
+            await asyncio.gather(*pending, return_exceptions=True)
+            raise failure
+        _phone_result, computer_result = await asyncio.gather(*tasks)
+    except BaseException:
+        # ``asyncio.gather`` does not cancel a still-running peer when one task
+        # raises. A failed audio/browser loop must not leave the other Agent
+        # blocked on its inbox, nor leave a PSTN/webhook transport open.
+        for task in tasks:
+            if not task.done():
+                task.cancel()
+        await asyncio.gather(*tasks, return_exceptions=True)
+        channel = agents.phone.channel
+        if hasattr(channel, "close") and not getattr(channel, "closed", False):
+            await channel.close()
+        raise
+    finished = await bus.receive("manager", timeout=5)
+    assert finished.type == "registration_finished"
+    return computer_result
+
+
+def timing_evidence(bus: MessageBus) -> Dict[str, object]:
+    questions = {
+        m.payload["field"]: m.monotonic_seconds
+        for m in bus.history if m.type == "question_asked" and m.payload.get("attempt") == 1
+    }
+    collected = {
+        m.payload["field"]: m.monotonic_seconds
+        for m in bus.history if m.type == "info_collected"
+    }
+    filled = {
+        m.payload["field"]: m.monotonic_seconds
+        for m in bus.history if m.type == "field_filled"
+    }
+    ordered = list(questions)
+    overlaps = []
+    expected_overlap_count = 0
+    for current, next_field in zip(ordered, ordered[1:]):
+        if current in collected and current in filled:
+            expected_overlap_count += 1
+            overlaps.append({
+                "field_being_filled": current,
+                "next_question": next_field,
+                "next_question_before_fill_completed": questions[next_field] < filled[current],
+                "next_question_at": questions[next_field],
+                "fill_completed_at": filled[current],
+            })
+    return {
+        "question_times": questions,
+        "collection_times": collected,
+        "fill_times": filled,
+        "overlap_checks": overlaps,
+        "expected_overlap_count": expected_overlap_count,
+        "independent_tasks": ["phone-agent-react-loop", "computer-agent-react-loop"],
+    }

chapter10/autonomous-phone-registration/requirements.txt

@@ -0,0 +1,11 @@
+openai>=1.30.0
+playwright>=1.44.0
+python-dotenv>=1.0.0
+sounddevice>=0.4.6
+numpy>=1.26.0
+pytest>=8.0.0
+pytest-asyncio>=0.23.0
+twilio>=9.0.0
+fastapi>=0.111.0
+uvicorn>=0.30.0
+python-multipart>=0.0.9

chapter10/autonomous-phone-registration/run_acceptance.py

@@ -0,0 +1,326 @@
+#!/usr/bin/env python3
+"""Run the complete safe current Experiment 10-3 acceptance scenario.
+
+The form and its submission endpoint are localhost-only.  Synthetic personal data is
+spoken by the configured TTS provider, crosses a real WebRTC audio track, is recorded
+at the remote peer, and goes through the configured ASR provider.  It is not injected
+as text.  One deliberately invalid email proves validation feedback and re-asking.
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import hashlib
+import json
+import re
+import threading
+import time
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from typing import ClassVar
+from urllib.parse import parse_qs
+
+import demo
+from validate_acceptance import validate_run
+
+FORM_HTML = """<!doctype html>
+<html lang="en"><meta charset="utf-8"><title>Safe local registration</title>
+<h1>Conference registration</h1>
+<form method="post" action="/register">
+  <label for="firstName">First name</label>
+  <input id="firstName" name="firstName" required>
+  <label for="lastName">Last name</label>
+  <input id="lastName" name="lastName" required>
+  <label for="email">Email address</label>
+  <input id="email" name="email" type="email" required placeholder="name@example.com">
+  <label for="userNumber">Phone number</label>
+  <input id="userNumber" name="userNumber" type="tel" required pattern="[0-9]{10}" title="10 digits">
+  <label for="gender">Gender</label>
+  <select id="gender" name="gender" required>
+    <option value="">Choose one</option><option>Female</option><option>Male</option><option>Non-binary</option>
+  </select>
+  <label for="address">Mailing address</label>
+  <textarea id="address" name="address" required></textarea>
+  <button type="submit">Register</button>
+</form></html>"""
+
+
+ANSWERS = {
+    "firstName": "Alice",
+    "lastName": "Tan",
+    "email": ["This is not an email address", "alice@example.com"],
+    "userNumber": "9123456789",
+    "gender": "Female",
+    "address": "One Example Street, Singapore",
+}
+
+CREDENTIAL_PATTERN = re.compile(
+    r"(?i)(?:sk-[A-Za-z0-9_-]{12,}|gho_[A-Za-z0-9_-]{12,}|"
+    r"github_pat_[A-Za-z0-9_-]{12,}|authorization.{0,16}bearer\s+[A-Za-z0-9._-]{12,})"
+)
+
+
+class _AcceptanceFormHandler(BaseHTTPRequestHandler):
+    submissions: ClassVar[list[dict[str, object]]] = []
+
+    def do_GET(self):
+        if self.path != "/register":
+            self.send_error(404)
+            return
+        body = FORM_HTML.encode("utf-8")
+        self.send_response(200)
+        self.send_header("Content-Type", "text/html; charset=utf-8")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def do_POST(self):
+        if self.path != "/register":
+            self.send_error(404)
+            return
+        length = int(self.headers.get("Content-Length", "0"))
+        parsed = parse_qs(self.rfile.read(length).decode("utf-8"), keep_blank_values=True)
+        self.__class__.submissions.append(
+            {
+                "field_names": sorted(parsed),
+                "field_count": len(parsed),
+                "all_values_redacted": True,
+            }
+        )
+        body = b"registration accepted by local test endpoint"
+        self.send_response(200)
+        self.send_header("Content-Type", "text/plain")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def log_message(self, _format, *_args):
+        return
+
+
+def sha256(path: Path) -> str:
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def _synthetic_values() -> list[str]:
+    values = []
+    for answer in ANSWERS.values():
+        if isinstance(answer, list):
+            values.extend(str(item) for item in answer)
+        else:
+            values.append(str(answer))
+    # Select options such as "Female" legitimately appear in the page observation
+    # before the participant answers; they are public schema, not collected PII.
+    return [value for value in values if value and value not in FORM_HTML]
+
+
+def _has_credential(value: str) -> bool:
+    return bool(CREDENTIAL_PATTERN.search(value))
+
+
+async def _git_head(root: Path) -> str:
+    process = await asyncio.create_subprocess_exec(
+        "git",
+        "rev-parse",
+        "HEAD",
+        cwd=root,
+        stdout=asyncio.subprocess.PIPE,
+        stderr=asyncio.subprocess.PIPE,
+    )
+    stdout, stderr = await process.communicate()
+    if process.returncode != 0:
+        raise RuntimeError(f"git rev-parse HEAD failed: {stderr.decode('utf-8').strip()}")
+    return stdout.decode("utf-8").strip()
+
+
+def parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(description="Safe full acceptance for current Experiment 10-3")
+    p.add_argument(
+        "--run-dir", default=None, help="output directory (default: timestamped validation run)"
+    )
+    return p
+
+
+async def run(run_dir: Path) -> int:
+    run_dir.mkdir(parents=True, exist_ok=False)
+    _AcceptanceFormHandler.submissions = []
+    server = ThreadingHTTPServer(("127.0.0.1", 0), _AcceptanceFormHandler)
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    url = f"http://127.0.0.1:{server.server_port}/register"
+    report_path = run_dir / "acceptance_report.json"
+    decision_path = run_dir / "decision.json"
+    timeline_path = run_dir / "message_timeline.json"
+    receipt_path = run_dir / "form_submission_receipt.json"
+    raw_request_path = run_dir / "raw_decision_request.json"
+    raw_response_path = run_dir / "raw_decision_response.json"
+    input_path = run_dir / "experiment_input.json"
+    validation_report_path = run_dir / "validation_report.json"
+    experiment_input = {
+        "schema_version": 1,
+        "experiment": "10-3",
+        "page_url": url,
+        "form_html": FORM_HTML,
+        "form_html_sha256": hashlib.sha256(FORM_HTML.encode("utf-8")).hexdigest(),
+        "field_answer_counts": {
+            name: len(value) if isinstance(value, list) else 1 for name, value in ANSWERS.items()
+        },
+        "participant": "safe synthesized voice over WebRTC RTP",
+        "participant_values_retained": False,
+    }
+    input_path.write_text(
+        json.dumps(experiment_input, ensure_ascii=False, indent=2), encoding="utf-8"
+    )
+    try:
+        args = demo.parser().parse_args(
+            [
+                "--url",
+                url,
+                "--headless",
+                "--submit",
+                "--phone-transport",
+                "webrtc",
+                "--webrtc-headless",
+                "--confirm-consent",
+                "--webrtc-answers-json",
+                json.dumps(ANSWERS),
+                "--trace",
+                str(timeline_path),
+                "--decision-trace",
+                str(decision_path),
+                "--raw-decision-request",
+                str(raw_request_path),
+                "--raw-decision-response",
+                str(raw_response_path),
+                "--acceptance-report",
+                str(report_path),
+            ]
+        )
+        exit_code = await demo.main(args)
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join(timeout=2)
+
+    receipt = {
+        "endpoint_scope": "localhost-only",
+        "submission_count": len(_AcceptanceFormHandler.submissions),
+        "submissions": _AcceptanceFormHandler.submissions,
+        "raw_values_retained": False,
+    }
+    receipt_path.write_text(json.dumps(receipt, indent=2), encoding="utf-8")
+    report = json.loads(report_path.read_text(encoding="utf-8"))
+    submission_pass = bool(
+        exit_code == 0
+        and receipt["submission_count"] == 1
+        and receipt["submissions"][0]["field_count"] == len(ANSWERS)
+        and set(receipt["submissions"][0]["field_names"]) == set(ANSWERS)
+    )
+    report["safe_local_submission_receipt"] = receipt
+    report["gates"]["real_form_submission"] = {
+        "status": "pass" if submission_pass else "fail",
+        "reason": None
+        if submission_pass
+        else "localhost endpoint did not receive exactly one complete submission",
+    }
+    persisted = "\n".join(
+        path.read_text(encoding="utf-8")
+        for path in (report_path, decision_path, timeline_path, receipt_path)
+    )
+    value_leak = any(value in persisted for value in _synthetic_values())
+    credential_leak = _has_credential(persisted)
+    privacy_pass = bool(
+        report["gates"]["privacy_redaction_and_ephemeral_audio"]["status"] == "pass"
+        and not value_leak
+        and not credential_leak
+    )
+    report["gates"]["privacy_redaction_and_ephemeral_audio"] = {
+        "status": "pass" if privacy_pass else "fail",
+        "reason": None if privacy_pass else "retained artifacts failed the credential/value scan",
+    }
+    all_gates_pass = all(item["status"] == "pass" for item in report["gates"].values())
+    report["overall_status"] = "pass" if all_gates_pass else "incomplete"
+    report_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
+
+    root = Path(__file__).parent
+    git_head = await _git_head(root)
+    runtime_files = [
+        "browser.py",
+        "bus.py",
+        "decision.py",
+        "demo.py",
+        "models.py",
+        "orchestration.py",
+        "run_acceptance.py",
+        "validate_acceptance.py",
+        "voice.py",
+        "webrtc_channel.py",
+    ]
+    artifacts = [
+        report_path,
+        decision_path,
+        timeline_path,
+        receipt_path,
+        raw_request_path,
+        raw_response_path,
+    ]
+    manifest = {
+        "schema_version": 2,
+        "experiment": "10-3",
+        "run_kind": "full_safe_webrtc_acceptance",
+        "generated_at": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+        "git_head_at_run": git_head,
+        "command": "python run_acceptance.py --run-dir <validation-run-directory>",
+        "providers": {
+            "decision_and_extraction": report["decision_provider"],
+            "speech": report["webrtc_receipt"]["speech_provider"],
+        },
+        "privacy": {
+            "phone_number_required": False,
+            "pstn_provider_required": False,
+            "participant": "safe synthesized voice",
+            "raw_audio_retained": False,
+            "transcripts_or_values_retained": False,
+            "form_values_retained": False,
+        },
+        "source_sha256": {name: sha256(root / name) for name in runtime_files},
+        "input_sha256": {input_path.name: sha256(input_path)},
+        "artifact_sha256": {path.name: sha256(path) for path in artifacts},
+        "acceptance": {
+            "overall_status": report["overall_status"],
+            "gate_count": len(report["gates"]),
+            "passed_gate_count": sum(item["status"] == "pass" for item in report["gates"].values()),
+        },
+    }
+    manifest_path = run_dir / "manifest.json"
+    manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
+    validation_report = validate_run(
+        run_dir,
+        source_root=root,
+        require_validation_report=False,
+    )
+    validation_report_path.write_text(
+        json.dumps(validation_report, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+    manifest["artifact_sha256"][validation_report_path.name] = sha256(validation_report_path)
+    manifest["retained_evidence_validation"] = validation_report["status"]
+    manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
+    final_validation = validate_run(run_dir, source_root=root)
+    if final_validation != validation_report:
+        raise RuntimeError("standalone validation result changed after manifest finalization")
+    print(
+        json.dumps({"run_dir": str(run_dir), "overall_status": report["overall_status"]}, indent=2)
+    )
+    return 0 if report["overall_status"] == "pass" else 1
+
+
+if __name__ == "__main__":
+    arguments = parser().parse_args()
+    destination = (
+        Path(arguments.run_dir)
+        if arguments.run_dir
+        else Path("validation/runs") / ("exp10-3-webrtc-" + time.strftime("%Y%m%dT%H%M%S%z"))
+    )
+    raise SystemExit(asyncio.run(run(destination)))

chapter10/autonomous-phone-registration/test_exp103_validation_evidence.py

@@ -0,0 +1,110 @@
+import hashlib
+import json
+from pathlib import Path
+
+ROOT = Path(__file__).parent
+WEBRTC_RUN = ROOT / "validation/runs/exp10-3-webrtc-raw-20260731-v4"
+
+
+def test_persisted_evidence_is_redacted_and_does_not_overclaim_voice():
+    report = json.loads((ROOT / "validation/real_browser_llm_2026-07-29.json").read_text())
+    timeline = json.loads((ROOT / "validation/message_timeline_2026-07-29.json").read_text())
+    assert report["gates"]["real_playwright_page_and_fill"]["status"] == "pass"
+    assert report["gates"]["autonomous_real_llm_tool_call"]["status"] == "pass"
+    assert report["gates"]["real_pstn_call"]["status"] == "not_run"
+    assert report["gates"]["real_audio_asr_tts"]["status"] == "not_run"
+    assert report["gates"]["real_form_submission"]["status"] == "not_run"
+    assert report["overall_status"] == "incomplete"
+    collected = [e for e in timeline["events"] if e["type"] == "info_collected"]
+    assert collected and all(e["payload"]["value"] == "<redacted>" for e in collected)
+
+
+def test_software_gate_record_preserves_live_acceptance_blockers():
+    data = json.loads((ROOT / "validation/software_gates_2026-07-29.json").read_text())
+    assert data["pstn_calls_placed"] == 0
+    assert data["human_audio_used"] is False
+    assert all(status == "pass" for status in data["gates"].values())
+    assert data["acceptance_boundary"]["real_pstn_call"] == "not_run"
+    assert data["acceptance_boundary"]["real_human_asr_tts"] == "not_run"
+    assert data["acceptance_boundary"]["real_external_form_submission"] == "not_run"
+    assert data["acceptance_boundary"]["overall_status"] == "incomplete"
+
+
+def test_latest_real_browser_llm_recheck_passes_only_safe_gates():
+    data = json.loads((ROOT / "validation/real_browser_llm_recheck_2026-07-29.json").read_text())
+    assert data["gates"]["real_playwright_page_and_fill"]["status"] == "pass"
+    assert data["gates"]["autonomous_real_llm_tool_call"]["status"] == "pass"
+    assert data["gates"]["ask_one_fill_one_concurrency"]["status"] == "pass"
+    assert (
+        len(data["timing_evidence"]["overlap_checks"])
+        == data["timing_evidence"]["expected_overlap_count"]
+        == 3
+    )
+    assert all(
+        item["next_question_before_fill_completed"]
+        for item in data["timing_evidence"]["overlap_checks"]
+    )
+    assert set(data["persisted_collected_values"]) == {"<redacted>"}
+    assert data["pstn_calls_placed"] == data["external_form_submissions"] == 0
+    assert data["human_audio_used"] is False
+    assert data["gates"]["real_form_submission"]["status"] == "not_run"
+    assert data["gates"]["real_pstn_call"]["status"] == "not_run"
+    assert data["gates"]["real_audio_asr_tts"]["status"] == "not_run"
+    assert data["overall_status"] == "incomplete"
+
+
+def test_formal_webrtc_acceptance_passes_every_gate_without_pstn():
+    report = json.loads((WEBRTC_RUN / "acceptance_report.json").read_text())
+    receipt = json.loads((WEBRTC_RUN / "form_submission_receipt.json").read_text())
+    timeline = json.loads((WEBRTC_RUN / "message_timeline.json").read_text())
+
+    assert report["overall_status"] == "pass"
+    assert all(gate["status"] == "pass" for gate in report["gates"].values())
+    assert report["provider_receipts"]["decision"]["response_id"]
+    assert len(report["provider_receipts"]["field_extractions"]) == 7
+    assert report["result"] == {
+        "filled": ["firstName", "lastName", "email", "userNumber", "gender", "address"],
+        "submitted": True,
+        "errors": [],
+    }
+    assert receipt["endpoint_scope"] == "localhost-only"
+    assert receipt["submission_count"] == 1
+    assert receipt["raw_values_retained"] is False
+
+    media = report["webrtc_receipt"]
+    assert media["offers"] == media["answers"] == 1
+    assert media["media_recordings"] == media["asr_count"] == 7
+    assert media["tts_prompt_count"] == 9
+    assert media["raw_audio_retained"] is media["transcripts_retained"] is False
+    assert all(item["packets"] > 0 and item["bytes"] > 0 for item in media["audio_rtp"])
+
+    assert sum(row["type"] == "format_invalid" for row in timeline) == 1
+    assert any(
+        row["type"] == "question_asked" and row["payload"] == {"field": "email", "attempt": 2}
+        for row in timeline
+    )
+    collected = [row for row in timeline if row["type"] == "info_collected"]
+    assert len(collected) == 6
+    assert {row["payload"]["value"] for row in collected} == {"<redacted>"}
+    overlaps = report["timing_evidence"]["overlap_checks"]
+    assert len(overlaps) == report["timing_evidence"]["expected_overlap_count"] == 5
+    assert all(row["next_question_before_fill_completed"] for row in overlaps)
+
+
+def test_formal_webrtc_manifest_hashes_runtime_and_artifacts():
+    manifest = json.loads((WEBRTC_RUN / "manifest.json").read_text())
+    assert manifest["schema_version"] == 2
+    assert manifest["retained_evidence_validation"] == "pass"
+    assert manifest["acceptance"] == {
+        "overall_status": "pass",
+        "gate_count": 9,
+        "passed_gate_count": 9,
+    }
+    assert manifest["privacy"]["phone_number_required"] is False
+    assert manifest["privacy"]["pstn_provider_required"] is False
+    for name, expected in manifest["artifact_sha256"].items():
+        assert hashlib.sha256((WEBRTC_RUN / name).read_bytes()).hexdigest() == expected
+    for name, expected in manifest["input_sha256"].items():
+        assert hashlib.sha256((WEBRTC_RUN / name).read_bytes()).hexdigest() == expected
+    for name, expected in manifest["source_sha256"].items():
+        assert hashlib.sha256((ROOT / name).read_bytes()).hexdigest() == expected

chapter10/autonomous-phone-registration/test_orchestration.py

@@ -0,0 +1,335 @@
+import asyncio
+from unittest.mock import AsyncMock, patch
+
+import pytest
+
+import demo
+from browser import RecoverableFillError
+from bus import MessageBus
+from models import DecisionRecord, FieldSpec
+from orchestration import (
+    ComputerAgent,
+    initiate_phone_call_agent,
+    run_parallel,
+    timing_evidence,
+)
+from voice import ScriptedPhoneChannel
+
+
+class FakeBrowser:
+    def __init__(self):
+        self.values = {}
+        self.submit_enabled = False
+
+    async def fill(self, field, value):
+        await asyncio.sleep(0.05)
+        self.values[field.name] = value
+
+    async def submit(self):
+        return False
+
+
+class SubmittingFakeBrowser(FakeBrowser):
+    def __init__(self):
+        super().__init__()
+        self.submit_enabled = True
+        self.submit_calls = 0
+
+    async def submit(self):
+        self.submit_calls += 1
+        return True
+
+
+class FailingFillBrowser(FakeBrowser):
+    def __init__(self):
+        super().__init__()
+        self.submit_calls = 0
+
+    async def fill(self, field, value):
+        raise RecoverableFillError(f"cannot fill {field.name}")
+
+    async def submit(self):
+        self.submit_calls += 1
+        return True
+
+
+@pytest.mark.asyncio
+async def test_ask_one_fill_one_runs_concurrently_and_reasks_invalid_format():
+    fields = [
+        FieldSpec("email", "邮箱", "email", format_hint="name@example.com"),
+        FieldSpec("birth", "出生日期", "date", format_hint="YYYY-MM-DD"),
+    ]
+    decision = DecisionRecord(
+        page_url="https://example.test/register",
+        page_title="Register",
+        known_fields=[],
+        discovered_fields=fields,
+        tool_called="initiate_phone_call_agent",
+        purpose="协助填写注册表单",
+        required_info=fields,
+        rationale_summary="tool call",
+        model="test",
+        monotonic_seconds=0,
+    )
+    # First email answer is invalid, forcing format feedback and a real re-ask.
+    channel = ScriptedPhoneChannel(["bad", "me@example.com", "2020-01-02"])
+    bus = MessageBus()
+    browser = FakeBrowser()
+    extracted = AsyncMock(side_effect=["bad", "me@example.com", "2020-01-02"])
+    agents = initiate_phone_call_agent(
+        decision=decision,
+        bus=bus,
+        channel=channel,
+        browser=browser,
+        known_values={},
+    )
+    with patch("orchestration._extract_value", extracted):
+        result = await run_parallel(agents, bus)
+
+    assert result["errors"] == []
+    assert browser.values == {"email": "me@example.com", "birth": "2020-01-02"}
+    assert any(m.type == "format_invalid" for m in bus.history)
+    evidence = timing_evidence(bus)
+    assert any(c["next_question_before_fill_completed"] for c in evidence["overlap_checks"])
+
+
+def test_field_validation_handles_email_phone_date_and_page_pattern():
+    assert not FieldSpec("e", "email", "email").validate("bad")[0]
+    assert FieldSpec("e", "email", "email").validate("a@b.com")[0]
+    assert not FieldSpec("d", "date", "date").validate("01/02/2020")[0]
+    assert FieldSpec("p", "code", pattern=r"[A-Z]{2}\d{4}").validate("AB1234")[0]
+
+
+@pytest.mark.asyncio
+async def test_live_transport_without_consent_refuses_before_browser_or_audio_creation():
+    args = demo.parser().parse_args(["--headless", "--phone-transport", "local"])
+    with patch("demo.RegistrationBrowser") as browser_type, patch(
+        "demo.LiveMicrophoneChannel"
+    ) as audio_type:
+        with pytest.raises(SystemExit, match="confirm-consent"):
+            await demo.main(args)
+    browser_type.assert_not_called()
+    audio_type.assert_not_called()
+
+
+@pytest.mark.asyncio
+async def test_unexpected_phone_failure_cancels_peer_and_closes_channel():
+    class ExplodingChannel:
+        def __init__(self):
+            self.closed = False
+
+        async def say(self, _text):
+            raise RuntimeError("synthetic transport failure")
+
+        async def listen(self, *, timeout=30):  # pragma: no cover - say fails first
+            raise AssertionError(timeout)
+
+        async def close(self):
+            self.closed = True
+
+    fields = [FieldSpec("email", "邮箱", "email")]
+    decision = DecisionRecord(
+        page_url="https://example.test/register",
+        page_title="Register",
+        known_fields=[],
+        discovered_fields=fields,
+        tool_called="initiate_phone_call_agent",
+        purpose="协助填写注册表单",
+        required_info=fields,
+        rationale_summary="tool call",
+        model="test",
+        monotonic_seconds=0,
+    )
+    bus = MessageBus()
+    channel = ExplodingChannel()
+    agents = initiate_phone_call_agent(
+        decision=decision,
+        bus=bus,
+        channel=channel,
+        browser=FakeBrowser(),
+        known_values={},
+    )
+    with pytest.raises(RuntimeError, match="synthetic transport failure"):
+        await asyncio.wait_for(run_parallel(agents, bus), timeout=0.5)
+    assert channel.closed is True
+    assert not any(
+        task.get_name() in {"phone-agent-react-loop", "computer-agent-react-loop"}
+        and not task.done()
+        for task in asyncio.all_tasks()
+    )
+
+
+@pytest.mark.asyncio
+async def test_task_completed_triggers_submission_when_browser_is_opted_in():
+    fields = [FieldSpec("email", "邮箱", "email")]
+    decision = DecisionRecord(
+        page_url="https://example.test/register",
+        page_title="Register",
+        known_fields=[],
+        discovered_fields=fields,
+        tool_called="initiate_phone_call_agent",
+        purpose="协助填写注册表单",
+        required_info=fields,
+        rationale_summary="tool call",
+        model="test",
+        monotonic_seconds=0,
+    )
+    browser = SubmittingFakeBrowser()
+    bus = MessageBus()
+    agents = initiate_phone_call_agent(
+        decision=decision,
+        bus=bus,
+        channel=ScriptedPhoneChannel(["me@example.com"]),
+        browser=browser,
+        known_values={},
+    )
+    with patch("orchestration._extract_value", AsyncMock(return_value="me@example.com")):
+        result = await run_parallel(agents, bus)
+    assert result["submitted"] is True
+    assert browser.submit_calls == 1
+
+
+@pytest.mark.asyncio
+async def test_computer_fill_error_flows_back_to_phone_and_blocks_submission():
+    fields = [FieldSpec("email", "邮箱", "email")]
+    decision = DecisionRecord(
+        page_url="https://example.test/register",
+        page_title="Register",
+        known_fields=[],
+        discovered_fields=fields,
+        tool_called="initiate_phone_call_agent",
+        purpose="协助填写注册表单",
+        required_info=fields,
+        rationale_summary="tool call",
+        model="test",
+        monotonic_seconds=0,
+    )
+    browser = FailingFillBrowser()
+    channel = ScriptedPhoneChannel(["me@example.com"])
+    bus = MessageBus()
+    agents = initiate_phone_call_agent(
+        decision=decision,
+        bus=bus,
+        channel=channel,
+        browser=browser,
+        known_values={},
+    )
+    with patch("orchestration._extract_value", AsyncMock(return_value="me@example.com")):
+        result = await run_parallel(agents, bus)
+
+    assert result["submitted"] is False
+    assert browser.submit_calls == 0
+    assert agents.phone.browser_feedback == [
+        {"field": "email", "error": "cannot fill email"}
+    ]
+    fill_error = next(message for message in bus.history if message.type == "fill_error")
+    assert getattr(fill_error, "_sensitive_keys") == ("error",)
+    assert "填写遇到问题" in channel.prompts[-1]
+
+
+@pytest.mark.asyncio
+async def test_malformed_info_collected_fails_before_fill_error_handling():
+    bus = MessageBus()
+    computer = ComputerAgent(bus, FakeBrowser(), [], {})
+    task = asyncio.create_task(computer.run())
+    await bus.send(
+        "phone_agent",
+        "computer_agent",
+        "info_collected",
+        sensitive_keys=("value",),
+        value="secret",
+    )
+
+    with pytest.raises(ValueError, match="non-empty field"):
+        await task
+    assert not any(message.type == "fill_error" for message in bus.history)
+
+
+@pytest.mark.asyncio
+async def test_optional_blank_is_audited_skip_and_never_written_to_browser():
+    fields = [
+        FieldSpec("email", "邮箱", "email"),
+        FieldSpec("birthday", "生日", "text", required=False),
+        FieldSpec("address", "地址", "textarea", required=False),
+    ]
+    decision = DecisionRecord(
+        page_url="https://example.test/register",
+        page_title="Register",
+        known_fields=[],
+        discovered_fields=fields,
+        tool_called="initiate_phone_call_agent",
+        purpose="协助填写注册表单",
+        required_info=fields,
+        rationale_summary="tool call",
+        model="test",
+        monotonic_seconds=0,
+    )
+    browser = FakeBrowser()
+    bus = MessageBus()
+    agents = initiate_phone_call_agent(
+        decision=decision,
+        bus=bus,
+        channel=ScriptedPhoneChannel(["me@example.com", "", ""]),
+        browser=browser,
+        known_values={},
+    )
+    with patch("orchestration._extract_value", AsyncMock(return_value="me@example.com")) as extract:
+        result = await run_parallel(agents, bus)
+
+    assert result["errors"] == []
+    assert browser.values == {"email": "me@example.com"}
+    assert extract.await_count == 1
+    skipped = [message for message in bus.history if message.type == "info_skipped"]
+    assert [message.payload["field"] for message in skipped] == ["birthday", "address"]
+    evidence = timing_evidence(bus)
+    assert evidence["expected_overlap_count"] == 1
+    assert len(evidence["overlap_checks"]) == 1
+
+
+class CountryFailBrowser(SubmittingFakeBrowser):
+    """Fails only the pre-filled known field, succeeds on the phone-collected one."""
+
+    async def fill(self, field, value):
+        if field.name == "country":
+            raise RecoverableFillError(f"cannot fill {field.name}")
+        await asyncio.sleep(0.05)
+        self.values[field.name] = value
+
+
+@pytest.mark.asyncio
+async def test_known_value_fill_error_flows_back_to_phone_and_blocks_submission():
+    # A known_values field that fails to pre-fill must surface a fill_error (like
+    # the in-dialogue path), so the phone agent reports the failure instead of
+    # telling the user registration completed.
+    email = FieldSpec("email", "邮箱", "email")
+    country = FieldSpec("country", "国家", "text")
+    decision = DecisionRecord(
+        page_url="https://example.test/register",
+        page_title="Register",
+        known_fields=[country.name],
+        discovered_fields=[email, country],
+        tool_called="initiate_phone_call_agent",
+        purpose="协助填写注册表单",
+        required_info=[email],
+        rationale_summary="tool call",
+        model="test",
+        monotonic_seconds=0,
+    )
+    browser = CountryFailBrowser()
+    channel = ScriptedPhoneChannel(["me@example.com"])
+    bus = MessageBus()
+    agents = initiate_phone_call_agent(
+        decision=decision,
+        bus=bus,
+        channel=channel,
+        browser=browser,
+        known_values={"country": "US"},
+    )
+    with patch("orchestration._extract_value", AsyncMock(return_value="me@example.com")):
+        result = await run_parallel(agents, bus)
+
+    assert result["submitted"] is False
+    assert browser.submit_calls == 0
+    assert {"field": "country", "error": "cannot fill country"} in agents.phone.browser_feedback
+    assert any(message.type == "fill_error" for message in bus.history)
+    assert "填写遇到问题" in channel.prompts[-1]

chapter10/autonomous-phone-registration/test_validate_acceptance.py

@@ -0,0 +1,77 @@
+import hashlib
+import json
+import shutil
+from pathlib import Path
+
+import pytest
+from validate_acceptance import ValidationFailure, validate_run
+
+ROOT = Path(__file__).parent
+RUN = ROOT / "validation/runs/exp10-3-webrtc-raw-20260731-v4"
+
+
+def _write(path: Path, value: dict) -> None:
+    path.write_text(json.dumps(value, ensure_ascii=False, indent=2), encoding="utf-8")
+
+
+def _rehash_artifact(run_dir: Path, name: str) -> None:
+    manifest_path = run_dir / "manifest.json"
+    manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+    manifest["artifact_sha256"][name] = hashlib.sha256((run_dir / name).read_bytes()).hexdigest()
+    _write(manifest_path, manifest)
+
+
+def _copy_run(tmp_path: Path) -> Path:
+    destination = tmp_path / "run"
+    shutil.copytree(RUN, destination)
+    return destination
+
+
+def test_standalone_validator_proves_raw_receipt_consistency() -> None:
+    result = validate_run(RUN, source_root=ROOT)
+    assert result["status"] == "pass"
+    assert result["checks"]["raw_ark_request_tool_choice_auto"] == "pass"
+    assert result["checks"]["raw_arguments_normalize_to_decision"] == "pass"
+
+
+def test_validator_rejects_semantically_modified_raw_receipt(tmp_path: Path) -> None:
+    run_dir = _copy_run(tmp_path)
+    path = run_dir / "raw_decision_response.json"
+    receipt = json.loads(path.read_text(encoding="utf-8"))
+    receipt["response"]["id"] = "tampered-response-id"
+    _write(path, receipt)
+    _rehash_artifact(run_dir, path.name)
+
+    with pytest.raises(ValidationFailure, match="response ID differs"):
+        validate_run(run_dir, source_root=ROOT)
+
+
+def test_validator_rejects_semantically_modified_normalized_decision(tmp_path: Path) -> None:
+    run_dir = _copy_run(tmp_path)
+    path = run_dir / "decision.json"
+    decision = json.loads(path.read_text(encoding="utf-8"))
+    decision["purpose"] = "tampered normalized purpose"
+    _write(path, decision)
+    _rehash_artifact(run_dir, path.name)
+
+    with pytest.raises(ValidationFailure, match="raw purpose differs"):
+        validate_run(run_dir, source_root=ROOT)
+
+
+def test_validator_rejects_modified_manifest_hash(tmp_path: Path) -> None:
+    run_dir = _copy_run(tmp_path)
+    path = run_dir / "manifest.json"
+    manifest = json.loads(path.read_text(encoding="utf-8"))
+    manifest["artifact_sha256"]["raw_decision_request.json"] = "0" * 64
+    _write(path, manifest)
+
+    with pytest.raises(ValidationFailure, match="artifact_sha256 hash mismatch"):
+        validate_run(run_dir, source_root=ROOT)
+
+
+def test_validator_rejects_unbound_retained_artifact(tmp_path: Path) -> None:
+    run_dir = _copy_run(tmp_path)
+    (run_dir / "unbound_transcript.txt").write_text("unexpected", encoding="utf-8")
+
+    with pytest.raises(ValidationFailure, match="retained run files differ"):
+        validate_run(run_dir, source_root=ROOT)

chapter10/autonomous-phone-registration/test_webrtc_channel.py

@@ -0,0 +1,87 @@
+import io
+import math
+import struct
+import wave
+
+import pytest
+
+from demo import _rtp_is_bidirectional, _webrtc_answer_plan
+from models import FieldSpec
+from run_acceptance import _has_credential, _synthetic_values
+from webrtc_channel import CALL_PAGE, WebRTCPhoneChannel
+
+
+class ToneSpeechBackend:
+    provider = "deterministic-test-tone"
+
+    async def synthesize(self, _text):
+        stream = io.BytesIO()
+        sample_rate = 16_000
+        with wave.open(stream, "wb") as wav:
+            wav.setnchannels(1)
+            wav.setsampwidth(2)
+            wav.setframerate(sample_rate)
+            samples = [
+                int(10_000 * math.sin(2 * math.pi * 440 * i / sample_rate))
+                for i in range(sample_rate // 2)
+            ]
+            wav.writeframes(b"".join(struct.pack("<h", sample) for sample in samples))
+        return stream.getvalue(), "audio/wav", {
+            "operation": "tts", "provider": self.provider, "latency_seconds": 0.0,
+        }
+
+    async def transcribe(self, audio, mime):
+        assert mime.startswith("audio/webm")
+        assert len(audio) > 256
+        return "accepted answer", {
+            "operation": "asr", "provider": self.provider, "latency_seconds": 0.0,
+            "raw_audio_retained": False, "transcript_retained": False,
+        }
+
+
+def test_page_uses_real_webrtc_media_and_keeps_answers_off_control_channel():
+    assert "new RTCPeerConnection" in CALL_PAGE
+    assert "addTrack" in CALL_PAGE
+    assert "MediaRecorder(agentInput" in CALL_PAGE
+    assert "non-sensitive-control" in CALL_PAGE
+    assert "control.send(JSON.stringify({type: 'prompt', text}))" in CALL_PAGE
+    assert "answer" not in CALL_PAGE.split("control.send", 1)[1].split(";", 1)[0]
+
+
+def test_answer_plan_preserves_retry_answers_in_field_order():
+    fields = [FieldSpec("name", "Name"), FieldSpec("email", "Email", "email")]
+    assert _webrtc_answer_plan(
+        '{"email":["bad","me@example.com"],"name":"Alice"}', fields
+    ) == ["Alice", "bad", "me@example.com"]
+
+
+def test_privacy_markers_exclude_public_form_options_but_include_private_values():
+    markers = _synthetic_values()
+    assert "Female" not in markers
+    assert "alice@example.com" in markers
+    assert "9123456789" in markers
+    assert not _has_credential("ask_one_fill_one_concurrency")
+    assert _has_credential("sk-examplecredential123")
+
+
+@pytest.mark.asyncio
+async def test_real_browser_webrtc_offer_answer_and_bidirectional_rtp():
+    channel = WebRTCPhoneChannel(
+        headless=True,
+        synthetic_answers=["synthetic answer"],
+        speech_backend=ToneSpeechBackend(),
+    )
+    try:
+        await channel.start()
+        await channel.say("question")
+        assert await channel.listen(timeout=10) == "accepted answer"
+    finally:
+        await channel.close()
+
+    receipt = channel.acceptance_receipt()
+    assert receipt["offers"] == receipt["answers"] == 1
+    assert receipt["media_recordings"] == 1
+    assert receipt["status"] == "completed"
+    assert _rtp_is_bidirectional(receipt)
+    assert receipt["raw_audio_retained"] is False
+    assert receipt["transcripts_retained"] is False

chapter10/autonomous-phone-registration/twilio_channel.py

@@ -0,0 +1,146 @@
+"""Real PSTN transport for the Experiment 10-3 Phone Agent.
+
+Twilio places one outbound call. Its speech ``Gather`` provides ASR and ``Say``
+provides TTS; the call stays open while the Phone and Computer Agents work.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+from typing import List
+
+
+class TwilioPhoneChannel:
+    def __init__(self):
+        required = ["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN", "TWILIO_FROM_NUMBER",
+                    "PHONE_USER_NUMBER", "TWILIO_WEBHOOK_BASE_URL"]
+        missing = [name for name in required if not os.getenv(name)]
+        if missing:
+            raise RuntimeError(f"Twilio PSTN 缺少环境变量：{', '.join(missing)}")
+        self.sid = os.environ["TWILIO_ACCOUNT_SID"]
+        self.token = os.environ["TWILIO_AUTH_TOKEN"]
+        self.from_number = os.environ["TWILIO_FROM_NUMBER"]
+        self.to_number = os.environ["PHONE_USER_NUMBER"]
+        self.base_url = os.environ["TWILIO_WEBHOOK_BASE_URL"].rstrip("/")
+        self.port = int(os.getenv("TWILIO_LOCAL_PORT", "8765"))
+        self.language = os.getenv("TWILIO_LANGUAGE", "zh-CN")
+        self.voice = os.getenv("TWILIO_VOICE", "Google.zh-CN-Standard-A")
+        self._pending: List[str] = []
+        self._answers: asyncio.Queue[str] = asyncio.Queue()
+        self._closing = False
+        self._server = None
+        self._server_task = None
+        self.call_sid = None
+        self.asr_count = 0
+        self.tts_prompt_count = 0
+        self.call_status = "not_started"
+        self._client = None
+        self.closed = False
+
+    async def start(self):
+        from fastapi import FastAPI, Request, Response
+        from twilio.request_validator import RequestValidator
+        from twilio.rest import Client
+        from twilio.twiml.voice_response import Gather, VoiceResponse
+        import uvicorn
+
+        app = FastAPI()
+        validator = RequestValidator(self.token)
+
+        async def verified(request: Request, form) -> bool:
+            signature = request.headers.get("X-Twilio-Signature", "")
+            public_url = self.base_url + request.url.path
+            return validator.validate(public_url, dict(form), signature)
+
+        @app.post("/voice")
+        async def voice(request: Request):
+            form = await request.form()
+            if not await verified(request, form):
+                return Response("invalid signature", status_code=403)
+            response = VoiceResponse()
+            if self._closing:
+                for text in self._pending:
+                    response.say(text, language=self.language, voice=self.voice)
+                    self.tts_prompt_count += 1
+                self._pending.clear()
+                response.hangup()
+            elif self._pending:
+                text = " ".join(self._pending)
+                self._pending.clear()
+                self.tts_prompt_count += 1
+                gather = Gather(
+                    input="speech",
+                    action=f"{self.base_url}/gather",
+                    method="POST",
+                    language=self.language,
+                    speech_timeout="auto",
+                    timeout=8,
+                )
+                gather.say(text, language=self.language, voice=self.voice)
+                response.append(gather)
+                response.redirect(f"{self.base_url}/voice", method="POST")
+            else:
+                response.pause(length=1)
+                response.redirect(f"{self.base_url}/voice", method="POST")
+            return Response(str(response), media_type="application/xml")
+
+        @app.post("/gather")
+        async def gather_result(request: Request):
+            form = await request.form()
+            if not await verified(request, form):
+                return Response("invalid signature", status_code=403)
+            transcript = str(form.get("SpeechResult", "")).strip()
+            if transcript:
+                await self._answers.put(transcript)
+                self.asr_count += 1
+            response = VoiceResponse()
+            response.redirect(f"{self.base_url}/voice", method="POST")
+            return Response(str(response), media_type="application/xml")
+
+        config = uvicorn.Config(app, host="0.0.0.0", port=self.port, log_level="warning")
+        self._server = uvicorn.Server(config)
+        self._server_task = asyncio.create_task(self._server.serve())
+        while not self._server.started:
+            await asyncio.sleep(0.05)
+
+        client = Client(self.sid, self.token)
+        self._client = client
+        call = await asyncio.to_thread(
+            client.calls.create,
+            to=self.to_number,
+            from_=self.from_number,
+            url=f"{self.base_url}/voice",
+            method="POST",
+        )
+        self.call_sid = call.sid
+        self.call_status = call.status or "queued"
+        print(f"  [PSTN] outbound call initiated; call SID suffix={call.sid[-6:]}")
+
+    async def say(self, text: str) -> None:
+        self._pending.append(text)
+
+    async def listen(self, *, timeout: float = 45.0) -> str:
+        text = await asyncio.wait_for(self._answers.get(), timeout)
+        print(f"  [Twilio ASR] 用户：{text}")
+        return text
+
+    async def close(self):
+        if self.closed:
+            return
+        self._closing = True
+        try:
+            # Let the current Gather redirect once so the final queued Say + Hangup is served.
+            await asyncio.sleep(2)
+            if self._server:
+                self._server.should_exit = True
+            if self._server_task:
+                await self._server_task
+            if self._client and self.call_sid:
+                try:
+                    call = await asyncio.to_thread(self._client.calls(self.call_sid).fetch)
+                    self.call_status = call.status
+                except Exception as exc:
+                    self.call_status = f"status_check_failed:{type(exc).__name__}"
+        finally:
+            self.closed = True

chapter10/autonomous-phone-registration/validate_acceptance.py

@@ -0,0 +1,348 @@
+#!/usr/bin/env python3
+"""Fail-closed validator for retained historical 10-3 evidence of current Experiment 10-3."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import re
+from pathlib import Path
+from typing import Any
+
+TOOL_NAME = "initiate_phone_call_agent"
+ARK_PROVIDER = "Volcengine ARK"
+ARK_ENDPOINT = "https://ark.cn-beijing.volces.com/api/v3"
+INPUT_NAMES = {"experiment_input.json"}
+ARTIFACT_NAMES = {
+    "acceptance_report.json",
+    "decision.json",
+    "form_submission_receipt.json",
+    "message_timeline.json",
+    "raw_decision_request.json",
+    "raw_decision_response.json",
+    "validation_report.json",
+}
+SOURCE_NAMES = {
+    "browser.py",
+    "bus.py",
+    "decision.py",
+    "demo.py",
+    "models.py",
+    "orchestration.py",
+    "run_acceptance.py",
+    "validate_acceptance.py",
+    "voice.py",
+    "webrtc_channel.py",
+}
+CREDENTIAL_PATTERN = re.compile(
+    r"(?i)(?:sk-[A-Za-z0-9_-]{12,}|gho_[A-Za-z0-9_-]{12,}|"
+    r"github_pat_[A-Za-z0-9_-]{12,}|authorization.{0,16}bearer\s+[A-Za-z0-9._-]{12,})"
+)
+
+
+class ValidationFailure(RuntimeError):
+    """Raised when retained evidence does not prove its claims."""
+
+
+def _require(condition: bool, message: str) -> None:
+    if not condition:
+        raise ValidationFailure(message)
+
+
+def _load_json(path: Path) -> dict[str, Any]:
+    try:
+        value = json.loads(path.read_text(encoding="utf-8"))
+    except (OSError, UnicodeError, json.JSONDecodeError) as exc:
+        raise ValidationFailure(f"cannot read JSON evidence {path.name}: {exc}") from exc
+    _require(isinstance(value, dict), f"{path.name} must contain a JSON object")
+    return value
+
+
+def _sha256(path: Path) -> str:
+    try:
+        return hashlib.sha256(path.read_bytes()).hexdigest()
+    except OSError as exc:
+        raise ValidationFailure(f"cannot hash {path}: {exc}") from exc
+
+
+def _validate_hash_map(
+    *,
+    expected_names: set[str],
+    hashes: Any,
+    base: Path,
+    label: str,
+) -> None:
+    _require(isinstance(hashes, dict), f"manifest {label} must be an object")
+    names = set(hashes)
+    _require(names == expected_names, f"manifest {label} names differ: {sorted(names)}")
+    for name, expected in hashes.items():
+        _require(
+            isinstance(expected, str) and len(expected) == 64, f"invalid {label} hash for {name}"
+        )
+        _require(_sha256(base / name) == expected, f"{label} hash mismatch for {name}")
+
+
+def _tool_call(response: dict[str, Any]) -> dict[str, Any]:
+    choices = response.get("choices")
+    _require(isinstance(choices, list) and len(choices) == 1, "raw response must have one choice")
+    choice = choices[0]
+    _require(
+        choice.get("finish_reason") == "tool_calls", "raw response did not finish with tool_calls"
+    )
+    message = choice.get("message", {})
+    calls = message.get("tool_calls")
+    _require(isinstance(calls, list) and len(calls) == 1, "raw response must have one tool call")
+    call = calls[0]
+    _require(call.get("type") == "function", "raw response tool call must be a function")
+    function = call.get("function", {})
+    _require(function.get("name") == TOOL_NAME, "raw response selected the wrong tool")
+    return function
+
+
+def _normalized_required_info(
+    raw_arguments: dict[str, Any], decision: dict[str, Any]
+) -> list[dict[str, Any]]:
+    discovered = decision.get("discovered_fields")
+    _require(isinstance(discovered, list), "normalized decision lacks discovered_fields")
+    by_name = {str(field.get("name", "")): field for field in discovered}
+    by_label = {str(field.get("label", "")).casefold(): field for field in discovered}
+    known = set(decision.get("known_fields", []))
+    normalized = []
+    for item in raw_arguments.get("required_info", []):
+        _require(isinstance(item, dict), "raw required_info entries must be objects")
+        candidate = by_name.get(str(item.get("name", ""))) or by_label.get(
+            str(item.get("label", "")).casefold()
+        )
+        _require(candidate is not None, "raw tool arguments reference an unknown field")
+        if candidate["name"] not in known and candidate not in normalized:
+            normalized.append(candidate)
+    return normalized
+
+
+def validate_run(
+    run_dir: Path,
+    *,
+    source_root: Path | None = None,
+    require_validation_report: bool = True,
+) -> dict[str, Any]:
+    """Validate one retained run and return a deterministic validation report."""
+    run_dir = run_dir.resolve()
+    source_root = (source_root or Path(__file__).parent).resolve()
+    manifest = _load_json(run_dir / "manifest.json")
+    _require(manifest.get("schema_version") == 2, "manifest schema_version must be 2")
+    _require(manifest.get("experiment") == "10-3", "manifest experiment must be 10-3")
+
+    artifact_names = (
+        ARTIFACT_NAMES if require_validation_report else ARTIFACT_NAMES - {"validation_report.json"}
+    )
+    expected_run_names = {"manifest.json"} | INPUT_NAMES | artifact_names
+    actual_run_names = {path.name for path in run_dir.iterdir()}
+    _require(
+        actual_run_names == expected_run_names,
+        f"retained run files differ: {sorted(actual_run_names)}",
+    )
+    input_hashes = manifest.get("input_sha256")
+    _validate_hash_map(
+        expected_names=INPUT_NAMES,
+        hashes=input_hashes,
+        base=run_dir,
+        label="input_sha256",
+    )
+    _validate_hash_map(
+        expected_names=artifact_names,
+        hashes=manifest.get("artifact_sha256"),
+        base=run_dir,
+        label="artifact_sha256",
+    )
+    source_hashes = manifest.get("source_sha256")
+    _require(isinstance(source_hashes, dict), "manifest source_sha256 must be an object")
+    _require(set(source_hashes) == SOURCE_NAMES, "manifest source_sha256 names differ")
+    for name, expected in source_hashes.items():
+        _require(_sha256(source_root / name) == expected, f"source_sha256 hash mismatch for {name}")
+    _require(
+        re.fullmatch(r"[0-9a-f]{40}", str(manifest.get("git_head_at_run", ""))) is not None,
+        "manifest git_head_at_run is invalid",
+    )
+
+    experiment_input = _load_json(run_dir / "experiment_input.json")
+    raw_request = _load_json(run_dir / "raw_decision_request.json")
+    raw_response = _load_json(run_dir / "raw_decision_response.json")
+    decision = _load_json(run_dir / "decision.json")
+    acceptance = _load_json(run_dir / "acceptance_report.json")
+    form_receipt = _load_json(run_dir / "form_submission_receipt.json")
+    timeline = json.loads((run_dir / "message_timeline.json").read_text(encoding="utf-8"))
+
+    _require(raw_request.get("provider") == ARK_PROVIDER, "raw request is not an ARK request")
+    _require(raw_request.get("endpoint") == ARK_ENDPOINT, "raw request uses an unexpected endpoint")
+    _require(
+        raw_request.get("credential_fields_retained") == [],
+        "raw request retained credential fields",
+    )
+    request = raw_request.get("request", {})
+    _require(request.get("tool_choice") == "auto", "raw request did not use tool_choice=auto")
+    tools = request.get("tools")
+    _require(
+        isinstance(tools, list) and len(tools) == 1, "raw request must expose one optional tool"
+    )
+    _require(
+        tools[0].get("function", {}).get("name") == TOOL_NAME, "raw request tool schema differs"
+    )
+
+    _require(raw_response.get("provider") == ARK_PROVIDER, "raw response is not from ARK")
+    _require(decision.get("provider") == ARK_PROVIDER, "normalized decision is not from ARK")
+    latency = raw_response.get("latency_seconds")
+    _require(
+        isinstance(latency, (int, float)) and latency > 0, "raw response lacks positive latency"
+    )
+    response = raw_response.get("response", {})
+    _require(
+        response.get("id") == decision.get("provider_response_id"),
+        "response ID differs from decision",
+    )
+    _require(request.get("model") == decision.get("model"), "request model differs from decision")
+    _require(response.get("model") == decision.get("model"), "response model differs from decision")
+    usage = response.get("usage", {})
+    normalized_usage = {
+        key: usage[key]
+        for key in ("prompt_tokens", "completion_tokens", "total_tokens")
+        if key in usage
+    }
+    _require(
+        normalized_usage == decision.get("provider_usage"), "response usage differs from decision"
+    )
+
+    function = _tool_call(response)
+    try:
+        raw_arguments = json.loads(function["arguments"])
+    except (KeyError, TypeError, json.JSONDecodeError) as exc:
+        raise ValidationFailure("raw tool-call arguments are not valid JSON") from exc
+    _require(isinstance(raw_arguments, dict), "raw tool-call arguments must be an object")
+    _require(
+        set(raw_arguments) == {"purpose", "required_info"},
+        "raw tool-call arguments contain unexpected fields",
+    )
+    _require(isinstance(raw_arguments["required_info"], list), "raw required_info must be a list")
+    _require(decision.get("tool_called") == TOOL_NAME, "normalized decision records the wrong tool")
+    _require(
+        raw_arguments.get("purpose") == decision.get("purpose"), "raw purpose differs from decision"
+    )
+    _require(
+        _normalized_required_info(raw_arguments, decision) == decision.get("required_info"),
+        "raw tool-call arguments do not normalize exactly to decision.json",
+    )
+
+    messages = request.get("messages")
+    _require(
+        isinstance(messages, list) and len(messages) == 2, "raw request messages are incomplete"
+    )
+    try:
+        user_observation = json.loads(messages[1]["content"])
+    except (KeyError, TypeError, json.JSONDecodeError) as exc:
+        raise ValidationFailure("raw request user observation is invalid") from exc
+    _require(
+        user_observation.get("page_url") == experiment_input.get("page_url"),
+        "input page URL differs",
+    )
+    visible_decision_fields = [
+        {
+            "name": field["name"],
+            "label": field["label"],
+            "type": field["input_type"],
+            "required": field["required"],
+            "format_hint": field["format_hint"],
+            "options": field["options"],
+        }
+        for field in decision["discovered_fields"]
+    ]
+    _require(
+        user_observation.get("form_fields") == visible_decision_fields,
+        "raw page observation differs",
+    )
+    _require(
+        experiment_input.get("form_html_sha256")
+        == hashlib.sha256(experiment_input.get("form_html", "").encode("utf-8")).hexdigest(),
+        "input form HTML hash differs",
+    )
+
+    _require(acceptance.get("overall_status") == "pass", "acceptance status is not pass")
+    gates = acceptance.get("gates", {})
+    _require(
+        gates and all(item.get("status") == "pass" for item in gates.values()),
+        "an acceptance gate failed",
+    )
+    _require(
+        manifest.get("acceptance")
+        == {
+            "overall_status": "pass",
+            "gate_count": len(gates),
+            "passed_gate_count": len(gates),
+        },
+        "manifest acceptance summary differs",
+    )
+    if require_validation_report:
+        _require(
+            manifest.get("retained_evidence_validation") == "pass",
+            "manifest retained-evidence status is not pass",
+        )
+    _require(isinstance(timeline, list), "message timeline must be a list")
+    collected = [row for row in timeline if row.get("type") == "info_collected"]
+    _require(collected, "message timeline has no collected fields")
+    _require(
+        all(row.get("payload", {}).get("value") == "<redacted>" for row in collected),
+        "participant values are not redacted",
+    )
+    _require(
+        acceptance.get("webrtc_receipt", {}).get("raw_audio_retained") is False,
+        "raw audio retained",
+    )
+    _require(
+        acceptance.get("webrtc_receipt", {}).get("transcripts_retained") is False,
+        "transcripts retained",
+    )
+    _require(
+        experiment_input.get("participant_values_retained") is False, "input claims values retained"
+    )
+    _require(form_receipt.get("raw_values_retained") is False, "form receipt retained raw values")
+    retained_text = "\n".join(
+        path.read_text(encoding="utf-8") for path in sorted(run_dir.iterdir()) if path.is_file()
+    )
+    _require(
+        not CREDENTIAL_PATTERN.search(retained_text), "retained evidence contains a credential"
+    )
+
+    result = {
+        "schema_version": 1,
+        "experiment": "10-3",
+        "status": "pass",
+        "checks": {
+            "source_hashes": "pass",
+            "artifact_hashes": "pass",
+            "input_hashes": "pass",
+            "raw_ark_request_tool_choice_auto": "pass",
+            "raw_ark_response_metadata": "pass",
+            "raw_arguments_normalize_to_decision": "pass",
+            "participant_privacy": "pass",
+            "acceptance_gates": "pass",
+        },
+    }
+    if require_validation_report:
+        _require(
+            _load_json(run_dir / "validation_report.json") == result,
+            "retained validation report differs from recomputed result",
+        )
+    return result
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("run_dir", type=Path)
+    parser.add_argument("--source-root", type=Path, default=Path(__file__).parent)
+    args = parser.parse_args()
+    report = validate_run(args.run_dir, source_root=args.source_root)
+    print(json.dumps(report, ensure_ascii=False, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

chapter10/autonomous-phone-registration/validation/message_timeline_2026-07-29.json

@@ -0,0 +1,23 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "redaction": "all collected values replaced with <redacted>",
+  "events": [
+    {"sequence": 1, "at": 19.711342, "sender": "phone_agent", "recipient": "computer_agent", "type": "call_started", "payload": {"fields": ["firstName", "lastName", "gender", "userNumber"]}},
+    {"sequence": 2, "at": 19.711714, "sender": "phone_agent", "recipient": "computer_agent", "type": "question_asked", "payload": {"field": "firstName", "attempt": 1}},
+    {"sequence": 3, "at": 22.437785, "sender": "phone_agent", "recipient": "computer_agent", "type": "info_collected", "payload": {"field": "firstName", "value": "<redacted>", "attempt": 1}},
+    {"sequence": 4, "at": 22.43812, "sender": "phone_agent", "recipient": "computer_agent", "type": "question_asked", "payload": {"field": "lastName", "attempt": 1}},
+    {"sequence": 5, "at": 22.520051, "sender": "computer_agent", "recipient": "phone_agent", "type": "field_filled", "payload": {"field": "firstName"}},
+    {"sequence": 6, "at": 25.792242, "sender": "phone_agent", "recipient": "computer_agent", "type": "info_collected", "payload": {"field": "lastName", "value": "<redacted>", "attempt": 1}},
+    {"sequence": 7, "at": 25.793196, "sender": "phone_agent", "recipient": "computer_agent", "type": "question_asked", "payload": {"field": "gender", "attempt": 1}},
+    {"sequence": 8, "at": 25.874117, "sender": "computer_agent", "recipient": "phone_agent", "type": "field_filled", "payload": {"field": "lastName"}},
+    {"sequence": 9, "at": 29.852781, "sender": "phone_agent", "recipient": "computer_agent", "type": "info_collected", "payload": {"field": "gender", "value": "<redacted>", "attempt": 1}},
+    {"sequence": 10, "at": 29.854119, "sender": "phone_agent", "recipient": "computer_agent", "type": "question_asked", "payload": {"field": "userNumber", "attempt": 1}},
+    {"sequence": 11, "at": 29.965478, "sender": "computer_agent", "recipient": "phone_agent", "type": "field_filled", "payload": {"field": "gender"}},
+    {"sequence": 12, "at": 32.229468, "sender": "phone_agent", "recipient": "computer_agent", "type": "info_collected", "payload": {"field": "userNumber", "value": "<redacted>", "attempt": 1}},
+    {"sequence": 13, "at": 32.229907, "sender": "phone_agent", "recipient": "computer_agent", "type": "task_completed", "payload": {}},
+    {"sequence": 14, "at": 32.258196, "sender": "computer_agent", "recipient": "phone_agent", "type": "field_filled", "payload": {"field": "userNumber"}},
+    {"sequence": 15, "at": 32.258767, "sender": "computer_agent", "recipient": "phone_agent", "type": "form_ready", "payload": {"errors": 0, "submitted": false}},
+    {"sequence": 16, "at": 32.266255, "sender": "computer_agent", "recipient": "manager", "type": "registration_finished", "payload": {"filled": ["firstName", "lastName", "gender", "userNumber"], "submitted": false, "errors": []}}
+  ]
+}

chapter10/autonomous-phone-registration/validation/real_browser_llm_2026-07-29.json

@@ -0,0 +1,84 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "generated_at": "2026-07-29T18:39:03+0800",
+  "command_profile": "real Playwright + real LLM + synthetic scripted phone answers",
+  "transport": "scripted",
+  "synthetic_values_used": true,
+  "values_persisted": false,
+  "decision_provider": "Volcengine ARK",
+  "decision_model": "doubao-seed-1-6-250615",
+  "page_url": "https://demoqa.com/automation-practice-form",
+  "fields_discovered": 13,
+  "tool_called": "initiate_phone_call_agent",
+  "required_fields": ["firstName", "lastName", "gender", "userNumber"],
+  "result": {
+    "filled": ["firstName", "lastName", "gender", "userNumber"],
+    "submitted": false,
+    "errors": [],
+    "browser_closed": true
+  },
+  "timing_evidence": {
+    "question_times": {
+      "firstName": 19.711714,
+      "lastName": 22.43812,
+      "gender": 25.793196,
+      "userNumber": 29.854119
+    },
+    "collection_times": {
+      "firstName": 22.437785,
+      "lastName": 25.792242,
+      "gender": 29.852781,
+      "userNumber": 32.229468
+    },
+    "fill_times": {
+      "firstName": 22.520051,
+      "lastName": 25.874117,
+      "gender": 29.965478,
+      "userNumber": 32.258196
+    },
+    "overlap_checks": [
+      {
+        "field_being_filled": "firstName",
+        "next_question": "lastName",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 22.43812,
+        "fill_completed_at": 22.520051
+      },
+      {
+        "field_being_filled": "lastName",
+        "next_question": "gender",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 25.793196,
+        "fill_completed_at": 25.874117
+      },
+      {
+        "field_being_filled": "gender",
+        "next_question": "userNumber",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 29.854119,
+        "fill_completed_at": 29.965478
+      }
+    ],
+    "independent_tasks": ["phone-agent-react-loop", "computer-agent-react-loop"]
+  },
+  "gates": {
+    "real_playwright_page_and_fill": {"status": "pass"},
+    "autonomous_real_llm_tool_call": {"status": "pass"},
+    "ask_one_fill_one_concurrency": {"status": "pass"},
+    "browser_resource_cleanup": {"status": "pass"},
+    "real_form_submission": {
+      "status": "not_run",
+      "reason": "submission was intentionally disabled; no external form side effect was authorized"
+    },
+    "real_pstn_call": {
+      "status": "not_run",
+      "reason": "no authorized consenting endpoint and Twilio configuration was supplied"
+    },
+    "real_audio_asr_tts": {
+      "status": "not_run",
+      "reason": "scripted transport is explicitly non-acceptance"
+    }
+  },
+  "overall_status": "incomplete"
+}

chapter10/autonomous-phone-registration/validation/real_browser_llm_recheck_2026-07-29.json

@@ -0,0 +1,75 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "generated_at": "2026-07-29T20:59:26+0800",
+  "command_profile": "real Playwright + real LLM + synthetic scripted phone answers; no submit",
+  "transport": "scripted",
+  "synthetic_values_used": true,
+  "human_audio_used": false,
+  "pstn_calls_placed": 0,
+  "external_form_submissions": 0,
+  "decision_provider": "Volcengine ARK",
+  "decision_model": "doubao-seed-1-6-250615",
+  "page_url": "https://demoqa.com/automation-practice-form",
+  "fields_discovered": 13,
+  "required_fields": ["firstName", "lastName", "gender", "userNumber"],
+  "result": {
+    "filled": ["firstName", "lastName", "gender", "userNumber"],
+    "submitted": false,
+    "errors": [],
+    "browser_closed": true
+  },
+  "timing_evidence": {
+    "question_times": {
+      "firstName": 31.312472,
+      "lastName": 34.77661,
+      "gender": 38.136418,
+      "userNumber": 41.920582
+    },
+    "fill_times": {
+      "firstName": 34.859382,
+      "lastName": 38.218494,
+      "gender": 42.040402,
+      "userNumber": 48.001121
+    },
+    "overlap_checks": [
+      {
+        "field_being_filled": "firstName",
+        "next_question": "lastName",
+        "next_question_before_fill_completed": true
+      },
+      {
+        "field_being_filled": "lastName",
+        "next_question": "gender",
+        "next_question_before_fill_completed": true
+      },
+      {
+        "field_being_filled": "gender",
+        "next_question": "userNumber",
+        "next_question_before_fill_completed": true
+      }
+    ],
+    "expected_overlap_count": 3,
+    "independent_tasks": ["phone-agent-react-loop", "computer-agent-react-loop"]
+  },
+  "persisted_collected_values": ["<redacted>", "<redacted>", "<redacted>", "<redacted>"],
+  "gates": {
+    "real_playwright_page_and_fill": {"status": "pass"},
+    "autonomous_real_llm_tool_call": {"status": "pass"},
+    "ask_one_fill_one_concurrency": {"status": "pass"},
+    "browser_resource_cleanup": {"status": "pass"},
+    "real_form_submission": {
+      "status": "not_run",
+      "reason": "no external form side effect was authorized"
+    },
+    "real_pstn_call": {
+      "status": "not_run",
+      "reason": "no authorized consenting endpoint was supplied"
+    },
+    "real_audio_asr_tts": {
+      "status": "not_run",
+      "reason": "scripted transport is explicitly non-acceptance"
+    }
+  },
+  "overall_status": "incomplete"
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/acceptance_report.json

@@ -0,0 +1,544 @@
+{
+  "schema_version": 2,
+  "experiment": "10-3",
+  "generated_at": "2026-07-31T19:29:06+0800",
+  "transport": "webrtc",
+  "synthetic_values_used": true,
+  "decision_provider": "Volcengine ARK",
+  "decision_model": "doubao-seed-1-6-250615",
+  "page_url": "http://127.0.0.1:50624/register",
+  "fields_discovered": 6,
+  "required_fields": [
+    "firstName",
+    "lastName",
+    "email",
+    "userNumber",
+    "gender",
+    "address"
+  ],
+  "result": {
+    "filled": [
+      "firstName",
+      "lastName",
+      "email",
+      "userNumber",
+      "gender",
+      "address"
+    ],
+    "submitted": true,
+    "errors": []
+  },
+  "timing_evidence": {
+    "question_times": {
+      "firstName": 26.873284,
+      "lastName": 36.099655,
+      "email": 47.377071,
+      "userNumber": 92.855214,
+      "gender": 112.645667,
+      "address": 136.483352
+    },
+    "collection_times": {
+      "firstName": 36.09918,
+      "lastName": 47.376214,
+      "email": 92.854189,
+      "userNumber": 112.64446,
+      "gender": 136.4828,
+      "address": 150.994505
+    },
+    "fill_times": {
+      "firstName": 36.140925,
+      "lastName": 47.400183,
+      "email": 92.884465,
+      "userNumber": 112.666944,
+      "gender": 136.509506,
+      "address": 151.016117
+    },
+    "overlap_checks": [
+      {
+        "field_being_filled": "firstName",
+        "next_question": "lastName",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 36.099655,
+        "fill_completed_at": 36.140925
+      },
+      {
+        "field_being_filled": "lastName",
+        "next_question": "email",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 47.377071,
+        "fill_completed_at": 47.400183
+      },
+      {
+        "field_being_filled": "email",
+        "next_question": "userNumber",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 92.855214,
+        "fill_completed_at": 92.884465
+      },
+      {
+        "field_being_filled": "userNumber",
+        "next_question": "gender",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 112.645667,
+        "fill_completed_at": 112.666944
+      },
+      {
+        "field_being_filled": "gender",
+        "next_question": "address",
+        "next_question_before_fill_completed": true,
+        "next_question_at": 136.483352,
+        "fill_completed_at": 136.509506
+      }
+    ],
+    "expected_overlap_count": 5,
+    "independent_tasks": [
+      "phone-agent-react-loop",
+      "computer-agent-react-loop"
+    ]
+  },
+  "webrtc_receipt": {
+    "transport": "webrtc",
+    "signaling_scope": "in-page localhost offer/answer; no external relay",
+    "offers": 1,
+    "answers": 1,
+    "ice_candidates": 6,
+    "media_recordings": 7,
+    "agent_connection_state": "connected",
+    "participant_connection_state": "connected",
+    "audio_rtp": [
+      {
+        "side": "agent",
+        "type": "inbound-rtp",
+        "packets": 603,
+        "bytes": 46939
+      },
+      {
+        "side": "agent",
+        "type": "outbound-rtp",
+        "packets": 2520,
+        "bytes": 186654
+      },
+      {
+        "side": "participant",
+        "type": "inbound-rtp",
+        "packets": 2520,
+        "bytes": 186654
+      },
+      {
+        "side": "participant",
+        "type": "outbound-rtp",
+        "packets": 603,
+        "bytes": 46939
+      }
+    ],
+    "tts_prompt_count": 9,
+    "asr_count": 7,
+    "speech_provider": "local system TTS + local OpenAI Whisper",
+    "synthetic_participant": true,
+    "raw_audio_retained": false,
+    "transcripts_retained": false,
+    "status": "completed"
+  },
+  "provider_receipts": {
+    "decision": {
+      "provider": "Volcengine ARK",
+      "model": "doubao-seed-1-6-250615",
+      "response_id": "0217854971908780d00bd2433ad042948032361e92cd5cefed308",
+      "usage": {
+        "prompt_tokens": 934,
+        "completion_tokens": 515,
+        "total_tokens": 1449
+      }
+    },
+    "field_extractions": [
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "021785497223817a04a47ac73179d3e64c063d7b69771fc7c8bfa",
+        "usage": {
+          "prompt_tokens": 190,
+          "completion_tokens": 76,
+          "total_tokens": 266
+        },
+        "transcript_or_value_retained": false
+      },
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "021785497232231ea4a69cd411baf032d46b5f346221adc35d161",
+        "usage": {
+          "prompt_tokens": 191,
+          "completion_tokens": 152,
+          "total_tokens": 343
+        },
+        "transcript_or_value_retained": false
+      },
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "0217854972517657904abb9448890f9b94c80482cf61b2921cfdb",
+        "usage": {
+          "prompt_tokens": 201,
+          "completion_tokens": 126,
+          "total_tokens": 327
+        },
+        "transcript_or_value_retained": false
+      },
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "0217854972781593b2c90331db17702ff1ef9ef62383e604a6a8d",
+        "usage": {
+          "prompt_tokens": 198,
+          "completion_tokens": 168,
+          "total_tokens": 366
+        },
+        "transcript_or_value_retained": false
+      },
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "02178549729600444d62fd40b4606d08afe49bd4ea80f05614313",
+        "usage": {
+          "prompt_tokens": 207,
+          "completion_tokens": 220,
+          "total_tokens": 427
+        },
+        "transcript_or_value_retained": false
+      },
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "0217854973220839707f6c6d04f57819fc098a4ef7e1cdd8482fb",
+        "usage": {
+          "prompt_tokens": 203,
+          "completion_tokens": 156,
+          "total_tokens": 359
+        },
+        "transcript_or_value_retained": false
+      },
+      {
+        "operation": "field_value_extraction",
+        "provider": "Volcengine ARK",
+        "model": "doubao-seed-1-6-250615",
+        "response_id": "02178549733582814c42a60562dfb6c540ee32d577db5463fa61f",
+        "usage": {
+          "prompt_tokens": 196,
+          "completion_tokens": 178,
+          "total_tokens": 374
+        },
+        "transcript_or_value_retained": false
+      }
+    ],
+    "speech": [
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 398942,
+        "latency_seconds": 2.529,
+        "network_used": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 67626,
+        "latency_seconds": 1.423,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 25068,
+        "latency_seconds": 1.289,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 13235,
+        "latency_seconds": 1.705,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 35488,
+        "latency_seconds": 1.339,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 21662,
+        "latency_seconds": 1.275,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 11287,
+        "latency_seconds": 1.727,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 330654,
+        "latency_seconds": 2.382,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 66502,
+        "latency_seconds": 1.444,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 27189,
+        "latency_seconds": 1.736,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 666252,
+        "latency_seconds": 3.27,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 73564,
+        "latency_seconds": 1.533,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 28819,
+        "latency_seconds": 1.816,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 86570,
+        "latency_seconds": 1.567,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 252282,
+        "latency_seconds": 2.144,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 90189,
+        "latency_seconds": 1.844,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 568444,
+        "latency_seconds": 3.272,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 31108,
+        "latency_seconds": 1.335,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 15183,
+        "latency_seconds": 1.76,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 102124,
+        "latency_seconds": 1.544,
+        "network_used": false
+      },
+      {
+        "operation": "synthetic_participant_tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 89304,
+        "latency_seconds": 1.512,
+        "network_used": false
+      },
+      {
+        "operation": "asr",
+        "provider": "local OpenAI Whisper",
+        "model": "whisper-tiny",
+        "model_sha256": "65147644a518d12f04e32d6f3b26facc3f8dd46e5390956a9424a650c0ce22b9",
+        "runtime": {
+          "torch": "2.7.0",
+          "openai_whisper": "20231106"
+        },
+        "request_bytes": 34981,
+        "latency_seconds": 1.776,
+        "network_used": false,
+        "raw_audio_retained": false,
+        "transcript_retained": false
+      },
+      {
+        "operation": "tts",
+        "provider": "macOS say",
+        "model": "operating-system speech synthesizer",
+        "request_id": null,
+        "response_bytes": 116628,
+        "latency_seconds": 1.588,
+        "network_used": false
+      }
+    ]
+  },
+  "gates": {
+    "real_playwright_page_and_fill": {
+      "status": "pass"
+    },
+    "autonomous_real_llm_tool_call": {
+      "status": "pass"
+    },
+    "ask_one_fill_one_concurrency": {
+      "status": "pass"
+    },
+    "validation_feedback_and_reask": {
+      "status": "pass"
+    },
+    "privacy_redaction_and_ephemeral_audio": {
+      "status": "pass",
+      "reason": null
+    },
+    "browser_resource_cleanup": {
+      "status": "pass"
+    },
+    "real_form_submission": {
+      "status": "pass",
+      "reason": null
+    },
+    "real_webrtc_session": {
+      "status": "pass",
+      "reason": null
+    },
+    "bidirectional_webrtc_audio_and_real_asr_tts": {
+      "status": "pass",
+      "reason": null
+    }
+  },
+  "overall_status": "pass",
+  "safe_local_submission_receipt": {
+    "endpoint_scope": "localhost-only",
+    "submission_count": 1,
+    "submissions": [
+      {
+        "field_names": [
+          "address",
+          "email",
+          "firstName",
+          "gender",
+          "lastName",
+          "userNumber"
+        ],
+        "field_count": 6,
+        "all_values_redacted": true
+      }
+    ],
+    "raw_values_retained": false
+  }
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/decision.json

@@ -0,0 +1,152 @@
+{
+  "page_url": "http://127.0.0.1:50624/register",
+  "page_title": "Safe local registration",
+  "known_fields": [],
+  "discovered_fields": [
+    {
+      "name": "firstName",
+      "label": "First name",
+      "input_type": "text",
+      "required": true,
+      "selector": "#firstName",
+      "format_hint": "",
+      "pattern": "",
+      "options": []
+    },
+    {
+      "name": "lastName",
+      "label": "Last name",
+      "input_type": "text",
+      "required": true,
+      "selector": "#lastName",
+      "format_hint": "",
+      "pattern": "",
+      "options": []
+    },
+    {
+      "name": "email",
+      "label": "Email address",
+      "input_type": "email",
+      "required": true,
+      "selector": "#email",
+      "format_hint": "name@example.com",
+      "pattern": "",
+      "options": []
+    },
+    {
+      "name": "userNumber",
+      "label": "Phone number",
+      "input_type": "tel",
+      "required": true,
+      "selector": "#userNumber",
+      "format_hint": "10 digits",
+      "pattern": "[0-9]{10}",
+      "options": []
+    },
+    {
+      "name": "gender",
+      "label": "Gender",
+      "input_type": "select",
+      "required": true,
+      "selector": "#gender",
+      "format_hint": "",
+      "pattern": "",
+      "options": [
+        "Choose one",
+        "Female",
+        "Male",
+        "Non-binary"
+      ]
+    },
+    {
+      "name": "address",
+      "label": "Mailing address",
+      "input_type": "textarea",
+      "required": true,
+      "selector": "#address",
+      "format_hint": "",
+      "pattern": "",
+      "options": []
+    }
+  ],
+  "tool_called": "initiate_phone_call_agent",
+  "purpose": "Complete registration by collecting required user information",
+  "required_info": [
+    {
+      "name": "firstName",
+      "label": "First name",
+      "input_type": "text",
+      "required": true,
+      "selector": "#firstName",
+      "format_hint": "",
+      "pattern": "",
+      "options": []
+    },
+    {
+      "name": "lastName",
+      "label": "Last name",
+      "input_type": "text",
+      "required": true,
+      "selector": "#lastName",
+      "format_hint": "",
+      "pattern": "",
+      "options": []
+    },
+    {
+      "name": "email",
+      "label": "Email address",
+      "input_type": "email",
+      "required": true,
+      "selector": "#email",
+      "format_hint": "name@example.com",
+      "pattern": "",
+      "options": []
+    },
+    {
+      "name": "userNumber",
+      "label": "Phone number",
+      "input_type": "tel",
+      "required": true,
+      "selector": "#userNumber",
+      "format_hint": "10 digits",
+      "pattern": "[0-9]{10}",
+      "options": []
+    },
+    {
+      "name": "gender",
+      "label": "Gender",
+      "input_type": "select",
+      "required": true,
+      "selector": "#gender",
+      "format_hint": "",
+      "pattern": "",
+      "options": [
+        "Choose one",
+        "Female",
+        "Male",
+        "Non-binary"
+      ]
+    },
+    {
+      "name": "address",
+      "label": "Mailing address",
+      "input_type": "textarea",
+      "required": true,
+      "selector": "#address",
+      "format_hint": "",
+      "pattern": "",
+      "options": []
+    }
+  ],
+  "rationale_summary": "模型通过工具调用决定启动 Phone Agent",
+  "model": "doubao-seed-1-6-250615",
+  "monotonic_seconds": 14.374983,
+  "provider": "Volcengine ARK",
+  "provider_response_id": "0217854971908780d00bd2433ad042948032361e92cd5cefed308",
+  "provider_usage": {
+    "prompt_tokens": 934,
+    "completion_tokens": 515,
+    "total_tokens": 1449
+  },
+  "wall_time": "2026-07-31T11:26:44.053087+00:00"
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/experiment_input.json

@@ -0,0 +1,17 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "page_url": "http://127.0.0.1:50624/register",
+  "form_html": "<!doctype html>\n<html lang=\"en\"><meta charset=\"utf-8\"><title>Safe local registration</title>\n<h1>Conference registration</h1>\n<form method=\"post\" action=\"/register\">\n  <label for=\"firstName\">First name</label>\n  <input id=\"firstName\" name=\"firstName\" required>\n  <label for=\"lastName\">Last name</label>\n  <input id=\"lastName\" name=\"lastName\" required>\n  <label for=\"email\">Email address</label>\n  <input id=\"email\" name=\"email\" type=\"email\" required placeholder=\"name@example.com\">\n  <label for=\"userNumber\">Phone number</label>\n  <input id=\"userNumber\" name=\"userNumber\" type=\"tel\" required pattern=\"[0-9]{10}\" title=\"10 digits\">\n  <label for=\"gender\">Gender</label>\n  <select id=\"gender\" name=\"gender\" required>\n    <option value=\"\">Choose one</option><option>Female</option><option>Male</option><option>Non-binary</option>\n  </select>\n  <label for=\"address\">Mailing address</label>\n  <textarea id=\"address\" name=\"address\" required></textarea>\n  <button type=\"submit\">Register</button>\n</form></html>",
+  "form_html_sha256": "cb8f0ca7d7260ab8f32e306e7d5abbfc06387d9a19976a1cfffdcece391d0d3b",
+  "field_answer_counts": {
+    "firstName": 1,
+    "lastName": 1,
+    "email": 2,
+    "userNumber": 1,
+    "gender": 1,
+    "address": 1
+  },
+  "participant": "safe synthesized voice over WebRTC RTP",
+  "participant_values_retained": false
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/form_submission_receipt.json

@@ -0,0 +1,19 @@
+{
+  "endpoint_scope": "localhost-only",
+  "submission_count": 1,
+  "submissions": [
+    {
+      "field_names": [
+        "address",
+        "email",
+        "firstName",
+        "gender",
+        "lastName",
+        "userNumber"
+      ],
+      "field_count": 6,
+      "all_values_redacted": true
+    }
+  ],
+  "raw_values_retained": false
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/manifest.json

@@ -0,0 +1,50 @@
+{
+  "schema_version": 2,
+  "experiment": "10-3",
+  "run_kind": "full_safe_webrtc_acceptance",
+  "generated_at": "2026-07-31T19:29:06+0800",
+  "git_head_at_run": "f66e2fbdbe267f75bf28470ddc59fc1b7beb5c4c",
+  "command": "python run_acceptance.py --run-dir <validation-run-directory>",
+  "providers": {
+    "decision_and_extraction": "Volcengine ARK",
+    "speech": "local system TTS + local OpenAI Whisper"
+  },
+  "privacy": {
+    "phone_number_required": false,
+    "pstn_provider_required": false,
+    "participant": "safe synthesized voice",
+    "raw_audio_retained": false,
+    "transcripts_or_values_retained": false,
+    "form_values_retained": false
+  },
+  "source_sha256": {
+    "browser.py": "66768575e96e76c745c9d470bbad24c98e47a8a321e9e908cc6ba67df795fd97",
+    "bus.py": "d5ae473831426b4cd5c9f746f8f9c24f7e1f08f3d61173430d5bb582f92af1de",
+    "decision.py": "64e68c5a7c77d9404fcb451fbd8325f29d0b4ab58fedcfb0c4700653ee53c913",
+    "demo.py": "d491b0a8f5de02fa7da44f382005a108d9f252b864b57fc189ff0b6608ddd826",
+    "models.py": "1f28ed0b6edfbcde68146ba68f183cc0a3a9ce6aa64a3220026453628eb97f1d",
+    "orchestration.py": "0e2e3eadcd5158e9ac4faea7bb5179f53f56ca84ed7956d31f023b397619e58e",
+    "run_acceptance.py": "867b4b9b5f7ca6201517fbe4b2165d2946e322fe1cd8bbe0a2bd6a08cf22be0b",
+    "validate_acceptance.py": "4a89fc84407fd4813459872f973411fcc2beb871ba1fe7553e22d5e8ce2dc85f",
+    "voice.py": "5fb46942a38fa4a8aa338e8360dfb459a3b0b629ee07579f127cea6dd962d490",
+    "webrtc_channel.py": "47835d12ba988f017647bf4e527696ef69ab867f7bd730f30904b5ed9369040f"
+  },
+  "input_sha256": {
+    "experiment_input.json": "49df4da2dcf86a24c31a8cb447e1c94b98bde908cdb95ce902a319ddc768b51d"
+  },
+  "artifact_sha256": {
+    "acceptance_report.json": "78adfb166ecf10736dcf69afdaacca52356f266d2b95e83a1a7c364106222b38",
+    "decision.json": "69ac3b412a0e5921f50b1b31c6e7678f71a77a7ed8275b0f773602fe8ab99baa",
+    "message_timeline.json": "3ad8d949458e1be0f8c632624c1fc7582621c624a45a2d72f4c2d1cc88852ca8",
+    "form_submission_receipt.json": "d073943e7ebbe29f3945ac3597f5237a8e6258f842f872b0a926abe547d63f66",
+    "raw_decision_request.json": "893317a2f60dc41a0966af41355d4ccad1276f0c289aba7b5df1889582a76f23",
+    "raw_decision_response.json": "19d8efd662750b29129fe13dd69c6b64accd1ddf06d8a2ecdab50edb3d775e94",
+    "validation_report.json": "c9a02501d00f790f88c7d43ad699cdcc7a6aac72dd6dd5d1d695d4a092925724"
+  },
+  "acceptance": {
+    "overall_status": "pass",
+    "gate_count": 9,
+    "passed_gate_count": 9
+  },
+  "retained_evidence_validation": "pass"
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/message_timeline.json

@@ -0,0 +1,303 @@
+[
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "call_started",
+    "payload": {
+      "purpose": "Complete registration by collecting required user information",
+      "fields": [
+        "firstName",
+        "lastName",
+        "email",
+        "userNumber",
+        "gender",
+        "address"
+      ]
+    },
+    "sequence": 1,
+    "monotonic_seconds": 15.983982,
+    "wall_time": "2026-07-31T19:26:45+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "firstName",
+      "attempt": 1
+    },
+    "sequence": 2,
+    "monotonic_seconds": 26.873284,
+    "wall_time": "2026-07-31T19:26:56+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "info_collected",
+    "payload": {
+      "field": "firstName",
+      "value": "<redacted>",
+      "attempt": 1
+    },
+    "sequence": 3,
+    "monotonic_seconds": 36.09918,
+    "wall_time": "2026-07-31T19:27:05+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "lastName",
+      "attempt": 1
+    },
+    "sequence": 4,
+    "monotonic_seconds": 36.099655,
+    "wall_time": "2026-07-31T19:27:05+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "field_filled",
+    "payload": {
+      "field": "firstName"
+    },
+    "sequence": 5,
+    "monotonic_seconds": 36.140925,
+    "wall_time": "2026-07-31T19:27:05+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "info_collected",
+    "payload": {
+      "field": "lastName",
+      "value": "<redacted>",
+      "attempt": 1
+    },
+    "sequence": 6,
+    "monotonic_seconds": 47.376214,
+    "wall_time": "2026-07-31T19:27:17+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "email",
+      "attempt": 1
+    },
+    "sequence": 7,
+    "monotonic_seconds": 47.377071,
+    "wall_time": "2026-07-31T19:27:17+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "field_filled",
+    "payload": {
+      "field": "lastName"
+    },
+    "sequence": 8,
+    "monotonic_seconds": 47.400183,
+    "wall_time": "2026-07-31T19:27:17+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "format_invalid",
+    "payload": {
+      "field": "email",
+      "attempt": 1,
+      "reason": "该项为必填项，不能留空"
+    },
+    "sequence": 9,
+    "monotonic_seconds": 65.561405,
+    "wall_time": "2026-07-31T19:27:35+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "email",
+      "attempt": 2
+    },
+    "sequence": 10,
+    "monotonic_seconds": 65.562376,
+    "wall_time": "2026-07-31T19:27:35+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "info_collected",
+    "payload": {
+      "field": "email",
+      "value": "<redacted>",
+      "attempt": 2
+    },
+    "sequence": 11,
+    "monotonic_seconds": 92.854189,
+    "wall_time": "2026-07-31T19:28:02+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "userNumber",
+      "attempt": 1
+    },
+    "sequence": 12,
+    "monotonic_seconds": 92.855214,
+    "wall_time": "2026-07-31T19:28:02+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "field_filled",
+    "payload": {
+      "field": "email"
+    },
+    "sequence": 13,
+    "monotonic_seconds": 92.884465,
+    "wall_time": "2026-07-31T19:28:02+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "info_collected",
+    "payload": {
+      "field": "userNumber",
+      "value": "<redacted>",
+      "attempt": 1
+    },
+    "sequence": 14,
+    "monotonic_seconds": 112.64446,
+    "wall_time": "2026-07-31T19:28:22+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "gender",
+      "attempt": 1
+    },
+    "sequence": 15,
+    "monotonic_seconds": 112.645667,
+    "wall_time": "2026-07-31T19:28:22+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "field_filled",
+    "payload": {
+      "field": "userNumber"
+    },
+    "sequence": 16,
+    "monotonic_seconds": 112.666944,
+    "wall_time": "2026-07-31T19:28:22+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "info_collected",
+    "payload": {
+      "field": "gender",
+      "value": "<redacted>",
+      "attempt": 1
+    },
+    "sequence": 17,
+    "monotonic_seconds": 136.4828,
+    "wall_time": "2026-07-31T19:28:46+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "question_asked",
+    "payload": {
+      "field": "address",
+      "attempt": 1
+    },
+    "sequence": 18,
+    "monotonic_seconds": 136.483352,
+    "wall_time": "2026-07-31T19:28:46+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "field_filled",
+    "payload": {
+      "field": "gender"
+    },
+    "sequence": 19,
+    "monotonic_seconds": 136.509506,
+    "wall_time": "2026-07-31T19:28:46+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "info_collected",
+    "payload": {
+      "field": "address",
+      "value": "<redacted>",
+      "attempt": 1
+    },
+    "sequence": 20,
+    "monotonic_seconds": 150.994505,
+    "wall_time": "2026-07-31T19:29:00+0800"
+  },
+  {
+    "sender": "phone_agent",
+    "recipient": "computer_agent",
+    "type": "task_completed",
+    "payload": {},
+    "sequence": 21,
+    "monotonic_seconds": 150.995086,
+    "wall_time": "2026-07-31T19:29:00+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "field_filled",
+    "payload": {
+      "field": "address"
+    },
+    "sequence": 22,
+    "monotonic_seconds": 151.016117,
+    "wall_time": "2026-07-31T19:29:00+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "phone_agent",
+    "type": "form_ready",
+    "payload": {
+      "errors": 0,
+      "submitted": true
+    },
+    "sequence": 23,
+    "monotonic_seconds": 152.049317,
+    "wall_time": "2026-07-31T19:29:01+0800"
+  },
+  {
+    "sender": "computer_agent",
+    "recipient": "manager",
+    "type": "registration_finished",
+    "payload": {
+      "filled": [
+        "firstName",
+        "lastName",
+        "email",
+        "userNumber",
+        "gender",
+        "address"
+      ],
+      "submitted": true,
+      "errors": []
+    },
+    "sequence": 24,
+    "monotonic_seconds": 152.050405,
+    "wall_time": "2026-07-31T19:29:01+0800"
+  }
+]

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/raw_decision_request.json

@@ -0,0 +1,65 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "provider": "Volcengine ARK",
+  "endpoint": "https://ark.cn-beijing.volces.com/api/v3",
+  "credential_fields_retained": [],
+  "request": {
+    "model": "doubao-seed-1-6-250615",
+    "messages": [
+      {
+        "role": "system",
+        "content": "You are a Computer Use Agent completing a registration request. Inspect the real page observation and the information already in context. When you need to collect a large amount of structured information and it can be done step by step through conversation, consider calling the Phone Agent tool. Do not call it for one or two simple missing values. Never invent user data. Give only a short decision summary; do not reveal private chain-of-thought."
+      },
+      {
+        "role": "user",
+        "content": "{\"request\": \"帮我在这个网站上完成注册\", \"page_url\": \"http://127.0.0.1:50624/register\", \"page_title\": \"Safe local registration\", \"form_fields\": [{\"name\": \"firstName\", \"label\": \"First name\", \"type\": \"text\", \"required\": true, \"format_hint\": \"\", \"options\": []}, {\"name\": \"lastName\", \"label\": \"Last name\", \"type\": \"text\", \"required\": true, \"format_hint\": \"\", \"options\": []}, {\"name\": \"email\", \"label\": \"Email address\", \"type\": \"email\", \"required\": true, \"format_hint\": \"name@example.com\", \"options\": []}, {\"name\": \"userNumber\", \"label\": \"Phone number\", \"type\": \"tel\", \"required\": true, \"format_hint\": \"10 digits\", \"options\": []}, {\"name\": \"gender\", \"label\": \"Gender\", \"type\": \"select\", \"required\": true, \"format_hint\": \"\", \"options\": [\"Choose one\", \"Female\", \"Male\", \"Non-binary\"]}, {\"name\": \"address\", \"label\": \"Mailing address\", \"type\": \"textarea\", \"required\": true, \"format_hint\": \"\", \"options\": []}], \"known_context_fields\": []}"
+      }
+    ],
+    "tools": [
+      {
+        "type": "function",
+        "function": {
+          "name": "initiate_phone_call_agent",
+          "description": "Start a live Phone Agent when a user must provide many missing pieces of structured information conversationally. The Phone Agent asks, confirms, validates, and streams each collected field back to the browser Agent.",
+          "parameters": {
+            "type": "object",
+            "properties": {
+              "purpose": {
+                "type": "string"
+              },
+              "required_info": {
+                "type": "array",
+                "items": {
+                  "type": "object",
+                  "properties": {
+                    "name": {
+                      "type": "string"
+                    },
+                    "label": {
+                      "type": "string"
+                    },
+                    "format_hint": {
+                      "type": "string"
+                    }
+                  },
+                  "required": [
+                    "name",
+                    "label"
+                  ],
+                  "additionalProperties": false
+                }
+              }
+            },
+            "required": [
+              "purpose",
+              "required_info"
+            ],
+            "additionalProperties": false
+          }
+        }
+      }
+    ],
+    "tool_choice": "auto"
+  }
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/raw_decision_response.json

@@ -0,0 +1,57 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "provider": "Volcengine ARK",
+  "latency_seconds": 13.610751,
+  "response": {
+    "id": "0217854971908780d00bd2433ad042948032361e92cd5cefed308",
+    "choices": [
+      {
+        "finish_reason": "tool_calls",
+        "index": 0,
+        "logprobs": null,
+        "message": {
+          "content": "",
+          "refusal": null,
+          "role": "assistant",
+          "annotations": null,
+          "audio": null,
+          "function_call": null,
+          "tool_calls": [
+            {
+              "id": "call_huhht07ujt63g3g48jzg0nkz",
+              "function": {
+                "arguments": "{\"purpose\":\"Complete registration by collecting required user information\",\"required_info\":[{\"name\":\"firstName\",\"label\":\"First name\"},{\"name\":\"lastName\",\"label\":\"Last name\"},{\"name\":\"email\",\"label\":\"Email address\",\"format_hint\":\"name@example.com\"},{\"name\":\"userNumber\",\"label\":\"Phone number\",\"format_hint\":\"10 digits\"},{\"name\":\"gender\",\"label\":\"Gender\"},{\"name\":\"address\",\"label\":\"Mailing address\"}]}",
+                "name": "initiate_phone_call_agent"
+              },
+              "type": "function"
+            }
+          ],
+          "reasoning_content": "我现在需要处理用户的注册请求。用户希望在网站上完成注册，提供的页面信息显示有多个必填字段：firstName、lastName、email、userNumber、gender和address。所有这些字段都是必填的，而且用户的已知上下文字段是空的，说明目前没有收集到任何信息。\n\n根据系统提示，当需要收集大量结构化信息且可以通过对话逐步完成时，应该考虑调用Phone Agent工具。这里有六个必填字段，属于大量信息，所以符合调用Phone Agent的条件。不需要为一两个简单的缺失值调用，但这里显然超过了这个数量。\n\n接下来，我需要构造调用Phone Agent的参数。purpose应该是注册所需的信息收集，required_info需要列出所有必填字段，每个字段包括name、label，可能还有format_hint。检查每个字段：\n\n- firstName：name是firstName，label是First name，format_hint为空。\n- lastName：类似，label是Last name。\n- email：label是Email address，format_hint是name@example.com。\n- userNumber：label是Phone number，format_hint是10 digits。\n- gender：label是Gender，format_hint为空，选项有几个，但Phone Agent可能需要确认选项。\n- address：label是Mailing address，format_hint为空。\n\n确保每个required_info项都包含name和label，format_hint可选。然后按照工具要求的格式构造JSON数组，使用正确的名称和参数。不需要并行调用，因为只需要启动一个Phone Agent来处理所有字段的收集。\n\n最后，检查是否符合格式要求，使用<|FunctionCallBegin|>和<|FunctionCallEnd|>包裹，确保JSON结构正确，没有语法错误。确认所有必填字段都被包含，没有遗漏。这样就能正确调用Phone Agent来帮助用户逐步完成注册信息的收集。"
+        }
+      }
+    ],
+    "created": 1785497204,
+    "model": "doubao-seed-1-6-250615",
+    "object": "chat.completion",
+    "moderation": null,
+    "service_tier": "default",
+    "system_fingerprint": null,
+    "usage": {
+      "completion_tokens": 515,
+      "prompt_tokens": 934,
+      "total_tokens": 1449,
+      "completion_tokens_details": {
+        "accepted_prediction_tokens": null,
+        "audio_tokens": null,
+        "reasoning_tokens": 394,
+        "rejected_prediction_tokens": null
+      },
+      "prompt_tokens_details": {
+        "audio_tokens": null,
+        "cache_write_tokens": null,
+        "cached_tokens": 0
+      }
+    }
+  }
+}

chapter10/autonomous-phone-registration/validation/runs/exp10-3-webrtc-raw-20260731-v4/validation_report.json

@@ -0,0 +1,15 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "status": "pass",
+  "checks": {
+    "source_hashes": "pass",
+    "artifact_hashes": "pass",
+    "input_hashes": "pass",
+    "raw_ark_request_tool_choice_auto": "pass",
+    "raw_ark_response_metadata": "pass",
+    "raw_arguments_normalize_to_decision": "pass",
+    "participant_privacy": "pass",
+    "acceptance_gates": "pass"
+  }
+}

chapter10/autonomous-phone-registration/validation/software_gates_2026-07-29.json

@@ -0,0 +1,23 @@
+{
+  "schema_version": 1,
+  "experiment": "10-3",
+  "evidence_type": "focused_software_tests",
+  "human_audio_used": false,
+  "pstn_calls_placed": 0,
+  "gates": {
+    "live_transport_refuses_without_consent_before_browser_or_audio_creation": "pass",
+    "unexpected_agent_failure_cancels_peer": "pass",
+    "unexpected_agent_failure_closes_phone_transport": "pass",
+    "task_completed_triggers_opt_in_submission": "pass",
+    "computer_fill_error_returns_to_phone_and_blocks_submission": "pass",
+    "optional_blank_is_audited_and_never_written_to_browser": "pass",
+    "field_validation_and_reask": "pass",
+    "ask_next_before_prior_fill_completes": "pass"
+  },
+  "acceptance_boundary": {
+    "real_pstn_call": "not_run",
+    "real_human_asr_tts": "not_run",
+    "real_external_form_submission": "not_run",
+    "overall_status": "incomplete"
+  }
+}

chapter10/autonomous-phone-registration/voice.py

@@ -0,0 +1,144 @@
+"""Live microphone/ASR/TTS channel and a deterministic test channel."""
+
+from __future__ import annotations
+
+import asyncio
+import os
+import subprocess
+import tempfile
+import time
+import wave
+from pathlib import Path
+from typing import Dict, List, Protocol
+
+
+class PhoneChannel(Protocol):
+    async def say(self, text: str) -> None: ...
+    async def listen(self, *, timeout: float = 30.0) -> str: ...
+
+
+class LiveMicrophoneChannel:
+    """A real cascaded phone-audio loop: OpenAI TTS -> speaker -> mic -> OpenAI ASR.
+
+    Local microphone/speaker are the call transport. The provider boundary is kept
+    behind this class so a PSTN/WebRTC transport can implement the same two methods.
+    """
+
+    def __init__(self, *, language: str = "zh", voice: str = "coral"):
+        from openai import OpenAI
+
+        if not os.getenv("OPENAI_API_KEY"):
+            raise RuntimeError("实时语音需要 OPENAI_API_KEY（OpenRouter 不提供 ASR/TTS）")
+        self.client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=60, max_retries=1)
+        self.language = language
+        self.voice = voice
+        self.sample_rate = int(os.getenv("VOICE_SAMPLE_RATE", "16000"))
+        self.silence_seconds = float(os.getenv("VOICE_SILENCE_SECONDS", "0.9"))
+        self.threshold = float(os.getenv("VOICE_RMS_THRESHOLD", "0.012"))
+        self.latencies: List[Dict[str, float]] = []
+
+    async def say(self, text: str) -> None:
+        started = time.monotonic()
+        path = Path(tempfile.mkstemp(suffix=".mp3")[1])
+
+        def synthesize():
+            result = self.client.audio.speech.create(
+                model=os.getenv("OPENAI_TTS_MODEL", "tts-1"),
+                voice=self.voice,
+                input=text,
+            )
+            result.stream_to_file(path)
+
+        try:
+            await asyncio.to_thread(synthesize)
+            synth_done = time.monotonic()
+            player = os.getenv("AUDIO_PLAYER", "afplay")
+            proc = await asyncio.create_subprocess_exec(
+                player, str(path), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
+            )
+            await proc.wait()
+            self.latencies.append({
+                "tts_seconds": round(synth_done - started, 3),
+                "playback_seconds": round(time.monotonic() - synth_done, 3),
+            })
+        finally:
+            path.unlink(missing_ok=True)
+
+    async def listen(self, *, timeout: float = 30.0) -> str:
+        path = Path(tempfile.mkstemp(suffix=".wav")[1])
+        started = time.monotonic()
+        try:
+            await asyncio.to_thread(self._record_vad, path, timeout)
+            record_done = time.monotonic()
+
+            def transcribe() -> str:
+                with path.open("rb") as audio:
+                    response = self.client.audio.transcriptions.create(
+                        model=os.getenv("OPENAI_ASR_MODEL", "whisper-1"),
+                        file=audio,
+                        language=self.language,
+                    )
+                return response.text.strip()
+
+            text = await asyncio.to_thread(transcribe)
+            self.latencies.append({
+                "capture_seconds": round(record_done - started, 3),
+                "asr_seconds": round(time.monotonic() - record_done, 3),
+            })
+            print(f"  [ASR] 用户：{text}")
+            return text
+        finally:
+            path.unlink(missing_ok=True)
+
+    def _record_vad(self, path: Path, timeout: float) -> None:
+        import numpy as np
+        import sounddevice as sd
+
+        block = 1024
+        frames = []
+        heard_speech = False
+        silent_blocks = 0
+        required_silence = max(1, int(self.silence_seconds * self.sample_rate / block))
+        deadline = time.monotonic() + timeout
+        print("  [麦克风] 请开始回答；检测到句末静音后自动提交……")
+        with sd.InputStream(samplerate=self.sample_rate, channels=1, dtype="float32", blocksize=block) as stream:
+            while time.monotonic() < deadline:
+                data, overflowed = stream.read(block)
+                if overflowed:
+                    print("  [麦克风] 输入发生 overflow，继续采集")
+                mono = data[:, 0].copy()
+                frames.append(mono)
+                rms = float(np.sqrt(np.mean(np.square(mono))))
+                if rms >= self.threshold:
+                    heard_speech = True
+                    silent_blocks = 0
+                elif heard_speech:
+                    silent_blocks += 1
+                    if silent_blocks >= required_silence:
+                        break
+        if not heard_speech:
+            raise TimeoutError("未在规定时间内检测到语音")
+        pcm = (np.concatenate(frames).clip(-1, 1) * 32767).astype("<i2")
+        with wave.open(str(path), "wb") as wav:
+            wav.setnchannels(1)
+            wav.setsampwidth(2)
+            wav.setframerate(self.sample_rate)
+            wav.writeframes(pcm.tobytes())
+
+
+class ScriptedPhoneChannel:
+    """Non-audio supplement for tests and orchestration debugging only."""
+
+    def __init__(self, answers: List[str]):
+        self.answers = asyncio.Queue()
+        for answer in answers:
+            self.answers.put_nowait(answer)
+        self.prompts: List[str] = []
+
+    async def say(self, text: str) -> None:
+        self.prompts.append(text)
+        print(f"  [scripted-phone] {text}")
+        await asyncio.sleep(0)
+
+    async def listen(self, *, timeout: float = 30.0) -> str:
+        return await asyncio.wait_for(self.answers.get(), timeout)

chapter10/autonomous-phone-registration/webrtc_channel.py

@@ -0,0 +1,648 @@
+"""Local WebRTC transport for the Experiment 10-3 Phone Agent.
+
+The participant page contains the two ends of a standards-based WebRTC call.  The
+agent sends synthesized speech on one RTP audio track; the participant sends a
+microphone track in the other direction.  Only the peer-side recording is handed to
+ASR, and it is kept in memory.  A safe acceptance mode substitutes generated speech
+for the microphone without bypassing WebRTC, MediaRecorder, or ASR.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+import io
+import json
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+import urllib.request
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from typing import Dict, List, Optional, Protocol, Tuple
+
+
+CALL_PAGE = r"""<!doctype html>
+<html lang="zh-CN">
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width,initial-scale=1">
+<title>Experiment 10-3 · Private WebRTC call</title>
+<style>
+  :root { color-scheme: light dark; font: 16px/1.5 system-ui, sans-serif; }
+  body { max-width: 760px; margin: 4rem auto; padding: 0 1.25rem; }
+  .card { border: 1px solid #8886; border-radius: 16px; padding: 1.4rem; }
+  #status { font-weight: 700; }
+  #prompt { min-height: 4.5rem; font-size: 1.15rem; padding: 1rem; background: #8881; }
+  button { font: inherit; padding: .7rem 1rem; margin-right: .5rem; }
+  .privacy { color: #666; font-size: .9rem; }
+</style>
+<body>
+  <main class="card">
+    <h1>Registration assistant call</h1>
+    <p id="status">Connecting a private local WebRTC session…</p>
+    <p id="prompt" aria-live="polite">The assistant's question will appear here.</p>
+    <button id="start" disabled>Start answer</button>
+    <button id="stop" disabled>Finish answer</button>
+    <p class="privacy">Audio stays in this process: the received answer is transcribed
+      ephemerally and raw media is discarded. No phone number or PSTN provider is used.</p>
+    <audio id="remoteAudio" autoplay></audio>
+  </main>
+<script>
+(() => {
+  const q = new URLSearchParams(location.search);
+  const automated = q.get('automation') === '1';
+  const status = document.querySelector('#status');
+  const prompt = document.querySelector('#prompt');
+  const start = document.querySelector('#start');
+  const stop = document.querySelector('#stop');
+  const remoteAudio = document.querySelector('#remoteAudio');
+  let context, agentPeer, userPeer, agentOutput, userOutput;
+  let control, agentInput, currentRecorder, answerResolve, answerReject, answerTimer;
+  const call = { offers: 0, answers: 0, iceCandidates: 0, mediaRecordings: 0 };
+
+  const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
+  const asBytes = value => Uint8Array.from(atob(value), c => c.charCodeAt(0));
+  const asBase64 = blob => new Promise((resolve, reject) => {
+    const reader = new FileReader();
+    reader.onerror = reject;
+    reader.onload = () => resolve(reader.result.split(',')[1]);
+    reader.readAsDataURL(blob);
+  });
+  async function playInto(base64Audio, destination) {
+    const decoded = await context.decodeAudioData(asBytes(base64Audio).buffer);
+    const source = context.createBufferSource();
+    source.buffer = decoded;
+    source.connect(destination);
+    source.start();
+    await new Promise(resolve => source.onended = resolve);
+    return decoded.duration;
+  }
+  async function rtpStats() {
+    const rows = [];
+    for (const [side, peer] of [['agent', agentPeer], ['participant', userPeer]]) {
+      for (const item of (await peer.getStats()).values()) {
+        if (item.kind === 'audio' && (item.type === 'inbound-rtp' || item.type === 'outbound-rtp')) {
+          rows.push({
+            side, type: item.type,
+            packets: item.packetsReceived ?? item.packetsSent ?? 0,
+            bytes: item.bytesReceived ?? item.bytesSent ?? 0
+          });
+        }
+      }
+    }
+    return rows;
+  }
+  async function finishRecording(error) {
+    clearTimeout(answerTimer);
+    const recorder = currentRecorder;
+    if (!recorder) return;
+    const done = new Promise(resolve => recorder.onstop = resolve);
+    recorder.stop();
+    await done;
+    currentRecorder = null;
+    start.disabled = automated;
+    stop.disabled = true;
+    if (error) {
+      answerReject?.(error);
+    } else {
+      const blob = new Blob(recorder.__chunks || [], { type: recorder.mimeType });
+      answerResolve?.({ audio: await asBase64(blob), mime: blob.type, stats: await rtpStats() });
+    }
+  }
+  // Keep chunks on the recorder so finishRecording does not retain answer media globally.
+  function prepareRecording(timeoutMs) {
+    return new Promise((resolve, reject) => {
+      answerResolve = resolve; answerReject = reject;
+      if (!agentInput) return reject(new Error('participant audio track is unavailable'));
+      const chunks = [];
+      const type = MediaRecorder.isTypeSupported('audio/webm;codecs=opus')
+        ? 'audio/webm;codecs=opus' : 'audio/webm';
+      currentRecorder = new MediaRecorder(agentInput, { mimeType: type });
+      currentRecorder.__chunks = chunks;
+      currentRecorder.ondataavailable = event => { if (event.data.size) chunks.push(event.data); };
+      currentRecorder.start(100);
+      call.mediaRecordings += 1;
+      start.disabled = true; stop.disabled = false;
+      status.textContent = 'Listening over the WebRTC audio track…';
+      answerTimer = setTimeout(() => finishRecording(new Error('answer timed out')), timeoutMs);
+    });
+  }
+
+  window.agentSay = async ({audio, text}) => {
+    prompt.textContent = text;
+    if (control?.readyState === 'open') control.send(JSON.stringify({type: 'prompt', text}));
+    const duration = await playInto(audio, agentOutput);
+    return { duration, stats: await rtpStats() };
+  };
+  window.waitForHumanAnswer = timeoutMs => new Promise((resolve, reject) => {
+    status.textContent = 'Click Start answer, speak, then click Finish answer.';
+    start.disabled = false;
+    stop.disabled = true;
+    const startTimer = setTimeout(() => {
+      start.disabled = true;
+      reject(new Error('answer was not started before timeout'));
+    }, timeoutMs);
+    start.onclick = () => {
+      clearTimeout(startTimer);
+      prepareRecording(timeoutMs).then(resolve, reject);
+    };
+  });
+  window.acceptanceAnswer = async ({audio, timeoutMs}) => {
+    const result = prepareRecording(timeoutMs);
+    await wait(250);
+    await playInto(audio, userOutput);
+    await wait(300);
+    await finishRecording();
+    return result;
+  };
+  window.callReceipt = async () => ({
+    ...call,
+    agentConnectionState: agentPeer?.connectionState,
+    participantConnectionState: userPeer?.connectionState,
+    rtp: await rtpStats()
+  });
+  window.closeCall = async () => {
+    clearTimeout(answerTimer);
+    for (const peer of [agentPeer, userPeer]) {
+      peer?.getSenders().forEach(sender => sender.track?.stop());
+      peer?.close();
+    }
+    context?.close();
+  };
+  stop.onclick = () => finishRecording();
+
+  window.callReady = (async () => {
+    context = new AudioContext();
+    await context.resume();
+    agentPeer = new RTCPeerConnection({iceServers: []});
+    userPeer = new RTCPeerConnection({iceServers: []});
+    agentPeer.onicecandidate = event => {
+      if (event.candidate) { call.iceCandidates++; userPeer.addIceCandidate(event.candidate); }
+    };
+    userPeer.onicecandidate = event => {
+      if (event.candidate) { call.iceCandidates++; agentPeer.addIceCandidate(event.candidate); }
+    };
+    agentOutput = context.createMediaStreamDestination();
+    agentPeer.addTrack(agentOutput.stream.getAudioTracks()[0], agentOutput.stream);
+    if (automated) {
+      userOutput = context.createMediaStreamDestination();
+      userPeer.addTrack(userOutput.stream.getAudioTracks()[0], userOutput.stream);
+    } else {
+      const mic = await navigator.mediaDevices.getUserMedia({audio: true, video: false});
+      userPeer.addTrack(mic.getAudioTracks()[0], mic);
+    }
+    agentPeer.ontrack = event => { agentInput = event.streams[0]; };
+    userPeer.ontrack = event => { remoteAudio.srcObject = event.streams[0]; remoteAudio.play().catch(() => {}); };
+    control = agentPeer.createDataChannel('non-sensitive-control');
+    userPeer.ondatachannel = event => event.channel.onmessage = message => {
+      const payload = JSON.parse(message.data);
+      if (payload.type === 'prompt') prompt.textContent = payload.text;
+    };
+    const offer = await agentPeer.createOffer(); call.offers++;
+    await agentPeer.setLocalDescription(offer);
+    await userPeer.setRemoteDescription(offer);
+    const answer = await userPeer.createAnswer(); call.answers++;
+    await userPeer.setLocalDescription(answer);
+    await agentPeer.setRemoteDescription(answer);
+    for (let i = 0; i < 100 && (agentPeer.connectionState !== 'connected' || userPeer.connectionState !== 'connected'); i++) await wait(50);
+    if (agentPeer.connectionState !== 'connected' || userPeer.connectionState !== 'connected') throw new Error('WebRTC connection did not reach connected state');
+    status.textContent = automated ? 'Safe synthesized participant connected.' : 'Private WebRTC call connected.';
+    start.disabled = true;
+    return window.callReceipt();
+  })();
+})();
+</script>
+</body>
+</html>"""
+
+
+class SpeechBackend(Protocol):
+    provider: str
+
+    async def synthesize(self, text: str) -> Tuple[bytes, str, Dict[str, object]]: ...
+    async def transcribe(self, audio: bytes, mime: str) -> Tuple[str, Dict[str, object]]: ...
+
+
+class OpenAISpeechBackend:
+    """OpenAI speech provider with value-free receipts."""
+
+    provider = "OpenAI Audio API"
+
+    def __init__(self, *, language: str = "zh", voice: str = "coral"):
+        from openai import OpenAI
+
+        if not os.getenv("OPENAI_API_KEY"):
+            raise RuntimeError("WebRTC 实时语音需要 OPENAI_API_KEY")
+        self.client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=90, max_retries=1)
+        self.language = language
+        self.voice = voice
+
+    async def synthesize(self, text: str) -> Tuple[bytes, str, Dict[str, object]]:
+        started = time.monotonic()
+
+        def call():
+            response = self.client.audio.speech.create(
+                model=os.getenv("OPENAI_TTS_MODEL", "gpt-4o-mini-tts"),
+                voice=self.voice,
+                input=text,
+                response_format="mp3",
+            )
+            return response.content, getattr(response, "_request_id", None)
+
+        content, request_id = await asyncio.to_thread(call)
+        return content, "audio/mpeg", {
+            "operation": "tts",
+            "provider": self.provider,
+            "model": os.getenv("OPENAI_TTS_MODEL", "gpt-4o-mini-tts"),
+            "request_id": request_id,
+            "response_bytes": len(content),
+            "latency_seconds": round(time.monotonic() - started, 3),
+        }
+
+    async def transcribe(self, audio: bytes, mime: str) -> Tuple[str, Dict[str, object]]:
+        started = time.monotonic()
+
+        def call():
+            extension = ".webm" if "webm" in mime else ".wav"
+            stream = io.BytesIO(audio)
+            stream.name = f"ephemeral-answer{extension}"
+            response = self.client.audio.transcriptions.create(
+                model=os.getenv("OPENAI_ASR_MODEL", "gpt-4o-mini-transcribe"),
+                file=stream,
+                language=self.language,
+            )
+            return response.text.strip(), getattr(response, "_request_id", None)
+
+        text, request_id = await asyncio.to_thread(call)
+        return text, {
+            "operation": "asr",
+            "provider": self.provider,
+            "model": os.getenv("OPENAI_ASR_MODEL", "gpt-4o-mini-transcribe"),
+            "request_id": request_id,
+            "request_bytes": len(audio),
+            "latency_seconds": round(time.monotonic() - started, 3),
+            "raw_audio_retained": False,
+            "transcript_retained": False,
+        }
+
+
+class SystemGeminiSpeechBackend:
+    """Local OS speech synthesis plus Gemini audio transcription.
+
+    This backend keeps generated prompt audio local and uses the already-authorized
+    Gemini endpoint only for ASR.  It is useful when an OpenAI text key is available
+    but its separate Audio API quota is not.
+    """
+
+    provider = "local system TTS + Google Gemini ASR"
+
+    def __init__(self):
+        if not os.getenv("GEMINI_API_KEY"):
+            raise RuntimeError("Gemini ASR requires GEMINI_API_KEY")
+        self.say = shutil.which("say")
+        self.espeak = shutil.which("espeak-ng") or shutil.which("espeak")
+        self.ffmpeg = shutil.which("ffmpeg")
+        if not (self.say or self.espeak) or not self.ffmpeg:
+            raise RuntimeError("local TTS requires say/espeak and ffmpeg")
+
+    async def synthesize(self, text: str) -> Tuple[bytes, str, Dict[str, object]]:
+        started = time.monotonic()
+
+        def call() -> bytes:
+            with tempfile.TemporaryDirectory(prefix="exp10-3-tts-") as directory:
+                source = Path(directory) / ("speech.aiff" if self.say else "speech.wav")
+                target = Path(directory) / "speech.wav"
+                if self.say:
+                    subprocess.run([self.say, "-o", str(source), text], check=True, capture_output=True)
+                else:
+                    subprocess.run([self.espeak, "-w", str(source), text], check=True, capture_output=True)
+                converted = Path(directory) / "speech-24k.wav"
+                subprocess.run(
+                    [self.ffmpeg, "-nostdin", "-loglevel", "error", "-y", "-i", str(source),
+                     "-ac", "1", "-ar", "24000", str(converted)],
+                    check=True, capture_output=True,
+                )
+                return converted.read_bytes()
+
+        content = await asyncio.to_thread(call)
+        return content, "audio/wav", {
+            "operation": "tts",
+            "provider": "macOS say" if self.say else "espeak",
+            "model": "operating-system speech synthesizer",
+            "request_id": None,
+            "response_bytes": len(content),
+            "latency_seconds": round(time.monotonic() - started, 3),
+            "network_used": False,
+        }
+
+    async def transcribe(self, audio: bytes, mime: str) -> Tuple[str, Dict[str, object]]:
+        started = time.monotonic()
+        model = os.getenv("GEMINI_ASR_MODEL", "gemini-2.5-flash")
+
+        def call():
+            payload = json.dumps({
+                "contents": [{"parts": [
+                    {"text": (
+                        "Transcribe this single short form-field answer exactly. Return only the "
+                        "transcript, with no quotes, label, explanation, or Markdown. Preserve email "
+                        "addresses, digits, punctuation, and capitalization when audible."
+                    )},
+                    {"inline_data": {
+                        "mime_type": mime.split(";", 1)[0],
+                        "data": base64.b64encode(audio).decode("ascii"),
+                    }},
+                ]}],
+                "generationConfig": {"temperature": 0, "maxOutputTokens": 256},
+            }).encode("utf-8")
+            key = os.environ["GEMINI_API_KEY"]
+            request = urllib.request.Request(
+                f"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent?key={key}",
+                data=payload,
+                headers={"Content-Type": "application/json"},
+                method="POST",
+            )
+            with urllib.request.urlopen(request, timeout=90) as response:
+                data = json.loads(response.read().decode("utf-8"))
+                request_id = response.headers.get("x-request-id")
+            text = data["candidates"][0]["content"]["parts"][0]["text"].strip()
+            return text, request_id, data.get("usageMetadata", {})
+
+        text, request_id, usage = await asyncio.to_thread(call)
+        return text, {
+            "operation": "asr",
+            "provider": "Google Gemini",
+            "model": model,
+            "request_id": request_id,
+            "usage": usage,
+            "request_bytes": len(audio),
+            "latency_seconds": round(time.monotonic() - started, 3),
+            "raw_audio_retained": False,
+            "transcript_retained": False,
+        }
+
+
+class SystemWhisperSpeechBackend(SystemGeminiSpeechBackend):
+    """Local OS speech synthesis and a local OpenAI Whisper checkpoint."""
+
+    provider = "local system TTS + local OpenAI Whisper"
+
+    def __init__(self):
+        self.say = shutil.which("say")
+        self.espeak = shutil.which("espeak-ng") or shutil.which("espeak")
+        self.ffmpeg = shutil.which("ffmpeg")
+        if not (self.say or self.espeak) or not self.ffmpeg:
+            raise RuntimeError("local speech requires say/espeak and ffmpeg")
+        requested = os.getenv("WHISPER_PYTHON")
+        candidates = [requested] if requested else [sys.executable, shutil.which("python3")]
+        self.whisper_python = next(
+            (candidate for candidate in candidates if candidate and self._has_whisper(candidate)), None
+        )
+        if not self.whisper_python:
+            raise RuntimeError(
+                "local ASR requires openai-whisper; set WHISPER_PYTHON to an environment containing whisper and torch"
+            )
+
+    @staticmethod
+    def _has_whisper(python: str) -> bool:
+        try:
+            return subprocess.run(
+                [python, "-c", "import torch, whisper"],
+                capture_output=True, timeout=20,
+            ).returncode == 0
+        except (OSError, subprocess.SubprocessError):
+            return False
+
+    async def transcribe(self, audio: bytes, mime: str) -> Tuple[str, Dict[str, object]]:
+        started = time.monotonic()
+        model = os.getenv("WHISPER_MODEL", "tiny")
+
+        def call():
+            with tempfile.TemporaryDirectory(prefix="exp10-3-asr-") as directory:
+                source = Path(directory) / ("answer.webm" if "webm" in mime else "answer.wav")
+                target = Path(directory) / "answer-16k.wav"
+                source.write_bytes(audio)
+                subprocess.run(
+                    [self.ffmpeg, "-nostdin", "-loglevel", "error", "-y", "-i", str(source),
+                     "-ac", "1", "-ar", "16000", str(target)],
+                    check=True, capture_output=True,
+                )
+                script = "\n".join([
+                    "import hashlib, json, pathlib, sys, torch, whisper",
+                    "model_name, path = sys.argv[1:3]",
+                    "cache = pathlib.Path.home()/'.cache'/'whisper'/(model_name+'.pt')",
+                    "loaded = whisper.load_model(model_name)",
+                    "result = loaded.transcribe(path, language='en', fp16=False, verbose=False)",
+                    "print('EXPERIMENT_JSON='+json.dumps({",
+                    " 'text': str(result.get('text') or '').strip(),",
+                    " 'model_sha256': hashlib.sha256(cache.read_bytes()).hexdigest() if cache.exists() else None,",
+                    " 'torch': torch.__version__, 'whisper': getattr(whisper, '__version__', 'unknown')}, ensure_ascii=False))",
+                ])
+                process = subprocess.run(
+                    [self.whisper_python, "-c", script, model, str(target)],
+                    check=True, capture_output=True, text=True, timeout=180,
+                )
+            marker = next(
+                line for line in process.stdout.splitlines() if line.startswith("EXPERIMENT_JSON=")
+            )
+            return json.loads(marker.split("=", 1)[1])
+
+        result = await asyncio.to_thread(call)
+        return result["text"], {
+            "operation": "asr",
+            "provider": "local OpenAI Whisper",
+            "model": f"whisper-{model}",
+            "model_sha256": result["model_sha256"],
+            "runtime": {"torch": result["torch"], "openai_whisper": result["whisper"]},
+            "request_bytes": len(audio),
+            "latency_seconds": round(time.monotonic() - started, 3),
+            "network_used": False,
+            "raw_audio_retained": False,
+            "transcript_retained": False,
+        }
+
+def default_speech_backend() -> SpeechBackend:
+    requested = os.getenv("WEBRTC_SPEECH_PROVIDER", "auto").casefold()
+    if requested not in {"auto", "openai", "gemini-system", "local-whisper"}:
+        raise RuntimeError(
+            "WEBRTC_SPEECH_PROVIDER must be auto, openai, gemini-system, or local-whisper"
+        )
+    if requested == "local-whisper":
+        return SystemWhisperSpeechBackend()
+    if requested == "gemini-system" or (
+        requested == "auto" and os.getenv("GEMINI_API_KEY")
+        and (shutil.which("say") or shutil.which("espeak-ng") or shutil.which("espeak"))
+        and shutil.which("ffmpeg")
+    ):
+        return SystemGeminiSpeechBackend()
+    return OpenAISpeechBackend()
+
+
+class _CallPageHandler(BaseHTTPRequestHandler):
+    def do_GET(self):  # noqa: N802 - BaseHTTPRequestHandler API
+        if self.path.split("?", 1)[0] not in {"/", "/call"}:
+            self.send_error(404)
+            return
+        body = CALL_PAGE.encode("utf-8")
+        self.send_response(200)
+        self.send_header("Content-Type", "text/html; charset=utf-8")
+        self.send_header("Cache-Control", "no-store")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def log_message(self, _format, *_args):
+        return
+
+
+class WebRTCPhoneChannel:
+    """A browser-based, bidirectional WebRTC PhoneChannel."""
+
+    def __init__(
+        self,
+        *,
+        headless: bool = False,
+        port: int = 0,
+        synthetic_answers: Optional[List[str]] = None,
+        speech_backend: Optional[SpeechBackend] = None,
+    ):
+        self.headless = headless
+        self.port = port
+        self.synthetic_answers: asyncio.Queue[str] = asyncio.Queue()
+        for answer in synthetic_answers or []:
+            self.synthetic_answers.put_nowait(answer)
+        self.synthetic_participant = synthetic_answers is not None
+        self.speech = speech_backend or default_speech_backend()
+        self.provider_receipts: List[Dict[str, object]] = []
+        self.latencies: List[Dict[str, float]] = []
+        self.tts_prompt_count = 0
+        self.asr_count = 0
+        self.closed = False
+        self.call_status = "created"
+        self.call_url = ""
+        self.receipt: Dict[str, object] = {}
+        self._server = None
+        self._server_thread = None
+        self._playwright = None
+        self._browser = None
+        self._context = None
+        self._page = None
+
+    async def start(self) -> None:
+        from playwright.async_api import async_playwright
+
+        self._server = ThreadingHTTPServer(("127.0.0.1", self.port), _CallPageHandler)
+        self._server_thread = threading.Thread(target=self._server.serve_forever, daemon=True)
+        self._server_thread.start()
+        self.call_url = f"http://127.0.0.1:{self._server.server_port}/call"
+        self._playwright = await async_playwright().start()
+        self._browser = await self._playwright.chromium.launch(
+            headless=self.headless,
+            args=["--autoplay-policy=no-user-gesture-required"],
+        )
+        self._context = await self._browser.new_context(permissions=["microphone"])
+        self._page = await self._context.new_page()
+        url = self.call_url + ("?automation=1" if self.synthetic_participant else "")
+        print(f"  [WebRTC] participant page: {self.call_url}")
+        await self._page.goto(url, wait_until="domcontentloaded")
+        self.receipt = await self._page.evaluate("() => window.callReady")
+        self.call_status = "connected"
+
+    async def say(self, text: str) -> None:
+        if self.call_status != "connected":
+            raise RuntimeError("WebRTC call is not connected")
+        audio, _mime, provider_receipt = await self.speech.synthesize(text)
+        self.provider_receipts.append(provider_receipt)
+        started = time.monotonic()
+        result = await self._page.evaluate(
+            "payload => window.agentSay(payload)",
+            {"audio": base64.b64encode(audio).decode("ascii"), "text": text},
+        )
+        self.tts_prompt_count += 1
+        self.latencies.append({
+            "tts_seconds": float(provider_receipt.get("latency_seconds", 0)),
+            "webrtc_playback_seconds": round(time.monotonic() - started, 3),
+        })
+        self.receipt["rtp"] = result["stats"]
+
+    async def listen(self, *, timeout: float = 120.0) -> str:
+        if self.call_status != "connected":
+            raise RuntimeError("WebRTC call is not connected")
+        if self.synthetic_participant:
+            answer = await asyncio.wait_for(self.synthetic_answers.get(), timeout)
+            audio, _mime, tts_receipt = await self.speech.synthesize(answer)
+            tts_receipt = {**tts_receipt, "operation": "synthetic_participant_tts"}
+            self.provider_receipts.append(tts_receipt)
+            result = await self._page.evaluate(
+                "payload => window.acceptanceAnswer(payload)",
+                {
+                    "audio": base64.b64encode(audio).decode("ascii"),
+                    "timeoutMs": int(timeout * 1000),
+                },
+            )
+        else:
+            result = await self._page.evaluate(
+                "timeoutMs => window.waitForHumanAnswer(timeoutMs)", int(timeout * 1000)
+            )
+        captured = base64.b64decode(result["audio"])
+        if len(captured) < 256:
+            raise RuntimeError("WebRTC answer audio was empty")
+        text, asr_receipt = await self.speech.transcribe(captured, result["mime"])
+        # Delete the only Python reference before returning the transcript.  Raw
+        # audio and transcripts never enter call receipts or message traces.
+        captured = b""
+        self.provider_receipts.append(asr_receipt)
+        self.asr_count += 1
+        self.latencies.append({"asr_seconds": float(asr_receipt.get("latency_seconds", 0))})
+        self.receipt["rtp"] = result["stats"]
+        return text
+
+    async def close(self) -> None:
+        if self.closed:
+            return
+        try:
+            if self._page and not self._page.is_closed():
+                try:
+                    self.receipt = await self._page.evaluate("() => window.callReceipt()")
+                    await self._page.evaluate("() => window.closeCall()")
+                except Exception:
+                    pass
+            if self._context:
+                await self._context.close()
+            if self._browser:
+                await self._browser.close()
+            if self._playwright:
+                await self._playwright.stop()
+        finally:
+            if self._server:
+                await asyncio.to_thread(self._server.shutdown)
+                self._server.server_close()
+            if self._server_thread:
+                self._server_thread.join(timeout=2)
+            self.call_status = "completed"
+            self.closed = True
+
+    def acceptance_receipt(self) -> Dict[str, object]:
+        """Return only transport metadata; no prompt, answer, audio, or transcript."""
+        rtp = self.receipt.get("rtp", [])
+        return {
+            "transport": "webrtc",
+            "signaling_scope": "in-page localhost offer/answer; no external relay",
+            "offers": self.receipt.get("offers", 0),
+            "answers": self.receipt.get("answers", 0),
+            "ice_candidates": self.receipt.get("iceCandidates", 0),
+            "media_recordings": self.receipt.get("mediaRecordings", 0),
+            "agent_connection_state": self.receipt.get("agentConnectionState"),
+            "participant_connection_state": self.receipt.get("participantConnectionState"),
+            "audio_rtp": rtp,
+            "tts_prompt_count": self.tts_prompt_count,
+            "asr_count": self.asr_count,
+            "speech_provider": self.speech.provider,
+            "synthetic_participant": self.synthetic_participant,
+            "raw_audio_retained": False,
+            "transcripts_retained": False,
+            "status": self.call_status,
+        }