liqiang
b119135836
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
16 lines
3.1 KiB
Markdown
16 lines
3.1 KiB
Markdown
# Chapter 1 experiment requirement/evidence ledger
|
||
|
||
The Chinese manuscript is authoritative. Runtime completion and manuscript
|
||
behavior claims are kept separate: a real, correctly controlled ablation may
|
||
finish even when one claimed degradation is not observed.
|
||
|
||
| Experiment | Exact manuscript gate | Status | Canonical evidence / qualification |
|
||
|---|---|---|---|
|
||
| 1-1 | One complete baseline plus removal of tool definitions, tool results, assistant reasoning, and prior history on the same multi-currency ReAct task | Exact five-arm real run completed; one prose behavior not reproduced | `context/validation/latest.json` retains direct Kimi K3 requests/responses, all tool observations, expected totals, and context contracts. Baseline was correct; no tools removed action; hidden tool results and history caused repeated calls. Removing reasoning still completed correctly, so the manuscript’s “contradictory decisions” claim is explicitly false for this run rather than fabricated. |
|
||
| 1-2 | Exact Kimi K3 with provider-hosted Formula web search, model-directed multiple search rounds, reasoning, current answer, and authoritative links | Passed | `web-search-agent/validation/latest.json`: direct Moonshot endpoint, exact model, 15 succeeded distinct Formula fibers over multiple rounds, official ASEAN/Indonesian sources, real response IDs and usage. |
|
||
| 1-3 | Multi-provider policy (author-mandated 2026-07-31): model-directed multi-round hosted web search + hosted code execution closed server-side on a Responses API, clarification-before-tools, and current answers with authoritative sources; the official OpenAI GPT-5.6 Sol path stays canonical, but acceptance is not gated on it — any provider with genuinely equivalent hosted tools is eligible | Passed on DashScope `qwen3.7-plus`; official OpenAI still quota-blocked | `search-codegen/validation/latest.json` (+ `validation/runs/real_20260731T170529Z/{evidence,receipts,manifest}.json`, SHA-256 sidecars, credential-free receipts). ASEAN: one hosted `web_search_call` batching 10 model-issued coordinate queries, then a hosted `code_interpreter_call` enumerating all 45 haversine pairs — Kuala Lumpur–Singapore 316.35 km, the same pair as the independent local reference (309.3 km with standard coordinates). Bitcoin: first turn clarified data source/indicators with no tools; continuation via `previous_response_id` ran 3 model-directed search rounds and 4 `code_interpreter_call`s computing MA7/MA20/RSI14/MACD, period return and max drawdown, and plotted a chart in the sandbox. The OpenAI `gpt-5.6-sol` attempt is recorded honestly as `credit_balance_exhausted` before inference. Qualifications: the DashScope sandbox has no outbound network, so daily closes were search-extracted (the model disclosed this in its report); the chart PNG stays inside the sandbox because the API returns logs only; qwen3.7-plus needed an explicit clarify-first system prompt to ask before using tools. |
|
||
|
||
Legacy demos and provider-compatible substitutes are teaching aids only. A
|
||
provider rejection before inference is not converted into a model failure and
|
||
does not authorize accepting a narrower proxy.
|