ai-agent-book 精选快照(<2MB 代码与文档,来自 github.com/bojieli/ai-agent-book)
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s

This commit is contained in:
2026-08-20 13:12:50 +00:00
commit b119135836
10275 changed files with 3284984 additions and 0 deletions
+189
View File
@@ -0,0 +1,189 @@
# GPT-5.6 Sol Deep Research / GPT-5.6 Sol 深度研究
> Responses API companion for Chapter 1, Experiment 1-3: hosted
> `web_search` + hosted `code_interpreter`, typed tool traces, citations, and an
> intent-clarification continuation. The canonical path is OpenAI GPT-5.6 Sol;
> acceptance is multi-provider and may be closed by any provider whose
> Responses API genuinely closes the search/code loop server-side — currently
> Alibaba Model Studio (DashScope) `qwen3.7-plus`.
← [Chapter 1 index / 返回第 1 章目录](../README.md) ·
📖 [Book experiment / 正文实验](../../book/chapter1.md)
## What this companion implements
The canonical path is the OpenAI **Responses API**, not a Chat Completions
request that merely contains similarly named tool objects. The active agent in
`agent.py` sends:
```json
{
"model": "gpt-5.6-sol",
"tools": [
{"type": "web_search", "search_context_size": "medium"},
{
"type": "code_interpreter",
"container": {"type": "auto", "memory_limit": "4g"}
}
],
"reasoning": {"effort": "high"},
"text": {"verbosity": "high"}
}
```
The DashScope backend speaks the same `/responses` protocol against
`{DASHSCOPE_BASE_URL}/responses` with the provider's hosted-tool shapes:
```json
{
"model": "qwen3.7-plus",
"tools": [{"type": "web_search"}, {"type": "code_interpreter"}],
"stream": true
}
```
DashScope runs thinking natively (no `reasoning.effort`/`text.verbosity`
knobs) and its gateway drops non-streaming requests that stay silent for
about 60 seconds, so the backend always streams and keeps the final
`response.completed` object, which has the same shape as a non-streaming
response.
Acceptance is based on provider output items. A successful ASEAN-capitals run
must contain completed `web_search_call` and `code_interpreter_call` items,
clickable URL citations, and the computed closest pair. A text answer that says
it used Python does not pass without the provider tool receipt.
The second scenario sends the deliberately ambiguous Bitcoin request used in
the chapter, requires the first response to clarify material preferences before
using tools, then continues with `previous_response_id` after the user supplies
the data source and indicators.
## Current evidence status
Run the complete validator with:
```bash
cd chapter1/search-codegen
python run_experiment_1_3.py --backends openai dashscope --reasoning high
```
The latest evidence is [validation/latest.json](validation/latest.json); raw
credential-free receipts, a manifest, and SHA-256 sidecars live in
`validation/runs/real_20260731T170529Z/`.
Result of the 2026-07-31 multi-provider acceptance run: **passed**, with
`dashscope` (`qwen3.7-plus`) as the acceptance backend.
- ASEAN capitals: one hosted `web_search_call` batching ten model-issued
coordinate queries, then a hosted `code_interpreter_call` that enumerated all
45 haversine pairs and found Kuala LumpurSingapore at 316.35 km — the same
pair as the independent local reference computed from standard coordinates.
- Bitcoin technical analysis: the first turn asked which data source and which
indicators to use **without calling any tool**; the continuation via
`previous_response_id` ran 3 model-directed search rounds and 4 hosted
`code_interpreter_call`s computing MA7/MA20, RSI14, MACD(12,26,9), period
return and max drawdown, and plotted a close-price chart in the sandbox.
- The official OpenAI `gpt-5.6-sol` path is still intact but remains
quota-blocked: both calls returned `credit_balance_exhausted` before any
hosted tool ran, which is recorded in the same evidence file.
- Honest qualifications: the DashScope sandbox has no outbound network, so the
daily closes were extracted through web search (the model disclosed this in
its report); the chart PNG stays inside the sandbox because this Responses
API returns execution logs only; and `qwen3.7-plus` only asks before acting
when the system prompt carries an explicit clarify-first rule — the shipped
prompt encodes it.
- The OpenRouter route is retained strictly as a diagnostic and is never
accepted. No fallback model, local Python replacement, fabricated tool
trace, or Chat-Completions approximation is counted as fulfillment.
Earlier blocked attempts are kept under `validation/real_20260729T155459Z/`
and `validation/real_20260730T033800Z/`.
## Setup and CLI
Python 3.9+ is required.
```bash
# From the repository root: use the shared Chapter 1 environment
uv sync --locked --extra ch1
# Activate it before changing directories:
source .venv/bin/activate
# pip fallback when uv is not installed:
# python -m pip install -e ".[ch1]"
cd chapter1/search-codegen
# Single-project compatibility path, still supported during migration:
# python -m pip install -r requirements.txt
export OPENAI_API_KEY=your-openai-api-key
# Exact official path
python main.py --backend openai --mode single \
--request "东盟 10 国首都之间最近的一对是哪两个?请搜索并用 Python 计算" \
--reasoning high --verbosity high --output result.json
# Equivalent-provider path (eligible for acceptance): Alibaba Model Studio
export DASHSCOPE_API_KEY=your-dashscope-api-key
python main.py --backend dashscope --mode single \
--request "东盟 10 国首都之间最近的一对是哪两个?请搜索并用 Python 计算" \
--output result.json
# Inspect the exact request without an API call
python main.py --backend openai --dry-run \
--request "东盟 10 国首都之间最近的一对?" \
--reasoning max --verbosity high
# Proxy diagnostic only; not canonical acceptance
export OPENROUTER_API_KEY=your-openrouter-api-key
python main.py --backend openrouter --mode single --request "Search current news"
```
Important options:
| Option | Meaning |
|---|---|
| `--backend openai` | Canonical `https://api.openai.com/v1/responses` path |
| `--backend dashscope` | Equivalent-provider path: DashScope Responses API, hosted `web_search` + `code_interpreter`, eligible for acceptance |
| `--backend openrouter` | Explicit proxy diagnostic; never silently substituted |
| `--reasoning` | `none`, `low`, `medium`, `high`, `xhigh`, or GPT-5.6 `max` |
| `--verbosity` | Responses `text.verbosity`: `low`, `medium`, or `high` |
| `--output` | Saves request, typed output items, citations, usage, and raw response |
## Verification
```bash
python -m pytest -q test_responses_agent.py
python -m py_compile agent.py config.py main.py run_experiment_1_3.py
```
The validator checks exact model identity, direct-vs-proxy provenance, both
hosted tool types, citations, clarification order, continuation linkage, token
usage, reported provider cost when available, and credential-free raw evidence.
## Official sources
- [GPT-5.6 Sol model](https://developers.openai.com/api/docs/models/gpt-5.6-sol)
- [Web search](https://developers.openai.com/api/docs/guides/tools-web-search)
- [Code Interpreter](https://developers.openai.com/api/docs/guides/tools-code-interpreter)
- [GPT-5.6 model guidance](https://developers.openai.com/api/docs/guides/model-guidance?model=gpt-5.6-sol)
- [Alibaba Model Studio code interpreter (DashScope)](https://help.aliyun.com/zh/model-studio/qwen-code-interpreter)
## 中文说明
本项目使用正文所述的**精确协议**:Responses API、托管 `web_search` 与托管
`code_interpreter`。验收依据是服务端返回的 `web_search_call` /
`code_interpreter_call` 和 URL 引用,而不是代码里“声明了工具”或答案里
“声称用过 Python”。
按作者批准的多提供商政策,验收不绑定官方 OpenAI 账号:官方 `gpt-5.6-sol` 路径
完整保留(当前 Key 推理返回 `credit_balance_exhausted`,已在证据中如实记录),
具备等价托管工具的提供商同样可以验收。2026-07-31 的正式运行用阿里云百炼
`qwen3.7-plus`DashScope Responses API)通过了全部验收门:东盟任务先搜索十个
首都坐标、再用托管 Python 枚举 45 对大圆距离(吉隆坡—新加坡 316.35 km,与独立
本地参考一致);比特币任务先在不用任何工具的情况下澄清数据源与指标,再通过
`previous_response_id` 继续,完成 3 轮模型主导的搜索与 4 次托管代码执行
MA7/MA20、RSI14、MACD、区间收益、最大回撤与走势图)。OpenRouter 只作为诊断
路径明确保留,不会被包装成替代品。
+396
View File
@@ -0,0 +1,396 @@
"""Exact GPT-5.6 Responses API agent for Experiment 1-3.
The previous companion sent Responses-style hosted tools to Chat Completions
through a proxy and then reported an empty ``tool_calls`` list. This module
uses the actual ``/v1/responses`` protocol and preserves its typed output items
(``web_search_call``, ``code_interpreter_call``, messages, and citations).
"""
from __future__ import annotations
import json
import logging
import time
from typing import Any, Dict, List, Literal, Optional
import requests
logger = logging.getLogger(__name__)
class GPT5NativeAgent:
"""GPT-5.6 Sol with OpenAI-hosted web search and Python tools."""
def __init__(
self,
api_key: str,
base_url: str = "https://api.openai.com/v1",
model: str = "gpt-5.6-sol",
):
if not api_key:
raise ValueError("An API key is required")
self.api_key = api_key
self.base_url = base_url.rstrip("/")
self.model = model
self.provider = (
"openai" if self.base_url == "https://api.openai.com/v1" else
"openrouter" if "openrouter.ai" in self.base_url else
"dashscope" if "dashscope" in self.base_url else
"custom"
)
self.conversation_history: List[Dict[str, Any]] = []
self.system_prompt = self._create_system_prompt()
self.previous_response_id: Optional[str] = None
self.api_turns: List[Dict[str, Any]] = []
@staticmethod
def _create_system_prompt() -> str:
return """You are a deep-research assistant. 你是一名深度研究助手。
Hard rule / 硬性规则: when the user's research request leaves material
preferences ambiguous — for example which data source to use or which
technical indicators to compute — ask a concise clarifying question FIRST
(for example “您偏好使用哪个数据源?需要分析哪些技术指标?”), and do NOT
call any tool until the user answers.
当用户的研究请求没有明确数据来源或具体分析指标时,必须先向用户提问澄清,
在用户回答之前不要调用任何工具。
After clarification, use hosted web search for current facts and cite
sources, and use the hosted Python/code-interpreter tool for quantitative
analysis; do not claim a calculation was run unless the response contains a
completed code_interpreter_call.
澄清之后:使用 web_search 获取最新事实并引用来源链接;所有定量计算必须通过
code_interpreter 实际执行,不得口算或声称运行了代码。"""
def _tools(self) -> List[Dict[str, Any]]:
if self.provider == "dashscope":
# Exact structures from the Alibaba Model Studio Responses API guides.
return [{"type": "web_search"}, {"type": "code_interpreter"}]
# Exact structures from the official OpenAI Responses API guides.
return [
{"type": "web_search", "search_context_size": "medium"},
{
"type": "code_interpreter",
"container": {"type": "auto", "memory_limit": "4g"},
},
]
def _build_responses_request(
self,
input_text: str,
*,
use_tools: bool = True,
tool_choice: Literal["auto", "none", "required"] = "auto",
reasoning_effort: str = "low",
verbosity: Optional[str] = None,
max_output_tokens: Optional[int] = None,
background: bool = False,
) -> Dict[str, Any]:
if self.provider != "dashscope":
if reasoning_effort not in {"none", "low", "medium", "high", "xhigh", "max"}:
raise ValueError("Unsupported GPT-5.6 reasoning effort")
if verbosity not in {None, "low", "medium", "high"}:
raise ValueError("verbosity must be low, medium, or high")
request: Dict[str, Any] = {
"model": self.model,
"instructions": self.system_prompt,
"input": input_text,
}
if self.provider == "dashscope":
# DashScope runs thinking natively and has no reasoning.effort or
# text.verbosity knobs; its gateway also drops non-streaming
# requests that stay silent for ~60s, so streaming is mandatory.
request["stream"] = True
else:
request["reasoning"] = {"effort": reasoning_effort}
request["background"] = background
request["store"] = True
if verbosity:
request["text"] = {"verbosity": verbosity}
if max_output_tokens:
request["max_output_tokens"] = max_output_tokens
if use_tools:
request["tools"] = self._tools()
request["tool_choice"] = tool_choice
if self.previous_response_id:
request["previous_response_id"] = self.previous_response_id
return request
@staticmethod
def _output_text(response: Dict[str, Any]) -> str:
if not isinstance(response, dict):
return ""
chunks: List[str] = []
for item in response.get("output") or []:
if not isinstance(item, dict) or item.get("type") != "message":
continue
for content in item.get("content") or []:
if isinstance(content, dict) and content.get("type") == "output_text" and content.get("text"):
chunks.append(content["text"])
return "\n".join(chunks).strip()
@staticmethod
def _tool_items(response: Dict[str, Any]) -> List[Dict[str, Any]]:
if not isinstance(response, dict):
return []
return [
item
for item in response.get("output") or []
if isinstance(item, dict)
and item.get("type") in {
"web_search_call",
"code_interpreter_call",
"hosted_tool_call",
}
]
@staticmethod
def _citations(response: Dict[str, Any]) -> List[Dict[str, Any]]:
citations = []
if not isinstance(response, dict):
return citations
for item in response.get("output") or []:
if not isinstance(item, dict):
continue
for content in item.get("content") or []:
if not isinstance(content, dict):
continue
for annotation in content.get("annotations") or []:
if isinstance(annotation, dict) and annotation.get("type") in {
"url_citation",
"container_file_citation",
}:
citations.append(annotation)
# DashScope reports sources on the web_search_call item itself
# instead of url_citation annotations; normalize them here.
if item.get("type") == "web_search_call":
action = item.get("action")
if isinstance(action, dict):
for source in action.get("sources") or []:
url = source if isinstance(source, str) else (source.get("url") if isinstance(source, dict) else None)
if url:
citations.append(
{"type": "url_citation", "url": url}
)
return citations
def _post_responses(
self, request: Dict[str, Any]
) -> tuple[int, Dict[str, Any], Optional[Dict[str, int]]]:
"""Send one Responses request and return (status, body, stream_events).
DashScope requires streaming; the final ``response.completed`` event
carries the same response object the non-streaming API returns, so both
paths converge on an identical shape.
"""
headers = {
"Authorization": f"Bearer {self.api_key}",
"Content-Type": "application/json",
}
if not request.get("stream"):
http_response = requests.post(
f"{self.base_url}/responses",
headers=headers,
json=request,
timeout=900,
)
try:
return http_response.status_code, http_response.json(), None
except ValueError:
return http_response.status_code, {"raw_text": http_response.text}, None
event_counts: Dict[str, int] = {}
final_response: Optional[Dict[str, Any]] = None
with requests.post(
f"{self.base_url}/responses",
headers=headers,
json=request,
stream=True,
timeout=900,
) as http_response:
status_code = http_response.status_code
if not http_response.ok:
return status_code, {"raw_text": http_response.text}, event_counts
for line in http_response.iter_lines(decode_unicode=True):
if not line or not line.startswith("data:"):
continue
data = line[len("data:"):].strip()
if data == "[DONE]":
break
try:
event = json.loads(data)
except ValueError:
continue
event_type = event.get("type") or "unknown"
event_counts[event_type] = event_counts.get(event_type, 0) + 1
if event_type in {"response.completed", "response.failed"}:
final_response = event.get("response")
if final_response is None:
return status_code, {"error": {"type": "stream_incomplete",
"message": "stream ended without response.completed"}}, event_counts
return status_code, final_response, event_counts
def process_request(
self,
user_request: str,
use_tools: bool = True,
tool_choice: Literal["auto", "none", "required"] = "auto",
temperature: float = 0.3,
max_tokens: Optional[int] = None,
reasoning_effort: str = "low",
verbosity: Optional[str] = None,
dry_run: bool = False,
background: bool = False,
) -> Dict[str, Any]:
"""Create one Responses API turn and retain its complete trace.
``temperature`` remains in the signature for legacy callers, but is not
sent: GPT-5.6 reasoning requests use ``reasoning.effort`` instead.
"""
request = self._build_responses_request(
user_request,
use_tools=use_tools,
tool_choice=tool_choice,
reasoning_effort=reasoning_effort,
verbosity=verbosity,
max_output_tokens=max_tokens,
background=background,
)
if dry_run:
return {
"success": True,
"dry_run": True,
"request": request,
"response": None,
"tool_calls": [],
"model": self.model,
"provider": self.provider,
}
started = time.monotonic()
try:
status_code, response, stream_events = self._post_responses(request)
elapsed = round(time.monotonic() - started, 6)
turn = {
"request": json.loads(json.dumps(request, ensure_ascii=False)),
"http_status": status_code,
"response": response,
"elapsed_seconds": elapsed,
}
if stream_events:
turn["stream_event_counts"] = stream_events
self.api_turns.append(turn)
if not isinstance(response, dict) or status_code >= 400 or response.get("error"):
error = (response.get("error") if isinstance(response, dict) else None) or {
"type": "http_error",
"message": (response.get("raw_text") if isinstance(response, dict) else None) or (json.dumps(response)[:500] if response is not None else "Empty response"),
}
return {
"success": False,
"error": error,
"response": None,
"request": request,
"raw_response": response,
"tool_calls": [],
"citations": [],
"usage": (response.get("usage") if isinstance(response, dict) else {}) or {},
"model": self.model,
"provider": self.provider,
"base_url": self.base_url,
"elapsed_seconds": elapsed,
}
self.previous_response_id = response.get("id")
text = self._output_text(response)
self.conversation_history.extend(
[
{"role": "user", "content": user_request},
{"role": "assistant", "content": text},
]
)
return {
"success": response.get("status") == "completed" and bool(text),
"error": response.get("error"),
"response": text,
"request": request,
"raw_response": response,
"output_items": response.get("output") or [],
"tool_calls": self._tool_items(response),
"citations": self._citations(response),
"usage": response.get("usage") or {},
"model": response.get("model") or self.model,
"requested_model": self.model,
"provider": self.provider,
"base_url": self.base_url,
"response_id": response.get("id"),
"status": response.get("status"),
"elapsed_seconds": elapsed,
"temperature_omitted_for_reasoning_model": temperature is not None,
}
except Exception as exc:
elapsed = round(time.monotonic() - started, 6)
self.api_turns.append(
{
"request": request,
"elapsed_seconds": elapsed,
"error": {"class": type(exc).__name__, "message": str(exc)},
}
)
return {
"success": False,
"error": {"class": type(exc).__name__, "message": str(exc)},
"response": None,
"request": request,
"tool_calls": [],
"citations": [],
"model": self.model,
"provider": self.provider,
"base_url": self.base_url,
"elapsed_seconds": elapsed,
}
def search_and_analyze(
self, topic: str, analysis_code: Optional[str] = None
) -> Dict[str, Any]:
code_requirement = (
f"Run this supplied Python in the hosted tool and inspect its output:\n{analysis_code}"
if analysis_code
else "Use the hosted Python tool for all quantitative processing."
)
return self.process_request(
f"Research current information about {topic}. {code_requirement} "
"Cite web sources and distinguish searched facts from computed results.",
use_tools=True,
reasoning_effort="medium",
)
def clear_history(self) -> None:
self.conversation_history = []
self.previous_response_id = None
self.api_turns = []
def get_history(self) -> List[Dict[str, Any]]:
return json.loads(json.dumps(self.conversation_history, ensure_ascii=False))
def set_system_prompt(self, prompt: str) -> None:
self.system_prompt = prompt
class GPT5AgentChain:
"""Sequential Responses turns linked with ``previous_response_id``."""
def __init__(self, agent: GPT5NativeAgent):
self.agent = agent
self.chain_results: List[Dict[str, Any]] = []
def add_step(self, request: str, **kwargs: Any) -> "GPT5AgentChain":
self.chain_results.append(
{"request": request, "result": self.agent.process_request(request, **kwargs)}
)
return self
def execute(self) -> List[Dict[str, Any]]:
return self.chain_results
def clear(self) -> None:
self.chain_results = []
+86
View File
@@ -0,0 +1,86 @@
"""Configuration for the exact GPT-5.6 Responses API companion."""
import os
from typing import Optional, Tuple
from dotenv import load_dotenv
load_dotenv()
def _optional_int_env(name: str) -> Optional[int]:
"""Read an optional integer without making module import configuration-fatal."""
raw_value = os.getenv(name)
if raw_value is None:
return None
cleaned = raw_value.strip()
if not cleaned or not cleaned.isascii() or not cleaned.isdecimal():
return None
try:
return int(cleaned)
except ValueError:
return None
class Config:
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")
OPENAI_BASE_URL = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
OPENROUTER_API_KEY = os.getenv("OPENROUTER_API_KEY", "")
OPENROUTER_BASE_URL = os.getenv(
"OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1"
)
DASHSCOPE_API_KEY = os.getenv("DASHSCOPE_API_KEY", "")
DASHSCOPE_BASE_URL = os.getenv(
"DASHSCOPE_BASE_URL",
"https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
)
DASHSCOPE_MODEL = os.getenv("DASHSCOPE_MODEL", "qwen3.7-plus")
BACKEND = os.getenv("BACKEND", "openai")
MODEL_NAME = os.getenv("MODEL_NAME", "gpt-5.6-sol")
DEFAULT_TEMPERATURE = 0.3 # legacy CLI compatibility; intentionally omitted
DEFAULT_MAX_TOKENS: Optional[int] = _optional_int_env("DEFAULT_MAX_TOKENS")
DEFAULT_TOOL_CHOICE = os.getenv("DEFAULT_TOOL_CHOICE", "auto")
LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO")
LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
RATE_LIMIT_RPM = int(os.getenv("RATE_LIMIT_RPM", "20"))
MAX_RETRIES = int(os.getenv("MAX_RETRIES", "3"))
RETRY_DELAY = float(os.getenv("RETRY_DELAY", "1.0"))
WEB_SEARCH_MAX_RESULTS = int(os.getenv("WEB_SEARCH_MAX_RESULTS", "5"))
CODE_INTERPRETER_TIMEOUT = int(os.getenv("CODE_INTERPRETER_TIMEOUT", "30"))
@classmethod
def resolve(
cls, backend: Optional[str] = None, model: Optional[str] = None
) -> Tuple[str, str, str]:
backend = backend or cls.BACKEND
if backend == "openai":
model = model or cls.MODEL_NAME
return cls.OPENAI_API_KEY, cls.OPENAI_BASE_URL, model.removeprefix("openai/")
if backend == "openrouter":
model = model or cls.MODEL_NAME
routed = model if model.startswith("openai/") else f"openai/{model}"
return cls.OPENROUTER_API_KEY, cls.OPENROUTER_BASE_URL, routed
if backend == "dashscope":
return cls.DASHSCOPE_API_KEY, cls.DASHSCOPE_BASE_URL, model or cls.DASHSCOPE_MODEL
raise ValueError("backend must be openai, openrouter, or dashscope")
@classmethod
def validate(cls, backend: Optional[str] = None) -> bool:
key, _, _ = cls.resolve(backend)
if not key:
print(f"Error: no API key for {backend or cls.BACKEND}")
return False
return True
@classmethod
def display(cls, backend: Optional[str] = None) -> None:
key, base_url, model = cls.resolve(backend)
print("=== GPT-5.6 Responses Configuration ===")
print(f"Backend: {backend or cls.BACKEND}")
print(f"API Base URL: {base_url}")
print(f"Model: {model}")
print(f"API Key configured: {bool(key)}")
def check_config() -> bool:
return Config.validate()
+44
View File
@@ -0,0 +1,44 @@
# OpenRouter API Configuration
# Get your API key from: https://openrouter.ai/keys
# Note: this experiment uses OpenRouter as its PRIMARY backend (no fallback
# needed). The same OPENROUTER_API_KEY also serves as the universal fallback
# for the other chapter1 experiments (context, learning-from-experience,
# web-search-agent) when their direct provider key is missing.
OPENROUTER_API_KEY=your-openrouter-api-key
# API Base URL (default is fine for most users)
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# OpenAI official path (canonical GPT-5.6 Sol); needs an account with quota
OPENAI_API_KEY=your-openai-api-key
# Alibaba Model Studio (DashScope) — eligible equivalent-provider path with
# hosted web_search + hosted code_interpreter on the Responses API.
# Get your API key from: https://www.alibabacloud.com/help/en/model-studio/get-api-key
# Use https://dashscope.aliyuncs.com/compatible-mode/v1 for a China-region key.
DASHSCOPE_API_KEY=your-dashscope-api-key
DASHSCOPE_BASE_URL=https://dashscope-intl.aliyuncs.com/compatible-mode/v1
DASHSCOPE_MODEL=qwen3.7-plus
# Model Configuration
# GPT-5 model identifier on OpenRouter
MODEL_NAME=openai/gpt-5.6-sol
# Request Configuration
DEFAULT_TEMPERATURE=0.3
DEFAULT_MAX_TOKENS=4000
DEFAULT_TOOL_CHOICE=auto
# Logging
LOG_LEVEL=INFO
# Rate Limiting
RATE_LIMIT_RPM=20
# Retry Configuration
MAX_RETRIES=3
RETRY_DELAY=1.0
# Tool-specific Configuration
WEB_SEARCH_MAX_RESULTS=5
CODE_INTERPRETER_TIMEOUT=30
+249
View File
@@ -0,0 +1,249 @@
#!/usr/bin/env python3
"""
Example showing the exact OpenRouter GPT-5 request format matching the Go implementation
"""
import json
import requests
import os
from typing import Dict, Any
try:
from dotenv import load_dotenv
load_dotenv()
except ImportError:
pass
def make_gpt5_openrouter_request(
api_key: str,
system_prompt: str,
user_prompt: str,
reasoning_effort: str = "low"
) -> Dict[str, Any]:
"""
Make a GPT-5 request using the exact format from the Go implementation
This matches the GPT5OpenRouterRequest structure from the Go code
"""
# Build messages (matching Go implementation)
messages = [
{
"role": "system",
"content": system_prompt
},
{
"role": "user",
"content": user_prompt
}
]
# Build web search tool configuration (matching Go GPT5OpenRouterWebSearchTool)
web_search_tool = {
"type": "web_search",
"search_context_size": "medium",
"user_location": {
"type": "approximate",
"country": "US"
}
}
# Build request with OpenRouter-specific parameters (matching Go GPT5OpenRouterRequest)
request_body = {
"model": "openai/gpt-5.6-sol", # Default from Go code
"messages": messages,
"tools": [web_search_tool],
"tool_choice": "auto",
"parallel_tool_calls": True,
"reasoning": {
"effort": reasoning_effort,
"generate_summary": False
},
"background": False,
"stream": False # Can be set to True for streaming
}
print("="*60)
print("GPT-5 OpenRouter Request (matching Go implementation):")
print("="*60)
print(json.dumps(request_body, indent=2))
print("="*60)
# Set headers (matching Go implementation)
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {api_key}"
}
# Make the request
url = "https://openrouter.ai/api/v1/chat/completions"
try:
response = requests.post(
url,
headers=headers,
json=request_body,
timeout=600 # Match Go timeout
)
print(f"\nResponse Status: {response.status_code}")
if response.status_code == 200:
response_data = response.json()
# Log usage (matching Go logging)
if "usage" in response_data:
usage = response_data["usage"]
input_tokens = usage.get(
"prompt_tokens", usage.get("input_tokens", 0)
)
output_tokens = usage.get(
"completion_tokens", usage.get("output_tokens", 0)
)
input_details = usage.get(
"prompt_tokens_details", usage.get("input_tokens_details")
)
output_details = usage.get(
"completion_tokens_details", usage.get("output_tokens_details")
)
print("\nGPT-5 OpenRouter Usage:")
print(f" Input: {input_tokens} tokens", end="")
if isinstance(input_details, dict):
print(f" (cached: {input_details.get('cached_tokens', 0)})")
else:
print()
print(f" Output: {output_tokens} tokens", end="")
if isinstance(output_details, dict):
print(f" (reasoning: {output_details.get('reasoning_tokens', 0)})")
else:
print()
print(f" Total: {usage.get('total_tokens', 0)}")
return response_data
else:
print(f"\nError: {response.text}")
return {"error": response.text, "status_code": response.status_code}
except Exception as e:
print(f"\nException: {str(e)}")
return {"error": str(e)}
def demonstrate_streaming_response():
"""
Demonstrate how streaming would work (matching Go handleStreamingResponse)
"""
print("\n" + "="*60)
print("Streaming Response Handler (pseudo-code matching Go):")
print("="*60)
streaming_code = '''
def handle_streaming_response(response):
"""
Handle streaming responses from GPT-5 OpenRouter API
Matches Go handleStreamingResponse function
"""
content_builder = []
reasoning_builder = []
reasoning_token_count = 0
for line in response.iter_lines():
if not line:
continue
line_str = line.decode('utf-8')
if not line_str.startswith("data: "):
continue
data = line_str[6:] # Remove "data: " prefix
if data == "[DONE]":
break
try:
chunk = json.loads(data)
if "choices" in chunk and len(chunk["choices"]) > 0:
delta = chunk["choices"][0].get("delta", {})
# Check for reasoning content
if "reasoning_content" in delta:
reasoning = delta["reasoning_content"]
reasoning_builder.append(reasoning)
reasoning_token_count += 1
print(f"🧠 [GPT-5 THINKING] {reasoning}")
# Check for regular content
if "content" in delta:
content = delta["content"]
content_builder.append(content)
except json.JSONDecodeError:
continue
final_content = "".join(content_builder)
return final_content
'''
print(streaming_code)
def main():
"""
Main demonstration
"""
print("\n" + "="*60)
print(" GPT-5 OpenRouter Request Format Demo")
print(" Exact match with Go implementation")
print("="*60)
# Get API key from environment
api_key = os.getenv("OPENROUTER_API_KEY")
if not api_key:
print("\n❌ Error: OPENROUTER_API_KEY not found in environment")
print("Please set: export OPENROUTER_API_KEY=your-openrouter-api-key")
return
# Example prompts
system_prompt = "You are a helpful AI assistant with web search capabilities."
user_prompt = "What are the latest developments in artificial intelligence?"
print("\n1. Making request with LOW reasoning effort:")
print("-"*60)
result_low = make_gpt5_openrouter_request(
api_key=api_key,
system_prompt=system_prompt,
user_prompt=user_prompt,
reasoning_effort="low"
)
if "choices" in result_low:
content = result_low["choices"][0]["message"]["content"]
print(f"\nResponse preview: {content[:200]}...")
print("\n2. Making request with HIGH reasoning effort:")
print("-"*60)
result_high = make_gpt5_openrouter_request(
api_key=api_key,
system_prompt=system_prompt,
user_prompt="Explain the implications of quantum computing on cryptography",
reasoning_effort="high"
)
if "choices" in result_high:
content = result_high["choices"][0]["message"]["content"]
print(f"\nResponse preview: {content[:200]}...")
# Show streaming handler
demonstrate_streaming_response()
print("\n" + "="*60)
print("Demo complete! This shows the exact request format from Go.")
print("="*60)
if __name__ == "__main__":
main()
@@ -0,0 +1,116 @@
# Go vs Python Implementation Comparison
This document shows how the Python implementation exactly matches the Go implementation for GPT-5 OpenRouter API calls.
## Request Structure Comparison
### Go Implementation
```go
// From the provided Go code
webSearchTool := GPT5OpenRouterWebSearchTool{
Type: "web_search",
SearchContextSize: "medium",
UserLocation: map[string]interface{}{
"type": "approximate",
"country": "US",
},
}
request := GPT5OpenRouterRequest{
Model: c.model,
Messages: messages,
Tools: []GPT5OpenRouterWebSearchTool{webSearchTool},
ToolChoice: "auto",
ParallelToolCalls: true,
Reasoning: &GPT5OpenRouterReasoning{
Effort: reasoningEffort,
GenerateSummary: false,
},
Background: false,
Stream: false,
}
```
### Python Implementation
```python
# From agent.py
web_search_tool = {
"type": "web_search",
"search_context_size": "medium",
"user_location": {
"type": "approximate",
"country": "US"
}
}
request_body = {
"model": self.model,
"messages": messages,
"tools": [web_search_tool],
"tool_choice": "auto",
"parallel_tool_calls": True,
"reasoning": {
"effort": reasoning_effort,
"generate_summary": False
},
"background": False,
"stream": False
}
```
## Key Matching Points
1. **Tool Structure**: Both implementations use the same tool structure with `type: "web_search"` and additional configuration fields.
2. **Request Parameters**: Identical parameters including:
- `model`
- `messages`
- `tools` (array of web_search tools)
- `tool_choice: "auto"`
- `parallel_tool_calls: true/True`
- `reasoning` with effort and generate_summary
- `background: false/False`
- `stream: false/False`
3. **Headers**: Both use simple headers:
```go
// Go
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", c.apiKey))
```
```python
# Python
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {self.api_key}"
}
```
4. **Model Default**: Both default to `openai/gpt-5.6-sol`
5. **Reasoning Levels**: Both support "low", "medium", and "high" reasoning effort
## Usage Comparison
### Go
```go
client := NewGPT5OpenRouterClientAdapter(apiKey, baseURL, model)
response, err := client.CallGPT5(ctx, systemPrompt, userPrompt, "medium")
```
### Python
```python
agent = GPT5NativeAgent(api_key, base_url, model)
result = agent.process_request(user_request, use_tools=True, reasoning_effort="medium")
```
## Response Handling
Both implementations:
- Handle streaming and non-streaming responses
- Log token usage including cached and reasoning tokens
- Extract content from the response choices
- Handle errors with appropriate status codes
The Python implementation is a direct port of the Go implementation, ensuring complete compatibility with the OpenRouter GPT-5 API.
+444
View File
@@ -0,0 +1,444 @@
"""
Main entry point for GPT-5 Native Tools Agent
Interactive CLI for using web_search and code_interpreter tools
"""
import sys
import json
import logging
from typing import Optional
from agent import GPT5NativeAgent, GPT5AgentChain
from config import Config
import argparse
# Set up logging
logging.basicConfig(
level=getattr(logging, Config.LOG_LEVEL),
format=Config.LOG_FORMAT
)
logger = logging.getLogger(__name__)
class InteractiveCLI:
"""Interactive command-line interface for GPT-5 Agent"""
def __init__(self, backend: str = None, model: str = None):
"""Initialize the CLI"""
if not Config.validate(backend):
raise ValueError("Invalid configuration. Please check your .env file")
api_key, base_url, resolved_model = Config.resolve(backend, model)
self.agent = GPT5NativeAgent(
api_key=api_key,
base_url=base_url,
model=resolved_model,
)
self.backend = backend or Config.BACKEND
self.commands = {
"/help": self.show_help,
"/clear": self.clear_history,
"/history": self.show_history,
"/tools": self.toggle_tools,
"/search": self.search_mode,
"/code": self.code_mode,
"/analyze": self.analyze_mode,
"/config": self.show_config,
"/reasoning": self.set_reasoning_effort,
"/exit": self.exit_cli,
"/quit": self.exit_cli,
}
self.use_tools = True
self.tool_choice = "auto"
self.reasoning_effort = "low" # Default reasoning effort
def show_help(self):
"""Display help information"""
help_text = """
Commands:
/help - Show this help message
/clear - Clear conversation history
/history - Show conversation history
/tools - Toggle tools on/off
/search - Enter web search mode
/code - Enter code interpreter mode
/analyze - Combined search + analysis mode
/config - Show current configuration
/reasoning - Set reasoning effort (low/medium/high)
/exit - Exit the application
Native Tools:
• web_search - Search the internet for real-time info
• code_interpreter - Execute Python code and analyze
Usage:
Simply type your request and the agent will use
appropriate tools automatically.
Examples:
"东盟 10 国首都之间,距离最近的两个首都是?给出你的详细分析推理过程。"
"搜索最近一年比特币的价格,计算收益率、最大回撤、年化波动等重要指标"
"""
print(help_text)
def clear_history(self):
"""Clear conversation history"""
self.agent.clear_history()
print("✅ Conversation history cleared")
def show_history(self):
"""Display conversation history"""
history = self.agent.get_history()
if not history:
print("📭 No conversation history")
return
print("\n" + "="*60)
print("CONVERSATION HISTORY")
print("="*60)
for i, msg in enumerate(history, 1):
role = msg["role"].upper()
content = msg["content"][:200] + "..." if len(msg["content"]) > 200 else msg["content"]
print(f"\n[{i}] {role}:\n{content}")
print("="*60)
def toggle_tools(self):
"""Toggle tool usage on/off"""
self.use_tools = not self.use_tools
status = "enabled" if self.use_tools else "disabled"
print(f"🔧 Tools {status}")
def search_mode(self):
"""Enter web search mode"""
print("\n🔍 Web Search Mode")
print("Enter your search query (or 'back' to return):")
query = input("> ").strip()
if query.lower() == "back":
return
request = f"Search the web for: {query}"
self._process_request(request, force_tools=True)
def code_mode(self):
"""Enter code interpreter mode"""
print("\n💻 Code Interpreter Mode")
print("Enter your code or computational request (or 'back' to return):")
request = input("> ").strip()
if request.lower() == "back":
return
enhanced_request = f"Use the code interpreter to: {request}"
self._process_request(enhanced_request, force_tools=True)
def analyze_mode(self):
"""Combined search and analysis mode"""
print("\n🔬 Search & Analyze Mode")
print("Enter topic to research and analyze (or 'back' to return):")
topic = input("> ").strip()
if topic.lower() == "back":
return
print("\nOptional: Enter Python code for analysis (press Enter to skip):")
code = input("> ").strip()
if code:
result = self.agent.search_and_analyze(topic, code)
else:
result = self.agent.search_and_analyze(topic)
self._display_result(result)
def show_config(self):
"""Display current configuration"""
Config.display()
print(f"\nCurrent Settings:")
print(f" Tools Enabled: {self.use_tools}")
print(f" Tool Choice: {self.tool_choice}")
print(f" Reasoning Effort: {self.reasoning_effort}")
def set_reasoning_effort(self):
"""Set the reasoning effort level"""
print("\n🧠 Set Reasoning Effort")
print("Options: low, medium, high")
print(f"Current: {self.reasoning_effort}")
effort = input("Enter new effort level: ").strip().lower()
if effort in ["low", "medium", "high"]:
self.reasoning_effort = effort
print(f"✅ Reasoning effort set to: {effort}")
else:
print(f"❌ Invalid effort level. Must be low, medium, or high")
def exit_cli(self):
"""Exit the application"""
print("\n👋 Goodbye!")
sys.exit(0)
def _process_request(self, request: str, force_tools: bool = False):
"""
Process a user request
Args:
request: User request
force_tools: Force tool usage regardless of settings
"""
use_tools = force_tools or self.use_tools
result = self.agent.process_request(
request,
use_tools=use_tools,
tool_choice=self.tool_choice if use_tools else "none",
temperature=Config.DEFAULT_TEMPERATURE,
max_tokens=Config.DEFAULT_MAX_TOKENS,
reasoning_effort=self.reasoning_effort
)
self._display_result(result)
def _display_result(self, result: dict):
"""
Display the result of a request
Args:
result: Result dictionary from agent
"""
print("\n" + "="*60)
if result["success"]:
# Display tool usage
if result["tool_calls"]:
print("🔧 Tools Used:")
for tool in result["tool_calls"]:
print(f"{tool.get('type', 'unknown_tool')}")
print()
# Display response
print("📝 Response:")
print("-"*60)
print(result["response"])
print("-"*60)
# Display token usage
if result.get("usage"):
usage = result["usage"]
total = usage.get("total_tokens", 0)
if total:
print(f"\n📊 Tokens used: {total}")
else:
print(f"❌ Error: {result.get('error', 'Unknown error')}")
print("="*60)
def run(self):
"""Run the interactive CLI"""
print("\n" + "="*60)
print(" 🤖 GPT-5 Native Tools Agent")
print(f" Responses API backend: {self.backend}")
print("="*60)
self.show_help()
while True:
try:
print("\n💬 Enter your request (or /help for commands):")
user_input = input("> ").strip()
if not user_input:
continue
# Check for commands
if user_input.startswith("/"):
command = user_input.split()[0].lower()
if command in self.commands:
self.commands[command]()
else:
print(f"❌ Unknown command: {command}")
print("Type /help for available commands")
else:
# Process as regular request
self._process_request(user_input)
except KeyboardInterrupt:
print("\n\n⚠️ Interrupted. Type /exit to quit or continue chatting.")
except Exception as e:
logger.error(f"Error: {str(e)}")
print(f"❌ An error occurred: {str(e)}")
def _run_single(args):
"""执行单次请求(single / dry-run 模式),打印可读轨迹并按需保存结果。"""
# dry-run 只组装请求体、不联网,因此无需真实 API Key
api_key, base_url, model = Config.resolve(args.backend, args.model)
api_key = api_key or ("DRYRUN-PLACEHOLDER" if args.dry_run else "")
agent = GPT5NativeAgent(
api_key=api_key,
base_url=base_url,
model=model,
)
result = agent.process_request(
args.request,
use_tools=not args.no_tools,
temperature=Config.DEFAULT_TEMPERATURE,
max_tokens=Config.DEFAULT_MAX_TOKENS,
reasoning_effort=args.reasoning,
verbosity=args.verbosity,
dry_run=args.dry_run
)
# dry-run:打印将要发送给模型的完整请求体(原生工具定义 + 参数)
if result.get("dry_run"):
print("\n" + "=" * 60)
print("🧪 Dry-run:以下是发送给 GPT-5 的请求体(未联网)")
print("=" * 60)
print(f"Model: {result['model']}")
print(f"任务: {args.request}")
print("-" * 60)
print(json.dumps(result["request"], indent=2, ensure_ascii=False))
print("=" * 60)
elif result["success"]:
print("\n" + "=" * 60)
print("📝 Response:")
print("-" * 60)
print(result["response"])
print("-" * 60)
usage = result.get("usage") or {}
if usage:
print(
f"📊 Tokens - Input: {usage.get('input_tokens', 'N/A')}, "
f"Output: {usage.get('output_tokens', 'N/A')}, "
f"Reasoning: {usage.get('output_tokens_details', {}).get('reasoning_tokens', 0)}, "
f"Total: {usage.get('total_tokens', 'N/A')}"
)
print("=" * 60)
else:
print(f"❌ Error: {result.get('error')}")
# 按需将完整结果(含轨迹/请求体)保存为 JSON,便于复盘
if args.output:
with open(args.output, "w", encoding="utf-8") as f:
json.dump(result, f, indent=2, ensure_ascii=False)
print(f"💾 结果已保存到: {args.output}")
if not result["success"]:
sys.exit(1)
def main():
"""主入口:解析命令行参数并分派到交互 / 单次 / 测试模式。"""
parser = argparse.ArgumentParser(
description="GPT-5 原生工具 Agent —— 演示实验 1.3:网络搜索 + 代码解释器的原生 Deep Research 能力",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""示例:
python main.py # 交互模式(默认)
python main.py --mode single --request "东盟 10 国首都之间距离最近的两个首都是?"
python main.py --mode single --request "分析比特币近一月走势" --reasoning high --verbosity high
python main.py --mode single --request "..." --output result.json
python main.py --dry-run --request "..." # 离线查看请求体(原生工具定义),无需 API Key
python main.py --mode test --test basic # 运行指定联网手动用例
""",
)
parser.add_argument(
"--mode",
choices=["interactive", "single", "test"],
default="interactive",
help="运行模式:interactive 交互对话(默认)/ single 单次请求 / test 联网手动用例",
)
parser.add_argument(
"--request",
type=str,
help="single / dry-run 模式下的任务或查询内容",
)
parser.add_argument(
"--backend",
choices=["openai", "openrouter", "dashscope"],
default=Config.BACKEND,
help="Responses API backend; openai is the exact canonical path, dashscope is the eligible equivalent-provider path",
)
parser.add_argument(
"--model",
type=str,
default=None,
help=f"覆盖模型名称(默认取配置 {Config.MODEL_NAME}",
)
parser.add_argument(
"--reasoning",
choices=["none", "low", "medium", "high", "xhigh", "max"],
default="low",
help="推理力度 Reasoning Effortlow/medium/high,默认 low",
)
parser.add_argument(
"--verbosity",
choices=["low", "medium", "high"],
default=None,
help="输出详略程度 Verbositylow/medium/high,默认跟随模型)",
)
parser.add_argument(
"--no-tools",
action="store_true",
help="禁用原生工具(web_search / code_interpreter",
)
parser.add_argument(
"--output",
type=str,
default=None,
help="将完整结果(含轨迹 / 请求体)保存为 JSON 文件的路径",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="离线组装并打印请求体(含原生工具定义),不调用 API、无需 API Key",
)
parser.add_argument(
"--test",
type=str,
help="test 模式下运行指定联网手动用例(basic/analysis/complex/code/reasoning/search_analyze/chain",
)
args = parser.parse_args()
# dry-run:离线路径,跳过 API Key 校验
if args.dry_run:
if not args.request:
print("❌ --dry-run 需要配合 --request 使用")
sys.exit(1)
_run_single(args)
return
# 其余模式需要有效配置
if not Config.validate(args.backend):
print("❌ 配置错误!")
print("请配置所选 backend 对应的 OPENAI_API_KEY / OPENROUTER_API_KEY / DASHSCOPE_API_KEY")
print("\n示例 .env")
print("DASHSCOPE_API_KEY=your-dashscope-api-key")
sys.exit(1)
if args.mode == "interactive":
cli = InteractiveCLI(args.backend, args.model)
cli.run()
elif args.mode == "single":
if not args.request:
print("❌ single 模式需要 --request 参数")
sys.exit(1)
_run_single(args)
elif args.mode == "test":
from tests.manual.agent_cases import TestGPT5Agent, run_single_test
if args.test:
run_single_test(args.test)
else:
tester = TestGPT5Agent()
tester.run_all_tests()
if __name__ == "__main__":
main()
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env python3
"""
Quick Start Demo for GPT-5 Native Tools Agent
Demonstrates basic usage of web_search and code_interpreter tools
"""
import os
import sys
from agent import GPT5NativeAgent
from config import Config
def demo_web_search():
"""Demonstrate web search capability"""
print("\n" + "="*60)
print("DEMO: Web Search Tool")
print("="*60)
agent = GPT5NativeAgent(
api_key=Config.OPENROUTER_API_KEY,
base_url=Config.OPENROUTER_BASE_URL
)
result = agent.process_request(
"What are the latest developments in GPT-5 and its capabilities?",
use_tools=True,
reasoning_effort="low"
)
if result["success"]:
print("\n✅ Web Search Result:")
print(result["response"][:500] + "...")
if result["tool_calls"]:
print(f"\n🔧 Tools used: {len(result['tool_calls'])}")
else:
print(f"❌ Error: {result['error']}")
def demo_code_interpreter():
"""Demonstrate code generation and analysis capability"""
print("\n" + "="*60)
print("DEMO: Code Generation and Analysis")
print("="*60)
agent = GPT5NativeAgent(
api_key=Config.OPENROUTER_API_KEY,
base_url=Config.OPENROUTER_BASE_URL
)
result = agent.process_request(
"""Create Python code to:
1. Generate the first 20 Fibonacci numbers
2. Calculate their sum and average
3. Find the golden ratio approximation using consecutive pairs
4. Explain the mathematical significance""",
use_tools=True,
reasoning_effort="medium"
)
if result["success"]:
print("\n✅ Code and Analysis Result:")
print(result["response"][:500] + "...")
if result["tool_calls"]:
print(f"\n🔧 Tools used: {len(result['tool_calls'])}")
else:
print(f"❌ Error: {result['error']}")
def demo_combined_tools():
"""Demonstrate using both tools together"""
print("\n" + "="*60)
print("DEMO: Combined Web Search + Code Analysis")
print("="*60)
agent = GPT5NativeAgent(
api_key=Config.OPENROUTER_API_KEY,
base_url=Config.OPENROUTER_BASE_URL
)
result = agent.search_and_analyze(
topic="Current S&P 500 performance and major tech stocks",
analysis_code="""
# Analyze market data
import random
import statistics
# Simulate stock prices based on search results
stocks = {
'AAPL': [175 + random.uniform(-5, 5) for _ in range(10)],
'GOOGL': [140 + random.uniform(-3, 3) for _ in range(10)],
'MSFT': [380 + random.uniform(-8, 8) for _ in range(10)]
}
# Calculate metrics
for symbol, prices in stocks.items():
avg = statistics.mean(prices)
vol = statistics.stdev(prices)
trend = "" if prices[-1] > prices[0] else ""
print(f"{symbol}: Avg=${avg:.2f}, Volatility=${vol:.2f}, Trend={trend}")
"""
)
if result["success"]:
print("\n✅ Combined Analysis Result:")
print(result["response"][:500] + "...")
if result["tool_calls"]:
print(f"\n🔧 Tools used: {len(result['tool_calls'])}")
else:
print(f"❌ Error: {result['error']}")
def main():
"""Run all demos"""
print("\n" + "="*60)
print(" GPT-5 Native Tools Agent - Quick Start Demo")
print("="*60)
# Check configuration
if not Config.validate():
print("\n❌ Configuration Error!")
print("Please set up your .env file with OPENROUTER_API_KEY")
print("\nSteps:")
print("1. Copy env.example to .env")
print("2. Add your OpenRouter API key")
print("3. Get a key at: https://openrouter.ai/keys")
sys.exit(1)
print("\n✅ Configuration valid")
print(f"Using model: {Config.MODEL_NAME}")
# Ask user which demo to run
print("\nSelect demo to run:")
print("1. Web Search only")
print("2. Code Generation and Analysis")
print("3. Combined Tools")
print("4. All demos")
choice = input("\nEnter choice (1-4): ").strip()
if choice == "1":
demo_web_search()
elif choice == "2":
demo_code_interpreter()
elif choice == "3":
demo_combined_tools()
elif choice == "4":
demo_web_search()
demo_code_interpreter()
demo_combined_tools()
else:
print("Invalid choice. Running all demos...")
demo_web_search()
demo_code_interpreter()
demo_combined_tools()
print("\n" + "="*60)
print("Demo complete! 🎉")
print("\nNext steps:")
print("- Run 'python main.py' for interactive mode")
print("- Run 'python main.py --mode test' for live manual cases")
print("- Check README.md for more examples")
print("="*60)
if __name__ == "__main__":
main()
+24
View File
@@ -0,0 +1,24 @@
# GPT-5 Native Tools Agent Requirements
# Core dependencies
openai>=1.35.0 # OpenAI client library (works with OpenRouter)
python-dotenv>=1.0.0 # Environment variable management
requests>=2.31.0 # HTTP requests for OpenRouter API calls
pydantic>=2.5.0 # Data validation and settings management
# Development and testing
pytest>=7.4.0 # Testing framework
pytest-asyncio>=0.21.0 # Async test support
black>=23.0.0 # Code formatting
flake8>=6.1.0 # Linting
mypy>=1.7.0 # Type checking
# Data processing (used by code_interpreter examples)
numpy>=1.24.0 # Numerical computing
pandas>=2.0.0 # Data analysis
matplotlib>=3.7.0 # Plotting (for code interpreter visualizations)
# Utilities
rich>=13.5.0 # Rich terminal output
tqdm>=4.66.0 # Progress bars
colorama>=0.4.6 # Cross-platform colored terminal text
@@ -0,0 +1,447 @@
#!/usr/bin/env python3
"""Run Experiment 1-3 on a hosted web-search + code-execution Responses API.
Acceptance policy (author-mandated, 2026-07-31): the experiment's essence is
model-directed multi-round web search + hosted code execution, clarification
before tools, and a current answer with authoritative sources. The canonical
OpenAI GPT-5.6 Sol path remains the reference implementation, but acceptance
is NOT gated on the official OpenAI account: any provider whose Responses API
genuinely closes the search/code loop server-side (currently Alibaba Model
Studio DashScope ``qwen3.7-plus``) is an eligible acceptance backend. The
OpenRouter route stays a diagnostic and is never accepted.
"""
from __future__ import annotations
import argparse
import hashlib
import itertools
import json
import math
import os
import platform
import shutil
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional, Tuple
from agent import GPT5NativeAgent
from config import Config
ASEAN_TASK = """Research the current official capitals and reliable coordinates
for the ten ASEAN member states. You must use hosted web search and cite the
sources. Then you must use the hosted Python tool—not mental arithmetic—to
enumerate all 45 capital pairs with the haversine formula and identify the
closest pair and distance. Include the coordinates, formula assumptions,
calculation result, retrieval date, and clickable citations. Do not say Python
was used unless a code_interpreter_call completes."""
AMBIGUOUS_TASK = "搜索最近一个月的比特币走势,做技术分析。"
CLARIFICATION_REPLY = (
"使用 CoinGecko 的 BTC/USD 日线收盘价;分析 MA7、MA20、RSI14、MACD(12,26,9)、"
"区间收益和最大回撤,如代码环境支持请绘制收盘价走势图。请搜索数据并用托管 "
"Python 工具实际计算,再给出含来源的报告和交易建议。"
)
# Backends whose runs may close the experiment, in priority order. The
# OpenRouter proxy is diagnostic-only and never appears here.
ACCEPTANCE_BACKENDS = ("openai", "dashscope")
# Independent reference: standard coordinates of the ten ASEAN capitals,
# used to verify the model's computed nearest pair without trusting it.
ASEAN_CAPITAL_COORDS: Dict[str, Tuple[float, float]] = {
"Bandar Seri Begawan": (4.9031, 114.9398),
"Phnom Penh": (11.5564, 104.9282),
"Jakarta": (-6.2088, 106.8456),
"Vientiane": (17.9757, 102.6331),
"Kuala Lumpur": (3.1390, 101.6869),
"Naypyidaw": (19.7633, 96.0785),
"Manila": (14.5995, 120.9842),
"Singapore": (1.3521, 103.8198),
"Bangkok": (13.7563, 100.5018),
"Hanoi": (21.0278, 105.8342),
}
def haversine_km(a: Tuple[float, float], b: Tuple[float, float]) -> float:
radius = 6371.0088
lat1, lon1 = map(math.radians, a)
lat2, lon2 = map(math.radians, b)
dlat, dlon = lat2 - lat1, lon2 - lon1
h = math.sin(dlat / 2) ** 2 + math.cos(lat1) * math.cos(lat2) * math.sin(dlon / 2) ** 2
return 2 * radius * math.asin(math.sqrt(h))
def independent_asean_reference() -> Dict[str, Any]:
"""Locally computed ground truth for the ASEAN nearest-pair check."""
pairs = [
(haversine_km(ca, cb), a, b)
for (a, ca), (b, cb) in itertools.combinations(ASEAN_CAPITAL_COORDS.items(), 2)
]
distance, first, second = min(pairs)
return {
"pair": sorted([first, second]),
"distance_km": round(distance, 1),
"pair_count": len(pairs),
"coordinates": ASEAN_CAPITAL_COORDS,
}
def git_value(*args: str) -> str | None:
try:
return subprocess.check_output(
["git", *args], text=True, stderr=subprocess.DEVNULL
).strip()
except (OSError, subprocess.CalledProcessError):
return None
def output_types(result: Dict[str, Any]) -> List[str]:
return [item.get("type") for item in result.get("output_items") or []]
def completed_calls(result: Dict[str, Any], kind: str) -> List[Dict[str, Any]]:
return [
item
for item in result.get("output_items") or []
if item.get("type") == kind and item.get("status") == "completed"
]
def url_citations(result: Dict[str, Any]) -> List[Dict[str, Any]]:
return [
item for item in result.get("citations") or [] if item.get("type") == "url_citation"
]
def model_identity_exact(result: Dict[str, Any]) -> bool:
"""The returned model must be exactly the requested model."""
requested = (result.get("requested_model") or result.get("model") or "").removeprefix(
"openai/"
)
returned = (result.get("model") or "").removeprefix("openai/")
return bool(requested) and requested == returned
def validate_asean(
result: Dict[str, Any], reference: Optional[Dict[str, Any]] = None
) -> Dict[str, Any]:
answer = result.get("response") or ""
reference = reference or independent_asean_reference()
pair_city, other_city = reference["pair"]
checks = {
"request_succeeded": result.get("success") is True,
"model_identity_exact": model_identity_exact(result),
"web_search_completed": bool(completed_calls(result, "web_search_call")),
"code_interpreter_completed": bool(
completed_calls(result, "code_interpreter_call")
),
"url_citations_present": len(url_citations(result)) >= 2,
"closest_pair_matches_independent_reference": (
pair_city.lower() in answer.lower() and other_city.lower() in answer.lower()
),
"distance_reported": any(unit in answer.lower() for unit in ("km", "公里", "千米")),
}
return {
"checks": checks,
"passed": all(checks.values()),
"output_types": output_types(result),
"independent_reference": {
"pair": reference["pair"],
"distance_km": reference["distance_km"],
"pair_count": reference["pair_count"],
},
}
def is_clarifying_question(result: Dict[str, Any]) -> bool:
text = result.get("response") or ""
return result.get("success") is True and not result.get("tool_calls") and (
"?" in text or "" in text
)
def validate_clarification(
first: Dict[str, Any], second: Dict[str, Any] | None
) -> Dict[str, Any]:
followup_text = (second or {}).get("response") or ""
lowered = followup_text.lower()
checks = {
"first_turn_clarified_before_tools": is_clarifying_question(first),
"continuation_used_previous_response_id": bool(
second and second.get("request", {}).get("previous_response_id") == first.get("response_id")
),
"followup_succeeded": bool(second and second.get("success")),
"followup_web_search_completed": bool(
second and completed_calls(second, "web_search_call")
),
"followup_code_interpreter_completed": bool(
second and completed_calls(second, "code_interpreter_call")
),
"followup_citations_present": bool(second and url_citations(second)),
"followup_reports_ma_rsi_macd": all(
token in lowered for token in ("ma", "rsi", "macd")
),
}
return {"checks": checks, "passed": all(checks.values())}
def total_usage(results: Iterable[Dict[str, Any] | None]) -> Dict[str, Any]:
totals: Dict[str, float] = {
"input_tokens": 0,
"output_tokens": 0,
"total_tokens": 0,
"reported_cost_usd": 0.0,
}
cost_reported = False
for result in results:
usage = (result or {}).get("usage") or {}
for name in ("input_tokens", "output_tokens", "total_tokens"):
totals[name] += int(usage.get(name) or 0)
if usage.get("cost") is not None:
cost_reported = True
totals["reported_cost_usd"] += float(usage["cost"])
totals["reported_cost_available"] = cost_reported
if not cost_reported:
totals["reported_cost_usd"] = None
return totals
def run_backend(backend: str, reasoning: str) -> Dict[str, Any]:
key, base_url, model = Config.resolve(backend)
if not key:
return {"backend": backend, "started": False, "error": "credential_missing"}
asean_agent = GPT5NativeAgent(key, base_url=base_url, model=model)
asean = asean_agent.process_request(
ASEAN_TASK,
reasoning_effort=reasoning,
verbosity="high",
max_tokens=16000,
)
clarification_agent = GPT5NativeAgent(key, base_url=base_url, model=model)
first = clarification_agent.process_request(
AMBIGUOUS_TASK,
reasoning_effort="medium",
verbosity="medium",
max_tokens=4000,
)
second = None
if is_clarifying_question(first):
second = clarification_agent.process_request(
CLARIFICATION_REPLY,
reasoning_effort=reasoning,
verbosity="high",
max_tokens=16000,
)
return {
"backend": backend,
"started": True,
"base_url": base_url,
"requested_model": model,
"asean": asean,
"asean_validation": validate_asean(asean),
"clarification": {
"ambiguous_task": AMBIGUOUS_TASK,
"first": first,
"user_reply": CLARIFICATION_REPLY if second else None,
"second": second,
"validation": validate_clarification(first, second),
},
"api_turns": asean_agent.api_turns + clarification_agent.api_turns,
"usage": total_usage((asean, first, second)),
}
def acceptance(runs: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Multi-provider policy: any eligible backend may close the experiment."""
per_backend = {}
for run in runs:
backend = run.get("backend")
if not run.get("started"):
per_backend[backend] = {"started": False, "error": run.get("error")}
continue
per_backend[backend] = {
"started": True,
"requested_model": run.get("requested_model"),
"asean_passed": run.get("asean_validation", {}).get("passed") is True,
"clarification_passed": run.get("clarification", {})
.get("validation", {})
.get("passed")
is True,
}
accepting = next(
(
backend
for backend in ACCEPTANCE_BACKENDS
if per_backend.get(backend, {}).get("asean_passed")
and per_backend.get(backend, {}).get("clarification_passed")
),
None,
)
eligible_attempted = [
backend for backend in ACCEPTANCE_BACKENDS if backend in per_backend
]
return {
"policy": (
"multi-provider: acceptance is not gated on the official OpenAI "
"account; any provider whose Responses API closes the hosted "
"search + code-execution loop server-side is eligible"
),
"eligible_acceptance_backends": list(ACCEPTANCE_BACKENDS),
"eligible_backends_attempted": eligible_attempted,
"acceptance_backend": accepting,
"per_backend": per_backend,
"openrouter_is_diagnostic_not_acceptance": "openrouter" in per_backend,
"passed": accepting is not None,
"reference_docs": [
"https://developers.openai.com/api/docs/guides/tools-web-search",
"https://developers.openai.com/api/docs/guides/tools-code-interpreter",
"https://help.aliyun.com/zh/model-studio/qwen-code-interpreter",
],
}
def write_json(path: Path, value: Dict[str, Any]) -> str:
path.parent.mkdir(parents=True, exist_ok=True)
payload = json.dumps(value, ensure_ascii=False, indent=2) + "\n"
path.write_text(payload, encoding="utf-8")
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
def assert_credential_free(payloads: Iterable[str]) -> None:
"""Refuse to write evidence that embeds any configured API key."""
secrets = [
value
for value in (
Config.OPENAI_API_KEY,
Config.OPENROUTER_API_KEY,
Config.DASHSCOPE_API_KEY,
os.getenv("MOONSHOT_API_KEY", ""),
os.getenv("KIMI_API_KEY", ""),
os.getenv("ARK_API_KEY", ""),
os.getenv("SILICONFLOW_API_KEY", ""),
os.getenv("GEMINI_API_KEY", ""),
)
if value
]
for payload in payloads:
for secret in secrets:
if secret in payload:
raise SystemExit(
"Refusing to write evidence: an API key value appears in the payload"
)
if "authorization" in payload.lower() and "bearer" in payload.lower():
raise SystemExit(
"Refusing to write evidence: an Authorization header appears in the payload"
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--backends",
nargs="+",
choices=["openai", "openrouter", "dashscope"],
default=["openai", "dashscope"],
)
parser.add_argument(
"--reasoning", choices=["low", "medium", "high", "xhigh", "max"], default="high"
)
parser.add_argument("--output-dir", type=Path)
args = parser.parse_args()
runs = [run_backend(backend, args.reasoning) for backend in args.backends]
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
output_dir = args.output_dir or Path("validation") / "runs" / f"real_{stamp}"
evidence = {
"schema_version": "1.1",
"experiment_id": "1-3",
"evidence_mode": "real_api",
"created_at": datetime.now(timezone.utc).isoformat(),
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
"host": {
"platform": platform.platform(),
"python": sys.version,
"machine": platform.machine(),
},
"repository": {
"commit": git_value("rev-parse", "HEAD"),
"branch": git_value("branch", "--show-current"),
"worktree_dirty": bool(git_value("status", "--porcelain")),
},
"credentials_recorded": False,
"independent_asean_reference": independent_asean_reference(),
"runs": runs,
}
evidence["acceptance"] = acceptance(runs)
receipts = {
"schema_version": "1.0",
"experiment_id": "1-3",
"created_at": evidence["created_at"],
"note": "Raw credential-free provider turns; no API keys or Authorization headers.",
"turns": [
{"backend": run.get("backend"), "api_turns": run.get("api_turns") or []}
for run in runs
],
}
evidence_json = json.dumps(evidence, ensure_ascii=False, indent=2)
receipts_json = json.dumps(receipts, ensure_ascii=False, indent=2)
assert_credential_free((evidence_json, receipts_json))
evidence_path = output_dir / "evidence.json"
evidence_digest = write_json(evidence_path, evidence)
receipts_digest = write_json(output_dir / "receipts.json", receipts)
(output_dir / "evidence.sha256").write_text(
f"{evidence_digest} evidence.json\n", encoding="utf-8"
)
(output_dir / "receipts.sha256").write_text(
f"{receipts_digest} receipts.json\n", encoding="utf-8"
)
manifest = {
"schema_version": "1.0",
"experiment_id": "1-3",
"run_id": output_dir.name,
"created_at": evidence["created_at"],
"artifacts": {
"evidence.json": {"sha256": evidence_digest},
"receipts.json": {"sha256": receipts_digest},
},
"inputs": {
"canonical_source": evidence["canonical_source"],
"backends": args.backends,
"reasoning": args.reasoning,
},
"repository": evidence["repository"],
"acceptance_passed": evidence["acceptance"]["passed"],
"acceptance_backend": evidence["acceptance"]["acceptance_backend"],
}
manifest_digest = write_json(output_dir / "manifest.json", manifest)
Path("validation").mkdir(exist_ok=True)
shutil.copyfile(evidence_path, Path("validation/latest.json"))
latest = json.loads(Path("validation/latest.json").read_text(encoding="utf-8"))
latest["artifact_hashes"] = {
"evidence.json": evidence_digest,
"receipts.json": receipts_digest,
"manifest.json": manifest_digest,
"run_dir": str(output_dir),
}
Path("validation/latest.json").write_text(
json.dumps(latest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
)
print(json.dumps(evidence["acceptance"], ensure_ascii=False, indent=2))
print(f"Evidence: {evidence_path}")
return 0 if evidence["acceptance"]["passed"] else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,89 @@
import os
from pathlib import Path
import subprocess
import sys
import example_request
ROOT = Path(__file__).parent
def _import_config_with(value: str | None) -> subprocess.CompletedProcess[str]:
env = os.environ.copy()
if value is None:
env.pop("DEFAULT_MAX_TOKENS", None)
else:
env["DEFAULT_MAX_TOKENS"] = value
return subprocess.run(
[
sys.executable,
"-c",
"from config import Config; print(repr(Config.DEFAULT_MAX_TOKENS))",
],
cwd=ROOT,
env=env,
capture_output=True,
check=False,
text=True,
)
def test_default_max_tokens_import_accepts_only_values_int_can_parse():
for value in (None, "", " ", "4000.0", "abc", "²", "-1", "+1"):
result = _import_config_with(value)
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == "None"
result = _import_config_with(" 4000 ")
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == "4000"
def test_chat_completions_usage_uses_chat_token_and_detail_keys(monkeypatch, capsys):
payload = {
"usage": {
"prompt_tokens": 123,
"completion_tokens": 45,
"total_tokens": 168,
"prompt_tokens_details": {"cached_tokens": 7},
"completion_tokens_details": {"reasoning_tokens": 9},
}
}
class FakeResponse:
status_code = 200
def json(self):
return payload
monkeypatch.setattr(
example_request.requests,
"post",
lambda *args, **kwargs: FakeResponse(),
)
result = example_request.make_gpt5_openrouter_request("key", "system", "user")
output = capsys.readouterr().out
assert result == payload
assert "Input: 123 tokens (cached: 7)" in output
assert "Output: 45 tokens (reasoning: 9)" in output
assert "Total: 168" in output
payload = {
"usage": {
"input_tokens": 210,
"output_tokens": 34,
"total_tokens": 244,
"input_tokens_details": {"cached_tokens": 11},
"output_tokens_details": {"reasoning_tokens": 13},
}
}
result = example_request.make_gpt5_openrouter_request("key", "system", "user")
output = capsys.readouterr().out
assert result == payload
assert "Input: 210 tokens (cached: 11)" in output
assert "Output: 34 tokens (reasoning: 13)" in output
assert "Total: 244" in output
@@ -0,0 +1,192 @@
from agent import GPT5NativeAgent
from config import Config
from run_experiment_1_3 import (
acceptance,
independent_asean_reference,
validate_asean,
validate_clarification,
)
DASHSCOPE_URL = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
def test_request_uses_official_responses_tool_shapes():
agent = GPT5NativeAgent("key")
request = agent._build_responses_request(
"task", reasoning_effort="max", verbosity="high"
)
assert request["reasoning"] == {"effort": "max"}
assert request["text"] == {"verbosity": "high"}
assert request["tools"] == [
{"type": "web_search", "search_context_size": "medium"},
{
"type": "code_interpreter",
"container": {"type": "auto", "memory_limit": "4g"},
},
]
def test_dashscope_request_uses_hosted_tool_shapes_and_streaming():
agent = GPT5NativeAgent("key", base_url=DASHSCOPE_URL, model="qwen3.7-plus")
assert agent.provider == "dashscope"
request = agent._build_responses_request(
"task", reasoning_effort="high", verbosity="high"
)
# DashScope runs thinking natively: no reasoning.effort or text.verbosity,
# and streaming is mandatory because its gateway drops idle connections.
assert "reasoning" not in request
assert "text" not in request
assert request["stream"] is True
assert request["tools"] == [
{"type": "web_search"},
{"type": "code_interpreter"},
]
def test_config_resolves_dashscope_backend():
key, base_url, model = Config.resolve("dashscope")
assert base_url == DASHSCOPE_URL
assert model == Config.DASHSCOPE_MODEL
assert isinstance(key, str)
def test_dashscope_citations_from_web_search_sources():
agent = GPT5NativeAgent("key", base_url=DASHSCOPE_URL, model="qwen3.7-plus")
response = {
"output": [
{
"type": "web_search_call",
"status": "completed",
"action": {
"query": "ASEAN capitals",
"sources": [
{"type": "url", "url": "https://asean.test/one"},
{"type": "url", "url": "https://asean.test/two"},
],
},
}
]
}
citations = agent._citations(response)
assert citations == [
{"type": "url_citation", "url": "https://asean.test/one"},
{"type": "url_citation", "url": "https://asean.test/two"},
]
def test_dashscope_citations_from_web_search_string_url_sources():
agent = GPT5NativeAgent("key", base_url=DASHSCOPE_URL, model="qwen3.7-plus")
response = {
"output": [
{
"type": "web_search_call",
"status": "completed",
"action": {
"query": "ASEAN capitals",
"sources": [
"https://asean.test/one",
"https://asean.test/two",
],
},
}
]
}
citations = agent._citations(response)
assert citations == [
{"type": "url_citation", "url": "https://asean.test/one"},
{"type": "url_citation", "url": "https://asean.test/two"},
]
def test_independent_asean_reference_is_kuala_lumpur_singapore():
reference = independent_asean_reference()
assert reference["pair"] == ["Kuala Lumpur", "Singapore"]
assert reference["pair_count"] == 45
assert 250 < reference["distance_km"] < 400
def test_asean_acceptance_requires_both_completed_hosted_tools():
result = {
"success": True,
"requested_model": "gpt-5.6-sol",
"model": "gpt-5.6-sol",
"response": "Singapore and Kuala Lumpur are 316 km apart.",
"output_items": [
{"type": "web_search_call", "status": "completed"},
{"type": "code_interpreter_call", "status": "completed"},
],
"citations": [
{"type": "url_citation", "url": "https://one.test"},
{"type": "url_citation", "url": "https://two.test"},
],
}
assert validate_asean(result)["passed"] is True
result["output_items"] = result["output_items"][:1]
assert validate_asean(result)["passed"] is False
def test_asean_acceptance_rejects_model_substitution():
result = {
"success": True,
"requested_model": "qwen3.7-plus",
"model": "qwen3.7-flash",
"response": "Singapore and Kuala Lumpur are 316 km apart.",
"output_items": [
{"type": "web_search_call", "status": "completed"},
{"type": "code_interpreter_call", "status": "completed"},
],
"citations": [
{"type": "url_citation", "url": "https://one.test"},
{"type": "url_citation", "url": "https://two.test"},
],
}
assert validate_asean(result)["checks"]["model_identity_exact"] is False
assert validate_asean(result)["passed"] is False
def test_clarification_requires_no_tools_then_linked_tool_run():
first = {
"success": True,
"response": "Which source and indicators do you prefer?",
"tool_calls": [],
"response_id": "resp_1",
}
second = {
"success": True,
"request": {"previous_response_id": "resp_1"},
"response": "MA7, MA20, RSI14 and MACD(12,26,9) were computed.",
"output_items": [
{"type": "web_search_call", "status": "completed"},
{"type": "code_interpreter_call", "status": "completed"},
],
"citations": [{"type": "url_citation"}],
}
assert validate_clarification(first, second)["passed"] is True
second_without_indicators = dict(second, response="Here is the report.")
assert validate_clarification(first, second_without_indicators)["passed"] is False
def test_acceptance_is_multi_provider_not_openai_gated():
passing_run = {
"backend": "dashscope",
"started": True,
"requested_model": "qwen3.7-plus",
"asean_validation": {"passed": True},
"clarification": {"validation": {"passed": True}},
}
blocked_openai = {"backend": "openai", "started": True,
"requested_model": "gpt-5.6-sol",
"asean_validation": {"passed": False},
"clarification": {"validation": {"passed": False}}}
result = acceptance([blocked_openai, passing_run])
assert result["passed"] is True
assert result["acceptance_backend"] == "dashscope"
# OpenAI keeps priority when both pass.
blocked_openai["asean_validation"] = {"passed": True}
blocked_openai["clarification"] = {"validation": {"passed": True}}
result = acceptance([blocked_openai, passing_run])
assert result["acceptance_backend"] == "openai"
# OpenRouter alone can never close the experiment.
openrouter_run = dict(passing_run, backend="openrouter")
result = acceptance([openrouter_run])
assert result["passed"] is False
assert result["openrouter_is_diagnostic_not_acceptance"] is True
@@ -0,0 +1,363 @@
"""
Live manual cases for GPT-5 Native Tools Agent.
These cases demonstrate web_search with the OpenRouter format and require
OPENROUTER_API_KEY.
"""
import json
import logging
import sys
from typing import Dict, Any, List
from datetime import datetime
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[2]
if str(PROJECT_ROOT) not in sys.path:
sys.path.insert(0, str(PROJECT_ROOT))
from agent import GPT5NativeAgent, GPT5AgentChain
from config import Config
# Set up logging
logging.basicConfig(
level=getattr(logging, Config.LOG_LEVEL),
format=Config.LOG_FORMAT
)
logger = logging.getLogger(__name__)
class TestGPT5Agent:
"""Live manual case suite for GPT-5 Native Tools Agent"""
def __init__(self):
"""Initialize manual case suite"""
if not Config.validate():
raise ValueError("Invalid configuration. Please check your .env file")
self.agent = GPT5NativeAgent(
api_key=Config.OPENROUTER_API_KEY,
base_url=Config.OPENROUTER_BASE_URL,
model=Config.MODEL_NAME
)
self.results = []
def test_web_search_basic(self) -> Dict[str, Any]:
"""
Test Case 1: Basic web search
"""
print("\n" + "="*60)
print("TEST 1: Basic Web Search")
print("="*60)
request = """Search for the latest information about GPT-5 capabilities and features."""
result = self.agent.process_request(
request,
use_tools=True,
reasoning_effort="low"
)
self._print_result(result)
return result
def test_web_search_with_analysis(self) -> Dict[str, Any]:
"""
Test Case 2: Web search with analysis request
"""
print("\n" + "="*60)
print("TEST 2: Web Search with Analysis")
print("="*60)
request = """Search for current cryptocurrency market trends and Bitcoin price.
Then analyze the data to identify patterns and provide insights."""
result = self.agent.process_request(
request,
use_tools=True,
reasoning_effort="medium"
)
self._print_result(result)
return result
def test_complex_research(self) -> Dict[str, Any]:
"""
Test Case 3: Complex research task
"""
print("\n" + "="*60)
print("TEST 3: Complex Research Task")
print("="*60)
request = """Research the current state of renewable energy adoption globally.
Find statistics on solar, wind, and hydroelectric capacity.
Analyze growth trends and project future adoption rates.
Provide a comprehensive summary with data-driven insights."""
result = self.agent.process_request(
request,
use_tools=True,
reasoning_effort="high"
)
self._print_result(result)
return result
def test_search_and_code(self) -> Dict[str, Any]:
"""
Test Case 4: Search and code generation
"""
print("\n" + "="*60)
print("TEST 4: Search and Code Generation")
print("="*60)
request = """Search for the latest Python web frameworks in 2025.
Then create a simple comparison table and sample code for the top 3 frameworks."""
result = self.agent.process_request(
request,
use_tools=True,
reasoning_effort="medium"
)
self._print_result(result)
return result
def test_reasoning_efforts(self) -> List[Dict[str, Any]]:
"""
Test Case 5: Compare different reasoning efforts
"""
print("\n" + "="*60)
print("TEST 5: Reasoning Effort Comparison")
print("="*60)
request = "What are the implications of quantum computing on current encryption methods?"
results = []
for effort in ["low", "medium", "high"]:
print(f"\n--- Testing with {effort} reasoning effort ---")
result = self.agent.process_request(
request,
use_tools=True,
reasoning_effort=effort
)
self._print_result(result)
results.append({
"effort": effort,
"result": result
})
return results
def test_search_and_analyze_method(self) -> Dict[str, Any]:
"""
Test Case 6: Using the search_and_analyze convenience method
"""
print("\n" + "="*60)
print("TEST 6: Search and Analyze Method")
print("="*60)
analysis_code = """
# Analyze stock market data
import statistics
# Sample data processing
prices = [100, 102, 98, 105, 103, 107, 104]
returns = [(prices[i] - prices[i-1])/prices[i-1] * 100 for i in range(1, len(prices))]
avg_return = statistics.mean(returns)
volatility = statistics.stdev(returns)
print(f"Average Return: {avg_return:.2f}%")
print(f"Volatility: {volatility:.2f}%")
"""
result = self.agent.search_and_analyze(
topic="Current S&P 500 performance and market outlook for 2025",
analysis_code=analysis_code
)
self._print_result(result)
return result
def test_agent_chain(self) -> List[Dict[str, Any]]:
"""
Test Case 7: Chain multiple requests
"""
print("\n" + "="*60)
print("TEST 7: Agent Chain")
print("="*60)
chain = GPT5AgentChain(self.agent)
# Step 1: Research
chain.add_step(
"Search for information about the latest AI developments in 2025",
use_tools=True,
reasoning_effort="low"
)
# Step 2: Deep dive
chain.add_step(
"Based on the previous findings, search for more details about the most promising AI breakthrough",
use_tools=True,
reasoning_effort="medium"
)
# Step 3: Analysis
chain.add_step(
"Analyze the impact of these AI developments on various industries",
use_tools=True,
reasoning_effort="high"
)
results = chain.execute()
for i, step_result in enumerate(results, 1):
print(f"\n--- Chain Step {i} ---")
self._print_result(step_result["result"])
return results
def _print_result(self, result: Dict[str, Any]):
"""
Pretty print test result
Args:
result: Test result dictionary
"""
if result["success"]:
print(f"\n✅ Test Passed")
print(f"\nResponse Preview:")
print("-"*60)
response = result["response"]
if len(response) > 500:
print(response[:500] + "...")
else:
print(response)
print("-"*60)
if result.get("usage"):
usage = result["usage"]
print(f"\n📊 Token Usage:")
print(f" - Input: {usage.get('input_tokens', 'N/A')}")
print(f" - Output: {usage.get('output_tokens', 'N/A')}")
print(f" - Total: {usage.get('total_tokens', 'N/A')}")
if usage.get("input_tokens_details"):
print(f" - Cached: {usage['input_tokens_details'].get('cached_tokens', 0)}")
if usage.get("output_tokens_details"):
print(f" - Reasoning: {usage['output_tokens_details'].get('reasoning_tokens', 0)}")
else:
print(f"\n❌ Test Failed")
print(f"Error: {result.get('error', 'Unknown error')}")
def run_all_tests(self):
"""Run all live manual cases"""
print("\n" + "="*60)
print("RUNNING GPT-5 NATIVE TOOLS MANUAL CASES")
print(f"Timestamp: {datetime.now().isoformat()}")
print(f"Model: {Config.MODEL_NAME}")
print("="*60)
case_methods = [
("Basic Web Search", self.test_web_search_basic),
("Web Search with Analysis", self.test_web_search_with_analysis),
("Complex Research", self.test_complex_research),
("Search and Code", self.test_search_and_code),
("Reasoning Efforts", self.test_reasoning_efforts),
("Search and Analyze Method", self.test_search_and_analyze_method),
("Agent Chain", self.test_agent_chain)
]
results_summary = []
for case_name, case_method in case_methods:
try:
print(f"\n🧪 Running: {case_name}")
result = case_method()
# Handle different result types
if isinstance(result, list):
# For tests that return multiple results
if all(isinstance(r, dict) and "result" in r for r in result):
success = all(r["result"]["success"] for r in result)
else:
success = all(r.get("success", False) for r in result if isinstance(r, dict))
else:
success = result.get("success", False)
results_summary.append({
"case": case_name,
"success": success,
"result": result
})
except Exception as e:
logger.error(f"Manual case {case_name} failed with exception: {str(e)}")
results_summary.append({
"case": case_name,
"success": False,
"error": str(e)
})
# Print summary
print("\n" + "="*60)
print("MANUAL CASE SUMMARY")
print("="*60)
passed = sum(1 for r in results_summary if r["success"])
total = len(results_summary)
for result in results_summary:
status = "✅ PASS" if result["success"] else "❌ FAIL"
print(f"{result['case']}: {status}")
print(f"\nTotal: {passed}/{total} manual cases passed")
print("="*60)
return results_summary
def run_single_test(test_name: str = "basic"):
"""
Run a single live manual case
Args:
test_name: Name of manual case to run
"""
tester = TestGPT5Agent()
test_map = {
"basic": tester.test_web_search_basic,
"analysis": tester.test_web_search_with_analysis,
"complex": tester.test_complex_research,
"code": tester.test_search_and_code,
"reasoning": tester.test_reasoning_efforts,
"search_analyze": tester.test_search_and_analyze_method,
"chain": tester.test_agent_chain
}
if test_name in test_map:
test_map[test_name]()
else:
print(f"Unknown test: {test_name}")
print(f"Available tests: {', '.join(test_map.keys())}")
if __name__ == "__main__":
# Check configuration first
Config.display()
if not Config.validate():
print("\n❌ Configuration validation failed!")
print("Please set up your .env file with OPENROUTER_API_KEY")
sys.exit(1)
# Run manual cases
if len(sys.argv) > 1:
# Run specific test
run_single_test(sys.argv[1])
else:
# Run all tests
tester = TestGPT5Agent()
tester.run_all_tests()
File diff suppressed because one or more lines are too long
@@ -0,0 +1,482 @@
{
"schema_version": "1.0",
"experiment_id": "1-3",
"evidence_mode": "real_api",
"created_at": "2026-07-29T15:54:59.247653+00:00",
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
"host": {
"platform": "macOS-26.3-arm64-arm-64bit",
"python": "3.11.4 (main, Jul 5 2023, 08:40:20) [Clang 14.0.6 ]",
"machine": "arm64"
},
"repository": {
"commit": "4a7f37cf278bd15948c409f14533017c4c7fbc29",
"branch": "main",
"worktree_dirty": true
},
"credentials_recorded": false,
"runs": [
{
"backend": "openai",
"started": true,
"base_url": "https://api.openai.com/v1",
"requested_model": "gpt-5.6-sol",
"asean": {
"success": false,
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
},
"response": null,
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
"reasoning": {
"effort": "high"
},
"background": false,
"store": true,
"text": {
"verbosity": "high"
},
"max_output_tokens": 16000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"raw_response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"tool_calls": [],
"citations": [],
"usage": {},
"model": "gpt-5.6-sol",
"provider": "openai",
"base_url": "https://api.openai.com/v1",
"elapsed_seconds": 3.478408
},
"asean_validation": {
"checks": {
"request_succeeded": false,
"exact_gpt_5_6_sol": true,
"web_search_completed": false,
"code_interpreter_completed": false,
"url_citations_present": false,
"closest_pair_reported": false,
"distance_reported": false
},
"passed": false,
"output_types": []
},
"clarification": {
"ambiguous_task": "搜索最近一个月的比特币走势,做技术分析。",
"first": {
"success": false,
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
},
"response": null,
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "搜索最近一个月的比特币走势,做技术分析。",
"reasoning": {
"effort": "medium"
},
"background": false,
"store": true,
"text": {
"verbosity": "medium"
},
"max_output_tokens": 4000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"raw_response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"tool_calls": [],
"citations": [],
"usage": {},
"model": "gpt-5.6-sol",
"provider": "openai",
"base_url": "https://api.openai.com/v1",
"elapsed_seconds": 3.775702
},
"user_reply": null,
"second": null,
"validation": {
"checks": {
"first_turn_clarified_before_tools": false,
"continuation_used_previous_response_id": false,
"followup_succeeded": false,
"followup_web_search_completed": false,
"followup_code_interpreter_completed": false,
"followup_citations_present": false
},
"passed": false
}
},
"api_turns": [
{
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
"reasoning": {
"effort": "high"
},
"background": false,
"store": true,
"text": {
"verbosity": "high"
},
"max_output_tokens": 16000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"http_status": 429,
"response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"elapsed_seconds": 3.478408
},
{
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "搜索最近一个月的比特币走势,做技术分析。",
"reasoning": {
"effort": "medium"
},
"background": false,
"store": true,
"text": {
"verbosity": "medium"
},
"max_output_tokens": 4000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"http_status": 429,
"response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"elapsed_seconds": 3.775702
}
],
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"total_tokens": 0,
"reported_cost_usd": null,
"reported_cost_available": false
}
},
{
"backend": "openrouter",
"started": true,
"base_url": "https://openrouter.ai/api/v1",
"requested_model": "openai/gpt-5.6-sol",
"asean": {
"success": false,
"error": {
"message": "User not found.",
"code": 401
},
"response": null,
"request": {
"model": "openai/gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
"reasoning": {
"effort": "high"
},
"background": false,
"store": true,
"text": {
"verbosity": "high"
},
"max_output_tokens": 16000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"raw_response": {
"error": {
"message": "User not found.",
"code": 401
}
},
"tool_calls": [],
"citations": [],
"usage": {},
"model": "openai/gpt-5.6-sol",
"provider": "openrouter",
"base_url": "https://openrouter.ai/api/v1",
"elapsed_seconds": 1.115963
},
"asean_validation": {
"checks": {
"request_succeeded": false,
"exact_gpt_5_6_sol": true,
"web_search_completed": false,
"code_interpreter_completed": false,
"url_citations_present": false,
"closest_pair_reported": false,
"distance_reported": false
},
"passed": false,
"output_types": []
},
"clarification": {
"ambiguous_task": "搜索最近一个月的比特币走势,做技术分析。",
"first": {
"success": false,
"error": {
"message": "User not found.",
"code": 401
},
"response": null,
"request": {
"model": "openai/gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "搜索最近一个月的比特币走势,做技术分析。",
"reasoning": {
"effort": "medium"
},
"background": false,
"store": true,
"text": {
"verbosity": "medium"
},
"max_output_tokens": 4000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"raw_response": {
"error": {
"message": "User not found.",
"code": 401
}
},
"tool_calls": [],
"citations": [],
"usage": {},
"model": "openai/gpt-5.6-sol",
"provider": "openrouter",
"base_url": "https://openrouter.ai/api/v1",
"elapsed_seconds": 1.025438
},
"user_reply": null,
"second": null,
"validation": {
"checks": {
"first_turn_clarified_before_tools": false,
"continuation_used_previous_response_id": false,
"followup_succeeded": false,
"followup_web_search_completed": false,
"followup_code_interpreter_completed": false,
"followup_citations_present": false
},
"passed": false
}
},
"api_turns": [
{
"request": {
"model": "openai/gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
"reasoning": {
"effort": "high"
},
"background": false,
"store": true,
"text": {
"verbosity": "high"
},
"max_output_tokens": 16000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"http_status": 401,
"response": {
"error": {
"message": "User not found.",
"code": 401
}
},
"elapsed_seconds": 1.115963
},
{
"request": {
"model": "openai/gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "搜索最近一个月的比特币走势,做技术分析。",
"reasoning": {
"effort": "medium"
},
"background": false,
"store": true,
"text": {
"verbosity": "medium"
},
"max_output_tokens": 4000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"http_status": 401,
"response": {
"error": {
"message": "User not found.",
"code": 401
}
},
"elapsed_seconds": 1.025438
}
],
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"total_tokens": 0,
"reported_cost_usd": null,
"reported_cost_available": false
}
}
],
"acceptance": {
"checks": {
"official_openai_asean_passed": false,
"official_openai_clarification_passed": false
},
"passed": false,
"openrouter_is_diagnostic_not_official_acceptance": true,
"official_docs": [
"https://developers.openai.com/api/docs/models/gpt-5.6-sol",
"https://developers.openai.com/api/docs/guides/tools-web-search",
"https://developers.openai.com/api/docs/guides/tools-code-interpreter",
"https://developers.openai.com/api/docs/guides/model-guidance?model=gpt-5.6-sol"
]
}
}
@@ -0,0 +1 @@
530316ce5e4d5de004b8076dd57a364bc0173aeb371dbc87f09f4fa913d11942 evidence.json
@@ -0,0 +1,264 @@
{
"schema_version": "1.0",
"experiment_id": "1-3",
"evidence_mode": "real_api",
"created_at": "2026-07-29T19:39:13.370389+00:00",
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
"host": {
"platform": "macOS-26.3-arm64-arm-64bit",
"python": "3.11.4 (main, Jul 5 2023, 08:40:20) [Clang 14.0.6 ]",
"machine": "arm64"
},
"repository": {
"commit": "4a7f37cf278bd15948c409f14533017c4c7fbc29",
"branch": "main",
"worktree_dirty": true
},
"credentials_recorded": false,
"runs": [
{
"backend": "openai",
"started": true,
"base_url": "https://api.openai.com/v1",
"requested_model": "gpt-5.6-sol",
"asean": {
"success": false,
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
},
"response": null,
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
"reasoning": {
"effort": "high"
},
"background": false,
"store": true,
"text": {
"verbosity": "high"
},
"max_output_tokens": 16000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"raw_response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"tool_calls": [],
"citations": [],
"usage": {},
"model": "gpt-5.6-sol",
"provider": "openai",
"base_url": "https://api.openai.com/v1",
"elapsed_seconds": 3.900986
},
"asean_validation": {
"checks": {
"request_succeeded": false,
"exact_gpt_5_6_sol": true,
"web_search_completed": false,
"code_interpreter_completed": false,
"url_citations_present": false,
"closest_pair_reported": false,
"distance_reported": false
},
"passed": false,
"output_types": []
},
"clarification": {
"ambiguous_task": "搜索最近一个月的比特币走势,做技术分析。",
"first": {
"success": false,
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
},
"response": null,
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "搜索最近一个月的比特币走势,做技术分析。",
"reasoning": {
"effort": "medium"
},
"background": false,
"store": true,
"text": {
"verbosity": "medium"
},
"max_output_tokens": 4000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"raw_response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"tool_calls": [],
"citations": [],
"usage": {},
"model": "gpt-5.6-sol",
"provider": "openai",
"base_url": "https://api.openai.com/v1",
"elapsed_seconds": 3.851438
},
"user_reply": null,
"second": null,
"validation": {
"checks": {
"first_turn_clarified_before_tools": false,
"continuation_used_previous_response_id": false,
"followup_succeeded": false,
"followup_web_search_completed": false,
"followup_code_interpreter_completed": false,
"followup_citations_present": false
},
"passed": false
}
},
"api_turns": [
{
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
"reasoning": {
"effort": "high"
},
"background": false,
"store": true,
"text": {
"verbosity": "high"
},
"max_output_tokens": 16000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"http_status": 429,
"response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"elapsed_seconds": 3.900986
},
{
"request": {
"model": "gpt-5.6-sol",
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
"input": "搜索最近一个月的比特币走势,做技术分析。",
"reasoning": {
"effort": "medium"
},
"background": false,
"store": true,
"text": {
"verbosity": "medium"
},
"max_output_tokens": 4000,
"tools": [
{
"type": "web_search",
"search_context_size": "medium"
},
{
"type": "code_interpreter",
"container": {
"type": "auto",
"memory_limit": "4g"
}
}
],
"tool_choice": "auto"
},
"http_status": 429,
"response": {
"error": {
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
"type": "insufficient_quota",
"param": null,
"code": "insufficient_quota"
}
},
"elapsed_seconds": 3.851438
}
],
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"total_tokens": 0,
"reported_cost_usd": null,
"reported_cost_available": false
}
}
],
"acceptance": {
"checks": {
"official_openai_asean_passed": false,
"official_openai_clarification_passed": false
},
"passed": false,
"openrouter_is_diagnostic_not_official_acceptance": false,
"official_docs": [
"https://developers.openai.com/api/docs/models/gpt-5.6-sol",
"https://developers.openai.com/api/docs/guides/tools-web-search",
"https://developers.openai.com/api/docs/guides/tools-code-interpreter",
"https://developers.openai.com/api/docs/guides/model-guidance?model=gpt-5.6-sol"
]
}
}
@@ -0,0 +1 @@
e2364d62b234e9691d0f69d09ea24335b9d685e17178577dfe1bade3573dfa1a evidence.json
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
aab1864446613463c936154eca2fc65df8496869c65d4c17dc1f12628eaff3cc evidence.json
@@ -0,0 +1,29 @@
{
"schema_version": "1.0",
"experiment_id": "1-3",
"run_id": "real_20260731T170529Z",
"created_at": "2026-07-31T17:05:29.580761+00:00",
"artifacts": {
"evidence.json": {
"sha256": "aab1864446613463c936154eca2fc65df8496869c65d4c17dc1f12628eaff3cc"
},
"receipts.json": {
"sha256": "9247d69b5915ceff7d27b9dbc0752d63791fc26309f8d69e17844508058edcbe"
}
},
"inputs": {
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
"backends": [
"openai",
"dashscope"
],
"reasoning": "high"
},
"repository": {
"commit": "8c2b7f55e4bd2e5348b15b823dd23a179ef3decf",
"branch": "codex/exp1-3-multiprovider-20260731",
"worktree_dirty": true
},
"acceptance_passed": true,
"acceptance_backend": "dashscope"
}
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
9247d69b5915ceff7d27b9dbc0752d63791fc26309f8d69e17844508058edcbe receipts.json