ai-agent-book 精选快照(<2MB 代码与文档,来自 github.com/bojieli/ai-agent-book)
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,189 @@
|
||||
# GPT-5.6 Sol Deep Research / GPT-5.6 Sol 深度研究
|
||||
|
||||
> Responses API companion for Chapter 1, Experiment 1-3: hosted
|
||||
> `web_search` + hosted `code_interpreter`, typed tool traces, citations, and an
|
||||
> intent-clarification continuation. The canonical path is OpenAI GPT-5.6 Sol;
|
||||
> acceptance is multi-provider and may be closed by any provider whose
|
||||
> Responses API genuinely closes the search/code loop server-side — currently
|
||||
> Alibaba Model Studio (DashScope) `qwen3.7-plus`.
|
||||
|
||||
← [Chapter 1 index / 返回第 1 章目录](../README.md) ·
|
||||
📖 [Book experiment / 正文实验](../../book/chapter1.md)
|
||||
|
||||
## What this companion implements
|
||||
|
||||
The canonical path is the OpenAI **Responses API**, not a Chat Completions
|
||||
request that merely contains similarly named tool objects. The active agent in
|
||||
`agent.py` sends:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gpt-5.6-sol",
|
||||
"tools": [
|
||||
{"type": "web_search", "search_context_size": "medium"},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {"type": "auto", "memory_limit": "4g"}
|
||||
}
|
||||
],
|
||||
"reasoning": {"effort": "high"},
|
||||
"text": {"verbosity": "high"}
|
||||
}
|
||||
```
|
||||
|
||||
The DashScope backend speaks the same `/responses` protocol against
|
||||
`{DASHSCOPE_BASE_URL}/responses` with the provider's hosted-tool shapes:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "qwen3.7-plus",
|
||||
"tools": [{"type": "web_search"}, {"type": "code_interpreter"}],
|
||||
"stream": true
|
||||
}
|
||||
```
|
||||
|
||||
DashScope runs thinking natively (no `reasoning.effort`/`text.verbosity`
|
||||
knobs) and its gateway drops non-streaming requests that stay silent for
|
||||
about 60 seconds, so the backend always streams and keeps the final
|
||||
`response.completed` object, which has the same shape as a non-streaming
|
||||
response.
|
||||
|
||||
Acceptance is based on provider output items. A successful ASEAN-capitals run
|
||||
must contain completed `web_search_call` and `code_interpreter_call` items,
|
||||
clickable URL citations, and the computed closest pair. A text answer that says
|
||||
it used Python does not pass without the provider tool receipt.
|
||||
|
||||
The second scenario sends the deliberately ambiguous Bitcoin request used in
|
||||
the chapter, requires the first response to clarify material preferences before
|
||||
using tools, then continues with `previous_response_id` after the user supplies
|
||||
the data source and indicators.
|
||||
|
||||
## Current evidence status
|
||||
|
||||
Run the complete validator with:
|
||||
|
||||
```bash
|
||||
cd chapter1/search-codegen
|
||||
python run_experiment_1_3.py --backends openai dashscope --reasoning high
|
||||
```
|
||||
|
||||
The latest evidence is [validation/latest.json](validation/latest.json); raw
|
||||
credential-free receipts, a manifest, and SHA-256 sidecars live in
|
||||
`validation/runs/real_20260731T170529Z/`.
|
||||
|
||||
Result of the 2026-07-31 multi-provider acceptance run: **passed**, with
|
||||
`dashscope` (`qwen3.7-plus`) as the acceptance backend.
|
||||
|
||||
- ASEAN capitals: one hosted `web_search_call` batching ten model-issued
|
||||
coordinate queries, then a hosted `code_interpreter_call` that enumerated all
|
||||
45 haversine pairs and found Kuala Lumpur–Singapore at 316.35 km — the same
|
||||
pair as the independent local reference computed from standard coordinates.
|
||||
- Bitcoin technical analysis: the first turn asked which data source and which
|
||||
indicators to use **without calling any tool**; the continuation via
|
||||
`previous_response_id` ran 3 model-directed search rounds and 4 hosted
|
||||
`code_interpreter_call`s computing MA7/MA20, RSI14, MACD(12,26,9), period
|
||||
return and max drawdown, and plotted a close-price chart in the sandbox.
|
||||
- The official OpenAI `gpt-5.6-sol` path is still intact but remains
|
||||
quota-blocked: both calls returned `credit_balance_exhausted` before any
|
||||
hosted tool ran, which is recorded in the same evidence file.
|
||||
- Honest qualifications: the DashScope sandbox has no outbound network, so the
|
||||
daily closes were extracted through web search (the model disclosed this in
|
||||
its report); the chart PNG stays inside the sandbox because this Responses
|
||||
API returns execution logs only; and `qwen3.7-plus` only asks before acting
|
||||
when the system prompt carries an explicit clarify-first rule — the shipped
|
||||
prompt encodes it.
|
||||
- The OpenRouter route is retained strictly as a diagnostic and is never
|
||||
accepted. No fallback model, local Python replacement, fabricated tool
|
||||
trace, or Chat-Completions approximation is counted as fulfillment.
|
||||
|
||||
Earlier blocked attempts are kept under `validation/real_20260729T155459Z/`
|
||||
and `validation/real_20260730T033800Z/`.
|
||||
|
||||
## Setup and CLI
|
||||
|
||||
Python 3.9+ is required.
|
||||
|
||||
```bash
|
||||
# From the repository root: use the shared Chapter 1 environment
|
||||
uv sync --locked --extra ch1
|
||||
|
||||
# Activate it before changing directories:
|
||||
source .venv/bin/activate
|
||||
|
||||
# pip fallback when uv is not installed:
|
||||
# python -m pip install -e ".[ch1]"
|
||||
|
||||
cd chapter1/search-codegen
|
||||
|
||||
# Single-project compatibility path, still supported during migration:
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
export OPENAI_API_KEY=your-openai-api-key
|
||||
|
||||
# Exact official path
|
||||
python main.py --backend openai --mode single \
|
||||
--request "东盟 10 国首都之间最近的一对是哪两个?请搜索并用 Python 计算" \
|
||||
--reasoning high --verbosity high --output result.json
|
||||
|
||||
# Equivalent-provider path (eligible for acceptance): Alibaba Model Studio
|
||||
export DASHSCOPE_API_KEY=your-dashscope-api-key
|
||||
python main.py --backend dashscope --mode single \
|
||||
--request "东盟 10 国首都之间最近的一对是哪两个?请搜索并用 Python 计算" \
|
||||
--output result.json
|
||||
|
||||
# Inspect the exact request without an API call
|
||||
python main.py --backend openai --dry-run \
|
||||
--request "东盟 10 国首都之间最近的一对?" \
|
||||
--reasoning max --verbosity high
|
||||
|
||||
# Proxy diagnostic only; not canonical acceptance
|
||||
export OPENROUTER_API_KEY=your-openrouter-api-key
|
||||
python main.py --backend openrouter --mode single --request "Search current news"
|
||||
```
|
||||
|
||||
Important options:
|
||||
|
||||
| Option | Meaning |
|
||||
|---|---|
|
||||
| `--backend openai` | Canonical `https://api.openai.com/v1/responses` path |
|
||||
| `--backend dashscope` | Equivalent-provider path: DashScope Responses API, hosted `web_search` + `code_interpreter`, eligible for acceptance |
|
||||
| `--backend openrouter` | Explicit proxy diagnostic; never silently substituted |
|
||||
| `--reasoning` | `none`, `low`, `medium`, `high`, `xhigh`, or GPT-5.6 `max` |
|
||||
| `--verbosity` | Responses `text.verbosity`: `low`, `medium`, or `high` |
|
||||
| `--output` | Saves request, typed output items, citations, usage, and raw response |
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
python -m pytest -q test_responses_agent.py
|
||||
python -m py_compile agent.py config.py main.py run_experiment_1_3.py
|
||||
```
|
||||
|
||||
The validator checks exact model identity, direct-vs-proxy provenance, both
|
||||
hosted tool types, citations, clarification order, continuation linkage, token
|
||||
usage, reported provider cost when available, and credential-free raw evidence.
|
||||
|
||||
## Official sources
|
||||
|
||||
- [GPT-5.6 Sol model](https://developers.openai.com/api/docs/models/gpt-5.6-sol)
|
||||
- [Web search](https://developers.openai.com/api/docs/guides/tools-web-search)
|
||||
- [Code Interpreter](https://developers.openai.com/api/docs/guides/tools-code-interpreter)
|
||||
- [GPT-5.6 model guidance](https://developers.openai.com/api/docs/guides/model-guidance?model=gpt-5.6-sol)
|
||||
- [Alibaba Model Studio code interpreter (DashScope)](https://help.aliyun.com/zh/model-studio/qwen-code-interpreter)
|
||||
|
||||
## 中文说明
|
||||
|
||||
本项目使用正文所述的**精确协议**:Responses API、托管 `web_search` 与托管
|
||||
`code_interpreter`。验收依据是服务端返回的 `web_search_call` /
|
||||
`code_interpreter_call` 和 URL 引用,而不是代码里“声明了工具”或答案里
|
||||
“声称用过 Python”。
|
||||
|
||||
按作者批准的多提供商政策,验收不绑定官方 OpenAI 账号:官方 `gpt-5.6-sol` 路径
|
||||
完整保留(当前 Key 推理返回 `credit_balance_exhausted`,已在证据中如实记录),
|
||||
具备等价托管工具的提供商同样可以验收。2026-07-31 的正式运行用阿里云百炼
|
||||
`qwen3.7-plus`(DashScope Responses API)通过了全部验收门:东盟任务先搜索十个
|
||||
首都坐标、再用托管 Python 枚举 45 对大圆距离(吉隆坡—新加坡 316.35 km,与独立
|
||||
本地参考一致);比特币任务先在不用任何工具的情况下澄清数据源与指标,再通过
|
||||
`previous_response_id` 继续,完成 3 轮模型主导的搜索与 4 次托管代码执行
|
||||
(MA7/MA20、RSI14、MACD、区间收益、最大回撤与走势图)。OpenRouter 只作为诊断
|
||||
路径明确保留,不会被包装成替代品。
|
||||
@@ -0,0 +1,396 @@
|
||||
"""Exact GPT-5.6 Responses API agent for Experiment 1-3.
|
||||
|
||||
The previous companion sent Responses-style hosted tools to Chat Completions
|
||||
through a proxy and then reported an empty ``tool_calls`` list. This module
|
||||
uses the actual ``/v1/responses`` protocol and preserves its typed output items
|
||||
(``web_search_call``, ``code_interpreter_call``, messages, and citations).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Any, Dict, List, Literal, Optional
|
||||
|
||||
import requests
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GPT5NativeAgent:
|
||||
"""GPT-5.6 Sol with OpenAI-hosted web search and Python tools."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
api_key: str,
|
||||
base_url: str = "https://api.openai.com/v1",
|
||||
model: str = "gpt-5.6-sol",
|
||||
):
|
||||
if not api_key:
|
||||
raise ValueError("An API key is required")
|
||||
self.api_key = api_key
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.model = model
|
||||
self.provider = (
|
||||
"openai" if self.base_url == "https://api.openai.com/v1" else
|
||||
"openrouter" if "openrouter.ai" in self.base_url else
|
||||
"dashscope" if "dashscope" in self.base_url else
|
||||
"custom"
|
||||
)
|
||||
self.conversation_history: List[Dict[str, Any]] = []
|
||||
self.system_prompt = self._create_system_prompt()
|
||||
self.previous_response_id: Optional[str] = None
|
||||
self.api_turns: List[Dict[str, Any]] = []
|
||||
|
||||
@staticmethod
|
||||
def _create_system_prompt() -> str:
|
||||
return """You are a deep-research assistant. 你是一名深度研究助手。
|
||||
|
||||
Hard rule / 硬性规则: when the user's research request leaves material
|
||||
preferences ambiguous — for example which data source to use or which
|
||||
technical indicators to compute — ask a concise clarifying question FIRST
|
||||
(for example “您偏好使用哪个数据源?需要分析哪些技术指标?”), and do NOT
|
||||
call any tool until the user answers.
|
||||
当用户的研究请求没有明确数据来源或具体分析指标时,必须先向用户提问澄清,
|
||||
在用户回答之前不要调用任何工具。
|
||||
|
||||
After clarification, use hosted web search for current facts and cite
|
||||
sources, and use the hosted Python/code-interpreter tool for quantitative
|
||||
analysis; do not claim a calculation was run unless the response contains a
|
||||
completed code_interpreter_call.
|
||||
澄清之后:使用 web_search 获取最新事实并引用来源链接;所有定量计算必须通过
|
||||
code_interpreter 实际执行,不得口算或声称运行了代码。"""
|
||||
|
||||
def _tools(self) -> List[Dict[str, Any]]:
|
||||
if self.provider == "dashscope":
|
||||
# Exact structures from the Alibaba Model Studio Responses API guides.
|
||||
return [{"type": "web_search"}, {"type": "code_interpreter"}]
|
||||
# Exact structures from the official OpenAI Responses API guides.
|
||||
return [
|
||||
{"type": "web_search", "search_context_size": "medium"},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {"type": "auto", "memory_limit": "4g"},
|
||||
},
|
||||
]
|
||||
|
||||
def _build_responses_request(
|
||||
self,
|
||||
input_text: str,
|
||||
*,
|
||||
use_tools: bool = True,
|
||||
tool_choice: Literal["auto", "none", "required"] = "auto",
|
||||
reasoning_effort: str = "low",
|
||||
verbosity: Optional[str] = None,
|
||||
max_output_tokens: Optional[int] = None,
|
||||
background: bool = False,
|
||||
) -> Dict[str, Any]:
|
||||
if self.provider != "dashscope":
|
||||
if reasoning_effort not in {"none", "low", "medium", "high", "xhigh", "max"}:
|
||||
raise ValueError("Unsupported GPT-5.6 reasoning effort")
|
||||
if verbosity not in {None, "low", "medium", "high"}:
|
||||
raise ValueError("verbosity must be low, medium, or high")
|
||||
request: Dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"instructions": self.system_prompt,
|
||||
"input": input_text,
|
||||
}
|
||||
if self.provider == "dashscope":
|
||||
# DashScope runs thinking natively and has no reasoning.effort or
|
||||
# text.verbosity knobs; its gateway also drops non-streaming
|
||||
# requests that stay silent for ~60s, so streaming is mandatory.
|
||||
request["stream"] = True
|
||||
else:
|
||||
request["reasoning"] = {"effort": reasoning_effort}
|
||||
request["background"] = background
|
||||
request["store"] = True
|
||||
if verbosity:
|
||||
request["text"] = {"verbosity": verbosity}
|
||||
if max_output_tokens:
|
||||
request["max_output_tokens"] = max_output_tokens
|
||||
if use_tools:
|
||||
request["tools"] = self._tools()
|
||||
request["tool_choice"] = tool_choice
|
||||
if self.previous_response_id:
|
||||
request["previous_response_id"] = self.previous_response_id
|
||||
return request
|
||||
|
||||
@staticmethod
|
||||
def _output_text(response: Dict[str, Any]) -> str:
|
||||
if not isinstance(response, dict):
|
||||
return ""
|
||||
chunks: List[str] = []
|
||||
for item in response.get("output") or []:
|
||||
if not isinstance(item, dict) or item.get("type") != "message":
|
||||
continue
|
||||
for content in item.get("content") or []:
|
||||
if isinstance(content, dict) and content.get("type") == "output_text" and content.get("text"):
|
||||
chunks.append(content["text"])
|
||||
return "\n".join(chunks).strip()
|
||||
|
||||
@staticmethod
|
||||
def _tool_items(response: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
if not isinstance(response, dict):
|
||||
return []
|
||||
return [
|
||||
item
|
||||
for item in response.get("output") or []
|
||||
if isinstance(item, dict)
|
||||
and item.get("type") in {
|
||||
"web_search_call",
|
||||
"code_interpreter_call",
|
||||
"hosted_tool_call",
|
||||
}
|
||||
]
|
||||
|
||||
@staticmethod
|
||||
def _citations(response: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
citations = []
|
||||
if not isinstance(response, dict):
|
||||
return citations
|
||||
for item in response.get("output") or []:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
for content in item.get("content") or []:
|
||||
if not isinstance(content, dict):
|
||||
continue
|
||||
for annotation in content.get("annotations") or []:
|
||||
if isinstance(annotation, dict) and annotation.get("type") in {
|
||||
"url_citation",
|
||||
"container_file_citation",
|
||||
}:
|
||||
citations.append(annotation)
|
||||
# DashScope reports sources on the web_search_call item itself
|
||||
# instead of url_citation annotations; normalize them here.
|
||||
if item.get("type") == "web_search_call":
|
||||
action = item.get("action")
|
||||
if isinstance(action, dict):
|
||||
for source in action.get("sources") or []:
|
||||
url = source if isinstance(source, str) else (source.get("url") if isinstance(source, dict) else None)
|
||||
if url:
|
||||
citations.append(
|
||||
{"type": "url_citation", "url": url}
|
||||
)
|
||||
return citations
|
||||
|
||||
def _post_responses(
|
||||
self, request: Dict[str, Any]
|
||||
) -> tuple[int, Dict[str, Any], Optional[Dict[str, int]]]:
|
||||
"""Send one Responses request and return (status, body, stream_events).
|
||||
|
||||
DashScope requires streaming; the final ``response.completed`` event
|
||||
carries the same response object the non-streaming API returns, so both
|
||||
paths converge on an identical shape.
|
||||
"""
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self.api_key}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
if not request.get("stream"):
|
||||
http_response = requests.post(
|
||||
f"{self.base_url}/responses",
|
||||
headers=headers,
|
||||
json=request,
|
||||
timeout=900,
|
||||
)
|
||||
try:
|
||||
return http_response.status_code, http_response.json(), None
|
||||
except ValueError:
|
||||
return http_response.status_code, {"raw_text": http_response.text}, None
|
||||
|
||||
event_counts: Dict[str, int] = {}
|
||||
final_response: Optional[Dict[str, Any]] = None
|
||||
with requests.post(
|
||||
f"{self.base_url}/responses",
|
||||
headers=headers,
|
||||
json=request,
|
||||
stream=True,
|
||||
timeout=900,
|
||||
) as http_response:
|
||||
status_code = http_response.status_code
|
||||
if not http_response.ok:
|
||||
return status_code, {"raw_text": http_response.text}, event_counts
|
||||
for line in http_response.iter_lines(decode_unicode=True):
|
||||
if not line or not line.startswith("data:"):
|
||||
continue
|
||||
data = line[len("data:"):].strip()
|
||||
if data == "[DONE]":
|
||||
break
|
||||
try:
|
||||
event = json.loads(data)
|
||||
except ValueError:
|
||||
continue
|
||||
event_type = event.get("type") or "unknown"
|
||||
event_counts[event_type] = event_counts.get(event_type, 0) + 1
|
||||
if event_type in {"response.completed", "response.failed"}:
|
||||
final_response = event.get("response")
|
||||
if final_response is None:
|
||||
return status_code, {"error": {"type": "stream_incomplete",
|
||||
"message": "stream ended without response.completed"}}, event_counts
|
||||
return status_code, final_response, event_counts
|
||||
|
||||
def process_request(
|
||||
self,
|
||||
user_request: str,
|
||||
use_tools: bool = True,
|
||||
tool_choice: Literal["auto", "none", "required"] = "auto",
|
||||
temperature: float = 0.3,
|
||||
max_tokens: Optional[int] = None,
|
||||
reasoning_effort: str = "low",
|
||||
verbosity: Optional[str] = None,
|
||||
dry_run: bool = False,
|
||||
background: bool = False,
|
||||
) -> Dict[str, Any]:
|
||||
"""Create one Responses API turn and retain its complete trace.
|
||||
|
||||
``temperature`` remains in the signature for legacy callers, but is not
|
||||
sent: GPT-5.6 reasoning requests use ``reasoning.effort`` instead.
|
||||
"""
|
||||
request = self._build_responses_request(
|
||||
user_request,
|
||||
use_tools=use_tools,
|
||||
tool_choice=tool_choice,
|
||||
reasoning_effort=reasoning_effort,
|
||||
verbosity=verbosity,
|
||||
max_output_tokens=max_tokens,
|
||||
background=background,
|
||||
)
|
||||
if dry_run:
|
||||
return {
|
||||
"success": True,
|
||||
"dry_run": True,
|
||||
"request": request,
|
||||
"response": None,
|
||||
"tool_calls": [],
|
||||
"model": self.model,
|
||||
"provider": self.provider,
|
||||
}
|
||||
|
||||
started = time.monotonic()
|
||||
try:
|
||||
status_code, response, stream_events = self._post_responses(request)
|
||||
elapsed = round(time.monotonic() - started, 6)
|
||||
turn = {
|
||||
"request": json.loads(json.dumps(request, ensure_ascii=False)),
|
||||
"http_status": status_code,
|
||||
"response": response,
|
||||
"elapsed_seconds": elapsed,
|
||||
}
|
||||
if stream_events:
|
||||
turn["stream_event_counts"] = stream_events
|
||||
self.api_turns.append(turn)
|
||||
if not isinstance(response, dict) or status_code >= 400 or response.get("error"):
|
||||
error = (response.get("error") if isinstance(response, dict) else None) or {
|
||||
"type": "http_error",
|
||||
"message": (response.get("raw_text") if isinstance(response, dict) else None) or (json.dumps(response)[:500] if response is not None else "Empty response"),
|
||||
}
|
||||
return {
|
||||
"success": False,
|
||||
"error": error,
|
||||
"response": None,
|
||||
"request": request,
|
||||
"raw_response": response,
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": (response.get("usage") if isinstance(response, dict) else {}) or {},
|
||||
"model": self.model,
|
||||
"provider": self.provider,
|
||||
"base_url": self.base_url,
|
||||
"elapsed_seconds": elapsed,
|
||||
}
|
||||
|
||||
self.previous_response_id = response.get("id")
|
||||
text = self._output_text(response)
|
||||
self.conversation_history.extend(
|
||||
[
|
||||
{"role": "user", "content": user_request},
|
||||
{"role": "assistant", "content": text},
|
||||
]
|
||||
)
|
||||
return {
|
||||
"success": response.get("status") == "completed" and bool(text),
|
||||
"error": response.get("error"),
|
||||
"response": text,
|
||||
"request": request,
|
||||
"raw_response": response,
|
||||
"output_items": response.get("output") or [],
|
||||
"tool_calls": self._tool_items(response),
|
||||
"citations": self._citations(response),
|
||||
"usage": response.get("usage") or {},
|
||||
"model": response.get("model") or self.model,
|
||||
"requested_model": self.model,
|
||||
"provider": self.provider,
|
||||
"base_url": self.base_url,
|
||||
"response_id": response.get("id"),
|
||||
"status": response.get("status"),
|
||||
"elapsed_seconds": elapsed,
|
||||
"temperature_omitted_for_reasoning_model": temperature is not None,
|
||||
}
|
||||
except Exception as exc:
|
||||
elapsed = round(time.monotonic() - started, 6)
|
||||
self.api_turns.append(
|
||||
{
|
||||
"request": request,
|
||||
"elapsed_seconds": elapsed,
|
||||
"error": {"class": type(exc).__name__, "message": str(exc)},
|
||||
}
|
||||
)
|
||||
return {
|
||||
"success": False,
|
||||
"error": {"class": type(exc).__name__, "message": str(exc)},
|
||||
"response": None,
|
||||
"request": request,
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"model": self.model,
|
||||
"provider": self.provider,
|
||||
"base_url": self.base_url,
|
||||
"elapsed_seconds": elapsed,
|
||||
}
|
||||
|
||||
def search_and_analyze(
|
||||
self, topic: str, analysis_code: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
code_requirement = (
|
||||
f"Run this supplied Python in the hosted tool and inspect its output:\n{analysis_code}"
|
||||
if analysis_code
|
||||
else "Use the hosted Python tool for all quantitative processing."
|
||||
)
|
||||
return self.process_request(
|
||||
f"Research current information about {topic}. {code_requirement} "
|
||||
"Cite web sources and distinguish searched facts from computed results.",
|
||||
use_tools=True,
|
||||
reasoning_effort="medium",
|
||||
)
|
||||
|
||||
def clear_history(self) -> None:
|
||||
self.conversation_history = []
|
||||
self.previous_response_id = None
|
||||
self.api_turns = []
|
||||
|
||||
def get_history(self) -> List[Dict[str, Any]]:
|
||||
return json.loads(json.dumps(self.conversation_history, ensure_ascii=False))
|
||||
|
||||
def set_system_prompt(self, prompt: str) -> None:
|
||||
self.system_prompt = prompt
|
||||
|
||||
|
||||
class GPT5AgentChain:
|
||||
"""Sequential Responses turns linked with ``previous_response_id``."""
|
||||
|
||||
def __init__(self, agent: GPT5NativeAgent):
|
||||
self.agent = agent
|
||||
self.chain_results: List[Dict[str, Any]] = []
|
||||
|
||||
def add_step(self, request: str, **kwargs: Any) -> "GPT5AgentChain":
|
||||
self.chain_results.append(
|
||||
{"request": request, "result": self.agent.process_request(request, **kwargs)}
|
||||
)
|
||||
return self
|
||||
|
||||
def execute(self) -> List[Dict[str, Any]]:
|
||||
return self.chain_results
|
||||
|
||||
def clear(self) -> None:
|
||||
self.chain_results = []
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Configuration for the exact GPT-5.6 Responses API companion."""
|
||||
|
||||
import os
|
||||
from typing import Optional, Tuple
|
||||
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
def _optional_int_env(name: str) -> Optional[int]:
|
||||
"""Read an optional integer without making module import configuration-fatal."""
|
||||
raw_value = os.getenv(name)
|
||||
if raw_value is None:
|
||||
return None
|
||||
cleaned = raw_value.strip()
|
||||
if not cleaned or not cleaned.isascii() or not cleaned.isdecimal():
|
||||
return None
|
||||
try:
|
||||
return int(cleaned)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
class Config:
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")
|
||||
OPENAI_BASE_URL = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
|
||||
OPENROUTER_API_KEY = os.getenv("OPENROUTER_API_KEY", "")
|
||||
OPENROUTER_BASE_URL = os.getenv(
|
||||
"OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1"
|
||||
)
|
||||
DASHSCOPE_API_KEY = os.getenv("DASHSCOPE_API_KEY", "")
|
||||
DASHSCOPE_BASE_URL = os.getenv(
|
||||
"DASHSCOPE_BASE_URL",
|
||||
"https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
|
||||
)
|
||||
DASHSCOPE_MODEL = os.getenv("DASHSCOPE_MODEL", "qwen3.7-plus")
|
||||
BACKEND = os.getenv("BACKEND", "openai")
|
||||
MODEL_NAME = os.getenv("MODEL_NAME", "gpt-5.6-sol")
|
||||
DEFAULT_TEMPERATURE = 0.3 # legacy CLI compatibility; intentionally omitted
|
||||
DEFAULT_MAX_TOKENS: Optional[int] = _optional_int_env("DEFAULT_MAX_TOKENS")
|
||||
DEFAULT_TOOL_CHOICE = os.getenv("DEFAULT_TOOL_CHOICE", "auto")
|
||||
LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO")
|
||||
LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
RATE_LIMIT_RPM = int(os.getenv("RATE_LIMIT_RPM", "20"))
|
||||
MAX_RETRIES = int(os.getenv("MAX_RETRIES", "3"))
|
||||
RETRY_DELAY = float(os.getenv("RETRY_DELAY", "1.0"))
|
||||
WEB_SEARCH_MAX_RESULTS = int(os.getenv("WEB_SEARCH_MAX_RESULTS", "5"))
|
||||
CODE_INTERPRETER_TIMEOUT = int(os.getenv("CODE_INTERPRETER_TIMEOUT", "30"))
|
||||
|
||||
@classmethod
|
||||
def resolve(
|
||||
cls, backend: Optional[str] = None, model: Optional[str] = None
|
||||
) -> Tuple[str, str, str]:
|
||||
backend = backend or cls.BACKEND
|
||||
if backend == "openai":
|
||||
model = model or cls.MODEL_NAME
|
||||
return cls.OPENAI_API_KEY, cls.OPENAI_BASE_URL, model.removeprefix("openai/")
|
||||
if backend == "openrouter":
|
||||
model = model or cls.MODEL_NAME
|
||||
routed = model if model.startswith("openai/") else f"openai/{model}"
|
||||
return cls.OPENROUTER_API_KEY, cls.OPENROUTER_BASE_URL, routed
|
||||
if backend == "dashscope":
|
||||
return cls.DASHSCOPE_API_KEY, cls.DASHSCOPE_BASE_URL, model or cls.DASHSCOPE_MODEL
|
||||
raise ValueError("backend must be openai, openrouter, or dashscope")
|
||||
|
||||
@classmethod
|
||||
def validate(cls, backend: Optional[str] = None) -> bool:
|
||||
key, _, _ = cls.resolve(backend)
|
||||
if not key:
|
||||
print(f"Error: no API key for {backend or cls.BACKEND}")
|
||||
return False
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def display(cls, backend: Optional[str] = None) -> None:
|
||||
key, base_url, model = cls.resolve(backend)
|
||||
print("=== GPT-5.6 Responses Configuration ===")
|
||||
print(f"Backend: {backend or cls.BACKEND}")
|
||||
print(f"API Base URL: {base_url}")
|
||||
print(f"Model: {model}")
|
||||
print(f"API Key configured: {bool(key)}")
|
||||
|
||||
|
||||
def check_config() -> bool:
|
||||
return Config.validate()
|
||||
@@ -0,0 +1,44 @@
|
||||
# OpenRouter API Configuration
|
||||
# Get your API key from: https://openrouter.ai/keys
|
||||
# Note: this experiment uses OpenRouter as its PRIMARY backend (no fallback
|
||||
# needed). The same OPENROUTER_API_KEY also serves as the universal fallback
|
||||
# for the other chapter1 experiments (context, learning-from-experience,
|
||||
# web-search-agent) when their direct provider key is missing.
|
||||
OPENROUTER_API_KEY=your-openrouter-api-key
|
||||
|
||||
# API Base URL (default is fine for most users)
|
||||
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
|
||||
|
||||
# OpenAI official path (canonical GPT-5.6 Sol); needs an account with quota
|
||||
OPENAI_API_KEY=your-openai-api-key
|
||||
|
||||
# Alibaba Model Studio (DashScope) — eligible equivalent-provider path with
|
||||
# hosted web_search + hosted code_interpreter on the Responses API.
|
||||
# Get your API key from: https://www.alibabacloud.com/help/en/model-studio/get-api-key
|
||||
# Use https://dashscope.aliyuncs.com/compatible-mode/v1 for a China-region key.
|
||||
DASHSCOPE_API_KEY=your-dashscope-api-key
|
||||
DASHSCOPE_BASE_URL=https://dashscope-intl.aliyuncs.com/compatible-mode/v1
|
||||
DASHSCOPE_MODEL=qwen3.7-plus
|
||||
|
||||
# Model Configuration
|
||||
# GPT-5 model identifier on OpenRouter
|
||||
MODEL_NAME=openai/gpt-5.6-sol
|
||||
|
||||
# Request Configuration
|
||||
DEFAULT_TEMPERATURE=0.3
|
||||
DEFAULT_MAX_TOKENS=4000
|
||||
DEFAULT_TOOL_CHOICE=auto
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO
|
||||
|
||||
# Rate Limiting
|
||||
RATE_LIMIT_RPM=20
|
||||
|
||||
# Retry Configuration
|
||||
MAX_RETRIES=3
|
||||
RETRY_DELAY=1.0
|
||||
|
||||
# Tool-specific Configuration
|
||||
WEB_SEARCH_MAX_RESULTS=5
|
||||
CODE_INTERPRETER_TIMEOUT=30
|
||||
@@ -0,0 +1,249 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Example showing the exact OpenRouter GPT-5 request format matching the Go implementation
|
||||
"""
|
||||
|
||||
import json
|
||||
import requests
|
||||
import os
|
||||
from typing import Dict, Any
|
||||
|
||||
try:
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv()
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
def make_gpt5_openrouter_request(
|
||||
api_key: str,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
reasoning_effort: str = "low"
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Make a GPT-5 request using the exact format from the Go implementation
|
||||
|
||||
This matches the GPT5OpenRouterRequest structure from the Go code
|
||||
"""
|
||||
|
||||
# Build messages (matching Go implementation)
|
||||
messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": system_prompt
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": user_prompt
|
||||
}
|
||||
]
|
||||
|
||||
# Build web search tool configuration (matching Go GPT5OpenRouterWebSearchTool)
|
||||
web_search_tool = {
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium",
|
||||
"user_location": {
|
||||
"type": "approximate",
|
||||
"country": "US"
|
||||
}
|
||||
}
|
||||
|
||||
# Build request with OpenRouter-specific parameters (matching Go GPT5OpenRouterRequest)
|
||||
request_body = {
|
||||
"model": "openai/gpt-5.6-sol", # Default from Go code
|
||||
"messages": messages,
|
||||
"tools": [web_search_tool],
|
||||
"tool_choice": "auto",
|
||||
"parallel_tool_calls": True,
|
||||
"reasoning": {
|
||||
"effort": reasoning_effort,
|
||||
"generate_summary": False
|
||||
},
|
||||
"background": False,
|
||||
"stream": False # Can be set to True for streaming
|
||||
}
|
||||
|
||||
print("="*60)
|
||||
print("GPT-5 OpenRouter Request (matching Go implementation):")
|
||||
print("="*60)
|
||||
print(json.dumps(request_body, indent=2))
|
||||
print("="*60)
|
||||
|
||||
# Set headers (matching Go implementation)
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {api_key}"
|
||||
}
|
||||
|
||||
# Make the request
|
||||
url = "https://openrouter.ai/api/v1/chat/completions"
|
||||
|
||||
try:
|
||||
response = requests.post(
|
||||
url,
|
||||
headers=headers,
|
||||
json=request_body,
|
||||
timeout=600 # Match Go timeout
|
||||
)
|
||||
|
||||
print(f"\nResponse Status: {response.status_code}")
|
||||
|
||||
if response.status_code == 200:
|
||||
response_data = response.json()
|
||||
|
||||
# Log usage (matching Go logging)
|
||||
if "usage" in response_data:
|
||||
usage = response_data["usage"]
|
||||
input_tokens = usage.get(
|
||||
"prompt_tokens", usage.get("input_tokens", 0)
|
||||
)
|
||||
output_tokens = usage.get(
|
||||
"completion_tokens", usage.get("output_tokens", 0)
|
||||
)
|
||||
input_details = usage.get(
|
||||
"prompt_tokens_details", usage.get("input_tokens_details")
|
||||
)
|
||||
output_details = usage.get(
|
||||
"completion_tokens_details", usage.get("output_tokens_details")
|
||||
)
|
||||
print("\nGPT-5 OpenRouter Usage:")
|
||||
print(f" Input: {input_tokens} tokens", end="")
|
||||
if isinstance(input_details, dict):
|
||||
print(f" (cached: {input_details.get('cached_tokens', 0)})")
|
||||
else:
|
||||
print()
|
||||
|
||||
print(f" Output: {output_tokens} tokens", end="")
|
||||
if isinstance(output_details, dict):
|
||||
print(f" (reasoning: {output_details.get('reasoning_tokens', 0)})")
|
||||
else:
|
||||
print()
|
||||
|
||||
print(f" Total: {usage.get('total_tokens', 0)}")
|
||||
|
||||
return response_data
|
||||
else:
|
||||
print(f"\nError: {response.text}")
|
||||
return {"error": response.text, "status_code": response.status_code}
|
||||
|
||||
except Exception as e:
|
||||
print(f"\nException: {str(e)}")
|
||||
return {"error": str(e)}
|
||||
|
||||
|
||||
def demonstrate_streaming_response():
|
||||
"""
|
||||
Demonstrate how streaming would work (matching Go handleStreamingResponse)
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("Streaming Response Handler (pseudo-code matching Go):")
|
||||
print("="*60)
|
||||
|
||||
streaming_code = '''
|
||||
def handle_streaming_response(response):
|
||||
"""
|
||||
Handle streaming responses from GPT-5 OpenRouter API
|
||||
Matches Go handleStreamingResponse function
|
||||
"""
|
||||
content_builder = []
|
||||
reasoning_builder = []
|
||||
reasoning_token_count = 0
|
||||
|
||||
for line in response.iter_lines():
|
||||
if not line:
|
||||
continue
|
||||
|
||||
line_str = line.decode('utf-8')
|
||||
|
||||
if not line_str.startswith("data: "):
|
||||
continue
|
||||
|
||||
data = line_str[6:] # Remove "data: " prefix
|
||||
|
||||
if data == "[DONE]":
|
||||
break
|
||||
|
||||
try:
|
||||
chunk = json.loads(data)
|
||||
|
||||
if "choices" in chunk and len(chunk["choices"]) > 0:
|
||||
delta = chunk["choices"][0].get("delta", {})
|
||||
|
||||
# Check for reasoning content
|
||||
if "reasoning_content" in delta:
|
||||
reasoning = delta["reasoning_content"]
|
||||
reasoning_builder.append(reasoning)
|
||||
reasoning_token_count += 1
|
||||
print(f"🧠 [GPT-5 THINKING] {reasoning}")
|
||||
|
||||
# Check for regular content
|
||||
if "content" in delta:
|
||||
content = delta["content"]
|
||||
content_builder.append(content)
|
||||
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
final_content = "".join(content_builder)
|
||||
return final_content
|
||||
'''
|
||||
print(streaming_code)
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Main demonstration
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print(" GPT-5 OpenRouter Request Format Demo")
|
||||
print(" Exact match with Go implementation")
|
||||
print("="*60)
|
||||
|
||||
# Get API key from environment
|
||||
api_key = os.getenv("OPENROUTER_API_KEY")
|
||||
|
||||
if not api_key:
|
||||
print("\n❌ Error: OPENROUTER_API_KEY not found in environment")
|
||||
print("Please set: export OPENROUTER_API_KEY=your-openrouter-api-key")
|
||||
return
|
||||
|
||||
# Example prompts
|
||||
system_prompt = "You are a helpful AI assistant with web search capabilities."
|
||||
user_prompt = "What are the latest developments in artificial intelligence?"
|
||||
|
||||
print("\n1. Making request with LOW reasoning effort:")
|
||||
print("-"*60)
|
||||
result_low = make_gpt5_openrouter_request(
|
||||
api_key=api_key,
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=user_prompt,
|
||||
reasoning_effort="low"
|
||||
)
|
||||
|
||||
if "choices" in result_low:
|
||||
content = result_low["choices"][0]["message"]["content"]
|
||||
print(f"\nResponse preview: {content[:200]}...")
|
||||
|
||||
print("\n2. Making request with HIGH reasoning effort:")
|
||||
print("-"*60)
|
||||
result_high = make_gpt5_openrouter_request(
|
||||
api_key=api_key,
|
||||
system_prompt=system_prompt,
|
||||
user_prompt="Explain the implications of quantum computing on cryptography",
|
||||
reasoning_effort="high"
|
||||
)
|
||||
|
||||
if "choices" in result_high:
|
||||
content = result_high["choices"][0]["message"]["content"]
|
||||
print(f"\nResponse preview: {content[:200]}...")
|
||||
|
||||
# Show streaming handler
|
||||
demonstrate_streaming_response()
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("Demo complete! This shows the exact request format from Go.")
|
||||
print("="*60)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,116 @@
|
||||
# Go vs Python Implementation Comparison
|
||||
|
||||
This document shows how the Python implementation exactly matches the Go implementation for GPT-5 OpenRouter API calls.
|
||||
|
||||
## Request Structure Comparison
|
||||
|
||||
### Go Implementation
|
||||
```go
|
||||
// From the provided Go code
|
||||
webSearchTool := GPT5OpenRouterWebSearchTool{
|
||||
Type: "web_search",
|
||||
SearchContextSize: "medium",
|
||||
UserLocation: map[string]interface{}{
|
||||
"type": "approximate",
|
||||
"country": "US",
|
||||
},
|
||||
}
|
||||
|
||||
request := GPT5OpenRouterRequest{
|
||||
Model: c.model,
|
||||
Messages: messages,
|
||||
Tools: []GPT5OpenRouterWebSearchTool{webSearchTool},
|
||||
ToolChoice: "auto",
|
||||
ParallelToolCalls: true,
|
||||
Reasoning: &GPT5OpenRouterReasoning{
|
||||
Effort: reasoningEffort,
|
||||
GenerateSummary: false,
|
||||
},
|
||||
Background: false,
|
||||
Stream: false,
|
||||
}
|
||||
```
|
||||
|
||||
### Python Implementation
|
||||
```python
|
||||
# From agent.py
|
||||
web_search_tool = {
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium",
|
||||
"user_location": {
|
||||
"type": "approximate",
|
||||
"country": "US"
|
||||
}
|
||||
}
|
||||
|
||||
request_body = {
|
||||
"model": self.model,
|
||||
"messages": messages,
|
||||
"tools": [web_search_tool],
|
||||
"tool_choice": "auto",
|
||||
"parallel_tool_calls": True,
|
||||
"reasoning": {
|
||||
"effort": reasoning_effort,
|
||||
"generate_summary": False
|
||||
},
|
||||
"background": False,
|
||||
"stream": False
|
||||
}
|
||||
```
|
||||
|
||||
## Key Matching Points
|
||||
|
||||
1. **Tool Structure**: Both implementations use the same tool structure with `type: "web_search"` and additional configuration fields.
|
||||
|
||||
2. **Request Parameters**: Identical parameters including:
|
||||
- `model`
|
||||
- `messages`
|
||||
- `tools` (array of web_search tools)
|
||||
- `tool_choice: "auto"`
|
||||
- `parallel_tool_calls: true/True`
|
||||
- `reasoning` with effort and generate_summary
|
||||
- `background: false/False`
|
||||
- `stream: false/False`
|
||||
|
||||
3. **Headers**: Both use simple headers:
|
||||
```go
|
||||
// Go
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", c.apiKey))
|
||||
```
|
||||
|
||||
```python
|
||||
# Python
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {self.api_key}"
|
||||
}
|
||||
```
|
||||
|
||||
4. **Model Default**: Both default to `openai/gpt-5.6-sol`
|
||||
|
||||
5. **Reasoning Levels**: Both support "low", "medium", and "high" reasoning effort
|
||||
|
||||
## Usage Comparison
|
||||
|
||||
### Go
|
||||
```go
|
||||
client := NewGPT5OpenRouterClientAdapter(apiKey, baseURL, model)
|
||||
response, err := client.CallGPT5(ctx, systemPrompt, userPrompt, "medium")
|
||||
```
|
||||
|
||||
### Python
|
||||
```python
|
||||
agent = GPT5NativeAgent(api_key, base_url, model)
|
||||
result = agent.process_request(user_request, use_tools=True, reasoning_effort="medium")
|
||||
```
|
||||
|
||||
## Response Handling
|
||||
|
||||
Both implementations:
|
||||
- Handle streaming and non-streaming responses
|
||||
- Log token usage including cached and reasoning tokens
|
||||
- Extract content from the response choices
|
||||
- Handle errors with appropriate status codes
|
||||
|
||||
The Python implementation is a direct port of the Go implementation, ensuring complete compatibility with the OpenRouter GPT-5 API.
|
||||
@@ -0,0 +1,444 @@
|
||||
"""
|
||||
Main entry point for GPT-5 Native Tools Agent
|
||||
Interactive CLI for using web_search and code_interpreter tools
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import logging
|
||||
from typing import Optional
|
||||
from agent import GPT5NativeAgent, GPT5AgentChain
|
||||
from config import Config
|
||||
import argparse
|
||||
|
||||
# Set up logging
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, Config.LOG_LEVEL),
|
||||
format=Config.LOG_FORMAT
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class InteractiveCLI:
|
||||
"""Interactive command-line interface for GPT-5 Agent"""
|
||||
|
||||
def __init__(self, backend: str = None, model: str = None):
|
||||
"""Initialize the CLI"""
|
||||
if not Config.validate(backend):
|
||||
raise ValueError("Invalid configuration. Please check your .env file")
|
||||
api_key, base_url, resolved_model = Config.resolve(backend, model)
|
||||
self.agent = GPT5NativeAgent(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=resolved_model,
|
||||
)
|
||||
self.backend = backend or Config.BACKEND
|
||||
|
||||
self.commands = {
|
||||
"/help": self.show_help,
|
||||
"/clear": self.clear_history,
|
||||
"/history": self.show_history,
|
||||
"/tools": self.toggle_tools,
|
||||
"/search": self.search_mode,
|
||||
"/code": self.code_mode,
|
||||
"/analyze": self.analyze_mode,
|
||||
"/config": self.show_config,
|
||||
"/reasoning": self.set_reasoning_effort,
|
||||
"/exit": self.exit_cli,
|
||||
"/quit": self.exit_cli,
|
||||
}
|
||||
|
||||
self.use_tools = True
|
||||
self.tool_choice = "auto"
|
||||
self.reasoning_effort = "low" # Default reasoning effort
|
||||
|
||||
def show_help(self):
|
||||
"""Display help information"""
|
||||
help_text = """
|
||||
Commands:
|
||||
/help - Show this help message
|
||||
/clear - Clear conversation history
|
||||
/history - Show conversation history
|
||||
/tools - Toggle tools on/off
|
||||
/search - Enter web search mode
|
||||
/code - Enter code interpreter mode
|
||||
/analyze - Combined search + analysis mode
|
||||
/config - Show current configuration
|
||||
/reasoning - Set reasoning effort (low/medium/high)
|
||||
/exit - Exit the application
|
||||
|
||||
Native Tools:
|
||||
• web_search - Search the internet for real-time info
|
||||
• code_interpreter - Execute Python code and analyze
|
||||
|
||||
Usage:
|
||||
Simply type your request and the agent will use
|
||||
appropriate tools automatically.
|
||||
|
||||
Examples:
|
||||
"东盟 10 国首都之间,距离最近的两个首都是?给出你的详细分析推理过程。"
|
||||
"搜索最近一年比特币的价格,计算收益率、最大回撤、年化波动等重要指标"
|
||||
"""
|
||||
print(help_text)
|
||||
|
||||
def clear_history(self):
|
||||
"""Clear conversation history"""
|
||||
self.agent.clear_history()
|
||||
print("✅ Conversation history cleared")
|
||||
|
||||
def show_history(self):
|
||||
"""Display conversation history"""
|
||||
history = self.agent.get_history()
|
||||
if not history:
|
||||
print("📭 No conversation history")
|
||||
return
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("CONVERSATION HISTORY")
|
||||
print("="*60)
|
||||
|
||||
for i, msg in enumerate(history, 1):
|
||||
role = msg["role"].upper()
|
||||
content = msg["content"][:200] + "..." if len(msg["content"]) > 200 else msg["content"]
|
||||
print(f"\n[{i}] {role}:\n{content}")
|
||||
|
||||
print("="*60)
|
||||
|
||||
def toggle_tools(self):
|
||||
"""Toggle tool usage on/off"""
|
||||
self.use_tools = not self.use_tools
|
||||
status = "enabled" if self.use_tools else "disabled"
|
||||
print(f"🔧 Tools {status}")
|
||||
|
||||
def search_mode(self):
|
||||
"""Enter web search mode"""
|
||||
print("\n🔍 Web Search Mode")
|
||||
print("Enter your search query (or 'back' to return):")
|
||||
|
||||
query = input("> ").strip()
|
||||
if query.lower() == "back":
|
||||
return
|
||||
|
||||
request = f"Search the web for: {query}"
|
||||
self._process_request(request, force_tools=True)
|
||||
|
||||
def code_mode(self):
|
||||
"""Enter code interpreter mode"""
|
||||
print("\n💻 Code Interpreter Mode")
|
||||
print("Enter your code or computational request (or 'back' to return):")
|
||||
|
||||
request = input("> ").strip()
|
||||
if request.lower() == "back":
|
||||
return
|
||||
|
||||
enhanced_request = f"Use the code interpreter to: {request}"
|
||||
self._process_request(enhanced_request, force_tools=True)
|
||||
|
||||
def analyze_mode(self):
|
||||
"""Combined search and analysis mode"""
|
||||
print("\n🔬 Search & Analyze Mode")
|
||||
print("Enter topic to research and analyze (or 'back' to return):")
|
||||
|
||||
topic = input("> ").strip()
|
||||
if topic.lower() == "back":
|
||||
return
|
||||
|
||||
print("\nOptional: Enter Python code for analysis (press Enter to skip):")
|
||||
code = input("> ").strip()
|
||||
|
||||
if code:
|
||||
result = self.agent.search_and_analyze(topic, code)
|
||||
else:
|
||||
result = self.agent.search_and_analyze(topic)
|
||||
|
||||
self._display_result(result)
|
||||
|
||||
def show_config(self):
|
||||
"""Display current configuration"""
|
||||
Config.display()
|
||||
print(f"\nCurrent Settings:")
|
||||
print(f" Tools Enabled: {self.use_tools}")
|
||||
print(f" Tool Choice: {self.tool_choice}")
|
||||
print(f" Reasoning Effort: {self.reasoning_effort}")
|
||||
|
||||
def set_reasoning_effort(self):
|
||||
"""Set the reasoning effort level"""
|
||||
print("\n🧠 Set Reasoning Effort")
|
||||
print("Options: low, medium, high")
|
||||
print(f"Current: {self.reasoning_effort}")
|
||||
|
||||
effort = input("Enter new effort level: ").strip().lower()
|
||||
if effort in ["low", "medium", "high"]:
|
||||
self.reasoning_effort = effort
|
||||
print(f"✅ Reasoning effort set to: {effort}")
|
||||
else:
|
||||
print(f"❌ Invalid effort level. Must be low, medium, or high")
|
||||
|
||||
def exit_cli(self):
|
||||
"""Exit the application"""
|
||||
print("\n👋 Goodbye!")
|
||||
sys.exit(0)
|
||||
|
||||
def _process_request(self, request: str, force_tools: bool = False):
|
||||
"""
|
||||
Process a user request
|
||||
|
||||
Args:
|
||||
request: User request
|
||||
force_tools: Force tool usage regardless of settings
|
||||
"""
|
||||
use_tools = force_tools or self.use_tools
|
||||
|
||||
result = self.agent.process_request(
|
||||
request,
|
||||
use_tools=use_tools,
|
||||
tool_choice=self.tool_choice if use_tools else "none",
|
||||
temperature=Config.DEFAULT_TEMPERATURE,
|
||||
max_tokens=Config.DEFAULT_MAX_TOKENS,
|
||||
reasoning_effort=self.reasoning_effort
|
||||
)
|
||||
|
||||
self._display_result(result)
|
||||
|
||||
def _display_result(self, result: dict):
|
||||
"""
|
||||
Display the result of a request
|
||||
|
||||
Args:
|
||||
result: Result dictionary from agent
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
|
||||
if result["success"]:
|
||||
# Display tool usage
|
||||
if result["tool_calls"]:
|
||||
print("🔧 Tools Used:")
|
||||
for tool in result["tool_calls"]:
|
||||
print(f" • {tool.get('type', 'unknown_tool')}")
|
||||
print()
|
||||
|
||||
# Display response
|
||||
print("📝 Response:")
|
||||
print("-"*60)
|
||||
print(result["response"])
|
||||
print("-"*60)
|
||||
|
||||
# Display token usage
|
||||
if result.get("usage"):
|
||||
usage = result["usage"]
|
||||
total = usage.get("total_tokens", 0)
|
||||
if total:
|
||||
print(f"\n📊 Tokens used: {total}")
|
||||
else:
|
||||
print(f"❌ Error: {result.get('error', 'Unknown error')}")
|
||||
|
||||
print("="*60)
|
||||
|
||||
def run(self):
|
||||
"""Run the interactive CLI"""
|
||||
print("\n" + "="*60)
|
||||
print(" 🤖 GPT-5 Native Tools Agent")
|
||||
print(f" Responses API backend: {self.backend}")
|
||||
print("="*60)
|
||||
|
||||
self.show_help()
|
||||
|
||||
while True:
|
||||
try:
|
||||
print("\n💬 Enter your request (or /help for commands):")
|
||||
user_input = input("> ").strip()
|
||||
|
||||
if not user_input:
|
||||
continue
|
||||
|
||||
# Check for commands
|
||||
if user_input.startswith("/"):
|
||||
command = user_input.split()[0].lower()
|
||||
if command in self.commands:
|
||||
self.commands[command]()
|
||||
else:
|
||||
print(f"❌ Unknown command: {command}")
|
||||
print("Type /help for available commands")
|
||||
else:
|
||||
# Process as regular request
|
||||
self._process_request(user_input)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
print("\n\n⚠️ Interrupted. Type /exit to quit or continue chatting.")
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {str(e)}")
|
||||
print(f"❌ An error occurred: {str(e)}")
|
||||
|
||||
|
||||
def _run_single(args):
|
||||
"""执行单次请求(single / dry-run 模式),打印可读轨迹并按需保存结果。"""
|
||||
# dry-run 只组装请求体、不联网,因此无需真实 API Key
|
||||
api_key, base_url, model = Config.resolve(args.backend, args.model)
|
||||
api_key = api_key or ("DRYRUN-PLACEHOLDER" if args.dry_run else "")
|
||||
|
||||
agent = GPT5NativeAgent(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=model,
|
||||
)
|
||||
|
||||
result = agent.process_request(
|
||||
args.request,
|
||||
use_tools=not args.no_tools,
|
||||
temperature=Config.DEFAULT_TEMPERATURE,
|
||||
max_tokens=Config.DEFAULT_MAX_TOKENS,
|
||||
reasoning_effort=args.reasoning,
|
||||
verbosity=args.verbosity,
|
||||
dry_run=args.dry_run
|
||||
)
|
||||
|
||||
# dry-run:打印将要发送给模型的完整请求体(原生工具定义 + 参数)
|
||||
if result.get("dry_run"):
|
||||
print("\n" + "=" * 60)
|
||||
print("🧪 Dry-run:以下是发送给 GPT-5 的请求体(未联网)")
|
||||
print("=" * 60)
|
||||
print(f"Model: {result['model']}")
|
||||
print(f"任务: {args.request}")
|
||||
print("-" * 60)
|
||||
print(json.dumps(result["request"], indent=2, ensure_ascii=False))
|
||||
print("=" * 60)
|
||||
elif result["success"]:
|
||||
print("\n" + "=" * 60)
|
||||
print("📝 Response:")
|
||||
print("-" * 60)
|
||||
print(result["response"])
|
||||
print("-" * 60)
|
||||
usage = result.get("usage") or {}
|
||||
if usage:
|
||||
print(
|
||||
f"📊 Tokens - Input: {usage.get('input_tokens', 'N/A')}, "
|
||||
f"Output: {usage.get('output_tokens', 'N/A')}, "
|
||||
f"Reasoning: {usage.get('output_tokens_details', {}).get('reasoning_tokens', 0)}, "
|
||||
f"Total: {usage.get('total_tokens', 'N/A')}"
|
||||
)
|
||||
print("=" * 60)
|
||||
else:
|
||||
print(f"❌ Error: {result.get('error')}")
|
||||
|
||||
# 按需将完整结果(含轨迹/请求体)保存为 JSON,便于复盘
|
||||
if args.output:
|
||||
with open(args.output, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, indent=2, ensure_ascii=False)
|
||||
print(f"💾 结果已保存到: {args.output}")
|
||||
|
||||
if not result["success"]:
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
"""主入口:解析命令行参数并分派到交互 / 单次 / 测试模式。"""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="GPT-5 原生工具 Agent —— 演示实验 1.3:网络搜索 + 代码解释器的原生 Deep Research 能力",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""示例:
|
||||
python main.py # 交互模式(默认)
|
||||
python main.py --mode single --request "东盟 10 国首都之间距离最近的两个首都是?"
|
||||
python main.py --mode single --request "分析比特币近一月走势" --reasoning high --verbosity high
|
||||
python main.py --mode single --request "..." --output result.json
|
||||
python main.py --dry-run --request "..." # 离线查看请求体(原生工具定义),无需 API Key
|
||||
python main.py --mode test --test basic # 运行指定联网手动用例
|
||||
""",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--mode",
|
||||
choices=["interactive", "single", "test"],
|
||||
default="interactive",
|
||||
help="运行模式:interactive 交互对话(默认)/ single 单次请求 / test 联网手动用例",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--request",
|
||||
type=str,
|
||||
help="single / dry-run 模式下的任务或查询内容",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--backend",
|
||||
choices=["openai", "openrouter", "dashscope"],
|
||||
default=Config.BACKEND,
|
||||
help="Responses API backend; openai is the exact canonical path, dashscope is the eligible equivalent-provider path",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--model",
|
||||
type=str,
|
||||
default=None,
|
||||
help=f"覆盖模型名称(默认取配置 {Config.MODEL_NAME})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--reasoning",
|
||||
choices=["none", "low", "medium", "high", "xhigh", "max"],
|
||||
default="low",
|
||||
help="推理力度 Reasoning Effort(low/medium/high,默认 low)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbosity",
|
||||
choices=["low", "medium", "high"],
|
||||
default=None,
|
||||
help="输出详略程度 Verbosity(low/medium/high,默认跟随模型)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-tools",
|
||||
action="store_true",
|
||||
help="禁用原生工具(web_search / code_interpreter)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
type=str,
|
||||
default=None,
|
||||
help="将完整结果(含轨迹 / 请求体)保存为 JSON 文件的路径",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="离线组装并打印请求体(含原生工具定义),不调用 API、无需 API Key",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--test",
|
||||
type=str,
|
||||
help="test 模式下运行指定联网手动用例(basic/analysis/complex/code/reasoning/search_analyze/chain)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# dry-run:离线路径,跳过 API Key 校验
|
||||
if args.dry_run:
|
||||
if not args.request:
|
||||
print("❌ --dry-run 需要配合 --request 使用")
|
||||
sys.exit(1)
|
||||
_run_single(args)
|
||||
return
|
||||
|
||||
# 其余模式需要有效配置
|
||||
if not Config.validate(args.backend):
|
||||
print("❌ 配置错误!")
|
||||
print("请配置所选 backend 对应的 OPENAI_API_KEY / OPENROUTER_API_KEY / DASHSCOPE_API_KEY")
|
||||
print("\n示例 .env:")
|
||||
print("DASHSCOPE_API_KEY=your-dashscope-api-key")
|
||||
sys.exit(1)
|
||||
|
||||
if args.mode == "interactive":
|
||||
cli = InteractiveCLI(args.backend, args.model)
|
||||
cli.run()
|
||||
|
||||
elif args.mode == "single":
|
||||
if not args.request:
|
||||
print("❌ single 模式需要 --request 参数")
|
||||
sys.exit(1)
|
||||
_run_single(args)
|
||||
|
||||
elif args.mode == "test":
|
||||
from tests.manual.agent_cases import TestGPT5Agent, run_single_test
|
||||
|
||||
if args.test:
|
||||
run_single_test(args.test)
|
||||
else:
|
||||
tester = TestGPT5Agent()
|
||||
tester.run_all_tests()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Quick Start Demo for GPT-5 Native Tools Agent
|
||||
Demonstrates basic usage of web_search and code_interpreter tools
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
from agent import GPT5NativeAgent
|
||||
from config import Config
|
||||
|
||||
def demo_web_search():
|
||||
"""Demonstrate web search capability"""
|
||||
print("\n" + "="*60)
|
||||
print("DEMO: Web Search Tool")
|
||||
print("="*60)
|
||||
|
||||
agent = GPT5NativeAgent(
|
||||
api_key=Config.OPENROUTER_API_KEY,
|
||||
base_url=Config.OPENROUTER_BASE_URL
|
||||
)
|
||||
|
||||
result = agent.process_request(
|
||||
"What are the latest developments in GPT-5 and its capabilities?",
|
||||
use_tools=True,
|
||||
reasoning_effort="low"
|
||||
)
|
||||
|
||||
if result["success"]:
|
||||
print("\n✅ Web Search Result:")
|
||||
print(result["response"][:500] + "...")
|
||||
if result["tool_calls"]:
|
||||
print(f"\n🔧 Tools used: {len(result['tool_calls'])}")
|
||||
else:
|
||||
print(f"❌ Error: {result['error']}")
|
||||
|
||||
def demo_code_interpreter():
|
||||
"""Demonstrate code generation and analysis capability"""
|
||||
print("\n" + "="*60)
|
||||
print("DEMO: Code Generation and Analysis")
|
||||
print("="*60)
|
||||
|
||||
agent = GPT5NativeAgent(
|
||||
api_key=Config.OPENROUTER_API_KEY,
|
||||
base_url=Config.OPENROUTER_BASE_URL
|
||||
)
|
||||
|
||||
result = agent.process_request(
|
||||
"""Create Python code to:
|
||||
1. Generate the first 20 Fibonacci numbers
|
||||
2. Calculate their sum and average
|
||||
3. Find the golden ratio approximation using consecutive pairs
|
||||
4. Explain the mathematical significance""",
|
||||
use_tools=True,
|
||||
reasoning_effort="medium"
|
||||
)
|
||||
|
||||
if result["success"]:
|
||||
print("\n✅ Code and Analysis Result:")
|
||||
print(result["response"][:500] + "...")
|
||||
if result["tool_calls"]:
|
||||
print(f"\n🔧 Tools used: {len(result['tool_calls'])}")
|
||||
else:
|
||||
print(f"❌ Error: {result['error']}")
|
||||
|
||||
def demo_combined_tools():
|
||||
"""Demonstrate using both tools together"""
|
||||
print("\n" + "="*60)
|
||||
print("DEMO: Combined Web Search + Code Analysis")
|
||||
print("="*60)
|
||||
|
||||
agent = GPT5NativeAgent(
|
||||
api_key=Config.OPENROUTER_API_KEY,
|
||||
base_url=Config.OPENROUTER_BASE_URL
|
||||
)
|
||||
|
||||
result = agent.search_and_analyze(
|
||||
topic="Current S&P 500 performance and major tech stocks",
|
||||
analysis_code="""
|
||||
# Analyze market data
|
||||
import random
|
||||
import statistics
|
||||
|
||||
# Simulate stock prices based on search results
|
||||
stocks = {
|
||||
'AAPL': [175 + random.uniform(-5, 5) for _ in range(10)],
|
||||
'GOOGL': [140 + random.uniform(-3, 3) for _ in range(10)],
|
||||
'MSFT': [380 + random.uniform(-8, 8) for _ in range(10)]
|
||||
}
|
||||
|
||||
# Calculate metrics
|
||||
for symbol, prices in stocks.items():
|
||||
avg = statistics.mean(prices)
|
||||
vol = statistics.stdev(prices)
|
||||
trend = "↑" if prices[-1] > prices[0] else "↓"
|
||||
print(f"{symbol}: Avg=${avg:.2f}, Volatility=${vol:.2f}, Trend={trend}")
|
||||
"""
|
||||
)
|
||||
|
||||
if result["success"]:
|
||||
print("\n✅ Combined Analysis Result:")
|
||||
print(result["response"][:500] + "...")
|
||||
if result["tool_calls"]:
|
||||
print(f"\n🔧 Tools used: {len(result['tool_calls'])}")
|
||||
else:
|
||||
print(f"❌ Error: {result['error']}")
|
||||
|
||||
def main():
|
||||
"""Run all demos"""
|
||||
print("\n" + "="*60)
|
||||
print(" GPT-5 Native Tools Agent - Quick Start Demo")
|
||||
print("="*60)
|
||||
|
||||
# Check configuration
|
||||
if not Config.validate():
|
||||
print("\n❌ Configuration Error!")
|
||||
print("Please set up your .env file with OPENROUTER_API_KEY")
|
||||
print("\nSteps:")
|
||||
print("1. Copy env.example to .env")
|
||||
print("2. Add your OpenRouter API key")
|
||||
print("3. Get a key at: https://openrouter.ai/keys")
|
||||
sys.exit(1)
|
||||
|
||||
print("\n✅ Configuration valid")
|
||||
print(f"Using model: {Config.MODEL_NAME}")
|
||||
|
||||
# Ask user which demo to run
|
||||
print("\nSelect demo to run:")
|
||||
print("1. Web Search only")
|
||||
print("2. Code Generation and Analysis")
|
||||
print("3. Combined Tools")
|
||||
print("4. All demos")
|
||||
|
||||
choice = input("\nEnter choice (1-4): ").strip()
|
||||
|
||||
if choice == "1":
|
||||
demo_web_search()
|
||||
elif choice == "2":
|
||||
demo_code_interpreter()
|
||||
elif choice == "3":
|
||||
demo_combined_tools()
|
||||
elif choice == "4":
|
||||
demo_web_search()
|
||||
demo_code_interpreter()
|
||||
demo_combined_tools()
|
||||
else:
|
||||
print("Invalid choice. Running all demos...")
|
||||
demo_web_search()
|
||||
demo_code_interpreter()
|
||||
demo_combined_tools()
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("Demo complete! 🎉")
|
||||
print("\nNext steps:")
|
||||
print("- Run 'python main.py' for interactive mode")
|
||||
print("- Run 'python main.py --mode test' for live manual cases")
|
||||
print("- Check README.md for more examples")
|
||||
print("="*60)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,24 @@
|
||||
# GPT-5 Native Tools Agent Requirements
|
||||
|
||||
# Core dependencies
|
||||
openai>=1.35.0 # OpenAI client library (works with OpenRouter)
|
||||
python-dotenv>=1.0.0 # Environment variable management
|
||||
requests>=2.31.0 # HTTP requests for OpenRouter API calls
|
||||
pydantic>=2.5.0 # Data validation and settings management
|
||||
|
||||
# Development and testing
|
||||
pytest>=7.4.0 # Testing framework
|
||||
pytest-asyncio>=0.21.0 # Async test support
|
||||
black>=23.0.0 # Code formatting
|
||||
flake8>=6.1.0 # Linting
|
||||
mypy>=1.7.0 # Type checking
|
||||
|
||||
# Data processing (used by code_interpreter examples)
|
||||
numpy>=1.24.0 # Numerical computing
|
||||
pandas>=2.0.0 # Data analysis
|
||||
matplotlib>=3.7.0 # Plotting (for code interpreter visualizations)
|
||||
|
||||
# Utilities
|
||||
rich>=13.5.0 # Rich terminal output
|
||||
tqdm>=4.66.0 # Progress bars
|
||||
colorama>=0.4.6 # Cross-platform colored terminal text
|
||||
@@ -0,0 +1,447 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run Experiment 1-3 on a hosted web-search + code-execution Responses API.
|
||||
|
||||
Acceptance policy (author-mandated, 2026-07-31): the experiment's essence is
|
||||
model-directed multi-round web search + hosted code execution, clarification
|
||||
before tools, and a current answer with authoritative sources. The canonical
|
||||
OpenAI GPT-5.6 Sol path remains the reference implementation, but acceptance
|
||||
is NOT gated on the official OpenAI account: any provider whose Responses API
|
||||
genuinely closes the search/code loop server-side (currently Alibaba Model
|
||||
Studio DashScope ``qwen3.7-plus``) is an eligible acceptance backend. The
|
||||
OpenRouter route stays a diagnostic and is never accepted.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import itertools
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Iterable, List, Optional, Tuple
|
||||
|
||||
from agent import GPT5NativeAgent
|
||||
from config import Config
|
||||
|
||||
|
||||
ASEAN_TASK = """Research the current official capitals and reliable coordinates
|
||||
for the ten ASEAN member states. You must use hosted web search and cite the
|
||||
sources. Then you must use the hosted Python tool—not mental arithmetic—to
|
||||
enumerate all 45 capital pairs with the haversine formula and identify the
|
||||
closest pair and distance. Include the coordinates, formula assumptions,
|
||||
calculation result, retrieval date, and clickable citations. Do not say Python
|
||||
was used unless a code_interpreter_call completes."""
|
||||
|
||||
AMBIGUOUS_TASK = "搜索最近一个月的比特币走势,做技术分析。"
|
||||
CLARIFICATION_REPLY = (
|
||||
"使用 CoinGecko 的 BTC/USD 日线收盘价;分析 MA7、MA20、RSI14、MACD(12,26,9)、"
|
||||
"区间收益和最大回撤,如代码环境支持请绘制收盘价走势图。请搜索数据并用托管 "
|
||||
"Python 工具实际计算,再给出含来源的报告和交易建议。"
|
||||
)
|
||||
|
||||
# Backends whose runs may close the experiment, in priority order. The
|
||||
# OpenRouter proxy is diagnostic-only and never appears here.
|
||||
ACCEPTANCE_BACKENDS = ("openai", "dashscope")
|
||||
|
||||
# Independent reference: standard coordinates of the ten ASEAN capitals,
|
||||
# used to verify the model's computed nearest pair without trusting it.
|
||||
ASEAN_CAPITAL_COORDS: Dict[str, Tuple[float, float]] = {
|
||||
"Bandar Seri Begawan": (4.9031, 114.9398),
|
||||
"Phnom Penh": (11.5564, 104.9282),
|
||||
"Jakarta": (-6.2088, 106.8456),
|
||||
"Vientiane": (17.9757, 102.6331),
|
||||
"Kuala Lumpur": (3.1390, 101.6869),
|
||||
"Naypyidaw": (19.7633, 96.0785),
|
||||
"Manila": (14.5995, 120.9842),
|
||||
"Singapore": (1.3521, 103.8198),
|
||||
"Bangkok": (13.7563, 100.5018),
|
||||
"Hanoi": (21.0278, 105.8342),
|
||||
}
|
||||
|
||||
|
||||
def haversine_km(a: Tuple[float, float], b: Tuple[float, float]) -> float:
|
||||
radius = 6371.0088
|
||||
lat1, lon1 = map(math.radians, a)
|
||||
lat2, lon2 = map(math.radians, b)
|
||||
dlat, dlon = lat2 - lat1, lon2 - lon1
|
||||
h = math.sin(dlat / 2) ** 2 + math.cos(lat1) * math.cos(lat2) * math.sin(dlon / 2) ** 2
|
||||
return 2 * radius * math.asin(math.sqrt(h))
|
||||
|
||||
|
||||
def independent_asean_reference() -> Dict[str, Any]:
|
||||
"""Locally computed ground truth for the ASEAN nearest-pair check."""
|
||||
pairs = [
|
||||
(haversine_km(ca, cb), a, b)
|
||||
for (a, ca), (b, cb) in itertools.combinations(ASEAN_CAPITAL_COORDS.items(), 2)
|
||||
]
|
||||
distance, first, second = min(pairs)
|
||||
return {
|
||||
"pair": sorted([first, second]),
|
||||
"distance_km": round(distance, 1),
|
||||
"pair_count": len(pairs),
|
||||
"coordinates": ASEAN_CAPITAL_COORDS,
|
||||
}
|
||||
|
||||
|
||||
def git_value(*args: str) -> str | None:
|
||||
try:
|
||||
return subprocess.check_output(
|
||||
["git", *args], text=True, stderr=subprocess.DEVNULL
|
||||
).strip()
|
||||
except (OSError, subprocess.CalledProcessError):
|
||||
return None
|
||||
|
||||
|
||||
def output_types(result: Dict[str, Any]) -> List[str]:
|
||||
return [item.get("type") for item in result.get("output_items") or []]
|
||||
|
||||
|
||||
def completed_calls(result: Dict[str, Any], kind: str) -> List[Dict[str, Any]]:
|
||||
return [
|
||||
item
|
||||
for item in result.get("output_items") or []
|
||||
if item.get("type") == kind and item.get("status") == "completed"
|
||||
]
|
||||
|
||||
|
||||
def url_citations(result: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
return [
|
||||
item for item in result.get("citations") or [] if item.get("type") == "url_citation"
|
||||
]
|
||||
|
||||
|
||||
def model_identity_exact(result: Dict[str, Any]) -> bool:
|
||||
"""The returned model must be exactly the requested model."""
|
||||
requested = (result.get("requested_model") or result.get("model") or "").removeprefix(
|
||||
"openai/"
|
||||
)
|
||||
returned = (result.get("model") or "").removeprefix("openai/")
|
||||
return bool(requested) and requested == returned
|
||||
|
||||
|
||||
def validate_asean(
|
||||
result: Dict[str, Any], reference: Optional[Dict[str, Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
answer = result.get("response") or ""
|
||||
reference = reference or independent_asean_reference()
|
||||
pair_city, other_city = reference["pair"]
|
||||
checks = {
|
||||
"request_succeeded": result.get("success") is True,
|
||||
"model_identity_exact": model_identity_exact(result),
|
||||
"web_search_completed": bool(completed_calls(result, "web_search_call")),
|
||||
"code_interpreter_completed": bool(
|
||||
completed_calls(result, "code_interpreter_call")
|
||||
),
|
||||
"url_citations_present": len(url_citations(result)) >= 2,
|
||||
"closest_pair_matches_independent_reference": (
|
||||
pair_city.lower() in answer.lower() and other_city.lower() in answer.lower()
|
||||
),
|
||||
"distance_reported": any(unit in answer.lower() for unit in ("km", "公里", "千米")),
|
||||
}
|
||||
return {
|
||||
"checks": checks,
|
||||
"passed": all(checks.values()),
|
||||
"output_types": output_types(result),
|
||||
"independent_reference": {
|
||||
"pair": reference["pair"],
|
||||
"distance_km": reference["distance_km"],
|
||||
"pair_count": reference["pair_count"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def is_clarifying_question(result: Dict[str, Any]) -> bool:
|
||||
text = result.get("response") or ""
|
||||
return result.get("success") is True and not result.get("tool_calls") and (
|
||||
"?" in text or "?" in text
|
||||
)
|
||||
|
||||
|
||||
def validate_clarification(
|
||||
first: Dict[str, Any], second: Dict[str, Any] | None
|
||||
) -> Dict[str, Any]:
|
||||
followup_text = (second or {}).get("response") or ""
|
||||
lowered = followup_text.lower()
|
||||
checks = {
|
||||
"first_turn_clarified_before_tools": is_clarifying_question(first),
|
||||
"continuation_used_previous_response_id": bool(
|
||||
second and second.get("request", {}).get("previous_response_id") == first.get("response_id")
|
||||
),
|
||||
"followup_succeeded": bool(second and second.get("success")),
|
||||
"followup_web_search_completed": bool(
|
||||
second and completed_calls(second, "web_search_call")
|
||||
),
|
||||
"followup_code_interpreter_completed": bool(
|
||||
second and completed_calls(second, "code_interpreter_call")
|
||||
),
|
||||
"followup_citations_present": bool(second and url_citations(second)),
|
||||
"followup_reports_ma_rsi_macd": all(
|
||||
token in lowered for token in ("ma", "rsi", "macd")
|
||||
),
|
||||
}
|
||||
return {"checks": checks, "passed": all(checks.values())}
|
||||
|
||||
|
||||
def total_usage(results: Iterable[Dict[str, Any] | None]) -> Dict[str, Any]:
|
||||
totals: Dict[str, float] = {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"total_tokens": 0,
|
||||
"reported_cost_usd": 0.0,
|
||||
}
|
||||
cost_reported = False
|
||||
for result in results:
|
||||
usage = (result or {}).get("usage") or {}
|
||||
for name in ("input_tokens", "output_tokens", "total_tokens"):
|
||||
totals[name] += int(usage.get(name) or 0)
|
||||
if usage.get("cost") is not None:
|
||||
cost_reported = True
|
||||
totals["reported_cost_usd"] += float(usage["cost"])
|
||||
totals["reported_cost_available"] = cost_reported
|
||||
if not cost_reported:
|
||||
totals["reported_cost_usd"] = None
|
||||
return totals
|
||||
|
||||
|
||||
def run_backend(backend: str, reasoning: str) -> Dict[str, Any]:
|
||||
key, base_url, model = Config.resolve(backend)
|
||||
if not key:
|
||||
return {"backend": backend, "started": False, "error": "credential_missing"}
|
||||
|
||||
asean_agent = GPT5NativeAgent(key, base_url=base_url, model=model)
|
||||
asean = asean_agent.process_request(
|
||||
ASEAN_TASK,
|
||||
reasoning_effort=reasoning,
|
||||
verbosity="high",
|
||||
max_tokens=16000,
|
||||
)
|
||||
|
||||
clarification_agent = GPT5NativeAgent(key, base_url=base_url, model=model)
|
||||
first = clarification_agent.process_request(
|
||||
AMBIGUOUS_TASK,
|
||||
reasoning_effort="medium",
|
||||
verbosity="medium",
|
||||
max_tokens=4000,
|
||||
)
|
||||
second = None
|
||||
if is_clarifying_question(first):
|
||||
second = clarification_agent.process_request(
|
||||
CLARIFICATION_REPLY,
|
||||
reasoning_effort=reasoning,
|
||||
verbosity="high",
|
||||
max_tokens=16000,
|
||||
)
|
||||
|
||||
return {
|
||||
"backend": backend,
|
||||
"started": True,
|
||||
"base_url": base_url,
|
||||
"requested_model": model,
|
||||
"asean": asean,
|
||||
"asean_validation": validate_asean(asean),
|
||||
"clarification": {
|
||||
"ambiguous_task": AMBIGUOUS_TASK,
|
||||
"first": first,
|
||||
"user_reply": CLARIFICATION_REPLY if second else None,
|
||||
"second": second,
|
||||
"validation": validate_clarification(first, second),
|
||||
},
|
||||
"api_turns": asean_agent.api_turns + clarification_agent.api_turns,
|
||||
"usage": total_usage((asean, first, second)),
|
||||
}
|
||||
|
||||
|
||||
def acceptance(runs: List[Dict[str, Any]]) -> Dict[str, Any]:
|
||||
"""Multi-provider policy: any eligible backend may close the experiment."""
|
||||
per_backend = {}
|
||||
for run in runs:
|
||||
backend = run.get("backend")
|
||||
if not run.get("started"):
|
||||
per_backend[backend] = {"started": False, "error": run.get("error")}
|
||||
continue
|
||||
per_backend[backend] = {
|
||||
"started": True,
|
||||
"requested_model": run.get("requested_model"),
|
||||
"asean_passed": run.get("asean_validation", {}).get("passed") is True,
|
||||
"clarification_passed": run.get("clarification", {})
|
||||
.get("validation", {})
|
||||
.get("passed")
|
||||
is True,
|
||||
}
|
||||
accepting = next(
|
||||
(
|
||||
backend
|
||||
for backend in ACCEPTANCE_BACKENDS
|
||||
if per_backend.get(backend, {}).get("asean_passed")
|
||||
and per_backend.get(backend, {}).get("clarification_passed")
|
||||
),
|
||||
None,
|
||||
)
|
||||
eligible_attempted = [
|
||||
backend for backend in ACCEPTANCE_BACKENDS if backend in per_backend
|
||||
]
|
||||
return {
|
||||
"policy": (
|
||||
"multi-provider: acceptance is not gated on the official OpenAI "
|
||||
"account; any provider whose Responses API closes the hosted "
|
||||
"search + code-execution loop server-side is eligible"
|
||||
),
|
||||
"eligible_acceptance_backends": list(ACCEPTANCE_BACKENDS),
|
||||
"eligible_backends_attempted": eligible_attempted,
|
||||
"acceptance_backend": accepting,
|
||||
"per_backend": per_backend,
|
||||
"openrouter_is_diagnostic_not_acceptance": "openrouter" in per_backend,
|
||||
"passed": accepting is not None,
|
||||
"reference_docs": [
|
||||
"https://developers.openai.com/api/docs/guides/tools-web-search",
|
||||
"https://developers.openai.com/api/docs/guides/tools-code-interpreter",
|
||||
"https://help.aliyun.com/zh/model-studio/qwen-code-interpreter",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def write_json(path: Path, value: Dict[str, Any]) -> str:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
payload = json.dumps(value, ensure_ascii=False, indent=2) + "\n"
|
||||
path.write_text(payload, encoding="utf-8")
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def assert_credential_free(payloads: Iterable[str]) -> None:
|
||||
"""Refuse to write evidence that embeds any configured API key."""
|
||||
secrets = [
|
||||
value
|
||||
for value in (
|
||||
Config.OPENAI_API_KEY,
|
||||
Config.OPENROUTER_API_KEY,
|
||||
Config.DASHSCOPE_API_KEY,
|
||||
os.getenv("MOONSHOT_API_KEY", ""),
|
||||
os.getenv("KIMI_API_KEY", ""),
|
||||
os.getenv("ARK_API_KEY", ""),
|
||||
os.getenv("SILICONFLOW_API_KEY", ""),
|
||||
os.getenv("GEMINI_API_KEY", ""),
|
||||
)
|
||||
if value
|
||||
]
|
||||
for payload in payloads:
|
||||
for secret in secrets:
|
||||
if secret in payload:
|
||||
raise SystemExit(
|
||||
"Refusing to write evidence: an API key value appears in the payload"
|
||||
)
|
||||
if "authorization" in payload.lower() and "bearer" in payload.lower():
|
||||
raise SystemExit(
|
||||
"Refusing to write evidence: an Authorization header appears in the payload"
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--backends",
|
||||
nargs="+",
|
||||
choices=["openai", "openrouter", "dashscope"],
|
||||
default=["openai", "dashscope"],
|
||||
)
|
||||
parser.add_argument(
|
||||
"--reasoning", choices=["low", "medium", "high", "xhigh", "max"], default="high"
|
||||
)
|
||||
parser.add_argument("--output-dir", type=Path)
|
||||
args = parser.parse_args()
|
||||
|
||||
runs = [run_backend(backend, args.reasoning) for backend in args.backends]
|
||||
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
||||
output_dir = args.output_dir or Path("validation") / "runs" / f"real_{stamp}"
|
||||
evidence = {
|
||||
"schema_version": "1.1",
|
||||
"experiment_id": "1-3",
|
||||
"evidence_mode": "real_api",
|
||||
"created_at": datetime.now(timezone.utc).isoformat(),
|
||||
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
|
||||
"host": {
|
||||
"platform": platform.platform(),
|
||||
"python": sys.version,
|
||||
"machine": platform.machine(),
|
||||
},
|
||||
"repository": {
|
||||
"commit": git_value("rev-parse", "HEAD"),
|
||||
"branch": git_value("branch", "--show-current"),
|
||||
"worktree_dirty": bool(git_value("status", "--porcelain")),
|
||||
},
|
||||
"credentials_recorded": False,
|
||||
"independent_asean_reference": independent_asean_reference(),
|
||||
"runs": runs,
|
||||
}
|
||||
evidence["acceptance"] = acceptance(runs)
|
||||
|
||||
receipts = {
|
||||
"schema_version": "1.0",
|
||||
"experiment_id": "1-3",
|
||||
"created_at": evidence["created_at"],
|
||||
"note": "Raw credential-free provider turns; no API keys or Authorization headers.",
|
||||
"turns": [
|
||||
{"backend": run.get("backend"), "api_turns": run.get("api_turns") or []}
|
||||
for run in runs
|
||||
],
|
||||
}
|
||||
|
||||
evidence_json = json.dumps(evidence, ensure_ascii=False, indent=2)
|
||||
receipts_json = json.dumps(receipts, ensure_ascii=False, indent=2)
|
||||
assert_credential_free((evidence_json, receipts_json))
|
||||
|
||||
evidence_path = output_dir / "evidence.json"
|
||||
evidence_digest = write_json(evidence_path, evidence)
|
||||
receipts_digest = write_json(output_dir / "receipts.json", receipts)
|
||||
(output_dir / "evidence.sha256").write_text(
|
||||
f"{evidence_digest} evidence.json\n", encoding="utf-8"
|
||||
)
|
||||
(output_dir / "receipts.sha256").write_text(
|
||||
f"{receipts_digest} receipts.json\n", encoding="utf-8"
|
||||
)
|
||||
manifest = {
|
||||
"schema_version": "1.0",
|
||||
"experiment_id": "1-3",
|
||||
"run_id": output_dir.name,
|
||||
"created_at": evidence["created_at"],
|
||||
"artifacts": {
|
||||
"evidence.json": {"sha256": evidence_digest},
|
||||
"receipts.json": {"sha256": receipts_digest},
|
||||
},
|
||||
"inputs": {
|
||||
"canonical_source": evidence["canonical_source"],
|
||||
"backends": args.backends,
|
||||
"reasoning": args.reasoning,
|
||||
},
|
||||
"repository": evidence["repository"],
|
||||
"acceptance_passed": evidence["acceptance"]["passed"],
|
||||
"acceptance_backend": evidence["acceptance"]["acceptance_backend"],
|
||||
}
|
||||
manifest_digest = write_json(output_dir / "manifest.json", manifest)
|
||||
|
||||
Path("validation").mkdir(exist_ok=True)
|
||||
shutil.copyfile(evidence_path, Path("validation/latest.json"))
|
||||
latest = json.loads(Path("validation/latest.json").read_text(encoding="utf-8"))
|
||||
latest["artifact_hashes"] = {
|
||||
"evidence.json": evidence_digest,
|
||||
"receipts.json": receipts_digest,
|
||||
"manifest.json": manifest_digest,
|
||||
"run_dir": str(output_dir),
|
||||
}
|
||||
Path("validation/latest.json").write_text(
|
||||
json.dumps(latest, ensure_ascii=False, indent=2) + "\n", encoding="utf-8"
|
||||
)
|
||||
|
||||
print(json.dumps(evidence["acceptance"], ensure_ascii=False, indent=2))
|
||||
print(f"Evidence: {evidence_path}")
|
||||
return 0 if evidence["acceptance"]["passed"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,89 @@
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import example_request
|
||||
|
||||
|
||||
ROOT = Path(__file__).parent
|
||||
|
||||
|
||||
def _import_config_with(value: str | None) -> subprocess.CompletedProcess[str]:
|
||||
env = os.environ.copy()
|
||||
if value is None:
|
||||
env.pop("DEFAULT_MAX_TOKENS", None)
|
||||
else:
|
||||
env["DEFAULT_MAX_TOKENS"] = value
|
||||
return subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-c",
|
||||
"from config import Config; print(repr(Config.DEFAULT_MAX_TOKENS))",
|
||||
],
|
||||
cwd=ROOT,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
text=True,
|
||||
)
|
||||
|
||||
|
||||
def test_default_max_tokens_import_accepts_only_values_int_can_parse():
|
||||
for value in (None, "", " ", "4000.0", "abc", "²", "-1", "+1"):
|
||||
result = _import_config_with(value)
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip() == "None"
|
||||
|
||||
result = _import_config_with(" 4000 ")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip() == "4000"
|
||||
|
||||
|
||||
def test_chat_completions_usage_uses_chat_token_and_detail_keys(monkeypatch, capsys):
|
||||
payload = {
|
||||
"usage": {
|
||||
"prompt_tokens": 123,
|
||||
"completion_tokens": 45,
|
||||
"total_tokens": 168,
|
||||
"prompt_tokens_details": {"cached_tokens": 7},
|
||||
"completion_tokens_details": {"reasoning_tokens": 9},
|
||||
}
|
||||
}
|
||||
|
||||
class FakeResponse:
|
||||
status_code = 200
|
||||
|
||||
def json(self):
|
||||
return payload
|
||||
|
||||
monkeypatch.setattr(
|
||||
example_request.requests,
|
||||
"post",
|
||||
lambda *args, **kwargs: FakeResponse(),
|
||||
)
|
||||
|
||||
result = example_request.make_gpt5_openrouter_request("key", "system", "user")
|
||||
output = capsys.readouterr().out
|
||||
|
||||
assert result == payload
|
||||
assert "Input: 123 tokens (cached: 7)" in output
|
||||
assert "Output: 45 tokens (reasoning: 9)" in output
|
||||
assert "Total: 168" in output
|
||||
|
||||
payload = {
|
||||
"usage": {
|
||||
"input_tokens": 210,
|
||||
"output_tokens": 34,
|
||||
"total_tokens": 244,
|
||||
"input_tokens_details": {"cached_tokens": 11},
|
||||
"output_tokens_details": {"reasoning_tokens": 13},
|
||||
}
|
||||
}
|
||||
result = example_request.make_gpt5_openrouter_request("key", "system", "user")
|
||||
output = capsys.readouterr().out
|
||||
|
||||
assert result == payload
|
||||
assert "Input: 210 tokens (cached: 11)" in output
|
||||
assert "Output: 34 tokens (reasoning: 13)" in output
|
||||
assert "Total: 244" in output
|
||||
@@ -0,0 +1,192 @@
|
||||
from agent import GPT5NativeAgent
|
||||
from config import Config
|
||||
from run_experiment_1_3 import (
|
||||
acceptance,
|
||||
independent_asean_reference,
|
||||
validate_asean,
|
||||
validate_clarification,
|
||||
)
|
||||
|
||||
DASHSCOPE_URL = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
|
||||
|
||||
|
||||
def test_request_uses_official_responses_tool_shapes():
|
||||
agent = GPT5NativeAgent("key")
|
||||
request = agent._build_responses_request(
|
||||
"task", reasoning_effort="max", verbosity="high"
|
||||
)
|
||||
assert request["reasoning"] == {"effort": "max"}
|
||||
assert request["text"] == {"verbosity": "high"}
|
||||
assert request["tools"] == [
|
||||
{"type": "web_search", "search_context_size": "medium"},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {"type": "auto", "memory_limit": "4g"},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_dashscope_request_uses_hosted_tool_shapes_and_streaming():
|
||||
agent = GPT5NativeAgent("key", base_url=DASHSCOPE_URL, model="qwen3.7-plus")
|
||||
assert agent.provider == "dashscope"
|
||||
request = agent._build_responses_request(
|
||||
"task", reasoning_effort="high", verbosity="high"
|
||||
)
|
||||
# DashScope runs thinking natively: no reasoning.effort or text.verbosity,
|
||||
# and streaming is mandatory because its gateway drops idle connections.
|
||||
assert "reasoning" not in request
|
||||
assert "text" not in request
|
||||
assert request["stream"] is True
|
||||
assert request["tools"] == [
|
||||
{"type": "web_search"},
|
||||
{"type": "code_interpreter"},
|
||||
]
|
||||
|
||||
|
||||
def test_config_resolves_dashscope_backend():
|
||||
key, base_url, model = Config.resolve("dashscope")
|
||||
assert base_url == DASHSCOPE_URL
|
||||
assert model == Config.DASHSCOPE_MODEL
|
||||
assert isinstance(key, str)
|
||||
|
||||
|
||||
def test_dashscope_citations_from_web_search_sources():
|
||||
agent = GPT5NativeAgent("key", base_url=DASHSCOPE_URL, model="qwen3.7-plus")
|
||||
response = {
|
||||
"output": [
|
||||
{
|
||||
"type": "web_search_call",
|
||||
"status": "completed",
|
||||
"action": {
|
||||
"query": "ASEAN capitals",
|
||||
"sources": [
|
||||
{"type": "url", "url": "https://asean.test/one"},
|
||||
{"type": "url", "url": "https://asean.test/two"},
|
||||
],
|
||||
},
|
||||
}
|
||||
]
|
||||
}
|
||||
citations = agent._citations(response)
|
||||
assert citations == [
|
||||
{"type": "url_citation", "url": "https://asean.test/one"},
|
||||
{"type": "url_citation", "url": "https://asean.test/two"},
|
||||
]
|
||||
|
||||
def test_dashscope_citations_from_web_search_string_url_sources():
|
||||
agent = GPT5NativeAgent("key", base_url=DASHSCOPE_URL, model="qwen3.7-plus")
|
||||
response = {
|
||||
"output": [
|
||||
{
|
||||
"type": "web_search_call",
|
||||
"status": "completed",
|
||||
"action": {
|
||||
"query": "ASEAN capitals",
|
||||
"sources": [
|
||||
"https://asean.test/one",
|
||||
"https://asean.test/two",
|
||||
],
|
||||
},
|
||||
}
|
||||
]
|
||||
}
|
||||
citations = agent._citations(response)
|
||||
assert citations == [
|
||||
{"type": "url_citation", "url": "https://asean.test/one"},
|
||||
{"type": "url_citation", "url": "https://asean.test/two"},
|
||||
]
|
||||
|
||||
|
||||
def test_independent_asean_reference_is_kuala_lumpur_singapore():
|
||||
reference = independent_asean_reference()
|
||||
assert reference["pair"] == ["Kuala Lumpur", "Singapore"]
|
||||
assert reference["pair_count"] == 45
|
||||
assert 250 < reference["distance_km"] < 400
|
||||
|
||||
|
||||
def test_asean_acceptance_requires_both_completed_hosted_tools():
|
||||
result = {
|
||||
"success": True,
|
||||
"requested_model": "gpt-5.6-sol",
|
||||
"model": "gpt-5.6-sol",
|
||||
"response": "Singapore and Kuala Lumpur are 316 km apart.",
|
||||
"output_items": [
|
||||
{"type": "web_search_call", "status": "completed"},
|
||||
{"type": "code_interpreter_call", "status": "completed"},
|
||||
],
|
||||
"citations": [
|
||||
{"type": "url_citation", "url": "https://one.test"},
|
||||
{"type": "url_citation", "url": "https://two.test"},
|
||||
],
|
||||
}
|
||||
assert validate_asean(result)["passed"] is True
|
||||
result["output_items"] = result["output_items"][:1]
|
||||
assert validate_asean(result)["passed"] is False
|
||||
|
||||
|
||||
def test_asean_acceptance_rejects_model_substitution():
|
||||
result = {
|
||||
"success": True,
|
||||
"requested_model": "qwen3.7-plus",
|
||||
"model": "qwen3.7-flash",
|
||||
"response": "Singapore and Kuala Lumpur are 316 km apart.",
|
||||
"output_items": [
|
||||
{"type": "web_search_call", "status": "completed"},
|
||||
{"type": "code_interpreter_call", "status": "completed"},
|
||||
],
|
||||
"citations": [
|
||||
{"type": "url_citation", "url": "https://one.test"},
|
||||
{"type": "url_citation", "url": "https://two.test"},
|
||||
],
|
||||
}
|
||||
assert validate_asean(result)["checks"]["model_identity_exact"] is False
|
||||
assert validate_asean(result)["passed"] is False
|
||||
|
||||
|
||||
def test_clarification_requires_no_tools_then_linked_tool_run():
|
||||
first = {
|
||||
"success": True,
|
||||
"response": "Which source and indicators do you prefer?",
|
||||
"tool_calls": [],
|
||||
"response_id": "resp_1",
|
||||
}
|
||||
second = {
|
||||
"success": True,
|
||||
"request": {"previous_response_id": "resp_1"},
|
||||
"response": "MA7, MA20, RSI14 and MACD(12,26,9) were computed.",
|
||||
"output_items": [
|
||||
{"type": "web_search_call", "status": "completed"},
|
||||
{"type": "code_interpreter_call", "status": "completed"},
|
||||
],
|
||||
"citations": [{"type": "url_citation"}],
|
||||
}
|
||||
assert validate_clarification(first, second)["passed"] is True
|
||||
second_without_indicators = dict(second, response="Here is the report.")
|
||||
assert validate_clarification(first, second_without_indicators)["passed"] is False
|
||||
|
||||
|
||||
def test_acceptance_is_multi_provider_not_openai_gated():
|
||||
passing_run = {
|
||||
"backend": "dashscope",
|
||||
"started": True,
|
||||
"requested_model": "qwen3.7-plus",
|
||||
"asean_validation": {"passed": True},
|
||||
"clarification": {"validation": {"passed": True}},
|
||||
}
|
||||
blocked_openai = {"backend": "openai", "started": True,
|
||||
"requested_model": "gpt-5.6-sol",
|
||||
"asean_validation": {"passed": False},
|
||||
"clarification": {"validation": {"passed": False}}}
|
||||
result = acceptance([blocked_openai, passing_run])
|
||||
assert result["passed"] is True
|
||||
assert result["acceptance_backend"] == "dashscope"
|
||||
# OpenAI keeps priority when both pass.
|
||||
blocked_openai["asean_validation"] = {"passed": True}
|
||||
blocked_openai["clarification"] = {"validation": {"passed": True}}
|
||||
result = acceptance([blocked_openai, passing_run])
|
||||
assert result["acceptance_backend"] == "openai"
|
||||
# OpenRouter alone can never close the experiment.
|
||||
openrouter_run = dict(passing_run, backend="openrouter")
|
||||
result = acceptance([openrouter_run])
|
||||
assert result["passed"] is False
|
||||
assert result["openrouter_is_diagnostic_not_acceptance"] is True
|
||||
@@ -0,0 +1,363 @@
|
||||
"""
|
||||
Live manual cases for GPT-5 Native Tools Agent.
|
||||
|
||||
These cases demonstrate web_search with the OpenRouter format and require
|
||||
OPENROUTER_API_KEY.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
from typing import Dict, Any, List
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(PROJECT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from agent import GPT5NativeAgent, GPT5AgentChain
|
||||
from config import Config
|
||||
|
||||
# Set up logging
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, Config.LOG_LEVEL),
|
||||
format=Config.LOG_FORMAT
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TestGPT5Agent:
|
||||
"""Live manual case suite for GPT-5 Native Tools Agent"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize manual case suite"""
|
||||
if not Config.validate():
|
||||
raise ValueError("Invalid configuration. Please check your .env file")
|
||||
|
||||
self.agent = GPT5NativeAgent(
|
||||
api_key=Config.OPENROUTER_API_KEY,
|
||||
base_url=Config.OPENROUTER_BASE_URL,
|
||||
model=Config.MODEL_NAME
|
||||
)
|
||||
self.results = []
|
||||
|
||||
def test_web_search_basic(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Test Case 1: Basic web search
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 1: Basic Web Search")
|
||||
print("="*60)
|
||||
|
||||
request = """Search for the latest information about GPT-5 capabilities and features."""
|
||||
|
||||
result = self.agent.process_request(
|
||||
request,
|
||||
use_tools=True,
|
||||
reasoning_effort="low"
|
||||
)
|
||||
|
||||
self._print_result(result)
|
||||
return result
|
||||
|
||||
def test_web_search_with_analysis(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Test Case 2: Web search with analysis request
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 2: Web Search with Analysis")
|
||||
print("="*60)
|
||||
|
||||
request = """Search for current cryptocurrency market trends and Bitcoin price.
|
||||
Then analyze the data to identify patterns and provide insights."""
|
||||
|
||||
result = self.agent.process_request(
|
||||
request,
|
||||
use_tools=True,
|
||||
reasoning_effort="medium"
|
||||
)
|
||||
|
||||
self._print_result(result)
|
||||
return result
|
||||
|
||||
def test_complex_research(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Test Case 3: Complex research task
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 3: Complex Research Task")
|
||||
print("="*60)
|
||||
|
||||
request = """Research the current state of renewable energy adoption globally.
|
||||
Find statistics on solar, wind, and hydroelectric capacity.
|
||||
Analyze growth trends and project future adoption rates.
|
||||
Provide a comprehensive summary with data-driven insights."""
|
||||
|
||||
result = self.agent.process_request(
|
||||
request,
|
||||
use_tools=True,
|
||||
reasoning_effort="high"
|
||||
)
|
||||
|
||||
self._print_result(result)
|
||||
return result
|
||||
|
||||
def test_search_and_code(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Test Case 4: Search and code generation
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 4: Search and Code Generation")
|
||||
print("="*60)
|
||||
|
||||
request = """Search for the latest Python web frameworks in 2025.
|
||||
Then create a simple comparison table and sample code for the top 3 frameworks."""
|
||||
|
||||
result = self.agent.process_request(
|
||||
request,
|
||||
use_tools=True,
|
||||
reasoning_effort="medium"
|
||||
)
|
||||
|
||||
self._print_result(result)
|
||||
return result
|
||||
|
||||
def test_reasoning_efforts(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Test Case 5: Compare different reasoning efforts
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 5: Reasoning Effort Comparison")
|
||||
print("="*60)
|
||||
|
||||
request = "What are the implications of quantum computing on current encryption methods?"
|
||||
|
||||
results = []
|
||||
for effort in ["low", "medium", "high"]:
|
||||
print(f"\n--- Testing with {effort} reasoning effort ---")
|
||||
result = self.agent.process_request(
|
||||
request,
|
||||
use_tools=True,
|
||||
reasoning_effort=effort
|
||||
)
|
||||
self._print_result(result)
|
||||
results.append({
|
||||
"effort": effort,
|
||||
"result": result
|
||||
})
|
||||
|
||||
return results
|
||||
|
||||
def test_search_and_analyze_method(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Test Case 6: Using the search_and_analyze convenience method
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 6: Search and Analyze Method")
|
||||
print("="*60)
|
||||
|
||||
analysis_code = """
|
||||
# Analyze stock market data
|
||||
import statistics
|
||||
|
||||
# Sample data processing
|
||||
prices = [100, 102, 98, 105, 103, 107, 104]
|
||||
returns = [(prices[i] - prices[i-1])/prices[i-1] * 100 for i in range(1, len(prices))]
|
||||
|
||||
avg_return = statistics.mean(returns)
|
||||
volatility = statistics.stdev(returns)
|
||||
|
||||
print(f"Average Return: {avg_return:.2f}%")
|
||||
print(f"Volatility: {volatility:.2f}%")
|
||||
"""
|
||||
|
||||
result = self.agent.search_and_analyze(
|
||||
topic="Current S&P 500 performance and market outlook for 2025",
|
||||
analysis_code=analysis_code
|
||||
)
|
||||
|
||||
self._print_result(result)
|
||||
return result
|
||||
|
||||
def test_agent_chain(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Test Case 7: Chain multiple requests
|
||||
"""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 7: Agent Chain")
|
||||
print("="*60)
|
||||
|
||||
chain = GPT5AgentChain(self.agent)
|
||||
|
||||
# Step 1: Research
|
||||
chain.add_step(
|
||||
"Search for information about the latest AI developments in 2025",
|
||||
use_tools=True,
|
||||
reasoning_effort="low"
|
||||
)
|
||||
|
||||
# Step 2: Deep dive
|
||||
chain.add_step(
|
||||
"Based on the previous findings, search for more details about the most promising AI breakthrough",
|
||||
use_tools=True,
|
||||
reasoning_effort="medium"
|
||||
)
|
||||
|
||||
# Step 3: Analysis
|
||||
chain.add_step(
|
||||
"Analyze the impact of these AI developments on various industries",
|
||||
use_tools=True,
|
||||
reasoning_effort="high"
|
||||
)
|
||||
|
||||
results = chain.execute()
|
||||
|
||||
for i, step_result in enumerate(results, 1):
|
||||
print(f"\n--- Chain Step {i} ---")
|
||||
self._print_result(step_result["result"])
|
||||
|
||||
return results
|
||||
|
||||
def _print_result(self, result: Dict[str, Any]):
|
||||
"""
|
||||
Pretty print test result
|
||||
|
||||
Args:
|
||||
result: Test result dictionary
|
||||
"""
|
||||
if result["success"]:
|
||||
print(f"\n✅ Test Passed")
|
||||
print(f"\nResponse Preview:")
|
||||
print("-"*60)
|
||||
response = result["response"]
|
||||
if len(response) > 500:
|
||||
print(response[:500] + "...")
|
||||
else:
|
||||
print(response)
|
||||
print("-"*60)
|
||||
|
||||
if result.get("usage"):
|
||||
usage = result["usage"]
|
||||
print(f"\n📊 Token Usage:")
|
||||
print(f" - Input: {usage.get('input_tokens', 'N/A')}")
|
||||
print(f" - Output: {usage.get('output_tokens', 'N/A')}")
|
||||
print(f" - Total: {usage.get('total_tokens', 'N/A')}")
|
||||
if usage.get("input_tokens_details"):
|
||||
print(f" - Cached: {usage['input_tokens_details'].get('cached_tokens', 0)}")
|
||||
if usage.get("output_tokens_details"):
|
||||
print(f" - Reasoning: {usage['output_tokens_details'].get('reasoning_tokens', 0)}")
|
||||
else:
|
||||
print(f"\n❌ Test Failed")
|
||||
print(f"Error: {result.get('error', 'Unknown error')}")
|
||||
|
||||
def run_all_tests(self):
|
||||
"""Run all live manual cases"""
|
||||
print("\n" + "="*60)
|
||||
print("RUNNING GPT-5 NATIVE TOOLS MANUAL CASES")
|
||||
print(f"Timestamp: {datetime.now().isoformat()}")
|
||||
print(f"Model: {Config.MODEL_NAME}")
|
||||
print("="*60)
|
||||
|
||||
case_methods = [
|
||||
("Basic Web Search", self.test_web_search_basic),
|
||||
("Web Search with Analysis", self.test_web_search_with_analysis),
|
||||
("Complex Research", self.test_complex_research),
|
||||
("Search and Code", self.test_search_and_code),
|
||||
("Reasoning Efforts", self.test_reasoning_efforts),
|
||||
("Search and Analyze Method", self.test_search_and_analyze_method),
|
||||
("Agent Chain", self.test_agent_chain)
|
||||
]
|
||||
|
||||
results_summary = []
|
||||
|
||||
for case_name, case_method in case_methods:
|
||||
try:
|
||||
print(f"\n🧪 Running: {case_name}")
|
||||
result = case_method()
|
||||
|
||||
# Handle different result types
|
||||
if isinstance(result, list):
|
||||
# For tests that return multiple results
|
||||
if all(isinstance(r, dict) and "result" in r for r in result):
|
||||
success = all(r["result"]["success"] for r in result)
|
||||
else:
|
||||
success = all(r.get("success", False) for r in result if isinstance(r, dict))
|
||||
else:
|
||||
success = result.get("success", False)
|
||||
|
||||
results_summary.append({
|
||||
"case": case_name,
|
||||
"success": success,
|
||||
"result": result
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Manual case {case_name} failed with exception: {str(e)}")
|
||||
results_summary.append({
|
||||
"case": case_name,
|
||||
"success": False,
|
||||
"error": str(e)
|
||||
})
|
||||
|
||||
# Print summary
|
||||
print("\n" + "="*60)
|
||||
print("MANUAL CASE SUMMARY")
|
||||
print("="*60)
|
||||
|
||||
passed = sum(1 for r in results_summary if r["success"])
|
||||
total = len(results_summary)
|
||||
|
||||
for result in results_summary:
|
||||
status = "✅ PASS" if result["success"] else "❌ FAIL"
|
||||
print(f"{result['case']}: {status}")
|
||||
|
||||
print(f"\nTotal: {passed}/{total} manual cases passed")
|
||||
print("="*60)
|
||||
|
||||
return results_summary
|
||||
|
||||
|
||||
def run_single_test(test_name: str = "basic"):
|
||||
"""
|
||||
Run a single live manual case
|
||||
|
||||
Args:
|
||||
test_name: Name of manual case to run
|
||||
"""
|
||||
tester = TestGPT5Agent()
|
||||
|
||||
test_map = {
|
||||
"basic": tester.test_web_search_basic,
|
||||
"analysis": tester.test_web_search_with_analysis,
|
||||
"complex": tester.test_complex_research,
|
||||
"code": tester.test_search_and_code,
|
||||
"reasoning": tester.test_reasoning_efforts,
|
||||
"search_analyze": tester.test_search_and_analyze_method,
|
||||
"chain": tester.test_agent_chain
|
||||
}
|
||||
|
||||
if test_name in test_map:
|
||||
test_map[test_name]()
|
||||
else:
|
||||
print(f"Unknown test: {test_name}")
|
||||
print(f"Available tests: {', '.join(test_map.keys())}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Check configuration first
|
||||
Config.display()
|
||||
|
||||
if not Config.validate():
|
||||
print("\n❌ Configuration validation failed!")
|
||||
print("Please set up your .env file with OPENROUTER_API_KEY")
|
||||
sys.exit(1)
|
||||
|
||||
# Run manual cases
|
||||
if len(sys.argv) > 1:
|
||||
# Run specific test
|
||||
run_single_test(sys.argv[1])
|
||||
else:
|
||||
# Run all tests
|
||||
tester = TestGPT5Agent()
|
||||
tester.run_all_tests()
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,482 @@
|
||||
{
|
||||
"schema_version": "1.0",
|
||||
"experiment_id": "1-3",
|
||||
"evidence_mode": "real_api",
|
||||
"created_at": "2026-07-29T15:54:59.247653+00:00",
|
||||
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
|
||||
"host": {
|
||||
"platform": "macOS-26.3-arm64-arm-64bit",
|
||||
"python": "3.11.4 (main, Jul 5 2023, 08:40:20) [Clang 14.0.6 ]",
|
||||
"machine": "arm64"
|
||||
},
|
||||
"repository": {
|
||||
"commit": "4a7f37cf278bd15948c409f14533017c4c7fbc29",
|
||||
"branch": "main",
|
||||
"worktree_dirty": true
|
||||
},
|
||||
"credentials_recorded": false,
|
||||
"runs": [
|
||||
{
|
||||
"backend": "openai",
|
||||
"started": true,
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"requested_model": "gpt-5.6-sol",
|
||||
"asean": {
|
||||
"success": false,
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
},
|
||||
"response": null,
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
|
||||
"reasoning": {
|
||||
"effort": "high"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "high"
|
||||
},
|
||||
"max_output_tokens": 16000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"raw_response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": {},
|
||||
"model": "gpt-5.6-sol",
|
||||
"provider": "openai",
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"elapsed_seconds": 3.478408
|
||||
},
|
||||
"asean_validation": {
|
||||
"checks": {
|
||||
"request_succeeded": false,
|
||||
"exact_gpt_5_6_sol": true,
|
||||
"web_search_completed": false,
|
||||
"code_interpreter_completed": false,
|
||||
"url_citations_present": false,
|
||||
"closest_pair_reported": false,
|
||||
"distance_reported": false
|
||||
},
|
||||
"passed": false,
|
||||
"output_types": []
|
||||
},
|
||||
"clarification": {
|
||||
"ambiguous_task": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"first": {
|
||||
"success": false,
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
},
|
||||
"response": null,
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"reasoning": {
|
||||
"effort": "medium"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "medium"
|
||||
},
|
||||
"max_output_tokens": 4000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"raw_response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": {},
|
||||
"model": "gpt-5.6-sol",
|
||||
"provider": "openai",
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"elapsed_seconds": 3.775702
|
||||
},
|
||||
"user_reply": null,
|
||||
"second": null,
|
||||
"validation": {
|
||||
"checks": {
|
||||
"first_turn_clarified_before_tools": false,
|
||||
"continuation_used_previous_response_id": false,
|
||||
"followup_succeeded": false,
|
||||
"followup_web_search_completed": false,
|
||||
"followup_code_interpreter_completed": false,
|
||||
"followup_citations_present": false
|
||||
},
|
||||
"passed": false
|
||||
}
|
||||
},
|
||||
"api_turns": [
|
||||
{
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
|
||||
"reasoning": {
|
||||
"effort": "high"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "high"
|
||||
},
|
||||
"max_output_tokens": 16000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"http_status": 429,
|
||||
"response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"elapsed_seconds": 3.478408
|
||||
},
|
||||
{
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"reasoning": {
|
||||
"effort": "medium"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "medium"
|
||||
},
|
||||
"max_output_tokens": 4000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"http_status": 429,
|
||||
"response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"elapsed_seconds": 3.775702
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"total_tokens": 0,
|
||||
"reported_cost_usd": null,
|
||||
"reported_cost_available": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"backend": "openrouter",
|
||||
"started": true,
|
||||
"base_url": "https://openrouter.ai/api/v1",
|
||||
"requested_model": "openai/gpt-5.6-sol",
|
||||
"asean": {
|
||||
"success": false,
|
||||
"error": {
|
||||
"message": "User not found.",
|
||||
"code": 401
|
||||
},
|
||||
"response": null,
|
||||
"request": {
|
||||
"model": "openai/gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
|
||||
"reasoning": {
|
||||
"effort": "high"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "high"
|
||||
},
|
||||
"max_output_tokens": 16000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"raw_response": {
|
||||
"error": {
|
||||
"message": "User not found.",
|
||||
"code": 401
|
||||
}
|
||||
},
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": {},
|
||||
"model": "openai/gpt-5.6-sol",
|
||||
"provider": "openrouter",
|
||||
"base_url": "https://openrouter.ai/api/v1",
|
||||
"elapsed_seconds": 1.115963
|
||||
},
|
||||
"asean_validation": {
|
||||
"checks": {
|
||||
"request_succeeded": false,
|
||||
"exact_gpt_5_6_sol": true,
|
||||
"web_search_completed": false,
|
||||
"code_interpreter_completed": false,
|
||||
"url_citations_present": false,
|
||||
"closest_pair_reported": false,
|
||||
"distance_reported": false
|
||||
},
|
||||
"passed": false,
|
||||
"output_types": []
|
||||
},
|
||||
"clarification": {
|
||||
"ambiguous_task": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"first": {
|
||||
"success": false,
|
||||
"error": {
|
||||
"message": "User not found.",
|
||||
"code": 401
|
||||
},
|
||||
"response": null,
|
||||
"request": {
|
||||
"model": "openai/gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"reasoning": {
|
||||
"effort": "medium"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "medium"
|
||||
},
|
||||
"max_output_tokens": 4000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"raw_response": {
|
||||
"error": {
|
||||
"message": "User not found.",
|
||||
"code": 401
|
||||
}
|
||||
},
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": {},
|
||||
"model": "openai/gpt-5.6-sol",
|
||||
"provider": "openrouter",
|
||||
"base_url": "https://openrouter.ai/api/v1",
|
||||
"elapsed_seconds": 1.025438
|
||||
},
|
||||
"user_reply": null,
|
||||
"second": null,
|
||||
"validation": {
|
||||
"checks": {
|
||||
"first_turn_clarified_before_tools": false,
|
||||
"continuation_used_previous_response_id": false,
|
||||
"followup_succeeded": false,
|
||||
"followup_web_search_completed": false,
|
||||
"followup_code_interpreter_completed": false,
|
||||
"followup_citations_present": false
|
||||
},
|
||||
"passed": false
|
||||
}
|
||||
},
|
||||
"api_turns": [
|
||||
{
|
||||
"request": {
|
||||
"model": "openai/gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
|
||||
"reasoning": {
|
||||
"effort": "high"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "high"
|
||||
},
|
||||
"max_output_tokens": 16000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"http_status": 401,
|
||||
"response": {
|
||||
"error": {
|
||||
"message": "User not found.",
|
||||
"code": 401
|
||||
}
|
||||
},
|
||||
"elapsed_seconds": 1.115963
|
||||
},
|
||||
{
|
||||
"request": {
|
||||
"model": "openai/gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"reasoning": {
|
||||
"effort": "medium"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "medium"
|
||||
},
|
||||
"max_output_tokens": 4000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"http_status": 401,
|
||||
"response": {
|
||||
"error": {
|
||||
"message": "User not found.",
|
||||
"code": 401
|
||||
}
|
||||
},
|
||||
"elapsed_seconds": 1.025438
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"total_tokens": 0,
|
||||
"reported_cost_usd": null,
|
||||
"reported_cost_available": false
|
||||
}
|
||||
}
|
||||
],
|
||||
"acceptance": {
|
||||
"checks": {
|
||||
"official_openai_asean_passed": false,
|
||||
"official_openai_clarification_passed": false
|
||||
},
|
||||
"passed": false,
|
||||
"openrouter_is_diagnostic_not_official_acceptance": true,
|
||||
"official_docs": [
|
||||
"https://developers.openai.com/api/docs/models/gpt-5.6-sol",
|
||||
"https://developers.openai.com/api/docs/guides/tools-web-search",
|
||||
"https://developers.openai.com/api/docs/guides/tools-code-interpreter",
|
||||
"https://developers.openai.com/api/docs/guides/model-guidance?model=gpt-5.6-sol"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
530316ce5e4d5de004b8076dd57a364bc0173aeb371dbc87f09f4fa913d11942 evidence.json
|
||||
@@ -0,0 +1,264 @@
|
||||
{
|
||||
"schema_version": "1.0",
|
||||
"experiment_id": "1-3",
|
||||
"evidence_mode": "real_api",
|
||||
"created_at": "2026-07-29T19:39:13.370389+00:00",
|
||||
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
|
||||
"host": {
|
||||
"platform": "macOS-26.3-arm64-arm-64bit",
|
||||
"python": "3.11.4 (main, Jul 5 2023, 08:40:20) [Clang 14.0.6 ]",
|
||||
"machine": "arm64"
|
||||
},
|
||||
"repository": {
|
||||
"commit": "4a7f37cf278bd15948c409f14533017c4c7fbc29",
|
||||
"branch": "main",
|
||||
"worktree_dirty": true
|
||||
},
|
||||
"credentials_recorded": false,
|
||||
"runs": [
|
||||
{
|
||||
"backend": "openai",
|
||||
"started": true,
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"requested_model": "gpt-5.6-sol",
|
||||
"asean": {
|
||||
"success": false,
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
},
|
||||
"response": null,
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
|
||||
"reasoning": {
|
||||
"effort": "high"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "high"
|
||||
},
|
||||
"max_output_tokens": 16000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"raw_response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": {},
|
||||
"model": "gpt-5.6-sol",
|
||||
"provider": "openai",
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"elapsed_seconds": 3.900986
|
||||
},
|
||||
"asean_validation": {
|
||||
"checks": {
|
||||
"request_succeeded": false,
|
||||
"exact_gpt_5_6_sol": true,
|
||||
"web_search_completed": false,
|
||||
"code_interpreter_completed": false,
|
||||
"url_citations_present": false,
|
||||
"closest_pair_reported": false,
|
||||
"distance_reported": false
|
||||
},
|
||||
"passed": false,
|
||||
"output_types": []
|
||||
},
|
||||
"clarification": {
|
||||
"ambiguous_task": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"first": {
|
||||
"success": false,
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
},
|
||||
"response": null,
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"reasoning": {
|
||||
"effort": "medium"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "medium"
|
||||
},
|
||||
"max_output_tokens": 4000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"raw_response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"tool_calls": [],
|
||||
"citations": [],
|
||||
"usage": {},
|
||||
"model": "gpt-5.6-sol",
|
||||
"provider": "openai",
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"elapsed_seconds": 3.851438
|
||||
},
|
||||
"user_reply": null,
|
||||
"second": null,
|
||||
"validation": {
|
||||
"checks": {
|
||||
"first_turn_clarified_before_tools": false,
|
||||
"continuation_used_previous_response_id": false,
|
||||
"followup_succeeded": false,
|
||||
"followup_web_search_completed": false,
|
||||
"followup_code_interpreter_completed": false,
|
||||
"followup_citations_present": false
|
||||
},
|
||||
"passed": false
|
||||
}
|
||||
},
|
||||
"api_turns": [
|
||||
{
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "Research the current official capitals and reliable coordinates\nfor the ten ASEAN member states. You must use hosted web search and cite the\nsources. Then you must use the hosted Python tool—not mental arithmetic—to\nenumerate all 45 capital pairs with the haversine formula and identify the\nclosest pair and distance. Include the coordinates, formula assumptions,\ncalculation result, retrieval date, and clickable citations. Do not say Python\nwas used unless a code_interpreter_call completes.",
|
||||
"reasoning": {
|
||||
"effort": "high"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "high"
|
||||
},
|
||||
"max_output_tokens": 16000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"http_status": 429,
|
||||
"response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"elapsed_seconds": 3.900986
|
||||
},
|
||||
{
|
||||
"request": {
|
||||
"model": "gpt-5.6-sol",
|
||||
"instructions": "You are a deep-research assistant. Use hosted web search for\ncurrent facts and cite sources. Use the hosted Python/code-interpreter tool for\nquantitative analysis; do not claim a calculation was run unless the response\ncontains a completed code_interpreter_call. Ask a concise clarifying question\nbefore research when a material user preference is genuinely ambiguous.",
|
||||
"input": "搜索最近一个月的比特币走势,做技术分析。",
|
||||
"reasoning": {
|
||||
"effort": "medium"
|
||||
},
|
||||
"background": false,
|
||||
"store": true,
|
||||
"text": {
|
||||
"verbosity": "medium"
|
||||
},
|
||||
"max_output_tokens": 4000,
|
||||
"tools": [
|
||||
{
|
||||
"type": "web_search",
|
||||
"search_context_size": "medium"
|
||||
},
|
||||
{
|
||||
"type": "code_interpreter",
|
||||
"container": {
|
||||
"type": "auto",
|
||||
"memory_limit": "4g"
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
},
|
||||
"http_status": 429,
|
||||
"response": {
|
||||
"error": {
|
||||
"message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.",
|
||||
"type": "insufficient_quota",
|
||||
"param": null,
|
||||
"code": "insufficient_quota"
|
||||
}
|
||||
},
|
||||
"elapsed_seconds": 3.851438
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"input_tokens": 0,
|
||||
"output_tokens": 0,
|
||||
"total_tokens": 0,
|
||||
"reported_cost_usd": null,
|
||||
"reported_cost_available": false
|
||||
}
|
||||
}
|
||||
],
|
||||
"acceptance": {
|
||||
"checks": {
|
||||
"official_openai_asean_passed": false,
|
||||
"official_openai_clarification_passed": false
|
||||
},
|
||||
"passed": false,
|
||||
"openrouter_is_diagnostic_not_official_acceptance": false,
|
||||
"official_docs": [
|
||||
"https://developers.openai.com/api/docs/models/gpt-5.6-sol",
|
||||
"https://developers.openai.com/api/docs/guides/tools-web-search",
|
||||
"https://developers.openai.com/api/docs/guides/tools-code-interpreter",
|
||||
"https://developers.openai.com/api/docs/guides/model-guidance?model=gpt-5.6-sol"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
e2364d62b234e9691d0f69d09ea24335b9d685e17178577dfe1bade3573dfa1a evidence.json
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
aab1864446613463c936154eca2fc65df8496869c65d4c17dc1f12628eaff3cc evidence.json
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"schema_version": "1.0",
|
||||
"experiment_id": "1-3",
|
||||
"run_id": "real_20260731T170529Z",
|
||||
"created_at": "2026-07-31T17:05:29.580761+00:00",
|
||||
"artifacts": {
|
||||
"evidence.json": {
|
||||
"sha256": "aab1864446613463c936154eca2fc65df8496869c65d4c17dc1f12628eaff3cc"
|
||||
},
|
||||
"receipts.json": {
|
||||
"sha256": "9247d69b5915ceff7d27b9dbc0752d63791fc26309f8d69e17844508058edcbe"
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"canonical_source": "book/chapter1.md#实验-1-3-gpt-5-6-原生-deep-research-能力",
|
||||
"backends": [
|
||||
"openai",
|
||||
"dashscope"
|
||||
],
|
||||
"reasoning": "high"
|
||||
},
|
||||
"repository": {
|
||||
"commit": "8c2b7f55e4bd2e5348b15b823dd23a179ef3decf",
|
||||
"branch": "codex/exp1-3-multiprovider-20260731",
|
||||
"worktree_dirty": true
|
||||
},
|
||||
"acceptance_passed": true,
|
||||
"acceptance_backend": "dashscope"
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
9247d69b5915ceff7d27b9dbc0752d63791fc26309f8d69e17844508058edcbe receipts.json
|
||||
Reference in New Issue
Block a user