ai-agent-book/chapter10/book-translation/validation/real_20260730T061500Z_v4/judge_receipts/unit-17.json

{
  "schema_version": 1,
  "credential_free": true,
  "attempts": [
    {
      "attempt": 1,
      "request": {
        "model": "doubao-seed-1-6-250615",
        "messages": [
          {
            "role": "user",
            "content": "You are an exacting bilingual technical-book translation evaluator. Compare two anonymous Chinese translations against the complete English Markdown source. Score both X and Y from 1 to 5 on exactly: accuracy (no omissions, inventions, or changed claims); fluency; terminology (consistent and technically correct); markdown_code_fidelity (figures, links, headings, equations, and fenced code preserved). Each score needs concrete quoted or located evidence. Prefer one only when evidence supports it. Return JSON only: {\"variants\":{\"X\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}},\"Y\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}}},\"preferred\":\"X|Y|tie\",\"preference_evidence\":\"...\"}.\n\nCOMPLETE ENGLISH SOURCE:\nSuccess rate estimation and amount calculation also need to be specified precisely enough to execute. The success rate should be evaluated step by step according to a fixed process, and the estimated probability should map directly to the billing model. For example, tasks with an estimated success probability above 60% might use the refundable model, while those below 30% might be rejected. Amount calculation must define the billing granularity—for example, phone calls are billed at $0.05 per minute, with the total rounded to the nearest whole dollar—and explicitly state that \"savings\" are calculated only from the existing bill. Otherwise, the model might reason, \"If the price rises to $180 next year without negotiation, and I help maintain it at $150, that saves $30,\" incorrectly counting the avoidance of a future price increase as savings.\n\nThese rules may seem trivial, but details like these determine the consistency of system behavior. In mature Agent teams, prompts are often designed by **product managers**, who iterate on rule definitions based on production data, user feedback, and operational experience. The engineer's role is to encode the rules accurately, ensure correct formatting and clear structure, and avoid making arbitrary business-logic decisions.\n\nThe core design philosophy is that large language models are strong at following complex instructions and extracting information from long contexts, but they should not be given excessive discretion in formulating business rules. By providing a clear operational framework, the model's cognitive resources are freed up to focus on parts that truly require reasoning. Effective training does not leave people to infer the process on their own; it provides detailed standard operating procedures that let people operate within a clear framework.\n\n### Few-Shot Examples: When to Show the Model Examples\n\nBeyond rules and processes, examples (few-shot examples) are another important type of system prompt content. When the desired output is difficult to describe precisely with rules—such as copywriting in a specific style, the format of a structured report, or the tone and nuance of customer service replies—it is often better to provide two or three high-quality input-output examples than to write long abstract descriptions. The model can adapt to these patterns within the current context, often more effectively than it can follow the same amount of abstract instruction (the internal mechanism behind this is discussed in the Context Compression section of this chapter). Conversely, for tasks the model already handles well and whose rules are easy to state, examples waste tokens.\n\nThere are two engineering decision points. First, **where to place the examples**: placing them in the system prompt makes them a static prefix effective for all requests; alternatively, a set of synthetic user/assistant messages can be placed in the first round of dialogue, suitable for scenarios where different example sets are needed for different conversation types. Second, **how examples affect KV Cache prefix stability**: regardless of where they are placed, examples appear early in the context. Once selected, they should remain byte-for-byte stable. Dynamically retrieving a different \"most relevant\" example for every request repeatedly invalidates the cache. Therefore, production systems typically prepare a fixed set of examples for each task type rather than selecting them on a per-request basis.\n\nMore examples are not always better: two or three carefully selected examples covering boundary cases are usually more useful than ten near-duplicates. Near-duplicates consume context and dilute the model's attention to the rules themselves.\n\n### Tool Definition Design\n\nIn addition to the system prompt, another important static component in the API request is the **tool definition** (the `tools` field). The quality of tool definitions directly determines the accuracy of the Agent's tool usage. A good tool definition functions like an operating manual, enabling a model that has never seen the tool to use it correctly from the outset and avoid common mistakes.\n\nClaude Code's tool definitions show that each tool description is carefully designed with usage boundaries (\"NEVER invoke grep or rg as a Bash command\"), concrete examples (`timezone: 'America/New_York'`), performance tips (\"Batch your tool calls together\"), and relationships between tools (\"Use the Read tool at least once before editing\"). Chapter 4 discusses the design principles and best practices for tool definitions in detail.\n\nTool definitions usually form a static prefix with the system prompt. Most LLM APIs send the `tools` field with every request, and providers cache it with the rest of the prefix. Since 2026, however, APIs have begun to support progressive disclosure natively. OpenAI's Responses API provides a `tool_search` tool and a `defer_loading: true` flag[^ch2-toolsearch-oai], allowing the model to load full schemas on demand through `tool_search_call` → `tool_search_output`. Anthropic provides Tool Search through `tool_reference` blocks, while Claude Code defers MCP tools by default: only tool names and server instructions are injected at session start, and full schemas are added after the model searches for them[^ch2-toolsearch-cc]. Codex CLI similarly uses `tool_search` with BM25 retrieval as part of its default architecture[^ch2-toolsearch-codex]. All these mechanisms follow the same pattern as the third Skills approach: the static prefix contains only tool names and brief descriptions, while the full schema is **appended to the end of the context** on demand and becomes part of the trajectory.\n\n[^ch2-toolsearch-oai]: OpenAI, \"Tool search\", Responses API documentation. https://developers.openai.com/api/docs/guides/tools-tool-search\n[^ch2-toolsearch-cc]: Anthropic, \"Scale with MCP tool search\", Claude Code documentation. https://code.claude.com/docs/en/mcp\n[^ch2-toolsearch-codex]: OpenAI Codex CLI source, `codex-rs/core/templates/search_tool/tool_description.md`: \"Some of the tools may not have been provided to you upfront, and you should use this tool (tool_search) to search for the required tools and load them.\"\n\nWhy does appending at the end not break the cache? This follows directly from the prefix property of the KV Cache discussed earlier: causal attention means each token's key-value pairs depend only on the tokens before it, so appending new content at the end changes none of the cached tokens' K and V—the newly added tool schema is computed once on its first appearance (a one-time cache write) and thereafter joins the ever-growing \"prefix,\" hitting the cache on every subsequent turn. This is not \"pre-compilation\" but append-only injection.\n\nOne point is easy to misunderstand: a discovered schema is appended only once. It then remains at its original position in the trajectory, and later messages are added **after** it; the schema is not moved to the end again on every turn. Re-injecting it each turn would require repeated prefilling and defeat the purpose of caching. Both APIs preserve the schema's original position in subsequent requests. OpenAI requires subsequent requests to retain the position of the `tool_search_output` item, and the same tool does not need to be loaded again in later turns. Anthropic expands the `tool_reference` block inline at its original position in the conversation history; in the documentation's words, you \"keep the same cache hit across every turn.\" Recomputation occurs only when the Prompt Cache TTL expires, which causes the entire prefix to be recomputed, or when the loaded tool set is modified, removed, or reordered, which invalidates the cache from that point onward.\n\nThe mechanism's other constraint is model capability: the model must have been trained on the pattern of \"tool definitions appearing mid-conversation\"—which is why only newer models (e.g., GPT-5.4+, the Claude 4.5+ series) currently support it, and why self-hosted open-source models need dedicated training. The full discussion of tool discovery is in Chapter 4's \"Proactive Tool Discovery\" section.\n\n\n\nANONYMOUS CHINESE X:\n### 上下文工程[第8/17部分]\n\n成功率估算和金额计算也需要精确到足以执行的程度。成功率应按照固定流程逐步评估，估算的概率应直接映射到计费模型。例如，估算成功率高于60%的任务可能采用可退款模型，而低于30%的可能被拒绝。金额计算必须明确计费粒度——例如，电话按每分钟0.05美元计费，总额四舍五入到最接近的整数美元——并明确说明“节省”仅从现有账单计算。否则，模型可能会推断：“如果明年不谈判价格涨到180美元，而我帮助维持在150美元，那节省了30美元”，错误地将避免未来价格上涨算作节省。\n\n这些规则可能看似微不足道，但诸如此类的细节决定了系统行为的一致性。在成熟的Agent团队中，提示词通常由产品经理设计，他们根据生产数据、用户反馈和运营经验迭代规则定义。工程师的角色是准确编码规则，确保格式正确和结构清晰，并避免随意做出业务逻辑决策。\n\n核心设计理念是大语言模型擅长遵循复杂指令和从长上下文中提取信息，但不应在制定业务规则时拥有过多的自由裁量权。通过提供清晰的操作框架，模型的认知资源得以解放，专注于真正需要推理的部分。有效的训练不会让人们自行推断流程；它提供详细的标准操作程序，让人们在清晰的框架内操作。\n\n### 少样本示例：何时向模型展示示例\n\n除了规则和流程，示例（少样本示例）是系统提示内容的另一种重要类型。当期望的输出难以用规则精确描述时——例如特定风格的文案、结构化报告的格式或客服回复的语气和细微差别——通常提供两三个高质量的输入输出示例比编写冗长的抽象描述更好。模型可以在当前上下文中适应这些模式，通常比遵循相同数量的抽象指令更有效（本章的上下文压缩部分讨论了这背后的内部机制）。相反，对于模型已经处理得很好且规则易于陈述的任务，示例会浪费词元。\n\n有两个工程决策点。首先，**示例放置在哪里**：将它们放在系统提示中使其成为对所有请求有效的静态前缀；或者在第一轮对话中放置一组合成的用户/助手消息，适用于不同对话类型需要不同示例集的场景。其次，**示例如何影响键值缓存前缀稳定性**：无论放置在哪里，示例都出现在上下文中的早期。一旦选定，它们应该逐字节稳定。为每个请求动态检索不同的“最相关”示例会反复使缓存失效。因此，生产系统通常为每种任务类型准备固定的示例集，而不是按需选择。\n\n更多示例并不总是更好：两三个精心挑选的涵盖边界情况的示例通常比十个近乎重复的示例更有用。近乎重复的示例消耗上下文并稀释模型对规则本身的注意力。\n\n### 工具定义设计\n\n除了系统提示，API请求中的另一个重要静态组件是**工具定义**（`tools`字段）。工具定义的质量直接决定了Agent使用工具的准确性。良好的工具定义就像操作手册，使从未见过该工具的模型从一开始就能正确使用它并避免常见错误。\n\nClaude Code的工具定义表明，每个工具描述都经过精心设计，包含使用边界（“绝不要将grep或rg作为Bash命令调用”）、具体示例（`timezone: 'America/New_York'`）、性能提示（“将工具调用批量在一起”）和工具之间的关系（“在编辑之前至少使用一次Read工具”）。第4章详细讨论了工具定义的设计原则和最佳实践。\n\n工具定义通常与系统提示形成静态前缀。大多数大语言模型API在每个请求中发送`tools`字段，提供商将其与前缀的其余部分一起缓存。然而，自2026年起，API开始原生支持渐进式披露。OpenAI的Responses API提供`tool_search`工具和`defer_loading: true`标志[^ch2-toolsearch-oai]，允许模型通过`tool_search_call`→`tool_search_output`按需加载完整架构。Anthropic通过`tool_reference`块提供工具搜索，而Claude Code默认延迟MCP工具：仅在会话开始时注入工具名称和服务器指令，完整架构在模型搜索后添加[^ch2-toolsearch-cc]。Codex CLI类似地将`tool_search`与BM25检索一起用作其默认架构的一部分[^ch2-toolsearch-codex]。所有这些机制都遵循与第三种技能方法相同的模式：静态前缀仅包含工具名称和简要描述，而完整架构按需**附加到上下文末尾**并成为轨迹的一部分。\n\n[^ch2-toolsearch-oai]: OpenAI，“工具搜索”，Responses API文档。https://developers.openai.com/api/docs/guides/tools-tool-search  \n[^ch2-toolsearch-cc]: Anthropic，“通过MCP工具搜索扩展”，Claude Code文档。https://code.claude.com/docs/en/mcp  \n[^ch2-toolsearch-codex]: OpenAI Codex CLI源代码，`codex-rs/core/templates/search_tool/tool_description.md`：“一些工具可能没有预先提供给你，你应该使用这个工具（tool_search）来搜索所需的工具并加载它们。”\n\n为什么附加在末尾不会破坏缓存？这直接遵循前面讨论的键值缓存的前缀属性：因果注意力意味着每个词元的键值对仅依赖于其之前的词元，因此在末尾附加新内容不会改变任何缓存词元的K和V——新添加的工具架构在首次出现时计算一次（一次性缓存写入），此后加入不断增长的“前缀”，在后续的每一轮中都命中缓存。这不是“预编译”，而是仅附加注入。\n\n有一点容易误解：发现的架构仅附加一次。然后它保留在轨迹中的原始位置，后续消息在其**之后**添加；架构不会在每一轮都移到末尾。每一轮重新注入它需要重复预填充，会违背缓存的目的。两个API都在后续请求中保留架构的原始位置。OpenAI要求后续请求保留`tool_search_output`项的位置，后续轮次不需要再次加载相同的工具。Anthropic在对话历史中的原始位置内联扩展`tool_reference`块；用文档中的话说，你“在每一轮都保持相同的缓存命中”。重新计算仅在提示缓存TTL过期时发生，这会导致整个前缀重新计算，或者在加载的工具集被修改、删除或重新排序时发生，从那时起缓存失效。\n\n该机制的另一个约束是模型能力：模型必须接受过“工具定义在对话中途出现”的模式训练——这就是为什么目前只有较新的模型（例如GPT-5.4+、Claude 4.5+系列）支持它，以及为什么自托管开源模型需要专门训练。工具发现的完整讨论在第4章的“主动工具发现”部分。\n\nANONYMOUS CHINESE Y:\n### 上下文工程[第8/17部分]\n成功率估计和金额计算也需要精确到足以执行的程度。成功率应根据固定流程逐步评估，估计概率应直接映射到计费模型。例如，估计成功率高于60%的任务可能使用可退款模型，而低于30%的任务可能被拒绝。金额计算必须定义计费粒度——例如，电话按每分钟0.05美元计费，总额四舍五入到最接近的整数美元——并明确声明“节省”仅从现有账单计算。否则，模型可能会推断：“如果明年不协商价格涨到180美元，而我帮助维持在150美元，那节省了30美元”，错误地将避免未来价格上涨算作节省。\n\n这些规则可能看似微不足道，但诸如此类的细节决定了系统行为的一致性。在成熟的代理团队中，提示通常由**产品经理**设计，他们根据生产数据、用户反馈和运营经验迭代规则定义。工程师的角色是准确编码规则，确保正确的格式和清晰的结构，避免随意做出业务逻辑决策。\n\n核心设计理念是，大语言模型擅长遵循复杂指令和从长上下文中提取信息，但不应在制定业务规则时被赋予过多的自由裁量权。通过提供清晰的操作框架，模型的认知资源被解放出来，专注于真正需要推理的部分。有效的培训不会让人们自己推断流程；它提供详细的标准操作程序，让人们在清晰的框架内操作。\n\n### 少样本示例：何时向模型展示示例\n除了规则和流程，示例（少样本示例）是系统提示内容的另一种重要类型。当所需输出难以用规则精确描述时——例如特定风格的文案、结构化报告的格式或客服回复的语气和细微差别——提供两三个高质量的输入输出示例通常比编写冗长的抽象描述更有效。模型可以在当前上下文中适应这些模式，通常比遵循相同数量的抽象指令更有效（本章上下文压缩部分讨论了其内部机制）。相反，对于模型已经处理得很好且规则容易陈述的任务，示例会浪费标记。\n\n有两个工程决策点。首先，**示例放置在哪里**：将它们放在系统提示中使其成为对所有请求有效的静态前缀；或者在第一轮对话中放置一组合成的用户/助手消息，适用于不同对话类型需要不同示例集的场景。其次，**示例如何影响KV缓存前缀稳定性**：无论放置在哪里，示例都出现在上下文中的早期位置。一旦选定，它们应保持字节完全稳定。每次请求动态检索不同的“最相关”示例会反复使缓存失效。因此，生产系统通常为每种任务类型准备固定的示例集，而不是在每次请求时选择。\n\n更多示例并不总是更好：两三个精心挑选的涵盖边界情况的示例通常比十个近乎重复的示例更有用。近乎重复的示例消耗上下文并稀释模型对规则本身的注意力。\n\n### 工具定义设计\n除了系统提示，API请求中另一个重要的静态组件是**工具定义**（`tools`字段）。工具定义的质量直接决定代理工具使用的准确性。好的工具定义就像操作手册，使从未见过该工具的模型从一开始就能正确使用它并避免常见错误。\n\nClaude Code的工具定义表明，每个工具描述都经过精心设计，包括使用边界（“NEVER invoke grep or rg as a Bash command”）、具体示例（`timezone: 'America/New_York'`）、性能提示（“Batch your tool calls together”）和工具之间的关系（“Use the Read tool at least once before editing”）。第4章详细讨论了工具定义的设计原则和最佳实践。\n\n工具定义通常与系统提示形成静态前缀。大多数大语言模型API在每次请求时发送`tools`字段，提供商将其与前缀的其余部分一起缓存。然而，自2026年以来，API开始原生支持渐进披露。OpenAI的Responses API提供`tool_search`工具和`defer_loading: true`标志[^ch2-toolsearch-oai]，允许模型通过`tool_search_call`→`tool_search_output`按需加载完整架构。Anthropic通过`tool_reference`块提供工具搜索，而Claude Code默认延迟MCP工具：仅在会话开始时注入工具名称和服务器指令，完整架构在模型搜索后添加到上下文中[^ch2-toolsearch-cc]。Codex CLI类似地将`tool_search`与BM25检索一起用作其默认架构的一部分[^ch2-toolsearch-codex]。所有这些机制都遵循与第三种技能方法相同的模式：静态前缀仅包含工具名称和简要描述，而完整架构**按需附加到上下文末尾**并成为轨迹的一部分。\n\n[^ch2-toolsearch-oai]: OpenAI，“工具搜索”，Responses API文档。https://developers.openai.com/api/docs/guides/tools-tool-search\n[^ch2-toolsearch-cc]: Anthropic，“通过MCP工具搜索扩展”，Claude Code文档。https://code.claude.com/docs/en/mcp\n[^ch2-toolsearch-codex]: OpenAI Codex CLI源代码，`codex-rs/core/templates/search_tool/tool_description.md`：“某些工具可能没有预先提供给你，你应该使用这个工具（tool_search）来搜索所需的工具并加载它们。”\n\n为什么附加到末尾不会破坏缓存？这直接遵循前面讨论的KV缓存的前缀属性：因果注意力意味着每个标记的键值对仅依赖于其前面的标记，因此在末尾附加新内容不会改变任何缓存标记的K和V——新添加的工具架构在首次出现时计算一次（一次性缓存写入），此后加入不断增长的“前缀”，在后续每一轮都命中缓存。这不是“预编译”而是仅附加注入。\n\n有一点容易误解：发现的架构仅附加一次。然后它在轨迹中的原始位置保持不变，后续消息添加在它**之后**；架构不会在每一轮都再次移动到末尾。每次轮次重新注入它将需要重复预填充并违背缓存的目的。两个API都在后续请求中保留架构的原始位置。OpenAI要求后续请求保留`tool_search_output`项的位置，后续轮次不需要再次加载同一工具。Anthropic在对话历史的原始位置内联扩展`tool_reference`块；用文档中的话来说，你“在每一轮都保持相同的缓存命中”。仅当提示缓存TTL过期（导致整个前缀重新计算）或加载的工具集被修改、删除或重新排序时，才会导致从该点开始的缓存失效。\n\n该机制的另一个约束是模型能力：模型必须接受过“工具定义出现在对话中间”的模式训练——这就是为什么目前只有较新的模型（例如GPT-5.4+、Claude 4.5+系列）支持它，以及为什么自托管开源模型需要专门训练。工具发现的完整讨论在第4章的“主动工具发现”部分。"
          }
        ],
        "temperature": 0,
        "response_format": {
          "type": "json_object"
        },
        "extra_body": {
          "thinking": {
            "type": "disabled"
          }
        }
      },
      "response": {
        "id": "0217853908134573ea4fa82759f0cae4f2ffa723d4f9dd646794b",
        "model": "doubao-seed-1-6-250615",
        "created": 1785390828,
        "content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"完整保留了英文原文的所有信息，包括技术细节如\\\"键值缓存的前缀属性\\\"、\\\"因果注意力\\\"等，无遗漏或篡改。例如对\\\"the model must have been trained on the pattern of 'tool definitions appearing mid-conversation'\\\"的翻译准确无误。\"},\"fluency\":{\"score\":4,\"evidence\":\"整体流畅，但部分句子稍显生硬，如\\\"诸如此类的细节决定了系统行为的一致性\\\"可优化为更自然的表达。\"},\"terminology\":{\"score\":5,\"evidence\":\"技术术语翻译一致且准确，如\\\"few-shot examples\\\"译为\\\"少样本示例\\\"，\\\"KV Cache\\\"译为\\\"键值缓存\\\"，\\\"token\\\"译为\\\"词元\\\"，全文保持统一。\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"Markdown格式完全保留，包括三级标题、代码块（如`timezone: 'America/New_York'`）、引用标记（[^ch2-toolsearch-oai]）及链接，结构与原文一致。\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"准确传达了原文所有核心信息，技术概念如\\\"progressive disclosure\\\"译为\\\"渐进披露\\\"，\\\"cache invalidation\\\"译为\\\"缓存失效\\\"等均正确，无信息增减。\"},\"fluency\":{\"score\":5,\"evidence\":\"语言表达更自然流畅，如将\\\"details like these determine the consistency\\\"译为\\\"诸如此类的细节决定了系统行为的一致性\\\"比X更通顺，符合中文表达习惯。\"},\"terminology\":{\"score\":5,\"evidence\":\"术语翻译准确且一致，如\\\"token\\\"统一译为\\\"标记\\\"，\\\"KV Cache\\\"译为\\\"KV缓存\\\"，\\\"tool definition\\\"译为\\\"工具定义\\\"，与行业标准一致。\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"Markdown结构完整，标题层级、代码块（如`defer_loading: true`）、脚注及链接格式均与原文一致，未出现格式错误。\"}}},\"preferred\":\"Y\",\"preference_evidence\":\"Y在保持与X同等准确性、术语一致性和格式保真度的基础上，语言表达更流畅自然，如对复杂句式的处理（如\\\"the model can adapt to these patterns within the current context\\\"的翻译）更符合中文阅读习惯，提升了可读性。\"}",
        "usage": {
          "prompt_tokens": 4944,
          "completion_tokens": 508,
          "total_tokens": 5452
        }
      },
      "latency_milliseconds": 14920,
      "validation": {
        "valid": true
      }
    }
  ]
}