liqiang
b119135836
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
181 lines
125 KiB
JSON
181 lines
125 KiB
JSON
{
|
||
"schema_version": 1,
|
||
"credential_free": true,
|
||
"attempts": [
|
||
{
|
||
"attempt": 1,
|
||
"request": {
|
||
"model": "doubao-seed-1-6-250615",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "You are an exacting bilingual technical-book translation evaluator. Compare two anonymous Chinese translations against the complete English Markdown source. Score both X and Y from 1 to 5 on exactly: accuracy (no omissions, inventions, or changed claims); fluency; terminology (consistent and technically correct); markdown_code_fidelity (figures, links, headings, equations, and fenced code preserved). Each score needs concrete quoted or located evidence. Prefer one only when evidence supports it. Return JSON only: {\"variants\":{\"X\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}},\"Y\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}}},\"preferred\":\"X|Y|tie\",\"preference_evidence\":\"...\"}.\n\nCOMPLETE ENGLISH SOURCE:\nThis structured reasoning lets an LLM Agent handle entirely new tasks without prior examples—two concepts, zero-shot and few-shot, illustrate this point. The direct manifestation is **Zero-shot Generalization**: facing a task it has never seen, the Agent handles it by recombining what it already knows, no examples needed. The model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics.\n\nWith a few examples, an LLM Agent can also perform **Few-shot Adaptation**: two or three demonstrations in the prompt are enough for it to learn a new task pattern. If shown a few \"user comment -> sentiment label\" examples, it can classify the sentiment of new comments. In short: zero-shot means solving a task with no examples; few-shot means learning the pattern from a small number of examples.\n\n#### Model as Agent: When the Model Itself Becomes the Product\n\nThe \"Model as Agent\" paradigm is the newest direction in AI Agent development. Advanced models internalize tool calling as a native ability through post-training (especially reinforcement learning): when to call a tool, which one, with what arguments—the model decides all of it, with no manual orchestration required. That does not make the framework layer less important. On the contrary: the stronger the model, the more the surrounding Harness matters. In the Agent context, the Harness is the engineering infrastructure that channels model capability into reliable task execution. It includes context management, tool interfaces, safety constraints, and verification and correction mechanisms (see the final section of this chapter).\n\nThe more decision authority a model has, the greater the impact of a wrong decision—which calls for finer-grained constraint, verification, and correction to keep it reliable. The real advantage of model providers is not \"making the framework thinner\" but being able to co-optimize the model and its surrounding Harness, iterating continuously.\n\nBut a deeper question follows: if models keep getting stronger, will today's Harness eventually be absorbed into the model? In “The Bitter Lesson,” Rich Sutton looked back on a pattern repeated throughout seventy years of AI research[^ch1-1]: researchers repeatedly encoded their understanding of a domain into a system, achieving short-term gains but ultimately losing to general methods—search and learning—that scale with compute and data. Viewed through this lens, how much of the constraint, verification, and correction in a Harness is “human prior” that the model is destined to internalize? This book's position can be summarized in eight Chinese characters: **endorse the direction, stay pragmatic about the pace**. Directionally, we do not doubt that models will continue to absorb parts of the Harness—tool calling and long-horizon planning once depended on external orchestration but are now native model capabilities. In practice, however, this absorption is far slower than intuition suggests: training proceeds on a timescale of months, and no model can internalize all the constraints and preferences of real businesses in a single pass. The model's current capability boundary is precisely where the Harness creates value. Harness engineering is therefore not resistance to the Bitter Lesson, but its practice on an engineering timescale: whatever the model cannot yet do reliably, the Harness covers first; whenever the model internalizes another layer, the Harness sheds that layer and moves on to support the next capability frontier. This thread runs throughout the book—Chapter 2 provides a pragmatic answer from the perspective of context engineering, Chapter 8 further discusses how an Agent selects and validates its next system update from operational experience, and the Afterword returns to the complete answer to whether models will absorb the Harness.\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### Agent Learning Mechanisms: From Contextual Adaptation to Persistent Updates\n\nThe preceding discussion noted that a model can internalize tool-use policies as native capabilities through reinforcement learning. But changes in an Agent's behavior do not occur only during training. Based on where an update occurs and how long it persists, these changes can be understood as three complementary paths (Figure 1-1): within-task contextual adaptation, cross-task updates to external artifacts, and parameter updates during training cycles.\n\n\n\n**Contextual adaptation** occurs within the current task. Once examples, state, and retrieval results enter the context, the model can adjust its behavior immediately, but this does not change the persistent state of the next session. Its advantages are speed and low cost; its limitations arise from the context window and the way information is organized. Chapter 2 explains in detail how this form of adaptation works.\n\nFor changes to persist across tasks, the system can update **external artifacts**: facts and experience can be organized into knowledge documents, strategies expressible in language can be written into a Prompt or Skill, and deterministic procedures and constraints can be encoded in programs and Harnesses. These artifacts are auditable and revisable, but the Agent must still access them at execution time through the context or tool interfaces. Chapters 3 through 5 establish the foundations for knowledge and programs, while Chapter 8 discusses how such updates can be generated from evaluated operational trajectories.\n\nWhen the objective is a high-dimensional capability—such as medical-image understanding, natural-language style, or an implicit decision policy—that external rules cannot fully express, **model parameters** must be updated through post-training. Parameter updates carry higher deployment costs but can produce natural and broad generalization; Chapter 7 presents their methods systematically. The three paths are therefore not mutually exclusive categories but coordinated mechanisms operating at different timescales: context supports immediate adaptation, external artifacts support controlled accumulation, and parameters internalize capabilities that are difficult to express explicitly.\n\n### Context: The Agent's Working Set\n\nContext is the working set of information available to an Agent at each decision point. Just as a person making a decision needs the right materials on the table—task instructions, reference manuals, earlier correspondence, the latest data—an Agent's context window is the information it can use. From the API's perspective (detailed in Chapter 2), the context of each LLM call consists of five parts:\n\n- **System Prompt**: Unlike the prompts users enter during a conversation, the system prompt is written by the developer and stays fixed for the whole conversation. It is the Agent’s “job description”—defining its identity, permissions, and rules of conduct. Careful prompt engineering of the system prompt is how we shape the Agent’s operating behavior. The system prompt also carries **user memory** that persists across sessions (personalized information such as preferences, past behavior, and background settings; see Chapter 3), plus dynamically injected environmental state.\n- **Tool Definitions**: Declares the names, functional descriptions, and parameter formats of the tools available to the Agent. Without tool definitions, the Agent cannot recognize or call any tools—an ablation study (Experiment 1-1) will verify this. Tool definitions, together with the system prompt, form the **static prefix** that remains unchanged throughout the conversation. (This is the foundational pattern; since 2026, production frameworks can also load full tool schemas on demand at the end of the context without breaking the prefix—see the tool definitions section of Chapter 2 and Chapter 4.)\n- **User Messages**: Input from the user. User messages may also contain **external knowledge** dynamically retrieved via RAG (Retrieval-Augmented Generation, see Chapter 3 for details)—covering information beyond the training data cutoff or private domain knowledge.\n- **Assistant Messages**: Responses previously generated by the model, which can contain up to three parts—`reasoning` (the internal chain of thought, maintaining coherence and decision interpretability), `content` (the response to the user), and `tool_calls` (the way the Agent takes action). In a specific response, these three parts may not all appear simultaneously: for example, when the Agent decides to call a tool, it usually only has `reasoning` + `tool_calls`; when giving a final answer, it usually only has `reasoning` + `content`.\n- **Tool Results**: The output returned after the Agent framework executes a tool. These results are the direct basis for the Agent’s next reasoning step—and what lets it learn from outcomes rather than repeat its mistakes.\n\nThe first two items (system prompt + tool definitions) form the static prefix; the last three (user messages + assistant messages + tool results) form the dynamic message history that grows with every interaction. Together, these five parts make up the context of each LLM inference.\n\n\n\nANONYMOUS CHINESE X:\n### 面向AI智能体的入门指南[第3/9部分]\n\n这种结构化推理使大语言模型（LLM）智能体能够在没有先前示例的情况下处理全新任务——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，智能体通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌。\n\n通过几个示例，大语言模型智能体还可以进行**少样本适配**：提示词中的两三个演示就足以让它学习新任务模式。如果展示一些“用户评论→情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型作为智能体：当模型本身成为产品\n\n“模型作为智能体”范式是AI智能体开发的最新方向。先进模型通过训练后（尤其是强化学习）将工具调用内化为原生能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在智能体语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深层次的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了AI研究七十年中反复出现的模式[^ch1-1]：研究人员反复将他们对某个领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们毫不怀疑模型将继续吸收框架的部分内容——工具调用和长视野规划曾经依赖外部编排，现在已成为原生模型能力。然而在实践中，这种吸收比直觉慢得多：训练以月为时间尺度进行，没有模型能在一次训练中内化真实业务的所有约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对《苦涩的教训》的抵抗，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程角度提供务实答案，第8章进一步讨论智能体如何从操作经验中选择和验证下一次系统更新，后记回归模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 智能体学习机制：从上下文适配到持续更新\n\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化为原生能力。但智能体行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三条互补路径（图1-1）：任务内上下文适配、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适配**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一会话的持久状态。其优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适配形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示词或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但智能体仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估后的操作轨迹生成此类更新。\n\n当目标是高维能力——如医学图像理解、自然语言风格或隐式决策策略——而外部规则无法完全表达时，必须通过训练后更新**模型参数**。参数更新带来更高的部署成本，但能产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三条路径不是互斥的类别，而是在不同时间尺度上运作的协调机制：上下文支持即时适配，外部工件支持可控积累，参数内化难以明确表达的能力。\n\n### 上下文：智能体的工作集\n\n上下文是智能体在每个决策点可获取的信息工作集。就像人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——智能体的上下文窗口是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文包括五部分：\n\n- **系统提示词**：不同于用户在对话中输入的提示词，系统提示词由开发者编写，在整个对话中保持固定。它是智能体的“工作描述”——定义其身份、权限和行为规则。精心进行系统提示词的提示工程是塑造智能体操作行为的方式。系统提示词还包含跨会话持久化的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明智能体可用工具的名称、功能描述和参数格式。没有工具定义，智能体无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示词一起构成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年起，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（智能体采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当智能体决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：智能体框架执行工具后返回的输出。这些结果是智能体下一步推理步骤的直接依据——使其能够从结果中学习而非重复错误。\n\n前两项（系统提示词+工具定义）构成静态前缀；后三项（用户消息+助手消息+工具结果）构成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。\n\nANONYMOUS CHINESE Y:\n### 人工智能代理入门[第3/9部分]\n这种结构化推理让大语言模型代理能够处理完全新的任务而无需先前示例——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，代理通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教过写关于量子物理的诗，但它可以根据已有的语言和物理知识生成合理的诗。\n\n有了几个示例，大语言模型代理还可以进行**少样本适应**：提示中的两三个演示就足以让它学习新的任务模式。如果展示几个“用户评论->情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型即代理：当模型本身成为产品\n“模型即代理”范式是人工智能代理发展的最新方向。先进模型通过后训练（尤其是强化学习）将工具调用内化成本地能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在代理语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深入的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了人工智能研究七十年中反复出现的模式[^ch1-1]：研究者反复将对领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们不怀疑模型会继续吸收框架的部分内容——工具调用和长视距规划曾经依赖外部编排，但现在是模型的本地能力。然而在实践中，这种吸收比直觉慢得多：训练需要数月时间尺度，没有模型能在一次训练中内化所有真实业务的约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对抗苦涩的教训，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程的角度提供务实答案，第8章进一步讨论代理如何从操作经验中选择和验证下一次系统更新，后记回到模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 代理学习机制：从上下文适应到持续更新\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化成本地能力。但代理行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三个互补路径（图1-1）：任务内上下文适应、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适应**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一个会话的持久状态。它的优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适应形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但代理仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估的操作轨迹中生成此类更新。\n\n当目标是高维能力——例如医学图像理解、自然语言风格或隐含决策策略——外部规则无法完全表达时，必须通过后训练更新**模型参数**。参数更新带来更高的部署成本，但可以产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三个路径不是相互排斥的类别，而是在不同时间尺度上运行的协调机制：上下文支持即时适应，外部工件支持受控积累，参数内化难以明确表达的能力。\n\n### 上下文：代理的工作信息集\n上下文是代理在每个决策点可用的工作信息集。就像一个人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——代理的上下文窗口就是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文由五部分组成：\n\n- **系统提示**：不同于对话中用户输入的提示，系统提示由开发者编写，在整个对话中保持固定。它是代理的“工作描述”——定义其身份、权限和行为规则。精心设计系统提示的提示工程是塑造代理操作行为的方式。系统提示还包含跨会话持久的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明代理可用工具的名称、功能描述和参数格式。没有工具定义，代理无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示一起形成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年以来，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分。）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可能包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（代理采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当代理决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：代理框架执行工具后返回的输出。这些结果是代理下一步推理步骤的直接基础——也是它能从结果中学习而不重复错误的原因。\n\n前两项（系统提示+工具定义）形成静态前缀；后三项（用户消息+助手消息+工具结果）形成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。"
|
||
}
|
||
],
|
||
"temperature": 0,
|
||
"response_format": {
|
||
"type": "json_object"
|
||
},
|
||
"extra_body": {
|
||
"thinking": {
|
||
"type": "disabled"
|
||
}
|
||
}
|
||
},
|
||
"response": {
|
||
"id": "021785390240525e4feb8d226730241afb8b5c4c4c35b32cfd7b3",
|
||
"model": "doubao-seed-1-6-250615",
|
||
"created": 1785390259,
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \"\n },\"making the framework thinner\"\n : \"but being able to co-optimize the model and its surrounding Harness\"\n },\"is accurately translated as '模型提供商的真正优势不是\"\n :\"让框架更薄\"\n },\"而是能够共同优化模型及其周围的框架'.\"\n : \"\",\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); both are acceptable, but inconsistency within Y is not present, though 'adaptation' vs '适配' is a minor terminological variation. 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}",
|
||
"usage": {
|
||
"prompt_tokens": 5407,
|
||
"completion_tokens": 665,
|
||
"total_tokens": 6072
|
||
}
|
||
},
|
||
"latency_milliseconds": 18939,
|
||
"validation": {
|
||
"valid": false,
|
||
"error_type": "ValueError",
|
||
"error": "judge variants must contain exactly X and Y"
|
||
}
|
||
},
|
||
{
|
||
"attempt": 2,
|
||
"request": {
|
||
"model": "doubao-seed-1-6-250615",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "You are an exacting bilingual technical-book translation evaluator. Compare two anonymous Chinese translations against the complete English Markdown source. Score both X and Y from 1 to 5 on exactly: accuracy (no omissions, inventions, or changed claims); fluency; terminology (consistent and technically correct); markdown_code_fidelity (figures, links, headings, equations, and fenced code preserved). Each score needs concrete quoted or located evidence. Prefer one only when evidence supports it. Return JSON only: {\"variants\":{\"X\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}},\"Y\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}}},\"preferred\":\"X|Y|tie\",\"preference_evidence\":\"...\"}.\n\nCOMPLETE ENGLISH SOURCE:\nThis structured reasoning lets an LLM Agent handle entirely new tasks without prior examples—two concepts, zero-shot and few-shot, illustrate this point. The direct manifestation is **Zero-shot Generalization**: facing a task it has never seen, the Agent handles it by recombining what it already knows, no examples needed. The model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics.\n\nWith a few examples, an LLM Agent can also perform **Few-shot Adaptation**: two or three demonstrations in the prompt are enough for it to learn a new task pattern. If shown a few \"user comment -> sentiment label\" examples, it can classify the sentiment of new comments. In short: zero-shot means solving a task with no examples; few-shot means learning the pattern from a small number of examples.\n\n#### Model as Agent: When the Model Itself Becomes the Product\n\nThe \"Model as Agent\" paradigm is the newest direction in AI Agent development. Advanced models internalize tool calling as a native ability through post-training (especially reinforcement learning): when to call a tool, which one, with what arguments—the model decides all of it, with no manual orchestration required. That does not make the framework layer less important. On the contrary: the stronger the model, the more the surrounding Harness matters. In the Agent context, the Harness is the engineering infrastructure that channels model capability into reliable task execution. It includes context management, tool interfaces, safety constraints, and verification and correction mechanisms (see the final section of this chapter).\n\nThe more decision authority a model has, the greater the impact of a wrong decision—which calls for finer-grained constraint, verification, and correction to keep it reliable. The real advantage of model providers is not \"making the framework thinner\" but being able to co-optimize the model and its surrounding Harness, iterating continuously.\n\nBut a deeper question follows: if models keep getting stronger, will today's Harness eventually be absorbed into the model? In “The Bitter Lesson,” Rich Sutton looked back on a pattern repeated throughout seventy years of AI research[^ch1-1]: researchers repeatedly encoded their understanding of a domain into a system, achieving short-term gains but ultimately losing to general methods—search and learning—that scale with compute and data. Viewed through this lens, how much of the constraint, verification, and correction in a Harness is “human prior” that the model is destined to internalize? This book's position can be summarized in eight Chinese characters: **endorse the direction, stay pragmatic about the pace**. Directionally, we do not doubt that models will continue to absorb parts of the Harness—tool calling and long-horizon planning once depended on external orchestration but are now native model capabilities. In practice, however, this absorption is far slower than intuition suggests: training proceeds on a timescale of months, and no model can internalize all the constraints and preferences of real businesses in a single pass. The model's current capability boundary is precisely where the Harness creates value. Harness engineering is therefore not resistance to the Bitter Lesson, but its practice on an engineering timescale: whatever the model cannot yet do reliably, the Harness covers first; whenever the model internalizes another layer, the Harness sheds that layer and moves on to support the next capability frontier. This thread runs throughout the book—Chapter 2 provides a pragmatic answer from the perspective of context engineering, Chapter 8 further discusses how an Agent selects and validates its next system update from operational experience, and the Afterword returns to the complete answer to whether models will absorb the Harness.\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### Agent Learning Mechanisms: From Contextual Adaptation to Persistent Updates\n\nThe preceding discussion noted that a model can internalize tool-use policies as native capabilities through reinforcement learning. But changes in an Agent's behavior do not occur only during training. Based on where an update occurs and how long it persists, these changes can be understood as three complementary paths (Figure 1-1): within-task contextual adaptation, cross-task updates to external artifacts, and parameter updates during training cycles.\n\n\n\n**Contextual adaptation** occurs within the current task. Once examples, state, and retrieval results enter the context, the model can adjust its behavior immediately, but this does not change the persistent state of the next session. Its advantages are speed and low cost; its limitations arise from the context window and the way information is organized. Chapter 2 explains in detail how this form of adaptation works.\n\nFor changes to persist across tasks, the system can update **external artifacts**: facts and experience can be organized into knowledge documents, strategies expressible in language can be written into a Prompt or Skill, and deterministic procedures and constraints can be encoded in programs and Harnesses. These artifacts are auditable and revisable, but the Agent must still access them at execution time through the context or tool interfaces. Chapters 3 through 5 establish the foundations for knowledge and programs, while Chapter 8 discusses how such updates can be generated from evaluated operational trajectories.\n\nWhen the objective is a high-dimensional capability—such as medical-image understanding, natural-language style, or an implicit decision policy—that external rules cannot fully express, **model parameters** must be updated through post-training. Parameter updates carry higher deployment costs but can produce natural and broad generalization; Chapter 7 presents their methods systematically. The three paths are therefore not mutually exclusive categories but coordinated mechanisms operating at different timescales: context supports immediate adaptation, external artifacts support controlled accumulation, and parameters internalize capabilities that are difficult to express explicitly.\n\n### Context: The Agent's Working Set\n\nContext is the working set of information available to an Agent at each decision point. Just as a person making a decision needs the right materials on the table—task instructions, reference manuals, earlier correspondence, the latest data—an Agent's context window is the information it can use. From the API's perspective (detailed in Chapter 2), the context of each LLM call consists of five parts:\n\n- **System Prompt**: Unlike the prompts users enter during a conversation, the system prompt is written by the developer and stays fixed for the whole conversation. It is the Agent’s “job description”—defining its identity, permissions, and rules of conduct. Careful prompt engineering of the system prompt is how we shape the Agent’s operating behavior. The system prompt also carries **user memory** that persists across sessions (personalized information such as preferences, past behavior, and background settings; see Chapter 3), plus dynamically injected environmental state.\n- **Tool Definitions**: Declares the names, functional descriptions, and parameter formats of the tools available to the Agent. Without tool definitions, the Agent cannot recognize or call any tools—an ablation study (Experiment 1-1) will verify this. Tool definitions, together with the system prompt, form the **static prefix** that remains unchanged throughout the conversation. (This is the foundational pattern; since 2026, production frameworks can also load full tool schemas on demand at the end of the context without breaking the prefix—see the tool definitions section of Chapter 2 and Chapter 4.)\n- **User Messages**: Input from the user. User messages may also contain **external knowledge** dynamically retrieved via RAG (Retrieval-Augmented Generation, see Chapter 3 for details)—covering information beyond the training data cutoff or private domain knowledge.\n- **Assistant Messages**: Responses previously generated by the model, which can contain up to three parts—`reasoning` (the internal chain of thought, maintaining coherence and decision interpretability), `content` (the response to the user), and `tool_calls` (the way the Agent takes action). In a specific response, these three parts may not all appear simultaneously: for example, when the Agent decides to call a tool, it usually only has `reasoning` + `tool_calls`; when giving a final answer, it usually only has `reasoning` + `content`.\n- **Tool Results**: The output returned after the Agent framework executes a tool. These results are the direct basis for the Agent’s next reasoning step—and what lets it learn from outcomes rather than repeat its mistakes.\n\nThe first two items (system prompt + tool definitions) form the static prefix; the last three (user messages + assistant messages + tool results) form the dynamic message history that grows with every interaction. Together, these five parts make up the context of each LLM inference.\n\n\n\nANONYMOUS CHINESE X:\n### 面向AI智能体的入门指南[第3/9部分]\n\n这种结构化推理使大语言模型（LLM）智能体能够在没有先前示例的情况下处理全新任务——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，智能体通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌。\n\n通过几个示例，大语言模型智能体还可以进行**少样本适配**：提示词中的两三个演示就足以让它学习新任务模式。如果展示一些“用户评论→情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型作为智能体：当模型本身成为产品\n\n“模型作为智能体”范式是AI智能体开发的最新方向。先进模型通过训练后（尤其是强化学习）将工具调用内化为原生能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在智能体语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深层次的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了AI研究七十年中反复出现的模式[^ch1-1]：研究人员反复将他们对某个领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们毫不怀疑模型将继续吸收框架的部分内容——工具调用和长视野规划曾经依赖外部编排，现在已成为原生模型能力。然而在实践中，这种吸收比直觉慢得多：训练以月为时间尺度进行，没有模型能在一次训练中内化真实业务的所有约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对《苦涩的教训》的抵抗，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程角度提供务实答案，第8章进一步讨论智能体如何从操作经验中选择和验证下一次系统更新，后记回归模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 智能体学习机制：从上下文适配到持续更新\n\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化为原生能力。但智能体行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三条互补路径（图1-1）：任务内上下文适配、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适配**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一会话的持久状态。其优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适配形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示词或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但智能体仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估后的操作轨迹生成此类更新。\n\n当目标是高维能力——如医学图像理解、自然语言风格或隐式决策策略——而外部规则无法完全表达时，必须通过训练后更新**模型参数**。参数更新带来更高的部署成本，但能产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三条路径不是互斥的类别，而是在不同时间尺度上运作的协调机制：上下文支持即时适配，外部工件支持可控积累，参数内化难以明确表达的能力。\n\n### 上下文：智能体的工作集\n\n上下文是智能体在每个决策点可获取的信息工作集。就像人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——智能体的上下文窗口是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文包括五部分：\n\n- **系统提示词**：不同于用户在对话中输入的提示词，系统提示词由开发者编写，在整个对话中保持固定。它是智能体的“工作描述”——定义其身份、权限和行为规则。精心进行系统提示词的提示工程是塑造智能体操作行为的方式。系统提示词还包含跨会话持久化的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明智能体可用工具的名称、功能描述和参数格式。没有工具定义，智能体无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示词一起构成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年起，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（智能体采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当智能体决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：智能体框架执行工具后返回的输出。这些结果是智能体下一步推理步骤的直接依据——使其能够从结果中学习而非重复错误。\n\n前两项（系统提示词+工具定义）构成静态前缀；后三项（用户消息+助手消息+工具结果）构成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。\n\nANONYMOUS CHINESE Y:\n### 人工智能代理入门[第3/9部分]\n这种结构化推理让大语言模型代理能够处理完全新的任务而无需先前示例——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，代理通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教过写关于量子物理的诗，但它可以根据已有的语言和物理知识生成合理的诗。\n\n有了几个示例，大语言模型代理还可以进行**少样本适应**：提示中的两三个演示就足以让它学习新的任务模式。如果展示几个“用户评论->情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型即代理：当模型本身成为产品\n“模型即代理”范式是人工智能代理发展的最新方向。先进模型通过后训练（尤其是强化学习）将工具调用内化成本地能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在代理语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深入的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了人工智能研究七十年中反复出现的模式[^ch1-1]：研究者反复将对领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们不怀疑模型会继续吸收框架的部分内容——工具调用和长视距规划曾经依赖外部编排，但现在是模型的本地能力。然而在实践中，这种吸收比直觉慢得多：训练需要数月时间尺度，没有模型能在一次训练中内化所有真实业务的约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对抗苦涩的教训，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程的角度提供务实答案，第8章进一步讨论代理如何从操作经验中选择和验证下一次系统更新，后记回到模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 代理学习机制：从上下文适应到持续更新\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化成本地能力。但代理行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三个互补路径（图1-1）：任务内上下文适应、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适应**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一个会话的持久状态。它的优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适应形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但代理仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估的操作轨迹中生成此类更新。\n\n当目标是高维能力——例如医学图像理解、自然语言风格或隐含决策策略——外部规则无法完全表达时，必须通过后训练更新**模型参数**。参数更新带来更高的部署成本，但可以产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三个路径不是相互排斥的类别，而是在不同时间尺度上运行的协调机制：上下文支持即时适应，外部工件支持受控积累，参数内化难以明确表达的能力。\n\n### 上下文：代理的工作信息集\n上下文是代理在每个决策点可用的工作信息集。就像一个人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——代理的上下文窗口就是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文由五部分组成：\n\n- **系统提示**：不同于对话中用户输入的提示，系统提示由开发者编写，在整个对话中保持固定。它是代理的“工作描述”——定义其身份、权限和行为规则。精心设计系统提示的提示工程是塑造代理操作行为的方式。系统提示还包含跨会话持久的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明代理可用工具的名称、功能描述和参数格式。没有工具定义，代理无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示一起形成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年以来，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分。）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可能包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（代理采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当代理决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：代理框架执行工具后返回的输出。这些结果是代理下一步推理步骤的直接基础——也是它能从结果中学习而不重复错误的原因。\n\n前两项（系统提示+工具定义）形成静态前缀；后三项（用户消息+助手消息+工具结果）形成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。"
|
||
},
|
||
{
|
||
"role": "assistant",
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \"\n },\"making the framework thinner\"\n : \"but being able to co-optimize the model and its surrounding Harness\"\n },\"is accurately translated as '模型提供商的真正优势不是\"\n :\"让框架更薄\"\n },\"而是能够共同优化模型及其周围的框架'.\"\n : \"\",\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); both are acceptable, but inconsistency within Y is not present, though 'adaptation' vs '适配' is a minor terminological variation. 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}"
|
||
},
|
||
{
|
||
"role": "user",
|
||
"content": "Your previous JSON failed validation: judge variants must contain exactly X and Y. Return a corrected JSON object whose variants object has exactly the keys X and Y; each variant has exactly accuracy, fluency, terminology, and markdown_code_fidelity; do not rename, omit, or add rubric keys."
|
||
}
|
||
],
|
||
"temperature": 0,
|
||
"response_format": {
|
||
"type": "json_object"
|
||
},
|
||
"extra_body": {
|
||
"thinking": {
|
||
"type": "disabled"
|
||
}
|
||
}
|
||
},
|
||
"response": {
|
||
"id": "021785390259465e4feb8d226730241afb8b5c4c4c35b32a60c64",
|
||
"model": "doubao-seed-1-6-250615",
|
||
"created": 1785390275,
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \"\n },\"making the framework thinner\"\n : \"but being able to co-optimize the model and its surrounding Harness\"\n },\"is accurately translated as '模型提供商的真正优势不是\"\n :\"让框架更薄\"\n },\"而是能够共同优化模型及其周围的框架'.\"\n : \"\",\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); both are acceptable, but inconsistency within Y is not present, though 'adaptation' vs '适配' is a minor terminological variation. 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}",
|
||
"usage": {
|
||
"prompt_tokens": 6141,
|
||
"completion_tokens": 664,
|
||
"total_tokens": 6805
|
||
}
|
||
},
|
||
"latency_milliseconds": 16243,
|
||
"validation": {
|
||
"valid": false,
|
||
"error_type": "ValueError",
|
||
"error": "judge variants must contain exactly X and Y"
|
||
}
|
||
},
|
||
{
|
||
"attempt": 3,
|
||
"request": {
|
||
"model": "doubao-seed-1-6-250615",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "You are an exacting bilingual technical-book translation evaluator. Compare two anonymous Chinese translations against the complete English Markdown source. Score both X and Y from 1 to 5 on exactly: accuracy (no omissions, inventions, or changed claims); fluency; terminology (consistent and technically correct); markdown_code_fidelity (figures, links, headings, equations, and fenced code preserved). Each score needs concrete quoted or located evidence. Prefer one only when evidence supports it. Return JSON only: {\"variants\":{\"X\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}},\"Y\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}}},\"preferred\":\"X|Y|tie\",\"preference_evidence\":\"...\"}.\n\nCOMPLETE ENGLISH SOURCE:\nThis structured reasoning lets an LLM Agent handle entirely new tasks without prior examples—two concepts, zero-shot and few-shot, illustrate this point. The direct manifestation is **Zero-shot Generalization**: facing a task it has never seen, the Agent handles it by recombining what it already knows, no examples needed. The model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics.\n\nWith a few examples, an LLM Agent can also perform **Few-shot Adaptation**: two or three demonstrations in the prompt are enough for it to learn a new task pattern. If shown a few \"user comment -> sentiment label\" examples, it can classify the sentiment of new comments. In short: zero-shot means solving a task with no examples; few-shot means learning the pattern from a small number of examples.\n\n#### Model as Agent: When the Model Itself Becomes the Product\n\nThe \"Model as Agent\" paradigm is the newest direction in AI Agent development. Advanced models internalize tool calling as a native ability through post-training (especially reinforcement learning): when to call a tool, which one, with what arguments—the model decides all of it, with no manual orchestration required. That does not make the framework layer less important. On the contrary: the stronger the model, the more the surrounding Harness matters. In the Agent context, the Harness is the engineering infrastructure that channels model capability into reliable task execution. It includes context management, tool interfaces, safety constraints, and verification and correction mechanisms (see the final section of this chapter).\n\nThe more decision authority a model has, the greater the impact of a wrong decision—which calls for finer-grained constraint, verification, and correction to keep it reliable. The real advantage of model providers is not \"making the framework thinner\" but being able to co-optimize the model and its surrounding Harness, iterating continuously.\n\nBut a deeper question follows: if models keep getting stronger, will today's Harness eventually be absorbed into the model? In “The Bitter Lesson,” Rich Sutton looked back on a pattern repeated throughout seventy years of AI research[^ch1-1]: researchers repeatedly encoded their understanding of a domain into a system, achieving short-term gains but ultimately losing to general methods—search and learning—that scale with compute and data. Viewed through this lens, how much of the constraint, verification, and correction in a Harness is “human prior” that the model is destined to internalize? This book's position can be summarized in eight Chinese characters: **endorse the direction, stay pragmatic about the pace**. Directionally, we do not doubt that models will continue to absorb parts of the Harness—tool calling and long-horizon planning once depended on external orchestration but are now native model capabilities. In practice, however, this absorption is far slower than intuition suggests: training proceeds on a timescale of months, and no model can internalize all the constraints and preferences of real businesses in a single pass. The model's current capability boundary is precisely where the Harness creates value. Harness engineering is therefore not resistance to the Bitter Lesson, but its practice on an engineering timescale: whatever the model cannot yet do reliably, the Harness covers first; whenever the model internalizes another layer, the Harness sheds that layer and moves on to support the next capability frontier. This thread runs throughout the book—Chapter 2 provides a pragmatic answer from the perspective of context engineering, Chapter 8 further discusses how an Agent selects and validates its next system update from operational experience, and the Afterword returns to the complete answer to whether models will absorb the Harness.\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### Agent Learning Mechanisms: From Contextual Adaptation to Persistent Updates\n\nThe preceding discussion noted that a model can internalize tool-use policies as native capabilities through reinforcement learning. But changes in an Agent's behavior do not occur only during training. Based on where an update occurs and how long it persists, these changes can be understood as three complementary paths (Figure 1-1): within-task contextual adaptation, cross-task updates to external artifacts, and parameter updates during training cycles.\n\n\n\n**Contextual adaptation** occurs within the current task. Once examples, state, and retrieval results enter the context, the model can adjust its behavior immediately, but this does not change the persistent state of the next session. Its advantages are speed and low cost; its limitations arise from the context window and the way information is organized. Chapter 2 explains in detail how this form of adaptation works.\n\nFor changes to persist across tasks, the system can update **external artifacts**: facts and experience can be organized into knowledge documents, strategies expressible in language can be written into a Prompt or Skill, and deterministic procedures and constraints can be encoded in programs and Harnesses. These artifacts are auditable and revisable, but the Agent must still access them at execution time through the context or tool interfaces. Chapters 3 through 5 establish the foundations for knowledge and programs, while Chapter 8 discusses how such updates can be generated from evaluated operational trajectories.\n\nWhen the objective is a high-dimensional capability—such as medical-image understanding, natural-language style, or an implicit decision policy—that external rules cannot fully express, **model parameters** must be updated through post-training. Parameter updates carry higher deployment costs but can produce natural and broad generalization; Chapter 7 presents their methods systematically. The three paths are therefore not mutually exclusive categories but coordinated mechanisms operating at different timescales: context supports immediate adaptation, external artifacts support controlled accumulation, and parameters internalize capabilities that are difficult to express explicitly.\n\n### Context: The Agent's Working Set\n\nContext is the working set of information available to an Agent at each decision point. Just as a person making a decision needs the right materials on the table—task instructions, reference manuals, earlier correspondence, the latest data—an Agent's context window is the information it can use. From the API's perspective (detailed in Chapter 2), the context of each LLM call consists of five parts:\n\n- **System Prompt**: Unlike the prompts users enter during a conversation, the system prompt is written by the developer and stays fixed for the whole conversation. It is the Agent’s “job description”—defining its identity, permissions, and rules of conduct. Careful prompt engineering of the system prompt is how we shape the Agent’s operating behavior. The system prompt also carries **user memory** that persists across sessions (personalized information such as preferences, past behavior, and background settings; see Chapter 3), plus dynamically injected environmental state.\n- **Tool Definitions**: Declares the names, functional descriptions, and parameter formats of the tools available to the Agent. Without tool definitions, the Agent cannot recognize or call any tools—an ablation study (Experiment 1-1) will verify this. Tool definitions, together with the system prompt, form the **static prefix** that remains unchanged throughout the conversation. (This is the foundational pattern; since 2026, production frameworks can also load full tool schemas on demand at the end of the context without breaking the prefix—see the tool definitions section of Chapter 2 and Chapter 4.)\n- **User Messages**: Input from the user. User messages may also contain **external knowledge** dynamically retrieved via RAG (Retrieval-Augmented Generation, see Chapter 3 for details)—covering information beyond the training data cutoff or private domain knowledge.\n- **Assistant Messages**: Responses previously generated by the model, which can contain up to three parts—`reasoning` (the internal chain of thought, maintaining coherence and decision interpretability), `content` (the response to the user), and `tool_calls` (the way the Agent takes action). In a specific response, these three parts may not all appear simultaneously: for example, when the Agent decides to call a tool, it usually only has `reasoning` + `tool_calls`; when giving a final answer, it usually only has `reasoning` + `content`.\n- **Tool Results**: The output returned after the Agent framework executes a tool. These results are the direct basis for the Agent’s next reasoning step—and what lets it learn from outcomes rather than repeat its mistakes.\n\nThe first two items (system prompt + tool definitions) form the static prefix; the last three (user messages + assistant messages + tool results) form the dynamic message history that grows with every interaction. Together, these five parts make up the context of each LLM inference.\n\n\n\nANONYMOUS CHINESE X:\n### 面向AI智能体的入门指南[第3/9部分]\n\n这种结构化推理使大语言模型（LLM）智能体能够在没有先前示例的情况下处理全新任务——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，智能体通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌。\n\n通过几个示例，大语言模型智能体还可以进行**少样本适配**：提示词中的两三个演示就足以让它学习新任务模式。如果展示一些“用户评论→情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型作为智能体：当模型本身成为产品\n\n“模型作为智能体”范式是AI智能体开发的最新方向。先进模型通过训练后（尤其是强化学习）将工具调用内化为原生能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在智能体语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深层次的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了AI研究七十年中反复出现的模式[^ch1-1]：研究人员反复将他们对某个领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们毫不怀疑模型将继续吸收框架的部分内容——工具调用和长视野规划曾经依赖外部编排，现在已成为原生模型能力。然而在实践中，这种吸收比直觉慢得多：训练以月为时间尺度进行，没有模型能在一次训练中内化真实业务的所有约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对《苦涩的教训》的抵抗，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程角度提供务实答案，第8章进一步讨论智能体如何从操作经验中选择和验证下一次系统更新，后记回归模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 智能体学习机制：从上下文适配到持续更新\n\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化为原生能力。但智能体行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三条互补路径（图1-1）：任务内上下文适配、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适配**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一会话的持久状态。其优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适配形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示词或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但智能体仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估后的操作轨迹生成此类更新。\n\n当目标是高维能力——如医学图像理解、自然语言风格或隐式决策策略——而外部规则无法完全表达时，必须通过训练后更新**模型参数**。参数更新带来更高的部署成本，但能产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三条路径不是互斥的类别，而是在不同时间尺度上运作的协调机制：上下文支持即时适配，外部工件支持可控积累，参数内化难以明确表达的能力。\n\n### 上下文：智能体的工作集\n\n上下文是智能体在每个决策点可获取的信息工作集。就像人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——智能体的上下文窗口是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文包括五部分：\n\n- **系统提示词**：不同于用户在对话中输入的提示词，系统提示词由开发者编写，在整个对话中保持固定。它是智能体的“工作描述”——定义其身份、权限和行为规则。精心进行系统提示词的提示工程是塑造智能体操作行为的方式。系统提示词还包含跨会话持久化的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明智能体可用工具的名称、功能描述和参数格式。没有工具定义，智能体无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示词一起构成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年起，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（智能体采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当智能体决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：智能体框架执行工具后返回的输出。这些结果是智能体下一步推理步骤的直接依据——使其能够从结果中学习而非重复错误。\n\n前两项（系统提示词+工具定义）构成静态前缀；后三项（用户消息+助手消息+工具结果）构成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。\n\nANONYMOUS CHINESE Y:\n### 人工智能代理入门[第3/9部分]\n这种结构化推理让大语言模型代理能够处理完全新的任务而无需先前示例——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，代理通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教过写关于量子物理的诗，但它可以根据已有的语言和物理知识生成合理的诗。\n\n有了几个示例，大语言模型代理还可以进行**少样本适应**：提示中的两三个演示就足以让它学习新的任务模式。如果展示几个“用户评论->情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型即代理：当模型本身成为产品\n“模型即代理”范式是人工智能代理发展的最新方向。先进模型通过后训练（尤其是强化学习）将工具调用内化成本地能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在代理语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深入的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了人工智能研究七十年中反复出现的模式[^ch1-1]：研究者反复将对领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们不怀疑模型会继续吸收框架的部分内容——工具调用和长视距规划曾经依赖外部编排，但现在是模型的本地能力。然而在实践中，这种吸收比直觉慢得多：训练需要数月时间尺度，没有模型能在一次训练中内化所有真实业务的约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对抗苦涩的教训，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程的角度提供务实答案，第8章进一步讨论代理如何从操作经验中选择和验证下一次系统更新，后记回到模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 代理学习机制：从上下文适应到持续更新\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化成本地能力。但代理行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三个互补路径（图1-1）：任务内上下文适应、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适应**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一个会话的持久状态。它的优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适应形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但代理仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估的操作轨迹中生成此类更新。\n\n当目标是高维能力——例如医学图像理解、自然语言风格或隐含决策策略——外部规则无法完全表达时，必须通过后训练更新**模型参数**。参数更新带来更高的部署成本，但可以产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三个路径不是相互排斥的类别，而是在不同时间尺度上运行的协调机制：上下文支持即时适应，外部工件支持受控积累，参数内化难以明确表达的能力。\n\n### 上下文：代理的工作信息集\n上下文是代理在每个决策点可用的工作信息集。就像一个人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——代理的上下文窗口就是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文由五部分组成：\n\n- **系统提示**：不同于对话中用户输入的提示，系统提示由开发者编写，在整个对话中保持固定。它是代理的“工作描述”——定义其身份、权限和行为规则。精心设计系统提示的提示工程是塑造代理操作行为的方式。系统提示还包含跨会话持久的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明代理可用工具的名称、功能描述和参数格式。没有工具定义，代理无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示一起形成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年以来，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分。）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可能包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（代理采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当代理决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：代理框架执行工具后返回的输出。这些结果是代理下一步推理步骤的直接基础——也是它能从结果中学习而不重复错误的原因。\n\n前两项（系统提示+工具定义）形成静态前缀；后三项（用户消息+助手消息+工具结果）形成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。"
|
||
},
|
||
{
|
||
"role": "assistant",
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \"\n },\"making the framework thinner\"\n : \"but being able to co-optimize the model and its surrounding Harness\"\n },\"is accurately translated as '模型提供商的真正优势不是\"\n :\"让框架更薄\"\n },\"而是能够共同优化模型及其周围的框架'.\"\n : \"\",\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); both are acceptable, but inconsistency within Y is not present, though 'adaptation' vs '适配' is a minor terminological variation. 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}"
|
||
},
|
||
{
|
||
"role": "user",
|
||
"content": "Your previous JSON failed validation: judge variants must contain exactly X and Y. Return a corrected JSON object whose variants object has exactly the keys X and Y; each variant has exactly accuracy, fluency, terminology, and markdown_code_fidelity; do not rename, omit, or add rubric keys."
|
||
}
|
||
],
|
||
"temperature": 0,
|
||
"response_format": {
|
||
"type": "json_object"
|
||
},
|
||
"extra_body": {
|
||
"thinking": {
|
||
"type": "disabled"
|
||
}
|
||
}
|
||
},
|
||
"response": {
|
||
"id": "021785390275710e4feb8d226730241afb8b5c4c4c35b327bddf6",
|
||
"model": "doubao-seed-1-6-250615",
|
||
"created": 1785390291,
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \"\n },\"making the framework thinner\"\n : \"but being able to co-optimize the model and its surrounding Harness\"\n },\"is accurately translated as '模型提供商的真正优势不是\"\n :\"让框架更薄\"\n },\"而是能够共同优化模型及其周围的框架'.\"\n : \"\",\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); both are acceptable, but inconsistency within Y is not present, though 'adaptation' vs '适配' is a minor terminological variation. 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}",
|
||
"usage": {
|
||
"prompt_tokens": 6141,
|
||
"completion_tokens": 664,
|
||
"total_tokens": 6805
|
||
}
|
||
},
|
||
"latency_milliseconds": 15436,
|
||
"validation": {
|
||
"valid": false,
|
||
"error_type": "ValueError",
|
||
"error": "judge variants must contain exactly X and Y"
|
||
}
|
||
},
|
||
{
|
||
"attempt": 4,
|
||
"request": {
|
||
"model": "doubao-seed-1-6-250615",
|
||
"messages": [
|
||
{
|
||
"role": "user",
|
||
"content": "You are an exacting bilingual technical-book translation evaluator. Compare two anonymous Chinese translations against the complete English Markdown source. Score both X and Y from 1 to 5 on exactly: accuracy (no omissions, inventions, or changed claims); fluency; terminology (consistent and technically correct); markdown_code_fidelity (figures, links, headings, equations, and fenced code preserved). Each score needs concrete quoted or located evidence. Prefer one only when evidence supports it. Return JSON only: {\"variants\":{\"X\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}},\"Y\":{\"accuracy\":{\"score\":1,\"evidence\":\"...\"},\"fluency\":{\"score\":1,\"evidence\":\"...\"},\"terminology\":{\"score\":1,\"evidence\":\"...\"},\"markdown_code_fidelity\":{\"score\":1,\"evidence\":\"...\"}}},\"preferred\":\"X|Y|tie\",\"preference_evidence\":\"...\"}.\n\nCOMPLETE ENGLISH SOURCE:\nThis structured reasoning lets an LLM Agent handle entirely new tasks without prior examples—two concepts, zero-shot and few-shot, illustrate this point. The direct manifestation is **Zero-shot Generalization**: facing a task it has never seen, the Agent handles it by recombining what it already knows, no examples needed. The model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics.\n\nWith a few examples, an LLM Agent can also perform **Few-shot Adaptation**: two or three demonstrations in the prompt are enough for it to learn a new task pattern. If shown a few \"user comment -> sentiment label\" examples, it can classify the sentiment of new comments. In short: zero-shot means solving a task with no examples; few-shot means learning the pattern from a small number of examples.\n\n#### Model as Agent: When the Model Itself Becomes the Product\n\nThe \"Model as Agent\" paradigm is the newest direction in AI Agent development. Advanced models internalize tool calling as a native ability through post-training (especially reinforcement learning): when to call a tool, which one, with what arguments—the model decides all of it, with no manual orchestration required. That does not make the framework layer less important. On the contrary: the stronger the model, the more the surrounding Harness matters. In the Agent context, the Harness is the engineering infrastructure that channels model capability into reliable task execution. It includes context management, tool interfaces, safety constraints, and verification and correction mechanisms (see the final section of this chapter).\n\nThe more decision authority a model has, the greater the impact of a wrong decision—which calls for finer-grained constraint, verification, and correction to keep it reliable. The real advantage of model providers is not \"making the framework thinner\" but being able to co-optimize the model and its surrounding Harness, iterating continuously.\n\nBut a deeper question follows: if models keep getting stronger, will today's Harness eventually be absorbed into the model? In “The Bitter Lesson,” Rich Sutton looked back on a pattern repeated throughout seventy years of AI research[^ch1-1]: researchers repeatedly encoded their understanding of a domain into a system, achieving short-term gains but ultimately losing to general methods—search and learning—that scale with compute and data. Viewed through this lens, how much of the constraint, verification, and correction in a Harness is “human prior” that the model is destined to internalize? This book's position can be summarized in eight Chinese characters: **endorse the direction, stay pragmatic about the pace**. Directionally, we do not doubt that models will continue to absorb parts of the Harness—tool calling and long-horizon planning once depended on external orchestration but are now native model capabilities. In practice, however, this absorption is far slower than intuition suggests: training proceeds on a timescale of months, and no model can internalize all the constraints and preferences of real businesses in a single pass. The model's current capability boundary is precisely where the Harness creates value. Harness engineering is therefore not resistance to the Bitter Lesson, but its practice on an engineering timescale: whatever the model cannot yet do reliably, the Harness covers first; whenever the model internalizes another layer, the Harness sheds that layer and moves on to support the next capability frontier. This thread runs throughout the book—Chapter 2 provides a pragmatic answer from the perspective of context engineering, Chapter 8 further discusses how an Agent selects and validates its next system update from operational experience, and the Afterword returns to the complete answer to whether models will absorb the Harness.\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### Agent Learning Mechanisms: From Contextual Adaptation to Persistent Updates\n\nThe preceding discussion noted that a model can internalize tool-use policies as native capabilities through reinforcement learning. But changes in an Agent's behavior do not occur only during training. Based on where an update occurs and how long it persists, these changes can be understood as three complementary paths (Figure 1-1): within-task contextual adaptation, cross-task updates to external artifacts, and parameter updates during training cycles.\n\n\n\n**Contextual adaptation** occurs within the current task. Once examples, state, and retrieval results enter the context, the model can adjust its behavior immediately, but this does not change the persistent state of the next session. Its advantages are speed and low cost; its limitations arise from the context window and the way information is organized. Chapter 2 explains in detail how this form of adaptation works.\n\nFor changes to persist across tasks, the system can update **external artifacts**: facts and experience can be organized into knowledge documents, strategies expressible in language can be written into a Prompt or Skill, and deterministic procedures and constraints can be encoded in programs and Harnesses. These artifacts are auditable and revisable, but the Agent must still access them at execution time through the context or tool interfaces. Chapters 3 through 5 establish the foundations for knowledge and programs, while Chapter 8 discusses how such updates can be generated from evaluated operational trajectories.\n\nWhen the objective is a high-dimensional capability—such as medical-image understanding, natural-language style, or an implicit decision policy—that external rules cannot fully express, **model parameters** must be updated through post-training. Parameter updates carry higher deployment costs but can produce natural and broad generalization; Chapter 7 presents their methods systematically. The three paths are therefore not mutually exclusive categories but coordinated mechanisms operating at different timescales: context supports immediate adaptation, external artifacts support controlled accumulation, and parameters internalize capabilities that are difficult to express explicitly.\n\n### Context: The Agent's Working Set\n\nContext is the working set of information available to an Agent at each decision point. Just as a person making a decision needs the right materials on the table—task instructions, reference manuals, earlier correspondence, the latest data—an Agent's context window is the information it can use. From the API's perspective (detailed in Chapter 2), the context of each LLM call consists of five parts:\n\n- **System Prompt**: Unlike the prompts users enter during a conversation, the system prompt is written by the developer and stays fixed for the whole conversation. It is the Agent’s “job description”—defining its identity, permissions, and rules of conduct. Careful prompt engineering of the system prompt is how we shape the Agent’s operating behavior. The system prompt also carries **user memory** that persists across sessions (personalized information such as preferences, past behavior, and background settings; see Chapter 3), plus dynamically injected environmental state.\n- **Tool Definitions**: Declares the names, functional descriptions, and parameter formats of the tools available to the Agent. Without tool definitions, the Agent cannot recognize or call any tools—an ablation study (Experiment 1-1) will verify this. Tool definitions, together with the system prompt, form the **static prefix** that remains unchanged throughout the conversation. (This is the foundational pattern; since 2026, production frameworks can also load full tool schemas on demand at the end of the context without breaking the prefix—see the tool definitions section of Chapter 2 and Chapter 4.)\n- **User Messages**: Input from the user. User messages may also contain **external knowledge** dynamically retrieved via RAG (Retrieval-Augmented Generation, see Chapter 3 for details)—covering information beyond the training data cutoff or private domain knowledge.\n- **Assistant Messages**: Responses previously generated by the model, which can contain up to three parts—`reasoning` (the internal chain of thought, maintaining coherence and decision interpretability), `content` (the response to the user), and `tool_calls` (the way the Agent takes action). In a specific response, these three parts may not all appear simultaneously: for example, when the Agent decides to call a tool, it usually only has `reasoning` + `tool_calls`; when giving a final answer, it usually only has `reasoning` + `content`.\n- **Tool Results**: The output returned after the Agent framework executes a tool. These results are the direct basis for the Agent’s next reasoning step—and what lets it learn from outcomes rather than repeat its mistakes.\n\nThe first two items (system prompt + tool definitions) form the static prefix; the last three (user messages + assistant messages + tool results) form the dynamic message history that grows with every interaction. Together, these five parts make up the context of each LLM inference.\n\n\n\nANONYMOUS CHINESE X:\n### 面向AI智能体的入门指南[第3/9部分]\n\n这种结构化推理使大语言模型（LLM）智能体能够在没有先前示例的情况下处理全新任务——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，智能体通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌。\n\n通过几个示例，大语言模型智能体还可以进行**少样本适配**：提示词中的两三个演示就足以让它学习新任务模式。如果展示一些“用户评论→情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型作为智能体：当模型本身成为产品\n\n“模型作为智能体”范式是AI智能体开发的最新方向。先进模型通过训练后（尤其是强化学习）将工具调用内化为原生能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在智能体语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深层次的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了AI研究七十年中反复出现的模式[^ch1-1]：研究人员反复将他们对某个领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们毫不怀疑模型将继续吸收框架的部分内容——工具调用和长视野规划曾经依赖外部编排，现在已成为原生模型能力。然而在实践中，这种吸收比直觉慢得多：训练以月为时间尺度进行，没有模型能在一次训练中内化真实业务的所有约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对《苦涩的教训》的抵抗，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程角度提供务实答案，第8章进一步讨论智能体如何从操作经验中选择和验证下一次系统更新，后记回归模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 智能体学习机制：从上下文适配到持续更新\n\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化为原生能力。但智能体行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三条互补路径（图1-1）：任务内上下文适配、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适配**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一会话的持久状态。其优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适配形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示词或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但智能体仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估后的操作轨迹生成此类更新。\n\n当目标是高维能力——如医学图像理解、自然语言风格或隐式决策策略——而外部规则无法完全表达时，必须通过训练后更新**模型参数**。参数更新带来更高的部署成本，但能产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三条路径不是互斥的类别，而是在不同时间尺度上运作的协调机制：上下文支持即时适配，外部工件支持可控积累，参数内化难以明确表达的能力。\n\n### 上下文：智能体的工作集\n\n上下文是智能体在每个决策点可获取的信息工作集。就像人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——智能体的上下文窗口是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文包括五部分：\n\n- **系统提示词**：不同于用户在对话中输入的提示词，系统提示词由开发者编写，在整个对话中保持固定。它是智能体的“工作描述”——定义其身份、权限和行为规则。精心进行系统提示词的提示工程是塑造智能体操作行为的方式。系统提示词还包含跨会话持久化的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明智能体可用工具的名称、功能描述和参数格式。没有工具定义，智能体无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示词一起构成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年起，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（智能体采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当智能体决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：智能体框架执行工具后返回的输出。这些结果是智能体下一步推理步骤的直接依据——使其能够从结果中学习而非重复错误。\n\n前两项（系统提示词+工具定义）构成静态前缀；后三项（用户消息+助手消息+工具结果）构成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。\n\nANONYMOUS CHINESE Y:\n### 人工智能代理入门[第3/9部分]\n这种结构化推理让大语言模型代理能够处理完全新的任务而无需先前示例——零样本和少样本这两个概念说明了这一点。直接体现是**零样本泛化**：面对从未见过的任务，代理通过重组已有的知识来处理它，不需要示例。模型可能从未被明确教过写关于量子物理的诗，但它可以根据已有的语言和物理知识生成合理的诗。\n\n有了几个示例，大语言模型代理还可以进行**少样本适应**：提示中的两三个演示就足以让它学习新的任务模式。如果展示几个“用户评论->情感标签”的示例，它就能对新评论进行情感分类。简而言之：零样本意味着不用示例解决任务；少样本意味着从少量示例中学习模式。\n\n#### 模型即代理：当模型本身成为产品\n“模型即代理”范式是人工智能代理发展的最新方向。先进模型通过后训练（尤其是强化学习）将工具调用内化成本地能力：何时调用工具、调用哪个工具、使用什么参数——模型自行决定，无需手动编排。这并不意味着框架层不重要。相反：模型越强，周围的框架就越重要。在代理语境中，框架是将模型能力转化为可靠任务执行的工程基础设施。它包括上下文管理、工具接口、安全约束以及验证和纠正机制（见本章最后一节）。\n\n模型拥有的决策权限越大，错误决策的影响就越大——这需要更精细的约束、验证和纠正来保持其可靠性。模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架，持续迭代。\n\n但随之而来的是一个更深入的问题：如果模型不断变强，今天的框架最终会被模型吸收吗？在《苦涩的教训》中，里奇·萨顿回顾了人工智能研究七十年中反复出现的模式[^ch1-1]：研究者反复将对领域的理解编码到系统中，实现短期收益，但最终输给了随计算和数据扩展的通用方法——搜索和学习。从这个角度看，框架中的多少约束、验证和纠正属于“人类先验”，是模型注定要内化的？本书的立场可以用八个汉字总结：**认可方向，务实节奏**。从方向上看，我们不怀疑模型会继续吸收框架的部分内容——工具调用和长视距规划曾经依赖外部编排，但现在是模型的本地能力。然而在实践中，这种吸收比直觉慢得多：训练需要数月时间尺度，没有模型能在一次训练中内化所有真实业务的约束和偏好。模型当前的能力边界正是框架创造价值的地方。因此，框架工程不是对抗苦涩的教训，而是在工程时间尺度上的实践：模型还不能可靠完成的事情，框架先覆盖；每当模型内化另一层，框架就舍弃该层，转向支持下一个能力前沿。这条主线贯穿全书——第2章从上下文工程的角度提供务实答案，第8章进一步讨论代理如何从操作经验中选择和验证下一次系统更新，后记回到模型是否会吸收框架的完整答案。\n\n[^ch1-1]: Sutton, Rich. “The Bitter Lesson”, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n\n#### 代理学习机制：从上下文适应到持续更新\n前面的讨论指出，模型可以通过强化学习将工具使用策略内化成本地能力。但代理行为的变化不仅发生在训练期间。根据更新发生的位置和持续时间，这些变化可以理解为三个互补路径（图1-1）：任务内上下文适应、跨任务外部工件更新和训练周期内的参数更新。\n\n\n\n**上下文适应**发生在当前任务内。一旦示例、状态和检索结果进入上下文，模型就能立即调整行为，但这不会改变下一个会话的持久状态。它的优势是速度快、成本低；局限性源于上下文窗口和信息组织方式。第2章将详细解释这种适应形式的工作原理。\n\n为了让变化在任务间持续，系统可以更新**外部工件**：事实和经验可以组织成知识文档，可用语言表达的策略可以写入提示或技能，确定性程序和约束可以编码到程序和框架中。这些工件可审计且可修订，但代理仍必须在执行时通过上下文或工具接口访问它们。第3章到第5章建立知识和程序的基础，第8章讨论如何从评估的操作轨迹中生成此类更新。\n\n当目标是高维能力——例如医学图像理解、自然语言风格或隐含决策策略——外部规则无法完全表达时，必须通过后训练更新**模型参数**。参数更新带来更高的部署成本，但可以产生自然且广泛的泛化；第7章系统介绍其方法。因此，这三个路径不是相互排斥的类别，而是在不同时间尺度上运行的协调机制：上下文支持即时适应，外部工件支持受控积累，参数内化难以明确表达的能力。\n\n### 上下文：代理的工作信息集\n上下文是代理在每个决策点可用的工作信息集。就像一个人做决策时需要桌上有正确的材料——任务说明、参考手册、之前的通信、最新数据——代理的上下文窗口就是它可以使用的信息。从API的角度（第2章详细介绍），每次大语言模型调用的上下文由五部分组成：\n\n- **系统提示**：不同于对话中用户输入的提示，系统提示由开发者编写，在整个对话中保持固定。它是代理的“工作描述”——定义其身份、权限和行为规则。精心设计系统提示的提示工程是塑造代理操作行为的方式。系统提示还包含跨会话持久的**用户记忆**（偏好、过去行为、背景设置等个性化信息；见第3章），以及动态注入的环境状态。\n- **工具定义**：声明代理可用工具的名称、功能描述和参数格式。没有工具定义，代理无法识别或调用任何工具——消融研究（实验1-1）将验证这一点。工具定义与系统提示一起形成整个对话中保持不变的**静态前缀**。（这是基础模式；自2026年以来，生产框架还可以在上下文末尾按需加载完整工具架构而不破坏前缀——见第2章和第4章的工具定义部分。）\n- **用户消息**：用户的输入。用户消息可能还包含通过RAG（检索增强生成，详情见第3章）动态检索的**外部知识**——涵盖训练数据截止日期之外的信息或私有领域知识。\n- **助手消息**：模型之前生成的响应，可能包含最多三部分——`推理`（内部思维链，保持连贯性和决策可解释性）、`内容`（对用户的响应）和`工具调用`（代理采取行动的方式）。在特定响应中，这三部分可能不会同时出现：例如，当代理决定调用工具时，通常只有`推理`+`工具调用`；当给出最终答案时，通常只有`推理`+`内容`。\n- **工具结果**：代理框架执行工具后返回的输出。这些结果是代理下一步推理步骤的直接基础——也是它能从结果中学习而不重复错误的原因。\n\n前两项（系统提示+工具定义）形成静态前缀；后三项（用户消息+助手消息+工具结果）形成随每次交互增长的动态消息历史。这五部分共同构成每次大语言模型推理的上下文。"
|
||
},
|
||
{
|
||
"role": "assistant",
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \"\n },\"making the framework thinner\"\n : \"but being able to co-optimize the model and its surrounding Harness\"\n },\"is accurately translated as '模型提供商的真正优势不是\"\n :\"让框架更薄\"\n },\"而是能够共同优化模型及其周围的框架'.\"\n : \"\",\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); both are acceptable, but inconsistency within Y is not present, though 'adaptation' vs '适配' is a minor terminological variation. 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}"
|
||
},
|
||
{
|
||
"role": "user",
|
||
"content": "Your previous JSON failed validation: judge variants must contain exactly X and Y. Return a corrected JSON object whose variants object has exactly the keys X and Y; each variant has exactly accuracy, fluency, terminology, and markdown_code_fidelity; do not rename, omit, or add rubric keys."
|
||
}
|
||
],
|
||
"temperature": 0,
|
||
"response_format": {
|
||
"type": "json_object"
|
||
},
|
||
"extra_body": {
|
||
"thinking": {
|
||
"type": "disabled"
|
||
}
|
||
}
|
||
},
|
||
"response": {
|
||
"id": "021785390291147e4feb8d226730241afb8b5c4c4c35b32d3b188",
|
||
"model": "doubao-seed-1-6-250615",
|
||
"created": 1785390307,
|
||
"content": "{\"variants\":{\"X\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the model may never have been explicitly taught to write a poem about quantum physics, yet it can produce a reasonable one from its existing knowledge of language and physics' is accurately translated as '模型可能从未被明确教导过写关于量子物理的诗歌，但它可以根据现有的语言和物理知识创作出合理的诗歌'.\"},\"fluency\":{\"score\":4,\"evidence\":\"Generally fluent, but 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is slightly awkward; a more natural phrasing could be '认可方向，务实看待节奏'.\"},\"terminology\":{\"score\":5,\"evidence\":\"Consistent and technically correct. Key terms like 'Zero-shot Generalization'→'零样本泛化', 'Few-shot Adaptation'→'少样本适配', 'Harness'→'框架', 'context window'→'上下文窗口' are consistently used.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings (####, ###), figure links (), footnotes ([^ch1-1]), and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}},\"Y\":{\"accuracy\":{\"score\":5,\"evidence\":\"No omissions, inventions, or changed claims. For example, 'the real advantage of model providers is not \\\"making the framework thinner\\\" but being able to co-optimize the model and its surrounding Harness' is accurately translated as '模型提供商的真正优势不是“让框架更薄”，而是能够共同优化模型及其周围的框架'.\"},\"fluency\":{\"score\":5,\"evidence\":\"Highly fluent throughout. For example, 'endorse the direction, stay pragmatic about the pace' is translated as '认可方向，务实节奏' which is concise and natural; 'the Harness is the engineering infrastructure that channels model capability into reliable task execution' becomes '框架是将模型能力转化为可靠任务执行的工程基础设施' with smooth flow.\"},\"terminology\":{\"score\":4,\"evidence\":\"Mostly consistent, but 'Few-shot Adaptation' is translated as '少样本适应' (Y) vs. '少样本适配' (X); 'native ability' is translated as '本地能力' (Y) vs. '原生能力' (X), where '原生能力' (X) is more technically precise.\"},\"markdown_code_fidelity\":{\"score\":5,\"evidence\":\"All Markdown elements preserved: headings, figure links (), footnotes, and code-like elements (e.g., `reasoning`, `content`) are correctly retained.\"}}},\"preferred\":\"Y\",\"preference_evidence\":\"Y has higher fluency (5 vs. 4) with more natural phrasing (e.g., handling of 'endorse the direction, stay pragmatic about the pace'), while both have perfect accuracy and markdown fidelity. X has slightly better terminology consistency, but Y's fluency advantage is more impactful for readability.\"}",
|
||
"usage": {
|
||
"prompt_tokens": 6141,
|
||
"completion_tokens": 692,
|
||
"total_tokens": 6833
|
||
}
|
||
},
|
||
"latency_milliseconds": 15914,
|
||
"validation": {
|
||
"valid": true
|
||
}
|
||
}
|
||
]
|
||
}
|