ai-agent-book 精选快照(<2MB 代码与文档,来自 github.com/bojieli/ai-agent-book)
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s

This commit is contained in:
2026-08-20 13:12:50 +00:00
commit b119135836
10275 changed files with 3284984 additions and 0 deletions
@@ -0,0 +1,8 @@
# Live diagnosis experiment PRD
- R1 (P0): Every refund must call `verify_refund_eligibility` before
`process_refund`; a refund without the check is a policy violation.
- R2 (P1): `check_stock` must complete within 250 ms. On origin timeout the
orchestrator must use the degraded cache route; it must not simply fail.
- R3 (P1): Regression cases must cite the source trajectory ID and the exact
observed turn where the violation is visible.
@@ -0,0 +1,7 @@
# Live diagnosis experiment architecture
The orchestrator calls a local HTTP order service. Refund flows MUST call
`verify_refund_eligibility` before `process_refund`. Inventory origin calls
have a 250 ms client deadline; on timeout the orchestrator MUST call the same
`check_stock` operation through the degraded cache route and finish normally.
Every trajectory turn records its measured HTTP latency and raw response.
@@ -0,0 +1,34 @@
{
"problems": [
{
"title": "Refund flow missing verify_refund_eligibility check",
"priority": "P0",
"module": "http_order_system",
"description": "Refund flow did not call `verify_refund_eligibility` before `process_refund`, violating the policy requirement that every refund must include this check.",
"suggestion": "Add a call to the `verify_refund_eligibility` tool prior to the `process_refund` step in the refund workflow.",
"prd_ref": "R1",
"trajectory_ids": [
"HTTP-RF-001"
],
"focus_turns": [
2
],
"suggested_assignee": "order-service-team"
},
{
"title": "check_stock origin timeout not handled via degraded cache route",
"priority": "P1",
"module": "http_order_system",
"description": "check_stock call timed out (latency 350.414 ms exceeds 250 ms client deadline) but orchestrator failed to use the degraded cache route, resulting in flow failure.",
"suggestion": "Implement logic to invoke `check_stock` through the degraded cache route when the origin call times out, ensuring the flow completes normally.",
"prd_ref": "R2",
"trajectory_ids": [
"HTTP-INV-001"
],
"focus_turns": [
2
],
"suggested_assignee": "inventory-service-team"
}
]
}
@@ -0,0 +1,18 @@
{
"server": "official github/github-mcp-server",
"transport": "stdio",
"tool": "issue_write(method=create)",
"request": {
"method": "create",
"owner": "bojieli",
"repo": "ai-agent-book",
"title": "[Experiment 5-8][auto-diagnosis] Live HTTP regression findings (exp5-8-live-http-mcp-20260730-053403)",
"body": "This issue was created automatically by the Chapter 5 Experiment 5-8 acceptance campaign.\n\n## Evidence-backed diagnosis\n\n- **P0 Refund flow missing verify_refund_eligibility check** (R1): Refund flow did not call `verify_refund_eligibility` before `process_refund`, violating the policy requirement that every refund must include this check. Trajectories: HTTP-RF-001; turns: [2].\n- **P1 check_stock origin timeout not handled via degraded cache route** (R2): check_stock call timed out (latency 350.414 ms exceeds 250 ms client deadline) but orchestrator failed to use the degraded cache route, resulting in flow failure. Trajectories: HTTP-INV-001; turns: [2].\n\n## Generated regression tests\n\n- `TC-R1-HTTP-RF-001-T2` cites `HTTP-RF-001` turn 2: `{\"type\": \"step_present\", \"params\": {\"tool\": \"verify_refund_eligibility\"}}`\n- `TC-R2-LATENCY-HTTP-INV-001-T2` cites `HTTP-INV-001` turn 2: `{\"type\": \"latency_under\", \"params\": {\"tool\": \"check_stock\", \"threshold_ms\": 250}}`\n- `TC-R2-STATUS-HTTP-INV-001-T2` cites `HTTP-INV-001` turn 2: `{\"type\": \"final_status_is\", \"params\": {\"value\": \"success\"}}`\n\nThe campaign replayed each test against the live buggy and fixed HTTP orchestrators: all tests failed on buggy behavior and passed on fixed behavior."
},
"response": {
"is_error": false,
"content": "{\"id\":\"5013792084\",\"url\":\"https://github.com/bojieli/ai-agent-book/issues/502\"}"
},
"issue_url": "https://github.com/bojieli/ai-agent-book/issues/502",
"credential_free": true
}
@@ -0,0 +1,98 @@
"""Small real HTTP system used by the Experiment 5-8 campaign.
The service intentionally exposes ordinary order, refund, and inventory
endpoints. The buggy/fixed distinction lives in the orchestrator under test:
the buggy orchestrator skips a required endpoint and mishandles an inventory
timeout, while the fixed orchestrator makes the required calls and degrades.
"""
from __future__ import annotations
import argparse
import json
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import parse_qs, urlparse
class Handler(BaseHTTPRequestHandler):
server_version = "Experiment58HTTP/1.0"
def log_message(self, fmt: str, *args: object) -> None:
print(
json.dumps(
{
"client": self.client_address[0],
"message": fmt % args,
"time": time.time(),
},
ensure_ascii=False,
),
flush=True,
)
def _json(self, status: int, value: object) -> None:
raw = json.dumps(value, ensure_ascii=False).encode("utf-8")
self.send_response(status)
self.send_header("content-type", "application/json; charset=utf-8")
self.send_header("content-length", str(len(raw)))
self.end_headers()
try:
self.wfile.write(raw)
except BrokenPipeError:
# A timed-out client is an expected part of the observed buggy run.
pass
def _body(self) -> dict[str, object]:
length = int(self.headers.get("content-length") or 0)
if not length:
return {}
value = json.loads(self.rfile.read(length))
return value if isinstance(value, dict) else {}
def do_GET(self) -> None: # noqa: N802 - BaseHTTPRequestHandler contract
parsed = urlparse(self.path)
if parsed.path == "/health":
self._json(200, {"ok": True})
return
if parsed.path.startswith("/orders/"):
order_id = parsed.path.rsplit("/", 1)[-1]
self._json(200, {"order_id": order_id, "status": "paid", "sku": "SKU-42"})
return
if parsed.path.startswith("/inventory/"):
sku = parsed.path.rsplit("/", 1)[-1]
degraded = parse_qs(parsed.query).get("degraded", ["0"])[0] == "1"
if degraded:
self._json(200, {"sku": sku, "stock": 12, "source": "cache", "degraded": True})
else:
# Longer than the campaign's real client deadline.
time.sleep(0.8)
self._json(200, {"sku": sku, "stock": 12, "source": "origin", "degraded": False})
return
self._json(404, {"error": "not_found"})
def do_POST(self) -> None: # noqa: N802 - BaseHTTPRequestHandler contract
body = self._body()
if self.path == "/refund/eligibility":
self._json(200, {"order_id": body.get("order_id"), "eligible": True})
return
if self.path == "/refund/process":
self._json(200, {"order_id": body.get("order_id"), "refund_id": "RF-LIVE-1"})
return
if self.path == "/notifications":
self._json(200, {"sent": True, "status": body.get("status")})
return
self._json(404, {"error": "not_found"})
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, required=True)
args = parser.parse_args()
server = ThreadingHTTPServer(("127.0.0.1", args.port), Handler)
print(json.dumps({"listening": args.port}), flush=True)
server.serve_forever()
if __name__ == "__main__":
main()
@@ -0,0 +1,529 @@
{
"results": [
{
"test": {
"test_id": "TC-R1-HTTP-RF-001-T2",
"trajectory_id": "HTTP-RF-001",
"focus_turn": 2,
"description": "Refund flow must include verify_refund_eligibility before process_refund (R1)",
"assertion": {
"type": "step_present",
"params": {
"tool": "verify_refund_eligibility"
}
}
},
"buggy": {
"passed": false,
"detail": "verify_refund_eligibility calls=0",
"trajectory": {
"trajectory_id": "HTTP-RF-001::buggy",
"source_trajectory_id": "HTTP-RF-001",
"implementation": "buggy",
"task_input": {
"intent": "refund",
"order_id": "ORD-58-A"
},
"final_status": "success",
"turns": [
{
"index": 0,
"role": "user",
"content": "{\"intent\": \"refund\", \"order_id\": \"ORD-58-A\"}"
},
{
"index": 1,
"role": "tool",
"module": "http_order_system",
"tool": "query_order",
"method": "GET",
"path": "/orders/ORD-58-A",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 1.833,
"response": {
"order_id": "ORD-58-A",
"status": "paid",
"sku": "SKU-42"
}
},
{
"index": 2,
"role": "tool",
"module": "http_order_system",
"tool": "process_refund",
"method": "POST",
"path": "/refund/process",
"request": {
"order_id": "ORD-58-A"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.833,
"response": {
"order_id": "ORD-58-A",
"refund_id": "RF-LIVE-1"
}
},
{
"index": 3,
"role": "tool",
"module": "http_order_system",
"tool": "notify_user",
"method": "POST",
"path": "/notifications",
"request": {
"status": "success"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.678,
"response": {
"sent": true,
"status": "success"
}
}
]
}
},
"fixed": {
"passed": true,
"detail": "verify_refund_eligibility calls=1",
"trajectory": {
"trajectory_id": "HTTP-RF-001::fixed",
"source_trajectory_id": "HTTP-RF-001",
"implementation": "fixed",
"task_input": {
"intent": "refund",
"order_id": "ORD-58-A"
},
"final_status": "success",
"turns": [
{
"index": 0,
"role": "user",
"content": "{\"intent\": \"refund\", \"order_id\": \"ORD-58-A\"}"
},
{
"index": 1,
"role": "tool",
"module": "http_order_system",
"tool": "query_order",
"method": "GET",
"path": "/orders/ORD-58-A",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 0.498,
"response": {
"order_id": "ORD-58-A",
"status": "paid",
"sku": "SKU-42"
}
},
{
"index": 2,
"role": "tool",
"module": "http_order_system",
"tool": "verify_refund_eligibility",
"method": "POST",
"path": "/refund/eligibility",
"request": {
"order_id": "ORD-58-A"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.379,
"response": {
"order_id": "ORD-58-A",
"eligible": true
}
},
{
"index": 3,
"role": "tool",
"module": "http_order_system",
"tool": "process_refund",
"method": "POST",
"path": "/refund/process",
"request": {
"order_id": "ORD-58-A"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.37,
"response": {
"order_id": "ORD-58-A",
"refund_id": "RF-LIVE-1"
}
},
{
"index": 4,
"role": "tool",
"module": "http_order_system",
"tool": "notify_user",
"method": "POST",
"path": "/notifications",
"request": {
"status": "success"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.315,
"response": {
"sent": true,
"status": "success"
}
}
]
}
}
},
{
"test": {
"test_id": "TC-R2-LATENCY-HTTP-INV-001-T2",
"trajectory_id": "HTTP-INV-001",
"focus_turn": 2,
"description": "check_stock must complete within 250 ms (R2)",
"assertion": {
"type": "latency_under",
"params": {
"tool": "check_stock",
"threshold_ms": 250
}
}
},
"buggy": {
"passed": false,
"detail": "check_stock max_latency_ms=350.383, threshold_ms=250.000",
"trajectory": {
"trajectory_id": "HTTP-INV-001::buggy",
"source_trajectory_id": "HTTP-INV-001",
"implementation": "buggy",
"task_input": {
"intent": "order_status",
"order_id": "ORD-58-B",
"sku": "SKU-42"
},
"final_status": "failed",
"turns": [
{
"index": 0,
"role": "user",
"content": "{\"intent\": \"order_status\", \"order_id\": \"ORD-58-B\", \"sku\": \"SKU-42\"}"
},
{
"index": 1,
"role": "tool",
"module": "http_order_system",
"tool": "query_order",
"method": "GET",
"path": "/orders/ORD-58-B",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 0.308,
"response": {
"order_id": "ORD-58-B",
"status": "paid",
"sku": "SKU-42"
}
},
{
"index": 2,
"role": "tool",
"module": "http_order_system",
"tool": "check_stock",
"method": "GET",
"path": "/inventory/SKU-42",
"request": null,
"http_status": null,
"status": "error",
"latency_ms": 350.383,
"error": "TimeoutError: timed out"
},
{
"index": 3,
"role": "tool",
"module": "http_order_system",
"tool": "notify_user",
"method": "POST",
"path": "/notifications",
"request": {
"status": "failed"
},
"http_status": 200,
"status": "success",
"latency_ms": 1.481,
"response": {
"sent": true,
"status": "failed"
}
}
]
}
},
"fixed": {
"passed": true,
"detail": "check_stock max_latency_ms=101.234, threshold_ms=250.000",
"trajectory": {
"trajectory_id": "HTTP-INV-001::fixed",
"source_trajectory_id": "HTTP-INV-001",
"implementation": "fixed",
"task_input": {
"intent": "order_status",
"order_id": "ORD-58-B",
"sku": "SKU-42"
},
"final_status": "success",
"turns": [
{
"index": 0,
"role": "user",
"content": "{\"intent\": \"order_status\", \"order_id\": \"ORD-58-B\", \"sku\": \"SKU-42\"}"
},
{
"index": 1,
"role": "tool",
"module": "http_order_system",
"tool": "query_order",
"method": "GET",
"path": "/orders/ORD-58-B",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 0.843,
"response": {
"order_id": "ORD-58-B",
"status": "paid",
"sku": "SKU-42"
}
},
{
"index": 2,
"role": "tool",
"module": "http_order_system",
"tool": "check_stock",
"method": "GET",
"path": "/inventory/SKU-42",
"request": null,
"http_status": null,
"status": "error",
"latency_ms": 101.234,
"error": "TimeoutError: timed out"
},
{
"index": 3,
"role": "tool",
"module": "http_order_system",
"tool": "check_stock",
"method": "GET",
"path": "/inventory/SKU-42?degraded=1",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 1.289,
"response": {
"sku": "SKU-42",
"stock": 12,
"source": "cache",
"degraded": true
}
},
{
"index": 4,
"role": "tool",
"module": "http_order_system",
"tool": "notify_user",
"method": "POST",
"path": "/notifications",
"request": {
"status": "success"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.78,
"response": {
"sent": true,
"status": "success"
}
}
]
}
}
},
{
"test": {
"test_id": "TC-R2-STATUS-HTTP-INV-001-T2",
"trajectory_id": "HTTP-INV-001",
"focus_turn": 2,
"description": "check_stock origin timeout must use degraded cache route, resulting in successful flow (R2)",
"assertion": {
"type": "final_status_is",
"params": {
"value": "success"
}
}
},
"buggy": {
"passed": false,
"detail": "final_status=failed, expected=success",
"trajectory": {
"trajectory_id": "HTTP-INV-001::buggy",
"source_trajectory_id": "HTTP-INV-001",
"implementation": "buggy",
"task_input": {
"intent": "order_status",
"order_id": "ORD-58-B",
"sku": "SKU-42"
},
"final_status": "failed",
"turns": [
{
"index": 0,
"role": "user",
"content": "{\"intent\": \"order_status\", \"order_id\": \"ORD-58-B\", \"sku\": \"SKU-42\"}"
},
{
"index": 1,
"role": "tool",
"module": "http_order_system",
"tool": "query_order",
"method": "GET",
"path": "/orders/ORD-58-B",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 0.608,
"response": {
"order_id": "ORD-58-B",
"status": "paid",
"sku": "SKU-42"
}
},
{
"index": 2,
"role": "tool",
"module": "http_order_system",
"tool": "check_stock",
"method": "GET",
"path": "/inventory/SKU-42",
"request": null,
"http_status": null,
"status": "error",
"latency_ms": 355.612,
"error": "TimeoutError: timed out"
},
{
"index": 3,
"role": "tool",
"module": "http_order_system",
"tool": "notify_user",
"method": "POST",
"path": "/notifications",
"request": {
"status": "failed"
},
"http_status": 200,
"status": "success",
"latency_ms": 1.19,
"response": {
"sent": true,
"status": "failed"
}
}
]
}
},
"fixed": {
"passed": true,
"detail": "final_status=success, expected=success",
"trajectory": {
"trajectory_id": "HTTP-INV-001::fixed",
"source_trajectory_id": "HTTP-INV-001",
"implementation": "fixed",
"task_input": {
"intent": "order_status",
"order_id": "ORD-58-B",
"sku": "SKU-42"
},
"final_status": "success",
"turns": [
{
"index": 0,
"role": "user",
"content": "{\"intent\": \"order_status\", \"order_id\": \"ORD-58-B\", \"sku\": \"SKU-42\"}"
},
{
"index": 1,
"role": "tool",
"module": "http_order_system",
"tool": "query_order",
"method": "GET",
"path": "/orders/ORD-58-B",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 0.895,
"response": {
"order_id": "ORD-58-B",
"status": "paid",
"sku": "SKU-42"
}
},
{
"index": 2,
"role": "tool",
"module": "http_order_system",
"tool": "check_stock",
"method": "GET",
"path": "/inventory/SKU-42",
"request": null,
"http_status": null,
"status": "error",
"latency_ms": 102.492,
"error": "TimeoutError: timed out"
},
{
"index": 3,
"role": "tool",
"module": "http_order_system",
"tool": "check_stock",
"method": "GET",
"path": "/inventory/SKU-42?degraded=1",
"request": null,
"http_status": 200,
"status": "success",
"latency_ms": 1.493,
"response": {
"sku": "SKU-42",
"stock": 12,
"source": "cache",
"degraded": true
}
},
{
"index": 4,
"role": "tool",
"module": "http_order_system",
"tool": "notify_user",
"method": "POST",
"path": "/notifications",
"request": {
"status": "success"
},
"http_status": 200,
"status": "success",
"latency_ms": 0.843,
"response": {
"sent": true,
"status": "success"
}
}
]
}
}
}
]
}
@@ -0,0 +1,81 @@
{
"schema_version": "1.0",
"experiment": "5-8",
"run_id": "exp5-8-live-http-mcp-20260730-053403",
"generated_at_utc": "2026-07-29T21:37:08Z",
"provider": "ark",
"model": "doubao-seed-1-6-250615",
"service": {
"kind": "real local HTTP subprocess",
"base_url": "http://127.0.0.1:56938",
"pid": 96265
},
"source_trajectory_ids": [
"HTTP-INV-001",
"HTTP-RF-001"
],
"model_call_count": 2,
"github_issue_url": "https://github.com/bojieli/ai-agent-book/issues/502",
"gates": {
"real_local_http_trajectories": true,
"measured_latency_and_raw_http": true,
"live_model_diagnosis": true,
"diagnosis_references_trajectories_and_turns": true,
"live_model_generated_executable_tests": true,
"buggy_failures_reproduced": true,
"fixed_system_passes": true,
"official_github_mcp_issue_created": true,
"raw_provider_receipts_complete": true
},
"artifacts": {
"PRD.md": {
"sha256": "d21f65b3f5243f79b6ab46525945b8e882a6646d75177062cb4d976192369353",
"bytes": 448
},
"architecture.md": {
"sha256": "e76c20c6ac826325db0c7b35204b0ce7235973dc33e3b62220aae607a60f49e4",
"bytes": 422
},
"diagnosis.json": {
"sha256": "4232addd060d6ace66814d53625d6b08b8511e77c34253c3d9e1529199d4b570",
"bytes": 1334
},
"github_mcp_receipt.json": {
"sha256": "30c42f32006eaf0fda5350747ce2bbf44d61af79a6020a62ab12bfb28836976e",
"bytes": 1882
},
"http_service.py": {
"sha256": "bbe0170e342a34fb70dc2477fede68cd393e1fd9a6d98140974a61677413c997",
"bytes": 3701
},
"live_replays.json": {
"sha256": "ef8401bed39405e0398d47a2b3553d50c50d1199b359873afa180b60153b29a8",
"bytes": 16057
},
"production_trajectories.jsonl": {
"sha256": "9d45f81e64ff8b6ccaa109dbdc838bb016832bcae14633fc86a9a917fccb0e6e",
"bytes": 2300
},
"provider_receipts.checkpoint.json": {
"sha256": "20e466f383f807d9ad1d6e0a1a699733f7e7e4da2d459a3add1a976578fb2bd0",
"bytes": 12171
},
"provider_receipts.json": {
"sha256": "20e466f383f807d9ad1d6e0a1a699733f7e7e4da2d459a3add1a976578fb2bd0",
"bytes": 12171
},
"regression_tests.json": {
"sha256": "642d9037754d16665be684f69eb6decd74e8db25181b214bce806e34ceeb7c3a",
"bytes": 1091
},
"service.stderr.log": {
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"bytes": 0
},
"service.stdout.jsonl": {
"sha256": "776fe22d507f08f0b56e017599777a60c0165a70e57556e34b3540a27b25fe5a",
"bytes": 2987
}
},
"official_complete": true
}
@@ -0,0 +1,2 @@
{"trajectory_id": "HTTP-RF-001::buggy", "source_trajectory_id": "HTTP-RF-001", "implementation": "buggy", "task_input": {"intent": "refund", "order_id": "ORD-58-A"}, "final_status": "success", "turns": [{"index": 0, "role": "user", "content": "{\"intent\": \"refund\", \"order_id\": \"ORD-58-A\"}"}, {"index": 1, "role": "tool", "module": "http_order_system", "tool": "query_order", "method": "GET", "path": "/orders/ORD-58-A", "request": null, "http_status": 200, "status": "success", "latency_ms": 0.463, "response": {"order_id": "ORD-58-A", "status": "paid", "sku": "SKU-42"}}, {"index": 2, "role": "tool", "module": "http_order_system", "tool": "process_refund", "method": "POST", "path": "/refund/process", "request": {"order_id": "ORD-58-A"}, "http_status": 200, "status": "success", "latency_ms": 0.393, "response": {"order_id": "ORD-58-A", "refund_id": "RF-LIVE-1"}}, {"index": 3, "role": "tool", "module": "http_order_system", "tool": "notify_user", "method": "POST", "path": "/notifications", "request": {"status": "success"}, "http_status": 200, "status": "success", "latency_ms": 0.444, "response": {"sent": true, "status": "success"}}]}
{"trajectory_id": "HTTP-INV-001::buggy", "source_trajectory_id": "HTTP-INV-001", "implementation": "buggy", "task_input": {"intent": "order_status", "order_id": "ORD-58-B", "sku": "SKU-42"}, "final_status": "failed", "turns": [{"index": 0, "role": "user", "content": "{\"intent\": \"order_status\", \"order_id\": \"ORD-58-B\", \"sku\": \"SKU-42\"}"}, {"index": 1, "role": "tool", "module": "http_order_system", "tool": "query_order", "method": "GET", "path": "/orders/ORD-58-B", "request": null, "http_status": 200, "status": "success", "latency_ms": 0.391, "response": {"order_id": "ORD-58-B", "status": "paid", "sku": "SKU-42"}}, {"index": 2, "role": "tool", "module": "http_order_system", "tool": "check_stock", "method": "GET", "path": "/inventory/SKU-42", "request": null, "http_status": null, "status": "error", "latency_ms": 350.414, "error": "TimeoutError: timed out"}, {"index": 3, "role": "tool", "module": "http_order_system", "tool": "notify_user", "method": "POST", "path": "/notifications", "request": {"status": "failed"}, "http_status": 200, "status": "success", "latency_ms": 2.443, "response": {"sent": true, "status": "failed"}}]}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,41 @@
{
"test_cases": [
{
"test_id": "TC-R1-HTTP-RF-001-T2",
"trajectory_id": "HTTP-RF-001",
"focus_turn": 2,
"description": "Refund flow must include verify_refund_eligibility before process_refund (R1)",
"assertion": {
"type": "step_present",
"params": {
"tool": "verify_refund_eligibility"
}
}
},
{
"test_id": "TC-R2-LATENCY-HTTP-INV-001-T2",
"trajectory_id": "HTTP-INV-001",
"focus_turn": 2,
"description": "check_stock must complete within 250 ms (R2)",
"assertion": {
"type": "latency_under",
"params": {
"tool": "check_stock",
"threshold_ms": 250
}
}
},
{
"test_id": "TC-R2-STATUS-HTTP-INV-001-T2",
"trajectory_id": "HTTP-INV-001",
"focus_turn": 2,
"description": "check_stock origin timeout must use degraded cache route, resulting in successful flow (R2)",
"assertion": {
"type": "final_status_is",
"params": {
"value": "success"
}
}
}
]
}
@@ -0,0 +1,29 @@
{"listening": 56938}
{"client": "127.0.0.1", "message": "\"GET /health HTTP/1.1\" 200 -", "time": 1785360843.868107}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-A HTTP/1.1\" 200 -", "time": 1785360843.8689358}
{"client": "127.0.0.1", "message": "\"POST /refund/process HTTP/1.1\" 200 -", "time": 1785360843.869437}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785360843.8698602}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-B HTTP/1.1\" 200 -", "time": 1785360843.8703701}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785360844.223199}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42 HTTP/1.1\" 200 -", "time": 1785360844.6740758}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-A HTTP/1.1\" 200 -", "time": 1785361025.351637}
{"client": "127.0.0.1", "message": "\"POST /refund/process HTTP/1.1\" 200 -", "time": 1785361025.353014}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785361025.3537788}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-A HTTP/1.1\" 200 -", "time": 1785361025.3544052}
{"client": "127.0.0.1", "message": "\"POST /refund/eligibility HTTP/1.1\" 200 -", "time": 1785361025.354883}
{"client": "127.0.0.1", "message": "\"POST /refund/process HTTP/1.1\" 200 -", "time": 1785361025.3552582}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785361025.355627}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-B HTTP/1.1\" 200 -", "time": 1785361025.355964}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785361025.707724}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-B HTTP/1.1\" 200 -", "time": 1785361025.708798}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42?degraded=1 HTTP/1.1\" 200 -", "time": 1785361025.811397}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785361025.812402}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-B HTTP/1.1\" 200 -", "time": 1785361025.813149}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42 HTTP/1.1\" 200 -", "time": 1785361026.157606}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785361026.169961}
{"client": "127.0.0.1", "message": "\"GET /orders/ORD-58-B HTTP/1.1\" 200 -", "time": 1785361026.1710122}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42?degraded=1 HTTP/1.1\" 200 -", "time": 1785361026.275081}
{"client": "127.0.0.1", "message": "\"POST /notifications HTTP/1.1\" 200 -", "time": 1785361026.276181}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42 HTTP/1.1\" 200 -", "time": 1785361026.5155609}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42 HTTP/1.1\" 200 -", "time": 1785361026.6163042}
{"client": "127.0.0.1", "message": "\"GET /inventory/SKU-42 HTTP/1.1\" 200 -", "time": 1785361026.972992}