Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
2910 lines
140 KiB
JSON
2910 lines
140 KiB
JSON
{
|
|
"summary": {
|
|
"label": "base",
|
|
"split": "eval",
|
|
"count": 256,
|
|
"exact": 0.0,
|
|
"protected_preservation": 0.9868421052631579,
|
|
"curly_count_match": 0.67578125,
|
|
"by_kind": {
|
|
"comment": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"english": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 0.75
|
|
},
|
|
"go": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"java": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"javascript": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"json": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 0.0
|
|
},
|
|
"markdown": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"mixed": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 0.0
|
|
},
|
|
"nested": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 0.0
|
|
},
|
|
"python": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"quote": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 0.0625
|
|
},
|
|
"rust": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"shell": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"sql": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"yaml": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 1.0
|
|
},
|
|
"zh": {
|
|
"count": 16,
|
|
"exact": 0.0,
|
|
"curly_count_match": 0.0
|
|
}
|
|
}
|
|
},
|
|
"rows": [
|
|
{
|
|
"id": "eval-0000",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-000;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0001",
|
|
"kind": "english",
|
|
"prediction": "“Do not remove the assertion.” 这段英文必须保持原样。 \n(案例编号 eval-001;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "原文引用如下:\"Do not remove the assertion.\" 这段英文必须保持原样。\n(案例编号 eval-001;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 0,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0002",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“请求编号”\nname = \"dry_run\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “请求编号”\nname = \"dry_run\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-002;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0003",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-003;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0004",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-004;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0005",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-005;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0006",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-006;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0007",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“当前状态”\nSELECT \"API_KEY\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “当前状态”\nSELECT \"API_KEY\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-007;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0008",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-008;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0009",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“缓存策略”\nname: \"ROLLBACK\"",
|
|
"target": "```yaml\n# 中文注释:显示 “缓存策略”\nname: \"ROLLBACK\"\n```\n代码不能被改坏。\n(案例编号 eval-009;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0010",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。\n(案例编号 eval-010;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0011",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\" \n(案例编号 eval-011;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-011;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0012",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。\n(案例编号 eval-012;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0013",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"This value must remain unchanged.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"This value must remain unchanged.\" 也要原样保留。\n(案例编号 eval-013;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0014",
|
|
"kind": "json",
|
|
"prediction": "请求编号必须存在。请求体示例:{\"message\": \"degraded\"}。",
|
|
"target": "中文说明:“请求编号必须存在”。请求体示例:{\"message\": \"degraded\"}。\n(案例编号 eval-014;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0015",
|
|
"kind": "zh",
|
|
"prediction": "他宣布:\"今天发布 v3\",随后补充:\"请检查权限\"。 \n(案例编号:eval-015;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v3”,随后补充:“检查权限”。\n(案例编号 eval-015;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0016",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-016;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0017",
|
|
"kind": "english",
|
|
"prediction": "\"The file was not changed.\" 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"The file was not changed.\" 这段英文必须保持原样。\n(案例编号 eval-017;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0018",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“发布说明”\nname = \"status\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “发布说明”\nname = \"status\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-018;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0019",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-019;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0020",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-020;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0021",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-021;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0022",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-022;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0023",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“目标路径”\nSELECT \"DELETE\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “目标路径”\nSELECT \"DELETE\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-023;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0024",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-024;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0025",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“当前状态”\nname: \"API_KEY\"",
|
|
"target": "```yaml\n# 中文注释:显示 “当前状态”\nname: \"API_KEY\"\n```\n代码不能被改坏。\n(案例编号 eval-025;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0026",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。\n(案例编号 eval-026;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0027",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-027;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0028",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。\n(案例编号 eval-028;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0029",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Please restart the service before retrying.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Please restart the service before retrying.\" 也要原样保留。\n(案例编号 eval-029;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0030",
|
|
"kind": "json",
|
|
"prediction": "发布说明必须存在。请求体示例:{\"message\": \"retry\"}。",
|
|
"target": "中文说明:“发布说明必须存在”。请求体示例:{\"message\": \"retry\"}。\n(案例编号 eval-030;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0031",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 v2\",随后补充:\"检查权限\"。\n(案例编号 eval-031;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v2”,随后补充:“检查权限”。\n(案例编号 eval-031;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0032",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-032;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0033",
|
|
"kind": "english",
|
|
"prediction": "\"This value must remain unchanged.\" 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"This value must remain unchanged.\" 这段英文必须保持原样。\n(案例编号 eval-033;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0034",
|
|
"kind": "python",
|
|
"prediction": "# 中文注释:显示“错误原因”\nname = \"PATCH\"\nprint(name)",
|
|
"target": "```python\n# 中文注释:显示 “错误原因”\nname = \"PATCH\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-034;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0035",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-035;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0036",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-036;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0037",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-037;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0038",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-038;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0039",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-039;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0040",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-040;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0041",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“目标路径”\nname: \"DELETE\"",
|
|
"target": "```yaml\n# 中文注释:显示 “目标路径”\nname: \"DELETE\"\n```\n代码不能被改坏。\n(案例编号 eval-041;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0042",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。\n(案例编号 eval-042;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0043",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-043;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0044",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。\n(案例编号 eval-044;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0045",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Run the migration only once.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Run the migration only once.\" 也要原样保留。\n(案例编号 eval-045;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0046",
|
|
"kind": "json",
|
|
"prediction": "错误原因必须存在。请求体示例:{\"message\": \"ready\"}。",
|
|
"target": "中文说明:“错误原因必须存在”。请求体示例:{\"message\": \"ready\"}。\n(案例编号 eval-046;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0047",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 2026.08\",随后补充:\"检查权限\"。 \n(案例编号:eval-047;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 2026.08”,随后补充:“检查权限”。\n(案例编号 eval-047;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0048",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-048;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0049",
|
|
"kind": "english",
|
|
"prediction": "请在重试前重启服务。\"Please restart the service before retrying.\"(案例编号 eval-049;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "原文引用如下:\"Please restart the service before retrying.\" 这段英文必须保持原样。\n(案例编号 eval-049;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0050",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“请求编号”\nname = \"dry_run\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “请求编号”\nname = \"dry_run\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-050;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0051",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-051;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0052",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-052;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0053",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-053;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0054",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-054;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0055",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“当前状态”\nSELECT \"API_KEY\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “当前状态”\nSELECT \"API_KEY\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-055;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0056",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-056;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0057",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“缓存策略”\nname: \"ROLLBACK\"",
|
|
"target": "```yaml\n# 中文注释:显示 “缓存策略”\nname: \"ROLLBACK\"\n```\n代码不能被改坏。\n(案例编号 eval-057;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0058",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。\n(案例编号 eval-058;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0059",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-059;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0060",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。\n(案例编号 eval-060;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0061",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Do not remove the assertion.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Do not remove the assertion.\" 也要原样保留。\n(案例编号 eval-061;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0062",
|
|
"kind": "json",
|
|
"prediction": "请求编号必须存在。请求体示例:{\"message\": \"degraded\"}。",
|
|
"target": "中文说明:“请求编号必须存在”。请求体示例:{\"message\": \"degraded\"}。\n(案例编号 eval-062;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0063",
|
|
"kind": "zh",
|
|
"prediction": "他宣布:\"今天发布 v3\",随后补充:\"请检查权限\"。 \n(案例编号:eval-063;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v3”,随后补充:“检查权限”。\n(案例编号 eval-063;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0064",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-064;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0065",
|
|
"kind": "english",
|
|
"prediction": "\"Run the migration only once.\" \n在进行数据库迁移时,应确保迁移脚本仅执行一次,以避免数据重复或结构冲突。请根据实际环境判断是否需要重新运行迁移,若已成功执行,则无需再次操作。",
|
|
"target": "原文引用如下:\"Run the migration only once.\" 这段英文必须保持原样。\n(案例编号 eval-065;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0066",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“发布说明”\nname = \"status\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “发布说明”\nname = \"status\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-066;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0067",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-067;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0068",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-068;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0069",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-069;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0070",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-070;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0071",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“目标路径”\nSELECT \"DELETE\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “目标路径”\nSELECT \"DELETE\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-071;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0072",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-072;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0073",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“当前状态”\nname: \"API_KEY\"",
|
|
"target": "```yaml\n# 中文注释:显示 “当前状态”\nname: \"API_KEY\"\n```\n代码不能被改坏。\n(案例编号 eval-073;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0074",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。\n(案例编号 eval-074;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0075",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-075;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0076",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。\n(案例编号 eval-076;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0077",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"The file was not changed.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"The file was not changed.\" 也要原样保留。\n(案例编号 eval-077;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0078",
|
|
"kind": "json",
|
|
"prediction": "发布说明必须存在。请求体示例:{\"message\": \"retry\"}。",
|
|
"target": "中文说明:“发布说明必须存在”。请求体示例:{\"message\": \"retry\"}。\n(案例编号 eval-078;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0079",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 v2\",随后补充:\"检查权限\"。\n(案例编号 eval-079;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v2”,随后补充:“检查权限”。\n(案例编号 eval-079;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0080",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-080;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0081",
|
|
"kind": "english",
|
|
"prediction": "“Do not remove the assertion.” 这段英文必须保持原样。 \n(案例编号 eval-081;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "原文引用如下:\"Do not remove the assertion.\" 这段英文必须保持原样。\n(案例编号 eval-081;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 0,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0082",
|
|
"kind": "python",
|
|
"prediction": "# 中文注释:显示“错误原因”\nname = \"PATCH\"\nprint(name)",
|
|
"target": "```python\n# 中文注释:显示 “错误原因”\nname = \"PATCH\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-082;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0083",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-083;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0084",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-084;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0085",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-085;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0086",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-086;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0087",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-087;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0088",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-088;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0089",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“目标路径”\nname: \"DELETE\"",
|
|
"target": "```yaml\n# 中文注释:显示 “目标路径”\nname: \"DELETE\"\n```\n代码不能被改坏。\n(案例编号 eval-089;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0090",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。\n(案例编号 eval-090;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0091",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-091;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0092",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。\n(案例编号 eval-092;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0093",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"This value must remain unchanged.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"This value must remain unchanged.\" 也要原样保留。\n(案例编号 eval-093;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0094",
|
|
"kind": "json",
|
|
"prediction": "错误原因必须存在。请求体示例:{\"message\": \"ready\"}。",
|
|
"target": "中文说明:“错误原因必须存在”。请求体示例:{\"message\": \"ready\"}。\n(案例编号 eval-094;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0095",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 2026.08\",随后补充:\"检查权限\"。 \n(案例编号:eval-095;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 2026.08”,随后补充:“检查权限”。\n(案例编号 eval-095;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0096",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-096;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0097",
|
|
"kind": "english",
|
|
"prediction": "\"The file was not changed.\" 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"The file was not changed.\" 这段英文必须保持原样。\n(案例编号 eval-097;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0098",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“请求编号”\nname = \"dry_run\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “请求编号”\nname = \"dry_run\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-098;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0099",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-099;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0100",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-100;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0101",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-101;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0102",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-102;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0103",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“当前状态”\nSELECT \"API_KEY\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “当前状态”\nSELECT \"API_KEY\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-103;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0104",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-104;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0105",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“缓存策略”\nname: \"ROLLBACK\"",
|
|
"target": "```yaml\n# 中文注释:显示 “缓存策略”\nname: \"ROLLBACK\"\n```\n代码不能被改坏。\n(案例编号 eval-105;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0106",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。\n(案例编号 eval-106;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0107",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-107;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0108",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。\n(案例编号 eval-108;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0109",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Please restart the service before retrying.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Please restart the service before retrying.\" 也要原样保留。\n(案例编号 eval-109;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0110",
|
|
"kind": "json",
|
|
"prediction": "请求编号必须存在。请求体示例:{\"message\": \"degraded\"}。",
|
|
"target": "中文说明:“请求编号必须存在”。请求体示例:{\"message\": \"degraded\"}。\n(案例编号 eval-110;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0111",
|
|
"kind": "zh",
|
|
"prediction": "他宣布:\"今天发布 v3\",随后补充:\"请检查权限\"。 \n(案例编号:eval-111;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v3”,随后补充:“检查权限”。\n(案例编号 eval-111;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0112",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-112;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0113",
|
|
"kind": "english",
|
|
"prediction": "\"This value must remain unchanged.\" 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"This value must remain unchanged.\" 这段英文必须保持原样。\n(案例编号 eval-113;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0114",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“发布说明”\nname = \"status\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “发布说明”\nname = \"status\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-114;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0115",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-115;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0116",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-116;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0117",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-117;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0118",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-118;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0119",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“目标路径”\nSELECT \"DELETE\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “目标路径”\nSELECT \"DELETE\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-119;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0120",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-120;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0121",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“当前状态”\nname: \"API_KEY\"",
|
|
"target": "```yaml\n# 中文注释:显示 “当前状态”\nname: \"API_KEY\"\n```\n代码不能被改坏。\n(案例编号 eval-121;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0122",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。\n(案例编号 eval-122;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0123",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\" \n(案例编号 eval-123;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-123;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0124",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。\n(案例编号 eval-124;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0125",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Run the migration only once.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Run the migration only once.\" 也要原样保留。\n(案例编号 eval-125;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0126",
|
|
"kind": "json",
|
|
"prediction": "发布说明必须存在。请求体示例:{\"message\": \"retry\"}。",
|
|
"target": "中文说明:“发布说明必须存在”。请求体示例:{\"message\": \"retry\"}。\n(案例编号 eval-126;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0127",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 v2\",随后补充:\"检查权限\"。\n(案例编号 eval-127;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v2”,随后补充:“检查权限”。\n(案例编号 eval-127;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0128",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-128;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0129",
|
|
"kind": "english",
|
|
"prediction": "请在重试前重启服务。\"Please restart the service before retrying.\"(案例编号 eval-129;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "原文引用如下:\"Please restart the service before retrying.\" 这段英文必须保持原样。\n(案例编号 eval-129;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0130",
|
|
"kind": "python",
|
|
"prediction": "# 中文注释:显示“错误原因”\nname = \"PATCH\"\nprint(name)",
|
|
"target": "```python\n# 中文注释:显示 “错误原因”\nname = \"PATCH\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-130;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0131",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-131;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0132",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-132;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0133",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-133;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0134",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-134;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0135",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-135;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0136",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-136;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0137",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“目标路径”\nname: \"DELETE\"",
|
|
"target": "```yaml\n# 中文注释:显示 “目标路径”\nname: \"DELETE\"\n```\n代码不能被改坏。\n(案例编号 eval-137;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0138",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。\n(案例编号 eval-138;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0139",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-139;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0140",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。\n(案例编号 eval-140;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0141",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Do not remove the assertion.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Do not remove the assertion.\" 也要原样保留。\n(案例编号 eval-141;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0142",
|
|
"kind": "json",
|
|
"prediction": "错误原因必须存在。请求体示例:{\"message\": \"ready\"}。",
|
|
"target": "中文说明:“错误原因必须存在”。请求体示例:{\"message\": \"ready\"}。\n(案例编号 eval-142;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0143",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 2026.08\",随后补充:\"检查权限\"。 \n(案例编号:eval-143;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 2026.08”,随后补充:“检查权限”。\n(案例编号 eval-143;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0144",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-144;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0145",
|
|
"kind": "english",
|
|
"prediction": "\"Run the migration only once.\" \n在进行数据库迁移时,务必确保迁移脚本仅执行一次,以避免数据重复或结构冲突。请根据实际环境判断是否需要重新运行迁移,若已成功执行,则无需再次操作。",
|
|
"target": "原文引用如下:\"Run the migration only once.\" 这段英文必须保持原样。\n(案例编号 eval-145;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0146",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“请求编号”\nname = \"dry_run\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “请求编号”\nname = \"dry_run\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-146;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0147",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-147;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0148",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-148;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0149",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-149;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0150",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-150;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0151",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“当前状态”\nSELECT \"API_KEY\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “当前状态”\nSELECT \"API_KEY\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-151;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0152",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-152;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0153",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“缓存策略”\nname: \"ROLLBACK\"",
|
|
"target": "```yaml\n# 中文注释:显示 “缓存策略”\nname: \"ROLLBACK\"\n```\n代码不能被改坏。\n(案例编号 eval-153;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0154",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。\n(案例编号 eval-154;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0155",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-155;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0156",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。\n(案例编号 eval-156;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0157",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:“检查权限”。英文原句 \"The file was not changed.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"The file was not changed.\" 也要原样保留。\n(案例编号 eval-157;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0158",
|
|
"kind": "json",
|
|
"prediction": "请求编号必须存在。请求体示例:{\"message\": \"degraded\"}。",
|
|
"target": "中文说明:“请求编号必须存在”。请求体示例:{\"message\": \"degraded\"}。\n(案例编号 eval-158;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0159",
|
|
"kind": "zh",
|
|
"prediction": "他宣布:\"今天发布 v3\",随后补充:\"请检查权限\"。 \n(案例编号:eval-159;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v3”,随后补充:“检查权限”。\n(案例编号 eval-159;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0160",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-160;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0161",
|
|
"kind": "english",
|
|
"prediction": "“Do not remove the assertion.” 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"Do not remove the assertion.\" 这段英文必须保持原样。\n(案例编号 eval-161;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 0,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0162",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“发布说明”\nname = \"status\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “发布说明”\nname = \"status\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-162;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0163",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-163;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0164",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-164;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0165",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-165;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0166",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-166;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0167",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“目标路径”\nSELECT \"DELETE\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “目标路径”\nSELECT \"DELETE\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-167;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0168",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-168;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0169",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“当前状态”\nname: \"API_KEY\"",
|
|
"target": "```yaml\n# 中文注释:显示 “当前状态”\nname: \"API_KEY\"\n```\n代码不能被改坏。\n(案例编号 eval-169;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0170",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。\n(案例编号 eval-170;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0171",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-171;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0172",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。\n(案例编号 eval-172;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0173",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"This value must remain unchanged.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"This value must remain unchanged.\" 也要原样保留。\n(案例编号 eval-173;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0174",
|
|
"kind": "json",
|
|
"prediction": "发布说明必须存在。请求体示例:{\"message\": \"retry\"}。",
|
|
"target": "中文说明:“发布说明必须存在”。请求体示例:{\"message\": \"retry\"}。\n(案例编号 eval-174;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0175",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 v2\",随后补充:\"检查权限\"。\n(案例编号 eval-175;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v2”,随后补充:“检查权限”。\n(案例编号 eval-175;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0176",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-176;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0177",
|
|
"kind": "english",
|
|
"prediction": "\"The file was not changed.\" 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"The file was not changed.\" 这段英文必须保持原样。\n(案例编号 eval-177;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0178",
|
|
"kind": "python",
|
|
"prediction": "# 中文注释:显示“错误原因”\nname = \"PATCH\"\nprint(name)",
|
|
"target": "```python\n# 中文注释:显示 “错误原因”\nname = \"PATCH\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-178;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0179",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-179;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0180",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-180;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0181",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-181;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0182",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-182;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0183",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-183;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0184",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-184;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0185",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“目标路径”\nname: \"DELETE\"",
|
|
"target": "```yaml\n# 中文注释:显示 “目标路径”\nname: \"DELETE\"\n```\n代码不能被改坏。\n(案例编号 eval-185;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0186",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。\n(案例编号 eval-186;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0187",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-187;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0188",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。\n(案例编号 eval-188;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0189",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Please restart the service before retrying.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Please restart the service before retrying.\" 也要原样保留。\n(案例编号 eval-189;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0190",
|
|
"kind": "json",
|
|
"prediction": "错误原因必须存在。请求体示例:{\"message\": \"ready\"}。",
|
|
"target": "中文说明:“错误原因必须存在”。请求体示例:{\"message\": \"ready\"}。\n(案例编号 eval-190;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0191",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 2026.08\",随后补充:\"检查权限\"。 \n(案例编号:eval-191;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 2026.08”,随后补充:“检查权限”。\n(案例编号 eval-191;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0192",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-192;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0193",
|
|
"kind": "english",
|
|
"prediction": "\"This value must remain unchanged.\" 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"This value must remain unchanged.\" 这段英文必须保持原样。\n(案例编号 eval-193;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0194",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“请求编号”\nname = \"dry_run\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “请求编号”\nname = \"dry_run\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-194;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0195",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-195;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0196",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-196;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0197",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-197;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0198",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-198;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0199",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“当前状态”\nSELECT \"API_KEY\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “当前状态”\nSELECT \"API_KEY\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-199;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0200",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-200;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0201",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“缓存策略”\nname: \"ROLLBACK\"",
|
|
"target": "```yaml\n# 中文注释:显示 “缓存策略”\nname: \"ROLLBACK\"\n```\n代码不能被改坏。\n(案例编号 eval-201;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0202",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。\n(案例编号 eval-202;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0203",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-203;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0204",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。\n(案例编号 eval-204;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0205",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Run the migration only once.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Run the migration only once.\" 也要原样保留。\n(案例编号 eval-205;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0206",
|
|
"kind": "json",
|
|
"prediction": "请求编号必须存在。请求体示例:{\"message\": \"degraded\"}。",
|
|
"target": "中文说明:“请求编号必须存在”。请求体示例:{\"message\": \"degraded\"}。\n(案例编号 eval-206;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0207",
|
|
"kind": "zh",
|
|
"prediction": "他宣布:\"今天发布 v3\",随后补充:\"请检查权限\"。 \n(案例编号:eval-207;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v3”,随后补充:“检查权限”。\n(案例编号 eval-207;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0208",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `deploy()` 后,服务返回 {\"status\": \"ready\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-208;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0209",
|
|
"kind": "english",
|
|
"prediction": "请在重试前重启服务。\"Please restart the service before retrying.\"(案例编号 eval-209;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "原文引用如下:\"Please restart the service before retrying.\" 这段英文必须保持原样。\n(案例编号 eval-209;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0210",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“发布说明”\nname = \"status\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “发布说明”\nname = \"status\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-210;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0211",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “当前状态”\nconst name = \"API_KEY\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-211;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0212",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “请求编号”\nString name = \"dry_run\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-212;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0213",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “缓存策略”\nname := \"ROLLBACK\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-213;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0214",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “错误原因”\nlet name = \"PATCH\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-214;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0215",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“目标路径”\nSELECT \"DELETE\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “目标路径”\nSELECT \"DELETE\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-215;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0216",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “发布说明”\nname=\"status\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-216;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0217",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“当前状态”\nname: \"API_KEY\"",
|
|
"target": "```yaml\n# 中文注释:显示 “当前状态”\nname: \"API_KEY\"\n```\n代码不能被改坏。\n(案例编号 eval-217;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0218",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-validate` 必须保持。\n(案例编号 eval-218;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0219",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-219;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0220",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `dry_run` 的语法必须保持。\n(案例编号 eval-220;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0221",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"Do not remove the assertion.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"Do not remove the assertion.\" 也要原样保留。\n(案例编号 eval-221;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0222",
|
|
"kind": "json",
|
|
"prediction": "发布说明必须存在。请求体示例:{\"message\": \"retry\"}。",
|
|
"target": "中文说明:“发布说明必须存在”。请求体示例:{\"message\": \"retry\"}。\n(案例编号 eval-222;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0223",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 v2\",随后补充:\"检查权限\"。\n(案例编号 eval-223;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v2”,随后补充:“检查权限”。\n(案例编号 eval-223;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0224",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `validate()` 后,服务返回 {\"status\": \"degraded\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-224;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0225",
|
|
"kind": "english",
|
|
"prediction": "\"Run the migration only once.\" \n在进行数据库迁移时,应确保迁移脚本仅执行一次,以避免数据重复或结构冲突。请根据实际环境判断是否需要重新运行迁移,若已成功执行,则无需再次操作。",
|
|
"target": "原文引用如下:\"Run the migration only once.\" 这段英文必须保持原样。\n(案例编号 eval-225;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0226",
|
|
"kind": "python",
|
|
"prediction": "# 中文注释:显示“错误原因”\nname = \"PATCH\"\nprint(name)",
|
|
"target": "```python\n# 中文注释:显示 “错误原因”\nname = \"PATCH\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-226;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0227",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “目标路径”\nconst name = \"DELETE\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-227;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0228",
|
|
"kind": "java",
|
|
"prediction": "```java\n// 中文注释:显示“发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```",
|
|
"target": "```java\n// 中文注释:显示 “发布说明”\nString name = \"status\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-228;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0229",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “当前状态”\nname := \"API_KEY\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-229;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0230",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “请求编号”\nlet name = \"dry_run\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-230;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0231",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “缓存策略”\nSELECT \"ROLLBACK\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-231;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0232",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “错误原因”\nname=\"PATCH\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-232;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0233",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“目标路径”\nname: \"DELETE\"",
|
|
"target": "```yaml\n# 中文注释:显示 “目标路径”\nname: \"DELETE\"\n```\n代码不能被改坏。\n(案例编号 eval-233;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0234",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-inspect` 必须保持。\n(案例编号 eval-234;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0235",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-235;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0236",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `PATCH` 的语法必须保持。\n(案例编号 eval-236;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0237",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"The file was not changed.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"The file was not changed.\" 也要原样保留。\n(案例编号 eval-237;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0238",
|
|
"kind": "json",
|
|
"prediction": "错误原因必须存在。请求体示例:{\"message\": \"ready\"}。",
|
|
"target": "中文说明:“错误原因必须存在”。请求体示例:{\"message\": \"ready\"}。\n(案例编号 eval-238;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0239",
|
|
"kind": "zh",
|
|
"prediction": "他提到:\"今天发布 2026.08\",随后补充:\"检查权限\"。 \n(案例编号:eval-239;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 2026.08”,随后补充:“检查权限”。\n(案例编号 eval-239;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0240",
|
|
"kind": "mixed",
|
|
"prediction": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中提示:\"查看变更记录\"。",
|
|
"target": "调用 `inspect()` 后,服务返回 {\"status\": \"retry\"}。说明文字中写着:“查看变更记录”。\n(案例编号 eval-240;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 3,
|
|
"protected_total": 3
|
|
},
|
|
{
|
|
"id": "eval-0241",
|
|
"kind": "english",
|
|
"prediction": "“Do not remove the assertion.” 这段英文必须保持原样。",
|
|
"target": "原文引用如下:\"Do not remove the assertion.\" 这段英文必须保持原样。\n(案例编号 eval-241;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 0,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 0,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0242",
|
|
"kind": "python",
|
|
"prediction": "```python\n# 中文注释:显示“请求编号”\nname = \"dry_run\"\nprint(name)\n```",
|
|
"target": "```python\n# 中文注释:显示 “请求编号”\nname = \"dry_run\"\nprint(name)\n```\n代码不能被改坏。\n(案例编号 eval-242;体裁=教程;语言=Python;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0243",
|
|
"kind": "javascript",
|
|
"prediction": "```javascript\n// 中文注释:显示“缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```",
|
|
"target": "```javascript\n// 中文注释:显示 “缓存策略”\nconst name = \"ROLLBACK\";\nconsole.log(name);\n```\n代码不能被改坏。\n(案例编号 eval-243;体裁=故障复盘;语言=JavaScript;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0244",
|
|
"kind": "java",
|
|
"prediction": "// 中文注释:显示“错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);",
|
|
"target": "```java\n// 中文注释:显示 “错误原因”\nString name = \"PATCH\";\nSystem.out.println(name);\n```\n代码不能被改坏。\n(案例编号 eval-244;体裁=教程;语言=Java;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0245",
|
|
"kind": "go",
|
|
"prediction": "```go\n// 中文注释:显示“目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```",
|
|
"target": "```go\n// 中文注释:显示 “目标路径”\nname := \"DELETE\"\nfmt.Println(name)\n```\n代码不能被改坏。\n(案例编号 eval-245;体裁=故障复盘;语言=Go;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0246",
|
|
"kind": "rust",
|
|
"prediction": "```rust\n// 中文注释:显示“发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```",
|
|
"target": "```rust\n// 中文注释:显示 “发布说明”\nlet name = \"status\";\nprintln!(\"{}\", name);\n```\n代码不能被改坏。\n(案例编号 eval-246;体裁=教程;语言=Rust;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0247",
|
|
"kind": "sql",
|
|
"prediction": "```sql\n-- 中文注释:显示“当前状态”\nSELECT \"API_KEY\" AS status;\n```",
|
|
"target": "```sql\n-- 中文注释:显示 “当前状态”\nSELECT \"API_KEY\" AS status;\n```\n代码不能被改坏。\n(案例编号 eval-247;体裁=故障复盘;语言=SQL;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0248",
|
|
"kind": "shell",
|
|
"prediction": "```bash\n# 中文注释:显示“请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```",
|
|
"target": "```bash\n# 中文注释:显示 “请求编号”\nname=\"dry_run\"\nprintf '%s\\n' \"$name\"\n```\n代码不能被改坏。\n(案例编号 eval-248;体裁=教程;语言=Shell;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0249",
|
|
"kind": "yaml",
|
|
"prediction": "# 中文注释:显示“缓存策略”\nname: \"ROLLBACK\"",
|
|
"target": "```yaml\n# 中文注释:显示 “缓存策略”\nname: \"ROLLBACK\"\n```\n代码不能被改坏。\n(案例编号 eval-249;体裁=故障复盘;语言=YAML;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0250",
|
|
"kind": "markdown",
|
|
"prediction": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。",
|
|
"target": "说明文字写着:“查看变更记录”;命令 `run-deploy` 必须保持。\n(案例编号 eval-250;体裁=教程;语言=Markdown;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0251",
|
|
"kind": "nested",
|
|
"prediction": "她说:\"他说'检查权限',然后才修改。\"",
|
|
"target": "她说:“他说‘检查权限’,然后才修改。”\n(案例编号 eval-251;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
},
|
|
{
|
|
"id": "eval-0252",
|
|
"kind": "comment",
|
|
"prediction": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。",
|
|
"target": "代码注释里的自然语言引用是:“查看变更记录”;Python 字符串 `status` 的语法必须保持。\n(案例编号 eval-252;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 2,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0253",
|
|
"kind": "quote",
|
|
"prediction": "报告引用用户原话:\"检查权限\"。英文原句 \"This value must remain unchanged.\" 也要原样保留。",
|
|
"target": "报告引用用户原话:“检查权限”。英文原句 \"This value must remain unchanged.\" 也要原样保留。\n(案例编号 eval-253;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 1,
|
|
"protected_total": 1
|
|
},
|
|
{
|
|
"id": "eval-0254",
|
|
"kind": "json",
|
|
"prediction": "请求编号必须存在。请求体示例:{\"message\": \"degraded\"}。",
|
|
"target": "中文说明:“请求编号必须存在”。请求体示例:{\"message\": \"degraded\"}。\n(案例编号 eval-254;体裁=教程;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 2,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 2,
|
|
"protected_total": 2
|
|
},
|
|
{
|
|
"id": "eval-0255",
|
|
"kind": "zh",
|
|
"prediction": "他宣布:\"今天发布 v3\",随后补充:\"请检查权限\"。 \n(案例编号:eval-255;体裁:故障复盘;语言:中文/英文自然语言;请保持事实和代码不变。)",
|
|
"target": "他说:“今天发布 v3”,随后补充:“检查权限”。\n(案例编号 eval-255;体裁=故障复盘;语言=中文/英文自然语言;请保持事实和代码不变。)",
|
|
"exact": 0,
|
|
"curly_count_target": 4,
|
|
"curly_count_pred": 0,
|
|
"protected_ok": 0,
|
|
"protected_total": 0
|
|
}
|
|
]
|
|
} |