ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,13 @@
|
||||
# 生成产物（幻灯片/音频/分段/最终视频）
|
||||
output/
|
||||
*.mp4
|
||||
*.mp3
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# 环境变量
|
||||
.env
|
||||
@@ -0,0 +1,337 @@
|
||||
# Experiment 5-5: Paper Lecture Video / 实验 5-5：论文讲解视频的自动生成 ★★
|
||||
|
||||
> Companion lab for *AI Agents in Depth*, Chapter 5 — spoken narration per slide, TTS, ffmpeg page-synced lecture video.
|
||||
> 《深入理解 AI Agent》第 5 章：每页口语讲解词 + TTS + ffmpeg 逐页同步合成带旁白的讲解视频。
|
||||
|
||||
← [Chapter 5 index / 返回第 5 章目录](../README.md)
|
||||
|
||||
---
|
||||
|
||||
## English
|
||||
|
||||
### Overview
|
||||
|
||||
On top of “paper → PPT”, the Agent generates **spoken lecture scripts** per slide (guiding narration, not bullet recitation), calls **TTS** for audio, then uses **ffmpeg** to **mux each slide PNG with its audio** into a narrated video.
|
||||
|
||||
### Canonical manuscript campaign
|
||||
|
||||
`campaign.py` is the formal Experiment 5-5 runner. It consumes twelve real
|
||||
Slidev screenshots and source from the pinned Experiment 5-4 paper run, calls
|
||||
Kimi K3 for narration, independently checks every narration against the actual
|
||||
slide pixels with Qwen-VL-Max, synthesizes every accepted page with Fish Audio
|
||||
S1, and produces a 5–15 minute H.264/AAC video. It checkpoints every provider
|
||||
call so an interrupted run resumes without replacing missing pages with
|
||||
silence or generated placeholders.
|
||||
|
||||
```bash
|
||||
python campaign.py --output validation/runs/my-real-run --workers 1
|
||||
```
|
||||
|
||||
The formal gate requires 12 distinct rendered pages, live receipts for all
|
||||
three providers, per-page A/V drift at most 0.2 seconds, final-duration drift at
|
||||
most 0.75 seconds, and a 300–900 second final video. `experiment_protocol.json`
|
||||
pins the source pages, models, thresholds, and authorized voice manifest.
|
||||
|
||||
The completed canonical run is
|
||||
[`validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/manifest.json`](validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/manifest.json)
|
||||
(SHA-256 `93bb69a916a76d12de56270928971f6e39f47755214f7a135817d7effd8b3f09`).
|
||||
All formal gates passed. The H.264/AAC result is 513.010 seconds (8.55
|
||||
minutes), summed page audio is 512.913 seconds, and the maximum measured page
|
||||
drift is 0.024 seconds. The rejected real malformed-JSON response for page 12
|
||||
is retained beside the successful retry instead of being hidden.
|
||||
|
||||
The `demo.py` flow below is retained as a fast teaching/compatibility path. Its
|
||||
built-in five-page PIL deck and offline silent audio do **not** satisfy the
|
||||
formal manuscript campaign.
|
||||
|
||||
### Legacy quick-demo pipeline
|
||||
|
||||
```
|
||||
Paper bullets (built-in sample)
|
||||
│ PIL render
|
||||
▼
|
||||
Per-page PNG slides ──► gpt-5.6-luna spoken script ──► OpenAI tts-1 → mp3
|
||||
│ │
|
||||
└──────────────── ffmpeg: each PNG + that page's audio ──┘
|
||||
│ (page duration = audio duration)
|
||||
▼
|
||||
ffmpeg concat
|
||||
▼
|
||||
output/lecture.mp4
|
||||
```
|
||||
|
||||
- **Self-contained**; does not depend on experiment 5-4: built-in *Attention Is All You Need* outline rendered to 5 slide PNGs via PIL (or replace with 5-4 Slidev screenshots).
|
||||
- Scripts from `gpt-5.6-luna`; audio from OpenAI `tts-1` (`voice=alloy`).
|
||||
- Video via ffmpeg: one mp4 segment per page with duration = that page’s audio, then concat → **display time matches speech exactly**.
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
# From the repository root: use the shared Chapter 5 environment
|
||||
uv sync --locked --python 3.12 --extra ch5
|
||||
|
||||
# Activate it before changing directories:
|
||||
# macOS/Linux:
|
||||
source .venv/bin/activate
|
||||
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
|
||||
# Windows cmd: .venv\Scripts\activate.bat
|
||||
|
||||
# pip fallback when uv is not installed:
|
||||
# python -m pip install -e ".[ch5]"
|
||||
|
||||
cd chapter5/paper-to-video
|
||||
|
||||
# Single-project compatibility path, still supported during migration:
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
cp env.example .env # OPENAI_API_KEY (or OPENROUTER_API_KEY for script fallback; TTS degrades offline)
|
||||
python demo.py # full 5-page lecture video
|
||||
```
|
||||
|
||||
Common flags (`python demo.py --help` for all):
|
||||
|
||||
```bash
|
||||
python demo.py --check # env check: ffmpeg/ffprobe/fonts/config; no API
|
||||
python demo.py --quick # smoke: page 1 only (same as --limit 1)
|
||||
python demo.py --limit 2 # first 2 pages only
|
||||
python demo.py --offline # no API: silent placeholder audio; validates ffmpeg pipeline
|
||||
```
|
||||
|
||||
Full flags:
|
||||
|
||||
| Flag | Description |
|
||||
| --- | --- |
|
||||
| `--slides FILE` | Slide content JSON (`[{title, subtitle, bullets}, ...]`); replaces built-in sample |
|
||||
| `--script FILE` | Ready narration JSON (list of strings, one per page); **skips LLM script gen** |
|
||||
| `-o, --output FILE` | Final video path (default `output/lecture.mp4`) |
|
||||
| `--tts-provider {openai,offline}` | TTS provider; `offline` = ffmpeg silent placeholder (no API) |
|
||||
| `--offline` | Fully offline: same as `--tts-provider offline` + bullet placeholder scripts (zero API) |
|
||||
| `--text-model / --tts-model / --tts-voice` | Override model/voice (defaults from same-named env vars) |
|
||||
| `--limit N / --quick / --check` | First N pages / page 1 only / check only |
|
||||
|
||||
> **Offline validation**: `--offline` needs no key or network; uses `ffmpeg anullsrc` with duration estimated from script length—runs “render → estimate duration → per-page mux → concat” to verify **per-page time alignment** (silent placeholders, not real voice).
|
||||
|
||||
Artifacts:
|
||||
|
||||
- `output/slides/slide_*.png` — slides
|
||||
- `output/audio/audio_*.mp3` — per-page audio
|
||||
- `output/segments/seg_*.mp4` — per-page segments
|
||||
- `output/narration.json` — scripts + audio durations
|
||||
- `output/lecture.mp4` — final video
|
||||
|
||||
Probe metadata:
|
||||
|
||||
```bash
|
||||
ffprobe -v error -show_format -show_streams output/lecture.mp4
|
||||
```
|
||||
|
||||
### Sample outputs
|
||||
|
||||
Built-in 5-page *Attention Is All You Need* full run (real artifacts):
|
||||
|
||||
- `output/lecture.mp4`: ~**2.8 MB**, **166.97s** (≈2m 47s), **1280×720**, **H.264** + **AAC**.
|
||||
- `output/audio/audio_01.mp3 … audio_05.mp3`: ~**28.6s / 33.3s / 37.2s / 37.9s / 29.9s** (sum ≈166.9s = video duration).
|
||||
- `output/narration.json`: e.g. page 1 spoken intro about “Attention Is All You Need” and attention without recurrence/convolutions.
|
||||
|
||||
Logs print per-page “slide → script → audio duration”; end summary compares total audio duration to final video (should match closely).
|
||||
|
||||
### Dependencies
|
||||
|
||||
- **ffmpeg / ffprobe** (validated on 8.x). macOS: `brew install ffmpeg`.
|
||||
- **Python**: root `ch5` extra (`openai`, `Pillow`, `python-dotenv`) or the compatibility `requirements.txt` path.
|
||||
- **CJK fonts**: script falls back over common macOS fonts (PingFang / STHeiti / Hiragino / Arial Unicode).
|
||||
- **Env**: `OPENAI_API_KEY` for official OpenAI; without it, `OPENROUTER_API_KEY` can fall back for scripts (TTS not on OpenRouter → offline silent). See `env.example`.
|
||||
|
||||
### Adapt / extend
|
||||
|
||||
- **Model / provider** (env or CLI; no code change):
|
||||
- `TEXT_MODEL` / `--text-model` (default `gpt-5.6-luna`).
|
||||
- `TTS_MODEL` / `TTS_VOICE` (or `--tts-model` / `--tts-voice`; default `tts-1` / `alloy`; voices e.g. `nova` / `shimmer` / `echo`).
|
||||
- `--tts-provider offline` for local validation.
|
||||
- `OPENAI_BASE_URL` + matching `OPENAI_API_KEY` for compatible endpoints.
|
||||
- **Input**: `--slides my.json` or edit `SLIDES` in `demo.py`; with real PDF, use 5-4 “paper → PPT” then feed bullets/screenshots here.
|
||||
- **Own scripts**: `--script narr.json` (string list) skips LLM and goes TTS → mux.
|
||||
- **Longer video**: more `SLIDES` pages or longer scripts (often 5–15 min).
|
||||
- **Fast tuning**: `--quick` / `--limit N` before full runs.
|
||||
|
||||
### Limitations
|
||||
|
||||
- Script + TTS call real OpenAI APIs (`TEXT_MODEL`, `TTS_MODEL`) and **bill**; full 5 pages ≈ 2–3 min video. Prefer `--check` then `--quick`.
|
||||
- Slides are static PIL (no animation/transitions); non-macOS may need `FONT_CANDIDATES` tweaks.
|
||||
- Page duration = audio only; no silence pads or BGM. For richer layout/transitions, prefer 5-4 Slidev screenshots as input.
|
||||
|
||||
---
|
||||
|
||||
## 中文
|
||||
|
||||
### 概述
|
||||
|
||||
在“论文 → PPT”的基础上，Agent 为每一页幻灯片生成**口语化讲解词**（引导性叙述，
|
||||
而非逐条复述要点），调用 **TTS** 合成语音，再用 **ffmpeg** 把 PPT 截图与音频
|
||||
**逐页同步合成**为一段带旁白的讲解视频。
|
||||
|
||||
### 正式实验活动
|
||||
|
||||
`campaign.py` 是实验 5-5 的正式运行器：它读取实验 5-4 固定真实论文活动中的 12 张
|
||||
Slidev 截图与源码，用 Kimi K3 生成讲解词，再让 Qwen-VL-Max 对照真实页面像素逐页独立
|
||||
审核；通过后调用 Fish Audio S1 合成每页语音，最终用 ffmpeg 生成 5–15 分钟的 H.264/AAC
|
||||
视频。所有供应商调用都可续跑缓存；中断后不会用静音或占位内容冒充缺失页面。
|
||||
|
||||
```bash
|
||||
python campaign.py --output validation/runs/my-real-run --workers 1
|
||||
```
|
||||
|
||||
正式门禁要求：12 张互不相同的真实渲染页、三个真实供应商的完整收据、逐页音画误差不超过
|
||||
0.2 秒、总时长误差不超过 0.75 秒、最终视频时长 300–900 秒。固定来源页、模型、阈值与
|
||||
授权音色清单记录在 `experiment_protocol.json`。
|
||||
|
||||
已完成的正式证据是
|
||||
[`validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/manifest.json`](validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/manifest.json)
|
||||
（SHA-256 `93bb69a916a76d12de56270928971f6e39f47755214f7a135817d7effd8b3f09`）。
|
||||
所有门禁均通过；H.264/AAC 成片长 513.010 秒（8.55 分钟），逐页音频合计
|
||||
512.913 秒，最大逐页漂移 0.024 秒。第 12 页真实供应商返回的非法 JSON
|
||||
作为失败尝试与成功重试一并保留，没有被隐藏。
|
||||
|
||||
下述 `demo.py` 流程保留为快速教学/兼容入口。其内置 5 页 PIL 幻灯片和离线静音模式
|
||||
**不满足**正式实验门禁。
|
||||
|
||||
### 旧版快速演示流程
|
||||
|
||||
```
|
||||
论文要点(内置示例)
|
||||
│ PIL 渲染
|
||||
▼
|
||||
每页 PNG 幻灯片 ──► gpt-5.6-luna 生成口语化讲解词 ──► OpenAI tts-1 合成 mp3
|
||||
│ │
|
||||
└──────────────── ffmpeg：每页 PNG + 该页音频 ───────────┘
|
||||
│ (每页时长 = 该页音频时长)
|
||||
▼
|
||||
ffmpeg concat 拼接
|
||||
▼
|
||||
output/lecture.mp4
|
||||
```
|
||||
|
||||
- 本项目**自包含**，不依赖实验 5-4：内置一份《Attention Is All You Need》的论文要点，
|
||||
用 PIL 直接渲染出 5 页幻灯片 PNG（也可替换为 5-4 的 Slidev 截图）。
|
||||
- 讲解词由 `gpt-5.6-luna` 生成；语音由 OpenAI `tts-1`（`voice=alloy`）合成。
|
||||
- 视频由 `ffmpeg` 合成：每页做一段 mp4，段时长等于该页音频时长，最后 concat 拼接，
|
||||
因此**每页展示时间与语音时长精确匹配**。
|
||||
|
||||
### 运行命令
|
||||
|
||||
```bash
|
||||
# 在仓库根目录使用统一的第 5 章环境
|
||||
uv sync --locked --python 3.12 --extra ch5
|
||||
|
||||
# 切换目录前先激活环境：
|
||||
# macOS/Linux：
|
||||
source .venv/bin/activate
|
||||
# Windows PowerShell：.\.venv\Scripts\Activate.ps1
|
||||
# Windows cmd：.venv\Scripts\activate.bat
|
||||
|
||||
# 未安装 uv 时可用 pip 兜底：
|
||||
# python -m pip install -e ".[ch5]"
|
||||
|
||||
cd chapter5/paper-to-video
|
||||
|
||||
# 迁移期间仍支持单项目兼容路径：
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
cp env.example .env # 填入 OPENAI_API_KEY（未配置时设 OPENROUTER_API_KEY 兜底讲解词，TTS 降级为离线占位）
|
||||
python demo.py # 生成全部 5 页的完整讲解视频
|
||||
```
|
||||
|
||||
常用参数（`python demo.py --help` 查看全部）：
|
||||
|
||||
```bash
|
||||
python demo.py --check # 环境自检：检查 ffmpeg/ffprobe/字体/配置，不调用任何 API
|
||||
python demo.py --quick # 快速冒烟：只跑第 1 页（等价 --limit 1），省时省钱
|
||||
python demo.py --limit 2 # 只处理前 2 页
|
||||
python demo.py --offline # 无需 API：占位静音音轨，验证整条 ffmpeg 合成流水线
|
||||
```
|
||||
|
||||
完整参数：
|
||||
|
||||
| 参数 | 说明 |
|
||||
| --- | --- |
|
||||
| `--slides FILE` | 幻灯片内容 JSON（`[{title, subtitle, bullets}, ...]`），替换内置示例 |
|
||||
| `--script FILE` | 现成讲解词 JSON（字符串列表，每页一段），提供后**跳过 LLM 生成** |
|
||||
| `-o, --output FILE` | 最终视频输出路径（默认 `output/lecture.mp4`） |
|
||||
| `--tts-provider {openai,offline}` | TTS 供应商；`offline` 用 ffmpeg 生成占位静音音轨（无需 API） |
|
||||
| `--offline` | 完全离线：等价 `--tts-provider offline`，并用要点占位讲解词（零 API 调用） |
|
||||
| `--text-model / --tts-model / --tts-voice` | 覆盖模型/音色（默认取同名环境变量） |
|
||||
| `--limit N / --quick / --check` | 只跑前 N 页 / 只跑第 1 页 / 仅自检 |
|
||||
|
||||
> **离线验证**：`--offline` 不需要任何 API Key 或网络，用 `ffmpeg anullsrc` 按讲解词字数
|
||||
> 估算时长合成静音占位音轨，跑通「渲染 → 估时 → 逐页合成 → concat 拼接」全链路，
|
||||
> 专门用于验证 ffmpeg 的**逐页时长对齐**是否正确（音轨为静音占位，非真实配音）。
|
||||
|
||||
产物：
|
||||
- `output/slides/slide_*.png` 每页幻灯片
|
||||
- `output/audio/audio_*.mp3` 每页讲解音频
|
||||
- `output/segments/seg_*.mp4` 每页分段视频
|
||||
- `output/narration.json` 每页讲解词与音频时长清单
|
||||
- `output/lecture.mp4` 最终讲解视频
|
||||
|
||||
查看视频元信息：
|
||||
|
||||
```bash
|
||||
ffprobe -v error -show_format -show_streams output/lecture.mp4
|
||||
```
|
||||
|
||||
### 预期输出示例
|
||||
|
||||
以内置的 5 页《Attention Is All You Need》为例，一次完整运行的真实产物：
|
||||
|
||||
- `output/lecture.mp4`：约 **2.8 MB**，时长 **166.97s**（≈2 分 47 秒），
|
||||
分辨率 **1280×720**，视频 **H.264** + 音频 **AAC**。
|
||||
- `output/audio/audio_01.mp3 … audio_05.mp3`：每页一段旁白，
|
||||
单页时长约 **28.6s / 33.3s / 37.2s / 37.9s / 29.9s**（总计 ≈166.9s，与视频时长一致）。
|
||||
- `output/narration.json`：每页的口语化讲解词与音频时长清单，例如第 1 页：
|
||||
|
||||
> 今天，我们将一起探讨一个改变了自然语言处理领域的重要研究——"Attention Is All You Need"……
|
||||
> 它完全依赖于注意力机制，摒弃了传统的循环和卷积结构。
|
||||
|
||||
运行日志会逐页打印「幻灯片 → 讲解词 → 音频时长」，末尾汇总各页音频总时长与最终视频时长
|
||||
（二者应基本一致，说明每页展示时间与语音精确对齐）。
|
||||
|
||||
### 依赖
|
||||
|
||||
- **ffmpeg / ffprobe**：命令行工具（本项目用 8.x 验证）。macOS 可 `brew install ffmpeg`。
|
||||
- **Python 包**：根目录 `ch5` extra（`openai`、`Pillow`、`python-dotenv`），或兼容 `requirements.txt` 路径。
|
||||
- **中文字体**：渲染幻灯片需系统中文字体，脚本已按 macOS 常见字体
|
||||
（PingFang / STHeiti / Hiragino / Arial Unicode）自动回退。
|
||||
- **环境变量**：需 `OPENAI_API_KEY`（走官方 OpenAI）；未配置时可用 `OPENROUTER_API_KEY` 兜底讲解词生成（此时 TTS 因不在 OpenRouter 上而降级为离线静音占位）。可选项见 `env.example`。
|
||||
|
||||
### 如何适配 / 扩展
|
||||
|
||||
- **换模型 / 换供应商**：环境变量或命令行均可，无需改代码：
|
||||
- `TEXT_MODEL` / `--text-model`：讲解词生成模型（默认 `gpt-5.6-luna`，可换其它）。
|
||||
- `TTS_MODEL` / `TTS_VOICE`（或 `--tts-model` / `--tts-voice`）：语音模型与音色
|
||||
（默认 `tts-1` / `alloy`，音色可选 `nova` / `shimmer` / `echo` 等）。
|
||||
- `--tts-provider offline`：切到离线占位音轨（不产生任何 API 调用），用于本地验证。
|
||||
- `OPENAI_BASE_URL`：指向任何**兼容 OpenAI 协议**的自定义端点（自建网关、代理或
|
||||
第三方供应商）；配合对应的 `OPENAI_API_KEY` 即可切换后端。
|
||||
- **换输入（换论文 / PDF）**：用 `--slides my.json` 传入外部幻灯片内容，或直接编辑
|
||||
`demo.py` 中的 `SLIDES` 列表（标题 / 副标题 / 要点）；若已有真实 PDF，可先用 5-4 的
|
||||
「论文 → PPT」流程产出要点或 Slidev 截图，再喂给本脚本，其余流程不变。
|
||||
- **自带讲解词**：用 `--script narr.json`（每页一段的字符串列表）跳过 LLM 生成，
|
||||
直接进入「TTS → 合成」，便于人工润色脚本后重跑。
|
||||
- **更长视频**：增加 `SLIDES` 页数或加长每页讲解词即可（单次 5~15 分钟）。
|
||||
- **快速调参**：先用 `--quick` / `--limit N` 只渲染少量页，确认音色/风格满意后再跑全量。
|
||||
|
||||
### 局限
|
||||
|
||||
- 讲解词与 TTS 都会产生真实的 OpenAI API 调用（`TEXT_MODEL` 与 `TTS_MODEL`），会**计费**；
|
||||
全量 5 页约生成 2~3 分钟视频。建议先 `--check` 自检、再 `--quick` 冒烟。
|
||||
- 幻灯片为 PIL 纯静态渲染（无动画/转场），中文字体依赖系统字体，非 macOS 需自行调整
|
||||
`FONT_CANDIDATES`。
|
||||
- 每页时长严格等于该页音频时长，不做静音停顿或背景音乐；如需更精细的排版与转场，
|
||||
建议改用 5-4 的 Slidev 截图作为输入。
|
||||
|
||||
---
|
||||
|
||||
## Notes / 说明
|
||||
|
||||
- Use `--check` / `--offline` before spending API budget. / 花钱前先 `--check` / `--offline`。
|
||||
- Commands/code/paths/env vars are identical in both language sections. / 命令、代码、路径与环境变量在中英文两侧保持一致。
|
||||
@@ -0,0 +1,467 @@
|
||||
"""Canonical real-API paper-to-lecture-video campaign for Experiment 5-5."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from openai import OpenAI
|
||||
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
PROTOCOL_PATH = HERE / "experiment_protocol.json"
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def sha256_bytes(value: bytes) -> str:
|
||||
return hashlib.sha256(value).hexdigest()
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
return sha256_bytes(path.read_bytes())
|
||||
|
||||
|
||||
def atomic_json(path: Path, value: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_suffix(path.suffix + ".tmp")
|
||||
temporary.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
temporary.replace(path)
|
||||
|
||||
|
||||
def run(command: list[str]) -> str:
|
||||
process = subprocess.run(command, text=True, capture_output=True)
|
||||
if process.returncode:
|
||||
raise RuntimeError(f"command failed ({process.returncode}): {' '.join(command)}\n{process.stderr}")
|
||||
return process.stdout
|
||||
|
||||
|
||||
def probe(path: Path) -> dict[str, Any]:
|
||||
return json.loads(run([
|
||||
"ffprobe", "-v", "error", "-show_streams", "-show_format", "-of", "json", str(path)
|
||||
]))
|
||||
|
||||
|
||||
def duration(probe_result: dict[str, Any]) -> float:
|
||||
return float(probe_result["format"]["duration"])
|
||||
|
||||
|
||||
def parse_json_content(content: str) -> dict[str, Any]:
|
||||
text = content.strip()
|
||||
if text.startswith("```"):
|
||||
text = re.sub(r"^```(?:json)?\s*|\s*```$", "", text, flags=re.S)
|
||||
start, end = text.find("{"), text.rfind("}")
|
||||
if start < 0 or end < start:
|
||||
raise ValueError("provider response did not contain a JSON object")
|
||||
value = json.loads(text[start:end + 1])
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("provider response was not a JSON object")
|
||||
return value
|
||||
|
||||
|
||||
def cached_json_call(
|
||||
path: Path,
|
||||
client: OpenAI,
|
||||
*,
|
||||
provider: str,
|
||||
model: str,
|
||||
messages: list[dict[str, Any]],
|
||||
max_tokens: int,
|
||||
) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
signature = sha256_bytes(json.dumps({
|
||||
"provider": provider, "model": model, "messages": messages,
|
||||
}, ensure_ascii=False, sort_keys=True).encode())
|
||||
if path.exists():
|
||||
cached = json.loads(path.read_text(encoding="utf-8"))
|
||||
if cached.get("signature") != signature:
|
||||
raise RuntimeError(f"cached provider-call signature mismatch: {path}")
|
||||
return cached["parsed"], cached["receipt"]
|
||||
failed_attempts: list[dict[str, Any]] = []
|
||||
for attempt in range(1, 4):
|
||||
started = time.perf_counter()
|
||||
response = client.chat.completions.create(
|
||||
model=model,
|
||||
messages=messages,
|
||||
temperature=1,
|
||||
max_tokens=max_tokens,
|
||||
response_format={"type": "json_object"},
|
||||
)
|
||||
raw = response.model_dump(mode="json")
|
||||
receipt = {
|
||||
"provider": provider,
|
||||
"request": {"model": model, "messages": messages, "temperature": 1, "max_tokens": max_tokens, "response_format": {"type": "json_object"}},
|
||||
"response": raw,
|
||||
"elapsed_ms": round((time.perf_counter() - started) * 1000, 3),
|
||||
"json_attempt": attempt,
|
||||
}
|
||||
try:
|
||||
parsed = parse_json_content(response.choices[0].message.content or "")
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
# Providers occasionally ignore response_format or truncate an
|
||||
# object. Preserve the real failed receipt and retry the API; do
|
||||
# not fabricate a narration or visual judgment to fill the page.
|
||||
failed_attempts.append({"error": str(exc), "receipt": receipt})
|
||||
atomic_json(
|
||||
path.with_name(path.stem + ".failed-attempts.json"),
|
||||
{"signature": signature, "attempts": failed_attempts},
|
||||
)
|
||||
if attempt == 3:
|
||||
raise RuntimeError(
|
||||
f"provider returned malformed JSON three times: {path}"
|
||||
) from exc
|
||||
continue
|
||||
atomic_json(path, {"signature": signature, "parsed": parsed, "receipt": receipt})
|
||||
return parsed, receipt
|
||||
raise AssertionError("unreachable JSON retry loop")
|
||||
|
||||
|
||||
def slide_sections(markdown: str) -> list[str]:
|
||||
parts = markdown.split("\n---\n")
|
||||
return [part.strip() for part in parts if "#" in part and "theme:" not in part]
|
||||
|
||||
|
||||
def image_data_url(path: Path) -> str:
|
||||
encoded = base64.b64encode(path.read_bytes()).decode()
|
||||
return f"data:image/png;base64,{encoded}"
|
||||
|
||||
|
||||
def provider_receipt_valid(receipt: dict[str, Any], expected_model: str) -> bool:
|
||||
response = receipt.get("response") or {}
|
||||
usage = response.get("usage") or {}
|
||||
return bool(response.get("id") and response.get("model") == expected_model and usage.get("total_tokens") is not None)
|
||||
|
||||
|
||||
def narration_prompt(section: str, page_index: int, total: int, feedback: str | None) -> str:
|
||||
return f"""你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。
|
||||
这是第 {page_index}/{total} 页。下面给出该页真实 Slidev 源码：
|
||||
<slide>\n{section}\n</slide>
|
||||
请写一段 180–240 个中文字符左右的口语讲解。必须满足：
|
||||
1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；
|
||||
2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；
|
||||
3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；
|
||||
4. 只返回 JSON：{{"narration":"...","visual_references":["..."]}}。
|
||||
{('上一轮独立视觉审核反馈：' + feedback + '。请修正。') if feedback else ''}"""
|
||||
|
||||
|
||||
def judge_prompt(narration: str, section: str) -> str:
|
||||
return f"""你是独立的讲解视频视觉审核员。图片是当前真实幻灯片；源码和旁白如下。
|
||||
<slide_source>\n{section}\n</slide_source>
|
||||
<narration>\n{narration}\n</narration>
|
||||
请只按图片中实际可见内容审核：
|
||||
- visual_alignment: 1–5，旁白是否准确呼应可见标题/公式/表格/图片；
|
||||
- guiding_narrative: 1–5，是否是讲解而非逐条复述；
|
||||
- factual_fidelity: 1–5，是否无虚构；
|
||||
- visible_elements_referenced: 列出旁白确实提到的可见元素；
|
||||
- issues: 具体问题列表。
|
||||
只返回 JSON 对象。"""
|
||||
|
||||
|
||||
def synthesize_fish(text: str, reference_id: str, output: Path) -> dict[str, Any]:
|
||||
from fish_audio_sdk import Session, TTSRequest
|
||||
|
||||
started = time.perf_counter()
|
||||
request = TTSRequest(text=text, reference_id=reference_id, format="mp3")
|
||||
audio = b"".join(Session(os.environ["FISH_API_KEY"]).tts(request, backend="s1"))
|
||||
if len(audio) < 1024:
|
||||
raise RuntimeError("Fish Audio returned empty or implausibly small audio")
|
||||
output.write_bytes(audio)
|
||||
return {
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": sha256_bytes(text.encode()),
|
||||
"text_characters": len(text),
|
||||
"reference_id_sha256": sha256_bytes(reference_id.encode()),
|
||||
"format": "mp3",
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": output.name,
|
||||
"sha256": sha256_file(output),
|
||||
"bytes": output.stat().st_size,
|
||||
},
|
||||
"elapsed_ms": round((time.perf_counter() - started) * 1000, 3),
|
||||
}
|
||||
|
||||
|
||||
def make_segment(slide: Path, audio: Path, output: Path, audio_duration: float) -> None:
|
||||
run([
|
||||
"ffmpeg", "-y", "-loglevel", "error", "-loop", "1", "-i", str(slide),
|
||||
"-i", str(audio), "-c:v", "libx264", "-tune", "stillimage", "-pix_fmt", "yuv420p",
|
||||
"-r", "30", "-vf", "scale=1280:720", "-c:a", "aac", "-b:a", "192k",
|
||||
"-t", f"{audio_duration:.6f}", "-movflags", "+faststart", str(output),
|
||||
])
|
||||
|
||||
|
||||
def process_page(
|
||||
run_dir: Path,
|
||||
narration_client: OpenAI,
|
||||
vision_client: OpenAI,
|
||||
protocol: dict[str, Any],
|
||||
page_number: int,
|
||||
source_page: int,
|
||||
slide: Path,
|
||||
section: str,
|
||||
reference_id: str,
|
||||
) -> dict[str, Any]:
|
||||
cache = run_dir / "provider_calls"
|
||||
cache.mkdir(exist_ok=True)
|
||||
max_rounds = int(protocol["acceptance"]["maximum_generation_review_rounds"])
|
||||
narration_model = protocol["providers"]["narration"]["model"]
|
||||
vision_model = protocol["providers"]["independent_visual_reviewer"]["model"]
|
||||
attempts = []
|
||||
feedback = None
|
||||
selected = None
|
||||
for attempt in range(1, max_rounds + 1):
|
||||
messages = [{"role": "user", "content": narration_prompt(section, page_number, protocol["acceptance"]["pages"], feedback)}]
|
||||
generated, narration_receipt = cached_json_call(
|
||||
cache / f"page-{page_number:02d}-narration-{attempt}.json", narration_client,
|
||||
provider="Moonshot", model=narration_model, messages=messages, max_tokens=1200,
|
||||
)
|
||||
narration = str(generated.get("narration") or "").strip()
|
||||
if not narration:
|
||||
raise RuntimeError(f"empty narration on page {page_number}, attempt {attempt}")
|
||||
judge_messages = [{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": judge_prompt(narration, section)},
|
||||
{"type": "image_url", "image_url": {"url": image_data_url(slide)}},
|
||||
],
|
||||
}]
|
||||
judged, judge_receipt = cached_json_call(
|
||||
cache / f"page-{page_number:02d}-vision-{attempt}.json", vision_client,
|
||||
provider="DashScope", model=vision_model, messages=judge_messages, max_tokens=1200,
|
||||
)
|
||||
scores = {
|
||||
name: int(judged.get(name, 0) or 0)
|
||||
for name in ("visual_alignment", "guiding_narrative", "factual_fidelity")
|
||||
}
|
||||
current = {
|
||||
"attempt": attempt,
|
||||
"narration": narration,
|
||||
"visual_references": generated.get("visual_references") or [],
|
||||
"judge": judged,
|
||||
"scores": scores,
|
||||
"narration_receipt": narration_receipt,
|
||||
"vision_receipt": judge_receipt,
|
||||
}
|
||||
attempts.append(current)
|
||||
if min(scores.values()) >= int(protocol["acceptance"]["minimum_visual_alignment_score"]):
|
||||
selected = current
|
||||
break
|
||||
feedback = json.dumps(judged.get("issues") or judged, ensure_ascii=False)
|
||||
if selected is None:
|
||||
selected = max(attempts, key=lambda item: min(item["scores"].values()))
|
||||
|
||||
audio = run_dir / "audio" / f"page-{page_number:02d}.mp3"
|
||||
audio.parent.mkdir(exist_ok=True)
|
||||
tts_receipt_path = run_dir / "provider_calls" / f"page-{page_number:02d}-fish-tts.json"
|
||||
expected_text_hash = sha256_bytes(selected["narration"].encode())
|
||||
if tts_receipt_path.exists() and audio.exists():
|
||||
tts_receipt = json.loads(tts_receipt_path.read_text(encoding="utf-8"))
|
||||
if (
|
||||
tts_receipt["request"]["text_sha256"] != expected_text_hash
|
||||
or tts_receipt["response_artifact"]["sha256"] != sha256_file(audio)
|
||||
):
|
||||
raise RuntimeError(f"Fish TTS checkpoint mismatch for page {page_number}")
|
||||
else:
|
||||
tts_receipt = synthesize_fish(selected["narration"], reference_id, audio)
|
||||
atomic_json(tts_receipt_path, tts_receipt)
|
||||
audio_probe = probe(audio)
|
||||
audio_duration = duration(audio_probe)
|
||||
segment = run_dir / "segments" / f"page-{page_number:02d}.mp4"
|
||||
segment.parent.mkdir(exist_ok=True)
|
||||
make_segment(slide, audio, segment, audio_duration)
|
||||
segment_probe = probe(segment)
|
||||
segment_duration = duration(segment_probe)
|
||||
return {
|
||||
"page": page_number,
|
||||
"source_page": source_page,
|
||||
"slide": {"path": str(slide.relative_to(run_dir)), "sha256": sha256_file(slide), "bytes": slide.stat().st_size},
|
||||
"section_sha256": sha256_bytes(section.encode()),
|
||||
"selected_attempt": selected["attempt"],
|
||||
"narration": selected["narration"],
|
||||
"narration_characters": len(selected["narration"]),
|
||||
"visual_references": selected["visual_references"],
|
||||
"review": selected["judge"],
|
||||
"scores": selected["scores"],
|
||||
"attempts": attempts,
|
||||
"tts_receipt": tts_receipt,
|
||||
"audio": {"path": str(audio.relative_to(run_dir)), "sha256": sha256_file(audio), "bytes": audio.stat().st_size, "seconds": audio_duration, "probe": audio_probe},
|
||||
"segment": {"path": str(segment.relative_to(run_dir)), "sha256": sha256_file(segment), "bytes": segment.stat().st_size, "seconds": segment_duration, "probe": segment_probe},
|
||||
"av_delta_seconds": abs(segment_duration - audio_duration),
|
||||
}
|
||||
|
||||
|
||||
def concat_segments(run_dir: Path, pages: list[dict[str, Any]], output: Path) -> None:
|
||||
listing = run_dir / "segments" / "concat.txt"
|
||||
listing.write_text("".join(
|
||||
f"file '{(run_dir / page['segment']['path']).resolve()}'\n" for page in pages
|
||||
), encoding="utf-8")
|
||||
run([
|
||||
"ffmpeg", "-y", "-loglevel", "error", "-f", "concat", "-safe", "0", "-i", str(listing),
|
||||
"-c", "copy", "-movflags", "+faststart", str(output),
|
||||
])
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--output", type=Path)
|
||||
parser.add_argument("--workers", type=int, default=3)
|
||||
args = parser.parse_args()
|
||||
load_dotenv(HERE / ".env")
|
||||
required = ("MOONSHOT_API_KEY", "DASHSCOPE_API_KEY", "FISH_API_KEY")
|
||||
missing = [name for name in required if not os.getenv(name)]
|
||||
if missing:
|
||||
raise RuntimeError(f"missing required real-provider credentials: {missing}")
|
||||
if not shutil.which("ffmpeg") or not shutil.which("ffprobe"):
|
||||
raise RuntimeError("ffmpeg and ffprobe are required")
|
||||
|
||||
protocol_bytes = PROTOCOL_PATH.read_bytes()
|
||||
protocol = json.loads(protocol_bytes)
|
||||
protocol_hash = sha256_bytes(protocol_bytes)
|
||||
run_dir = (args.output or HERE / "validation" / "runs" / f"exp5-5-real-{datetime.now().strftime('%Y%m%d-%H%M%S')}").resolve()
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
protocol_copy = run_dir / "experiment_protocol.json"
|
||||
if protocol_copy.exists() and protocol_copy.read_bytes() != protocol_bytes:
|
||||
raise RuntimeError("run directory has a different frozen protocol")
|
||||
protocol_copy.write_bytes(protocol_bytes)
|
||||
|
||||
source_spec = protocol["source"]
|
||||
paper = (HERE / source_spec["paper"]).resolve()
|
||||
markdown_path = (HERE / source_spec["slide_markdown"]).resolve()
|
||||
rendered = (HERE / source_spec["rendered_slides"]).resolve()
|
||||
for path in (paper, markdown_path, rendered):
|
||||
if not path.exists():
|
||||
raise RuntimeError(f"required Experiment 5-4 source is missing: {path}")
|
||||
sections = slide_sections(markdown_path.read_text(encoding="utf-8"))
|
||||
selected_pages = [int(value) for value in source_spec["selected_pages"]]
|
||||
if len(selected_pages) != int(protocol["acceptance"]["pages"]):
|
||||
raise RuntimeError("selected page count differs from protocol")
|
||||
slides_dir = run_dir / "slides"
|
||||
slides_dir.mkdir(exist_ok=True)
|
||||
page_inputs = []
|
||||
for page_number, source_page in enumerate(selected_pages, 1):
|
||||
source_image = rendered / f"{source_page}.png"
|
||||
copied = slides_dir / f"page-{page_number:02d}-source-{source_page:02d}.png"
|
||||
if not copied.exists():
|
||||
shutil.copy2(source_image, copied)
|
||||
elif sha256_file(copied) != sha256_file(source_image):
|
||||
raise RuntimeError(f"copied slide mutated: {copied}")
|
||||
page_inputs.append((page_number, source_page, copied, sections[source_page - 1]))
|
||||
|
||||
voice_manifest_path = (HERE / protocol["providers"]["tts"]["authorized_reference_manifest"]).resolve()
|
||||
voice_manifest = json.loads(voice_manifest_path.read_text(encoding="utf-8"))
|
||||
reference_id = str(voice_manifest.get("source_reference_id") or "")
|
||||
if not reference_id:
|
||||
raise RuntimeError("authorized Fish voice manifest has no source_reference_id")
|
||||
narration_provider = protocol["providers"]["narration"]
|
||||
vision_provider = protocol["providers"]["independent_visual_reviewer"]
|
||||
narration_client = OpenAI(api_key=os.environ["MOONSHOT_API_KEY"], base_url=narration_provider["endpoint"], timeout=180, max_retries=3)
|
||||
vision_client = OpenAI(api_key=os.environ["DASHSCOPE_API_KEY"], base_url=vision_provider["endpoint"], timeout=180, max_retries=3)
|
||||
|
||||
page_results = []
|
||||
with ThreadPoolExecutor(max_workers=args.workers) as pool:
|
||||
futures = {
|
||||
pool.submit(
|
||||
process_page, run_dir, narration_client, vision_client, protocol,
|
||||
page_number, source_page, slide, section, reference_id,
|
||||
): page_number
|
||||
for page_number, source_page, slide, section in page_inputs
|
||||
}
|
||||
for future in as_completed(futures):
|
||||
page = futures[future]
|
||||
result = future.result()
|
||||
page_results.append(result)
|
||||
print(f"page {page}/12 complete: {result['audio']['seconds']:.2f}s", flush=True)
|
||||
page_results.sort(key=lambda item: item["page"])
|
||||
final_video = run_dir / "lecture.mp4"
|
||||
concat_segments(run_dir, page_results, final_video)
|
||||
video_probe = probe(final_video)
|
||||
video_duration = duration(video_probe)
|
||||
audio_total = sum(page["audio"]["seconds"] for page in page_results)
|
||||
streams = video_probe["streams"]
|
||||
video_streams = [stream for stream in streams if stream.get("codec_type") == "video"]
|
||||
audio_streams = [stream for stream in streams if stream.get("codec_type") == "audio"]
|
||||
min_score = int(protocol["acceptance"]["minimum_visual_alignment_score"])
|
||||
acceptance = {
|
||||
"real_paper_and_experiment_5_4_slides": len(paper.read_bytes()) > 1_000_000 and len(page_results) == 12,
|
||||
"twelve_selected_real_rendered_pages": len({page["slide"]["sha256"] for page in page_results}) == 12,
|
||||
"live_narration_provider_receipts": all(
|
||||
provider_receipt_valid(attempt["narration_receipt"], narration_provider["model"])
|
||||
for page in page_results for attempt in page["attempts"]
|
||||
),
|
||||
"independent_live_vision_review_receipts": all(
|
||||
provider_receipt_valid(attempt["vision_receipt"], vision_provider["model"])
|
||||
for page in page_results for attempt in page["attempts"]
|
||||
),
|
||||
"visual_narrative_alignment": all(min(page["scores"].values()) >= min_score for page in page_results),
|
||||
"real_fish_s1_audio_each_page": all(
|
||||
page["tts_receipt"]["provider"] == "Fish Audio"
|
||||
and page["tts_receipt"]["model"] == "s1"
|
||||
and page["audio"]["bytes"] > 1024
|
||||
for page in page_results
|
||||
),
|
||||
"per_page_display_matches_audio": all(
|
||||
page["av_delta_seconds"] <= float(protocol["acceptance"]["per_page_av_alignment_tolerance_seconds"])
|
||||
for page in page_results
|
||||
),
|
||||
"duration_5_to_15_minutes": float(protocol["acceptance"]["duration_seconds_min"]) <= video_duration <= float(protocol["acceptance"]["duration_seconds_max"]),
|
||||
"final_duration_matches_page_audio": abs(video_duration - audio_total) <= float(protocol["acceptance"]["final_duration_tolerance_seconds"]),
|
||||
"h264_video_and_aac_audio": bool(video_streams and audio_streams) and video_streams[0]["codec_name"] == "h264" and audio_streams[0]["codec_name"] == "aac",
|
||||
"raw_artifact_hashes": all(page["audio"]["sha256"] and page["segment"]["sha256"] and page["slide"]["sha256"] for page in page_results),
|
||||
}
|
||||
acceptance["passed"] = all(acceptance.values())
|
||||
evidence = {
|
||||
"experiment_id": "5-5",
|
||||
"status": "passed" if acceptance["passed"] else "partial",
|
||||
"created_at": utc_now(),
|
||||
"protocol_sha256": protocol_hash,
|
||||
"source": {
|
||||
"paper": str(paper), "paper_sha256": sha256_file(paper), "paper_bytes": paper.stat().st_size,
|
||||
"slide_markdown": str(markdown_path), "slide_markdown_sha256": sha256_file(markdown_path),
|
||||
"selected_source_pages": selected_pages,
|
||||
"voice_manifest_sha256": sha256_file(voice_manifest_path),
|
||||
"authorized_reference_id_sha256": sha256_bytes(reference_id.encode()),
|
||||
},
|
||||
"scope": {"pages": len(page_results), "narration_and_review_attempts": sum(len(page["attempts"]) for page in page_results)},
|
||||
"summary": {"audio_seconds_sum": audio_total, "video_seconds": video_duration, "minutes": video_duration / 60, "max_av_delta_seconds": max(page["av_delta_seconds"] for page in page_results)},
|
||||
"acceptance": acceptance,
|
||||
"pages": page_results,
|
||||
"final_video": {"path": final_video.name, "sha256": sha256_file(final_video), "bytes": final_video.stat().st_size, "probe": video_probe},
|
||||
}
|
||||
comparison = run_dir / "comparison.json"
|
||||
atomic_json(comparison, evidence)
|
||||
artifacts = {
|
||||
str(path.relative_to(run_dir)): {"sha256": sha256_file(path), "bytes": path.stat().st_size}
|
||||
for path in sorted(item for item in run_dir.rglob("*") if item.is_file() and item.name != "manifest.json")
|
||||
}
|
||||
manifest = {
|
||||
"experiment_id": "5-5", "status": evidence["status"], "acceptance": acceptance,
|
||||
"comparison_sha256": sha256_file(comparison), "artifacts": artifacts,
|
||||
}
|
||||
manifest_path = run_dir / "manifest.json"
|
||||
atomic_json(manifest_path, manifest)
|
||||
atomic_json(HERE / "validation" / "latest.json", {
|
||||
"experiment_id": "5-5", "status": evidence["status"],
|
||||
"run_dir": str(run_dir.relative_to(HERE)), "manifest_sha256": sha256_file(manifest_path),
|
||||
})
|
||||
print(json.dumps({"status": evidence["status"], "summary": evidence["summary"], "acceptance": acceptance}, ensure_ascii=False, indent=2))
|
||||
return 0 if acceptance["passed"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,631 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
实验 5-5：论文讲解视频的自动生成（★★）
|
||||
|
||||
流水线（端到端自包含，无需依赖 5-4）：
|
||||
1) 幻灯片：用 PIL 生成若干页带标题/要点的 PNG（模拟“论文 -> PPT”的产物），
|
||||
也可用 --slides 传入外部 JSON 替换内置示例。
|
||||
2) 讲解词：对每一页调用 gpt-5.6-luna 生成【口语化、引导性】的讲解文字
|
||||
（是叙述而非复述要点，负责承上启下）；也可用 --script 直接喂入现成脚本。
|
||||
3) TTS：用 OpenAI tts-1（voice=alloy）把讲解词合成为每页的语音 mp3；
|
||||
或用 --tts-provider offline 让 ffmpeg 生成占位静音音轨（无需任何 API）。
|
||||
4) 合成：用 ffmpeg 把「每页 PNG + 该页音频」合成为分段视频（每页时长=该页音频时长），
|
||||
再用 concat 拼接为一个 output/lecture.mp4（输出路径可用 --output 指定）。
|
||||
5) 校验：用 ffprobe 打印最终 mp4 的时长/分辨率/音视频流信息。
|
||||
|
||||
依赖：ffmpeg / ffprobe（命令行）、Python 包见 requirements.txt。
|
||||
环境变量：OPENAI_API_KEY（用 openai 供应商时必填；未配置时可用 OPENROUTER_API_KEY 兜底讲解词生成，TTS 降级为离线占位），
|
||||
可选 OPENAI_BASE_URL / TEXT_MODEL / TTS_MODEL / TTS_VOICE。
|
||||
提示：想在无 API / 无网络时验证整条 ffmpeg 合成流水线，用 `python demo.py --offline`。
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
try:
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 路径与配置
|
||||
# ---------------------------------------------------------------------------
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
OUTPUT_DIR = ROOT / "output"
|
||||
SLIDES_DIR = OUTPUT_DIR / "slides"
|
||||
AUDIO_DIR = OUTPUT_DIR / "audio"
|
||||
SEG_DIR = OUTPUT_DIR / "segments"
|
||||
FINAL_MP4 = OUTPUT_DIR / "lecture.mp4"
|
||||
|
||||
# 默认模型/音色：优先取环境变量，命令行 --text-model 等可再覆盖。
|
||||
DEFAULT_TEXT_MODEL = os.getenv("TEXT_MODEL", "gpt-5.6-luna")
|
||||
DEFAULT_TTS_MODEL = os.getenv("TTS_MODEL", "tts-1")
|
||||
DEFAULT_TTS_VOICE = os.getenv("TTS_VOICE", "alloy")
|
||||
|
||||
OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"
|
||||
|
||||
|
||||
def map_model_to_openrouter(model: str) -> str:
|
||||
"""把直连模型名映射为 OpenRouter 上的 id（非可映射 id 统一兜底到当前廉价旗舰）。"""
|
||||
if not model or "/" in model:
|
||||
return model or "openai/gpt-5.6-luna"
|
||||
m = model.lower()
|
||||
if m.startswith(("gpt-", "o1", "o3", "o4")):
|
||||
return "openai/" + model
|
||||
if m.startswith("claude"):
|
||||
if "haiku" in m:
|
||||
return "anthropic/claude-haiku-4.5"
|
||||
if "sonnet" in m:
|
||||
return "anthropic/claude-sonnet-4.6"
|
||||
return "anthropic/claude-opus-4.8"
|
||||
if m.startswith("gemini"):
|
||||
return "google/" + model
|
||||
return "openai/gpt-5.6-luna"
|
||||
|
||||
# 离线占位音轨的中文语速估算（字/秒），用于把讲解词长度换算成展示时长。
|
||||
OFFLINE_CHARS_PER_SEC = 4.5
|
||||
|
||||
# 视频参数
|
||||
WIDTH, HEIGHT = 1280, 720
|
||||
FPS = 30
|
||||
|
||||
# macOS 上可用的中文字体（按优先级回退）
|
||||
FONT_CANDIDATES = [
|
||||
"/System/Library/Fonts/PingFang.ttc",
|
||||
"/System/Library/Fonts/STHeiti Medium.ttc",
|
||||
"/System/Library/Fonts/Hiragino Sans GB.ttc",
|
||||
"/System/Library/Fonts/Supplemental/Arial Unicode.ttf",
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
"""一次运行的可调参数（由命令行/环境变量组装）。"""
|
||||
|
||||
provider: str = "openai" # openai | offline
|
||||
text_model: str = DEFAULT_TEXT_MODEL
|
||||
tts_model: str = DEFAULT_TTS_MODEL
|
||||
tts_voice: str = DEFAULT_TTS_VOICE
|
||||
limit: "int | None" = None
|
||||
output: Path = FINAL_MP4
|
||||
slides: "list[dict] | None" = None # 幻灯片内容（None=用内置示例）
|
||||
script: "list[str] | None" = None # 现成讲解词（None=按需生成）
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 模拟“论文 -> PPT”的产物：每页的标题与要点。
|
||||
# 这里用《Attention Is All You Need》（Transformer）作为示例论文。
|
||||
# 在真实的 5-4 流程中，这些数据由 Proposer/Reviewer Agent 从论文 PDF 生成。
|
||||
# 也可用 --slides your_slides.json 传入同样结构的外部数据替换本示例。
|
||||
# ---------------------------------------------------------------------------
|
||||
SLIDES = [
|
||||
{
|
||||
"title": "Attention Is All You Need",
|
||||
"subtitle": "Transformer：一种全新的序列建模架构",
|
||||
"bullets": [
|
||||
"Vaswani 等人，2017 年发表于 NeurIPS",
|
||||
"完全基于注意力机制，抛弃循环与卷积",
|
||||
"在机器翻译任务上取得当时最优效果",
|
||||
],
|
||||
},
|
||||
{
|
||||
"title": "研究背景与动机",
|
||||
"subtitle": "为什么要抛弃 RNN？",
|
||||
"bullets": [
|
||||
"RNN 按时间步串行计算，难以并行",
|
||||
"长距离依赖在梯度传播中容易衰减",
|
||||
"训练大模型时的计算效率成为瓶颈",
|
||||
],
|
||||
},
|
||||
{
|
||||
"title": "核心方法：自注意力",
|
||||
"subtitle": "Self-Attention 与多头机制",
|
||||
"bullets": [
|
||||
"用 Query / Key / Value 计算词与词的关联",
|
||||
"多头注意力从不同子空间捕捉多种关系",
|
||||
"位置编码为模型注入序列顺序信息",
|
||||
],
|
||||
},
|
||||
{
|
||||
"title": "实验结果",
|
||||
"subtitle": "更快、更准",
|
||||
"bullets": [
|
||||
"WMT14 英德翻译 BLEU 达 28.4，创新高",
|
||||
"训练成本显著低于此前的最优模型",
|
||||
"可高度并行，充分利用 GPU 算力",
|
||||
],
|
||||
},
|
||||
{
|
||||
"title": "总结与影响",
|
||||
"subtitle": "开启大模型时代",
|
||||
"bullets": [
|
||||
"Transformer 成为 NLP 的通用骨架",
|
||||
"催生 BERT、GPT 等预训练大模型",
|
||||
"影响扩展到视觉、语音、多模态领域",
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 工具函数
|
||||
# ---------------------------------------------------------------------------
|
||||
def load_font(size: int) -> ImageFont.FreeTypeFont:
|
||||
"""按候选列表加载一个可用字体（支持中文）。"""
|
||||
for path in FONT_CANDIDATES:
|
||||
if os.path.exists(path):
|
||||
try:
|
||||
return ImageFont.truetype(path, size)
|
||||
except Exception:
|
||||
continue
|
||||
return ImageFont.load_default()
|
||||
|
||||
|
||||
def run(cmd: list) -> str:
|
||||
"""执行命令并返回 stdout，失败则抛出异常并打印 stderr。"""
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"命令失败: {' '.join(cmd)}\nSTDERR:\n{proc.stderr}"
|
||||
)
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def ffprobe_duration(path: Path) -> float:
|
||||
"""用 ffprobe 读取媒体文件时长（秒）。缺少时长元数据时 ffprobe 输出 N/A，给出清晰报错。"""
|
||||
out = run(
|
||||
[
|
||||
"ffprobe", "-v", "error",
|
||||
"-show_entries", "format=duration",
|
||||
"-of", "default=noprint_wrappers=1:nokey=1",
|
||||
str(path),
|
||||
]
|
||||
)
|
||||
out = out.strip()
|
||||
if not out or out == "N/A":
|
||||
raise RuntimeError(f"ffprobe 无法读取时长（文件缺少时长元数据或不是音视频文件）：{path}")
|
||||
return float(out)
|
||||
|
||||
|
||||
def load_slides_file(path: Path) -> list:
|
||||
"""从 JSON 文件加载幻灯片内容（[{title, subtitle, bullets}, ...]）。"""
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(data, list) or not data:
|
||||
sys.exit(f"[错误] --slides 文件应是非空的 JSON 列表：{path}")
|
||||
for i, s in enumerate(data):
|
||||
if not all(k in s for k in ("title", "subtitle", "bullets")):
|
||||
sys.exit(f"[错误] --slides 第 {i + 1} 项缺少 title/subtitle/bullets 字段。")
|
||||
return data
|
||||
|
||||
|
||||
def load_script_file(path: Path) -> list:
|
||||
"""从 JSON 文件加载现成讲解词（每页一段的字符串列表）。"""
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(data, list) or not all(isinstance(x, str) for x in data):
|
||||
sys.exit(f"[错误] --script 文件应是 JSON 字符串列表（每页一段）：{path}")
|
||||
return data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 步骤 1：渲染幻灯片 PNG
|
||||
# ---------------------------------------------------------------------------
|
||||
def _slide_bullets(slide: dict) -> list[str]:
|
||||
"""Keep string bullets only; JSON null / non-str items are skipped."""
|
||||
bullets = slide.get("bullets") or []
|
||||
return [b for b in bullets if isinstance(b, str)]
|
||||
|
||||
|
||||
def render_slide(slide: dict, index: int, total: int) -> Path:
|
||||
"""把一页幻灯片渲染为 1280x720 的 PNG。"""
|
||||
img = Image.new("RGB", (WIDTH, HEIGHT), color=(23, 32, 56)) # 深蓝底
|
||||
draw = ImageDraw.Draw(img)
|
||||
|
||||
title_font = load_font(58)
|
||||
subtitle_font = load_font(34)
|
||||
bullet_font = load_font(32)
|
||||
footer_font = load_font(22)
|
||||
|
||||
# 顶部装饰条
|
||||
draw.rectangle([0, 0, WIDTH, 12], fill=(88, 166, 255))
|
||||
|
||||
# 标题（超宽自动换行）
|
||||
y = 90
|
||||
for line in textwrap.wrap(slide["title"], width=22):
|
||||
draw.text((90, y), line, font=title_font, fill=(255, 255, 255))
|
||||
y += 72
|
||||
|
||||
# 副标题
|
||||
y += 6
|
||||
draw.text((90, y), slide["subtitle"], font=subtitle_font, fill=(88, 166, 255))
|
||||
y += 70
|
||||
|
||||
# 要点
|
||||
for bullet in _slide_bullets(slide):
|
||||
draw.ellipse([94, y + 14, 110, y + 30], fill=(88, 166, 255))
|
||||
for j, line in enumerate(textwrap.wrap(bullet, width=30)):
|
||||
draw.text((130, y), line, font=bullet_font, fill=(220, 226, 240))
|
||||
y += 44
|
||||
y += 16
|
||||
|
||||
# 页脚：页码
|
||||
footer = f"第 {index + 1} / {total} 页"
|
||||
draw.text((90, HEIGHT - 50), footer, font=footer_font, fill=(120, 132, 160))
|
||||
|
||||
path = SLIDES_DIR / f"slide_{index + 1:02d}.png"
|
||||
img.save(path)
|
||||
return path
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 步骤 2：为每页生成口语化讲解词
|
||||
# ---------------------------------------------------------------------------
|
||||
def offline_narration(slide: dict) -> str:
|
||||
"""离线占位讲解词：不调用 LLM，用副标题+要点拼出一段可读文本（供占位音轨估时）。"""
|
||||
return f"{slide['subtitle']}。" + "；".join(_slide_bullets(slide)) + "。"
|
||||
|
||||
|
||||
def generate_narration(client, cfg: Config, slide: dict, index: int, total: int) -> str:
|
||||
"""调用文本模型（默认 gpt-5.6-luna），为当前页生成口语化、引导性的讲解文字。"""
|
||||
position = (
|
||||
"这是开场第一页，请自然地引入主题" if index == 0
|
||||
else "这是最后一页，请做收尾总结" if index == total - 1
|
||||
else "这是中间页，请与上一页自然衔接、承上启下"
|
||||
)
|
||||
prompt = (
|
||||
"你是一位科普讲师，正在为一段论文讲解视频配音。\n"
|
||||
f"当前是第 {index + 1}/{total} 页幻灯片。{position}。\n\n"
|
||||
f"幻灯片标题：{slide['title']}\n"
|
||||
f"副标题：{slide['subtitle']}\n"
|
||||
f"要点：\n- " + "\n- ".join(_slide_bullets(slide)) + "\n\n"
|
||||
"请生成这一页的口语化讲解词，要求：\n"
|
||||
"1) 是引导性的口语叙述，而不是逐条复述要点；\n"
|
||||
"2) 自然流畅、有过渡，像真人讲课；\n"
|
||||
"3) 长度控制在 3~4 句话（约 70~110 字）；\n"
|
||||
"4) 只输出讲解词正文，不要任何前后缀、标题或列表符号。"
|
||||
)
|
||||
# 推理模型（gpt-5 / o 系列等）可能不接受自定义 temperature，统一置 1。
|
||||
_reasoning = any(k in (cfg.text_model or "").lower()
|
||||
for k in ("gpt-5", "o1", "o3", "o4", "thinking", "reasoner", "kimi-k3"))
|
||||
resp = client.chat.completions.create(
|
||||
model=cfg.text_model,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
temperature=1 if _reasoning else 0.7,
|
||||
)
|
||||
return resp.choices[0].message.content.strip()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 步骤 3：TTS 合成语音
|
||||
# ---------------------------------------------------------------------------
|
||||
def synthesize_openai(client, cfg: Config, text: str, index: int) -> Path:
|
||||
"""用 OpenAI tts-1 把讲解词合成为 mp3。"""
|
||||
path = AUDIO_DIR / f"audio_{index + 1:02d}.mp3"
|
||||
# 使用流式写盘接口，避免把整段音频读进内存
|
||||
with client.audio.speech.with_streaming_response.create(
|
||||
model=cfg.tts_model,
|
||||
voice=cfg.tts_voice,
|
||||
input=text,
|
||||
) as response:
|
||||
response.stream_to_file(str(path))
|
||||
return path
|
||||
|
||||
|
||||
def synthesize_offline(text: str, index: int) -> Path:
|
||||
"""离线占位 TTS：用 ffmpeg 生成一段“静音” mp3，时长按讲解词字数估算。
|
||||
|
||||
这样无需任何 API/网络即可跑通「渲染 -> 估时 -> ffmpeg 合成」全链路，
|
||||
用于验证 ffmpeg 逐页对齐与拼接是否正确（音轨为静音占位，非真实配音）。
|
||||
"""
|
||||
path = AUDIO_DIR / f"audio_{index + 1:02d}.mp3"
|
||||
duration = max(2.0, len(text) / OFFLINE_CHARS_PER_SEC)
|
||||
run(
|
||||
[
|
||||
"ffmpeg", "-y",
|
||||
"-f", "lavfi",
|
||||
"-i", "anullsrc=channel_layout=mono:sample_rate=24000",
|
||||
"-t", f"{duration:.3f}",
|
||||
"-c:a", "libmp3lame", "-q:a", "9",
|
||||
str(path),
|
||||
]
|
||||
)
|
||||
return path
|
||||
|
||||
|
||||
def synthesize_speech(client, cfg: Config, text: str, index: int) -> Path:
|
||||
"""按供应商合成一段语音音频。"""
|
||||
if cfg.provider == "offline":
|
||||
return synthesize_offline(text, index)
|
||||
return synthesize_openai(client, cfg, text, index)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 步骤 4：ffmpeg 合成
|
||||
# ---------------------------------------------------------------------------
|
||||
def build_segment(png: Path, mp3: Path, index: int, duration: float) -> Path:
|
||||
"""把「一页 PNG + 该页音频」合成为一段 mp4。
|
||||
|
||||
用 -t 把整段时长精确锁定为该页音频时长，保证“每页展示时间与语音时长精确匹配”
|
||||
（仅靠 -loop + -shortest 会让静态图轨比音频多出约 1~2 秒）。
|
||||
"""
|
||||
out = SEG_DIR / f"seg_{index + 1:02d}.mp4"
|
||||
run(
|
||||
[
|
||||
"ffmpeg", "-y",
|
||||
"-loop", "1", "-i", str(png), # 静态图片循环作为视频轨
|
||||
"-i", str(mp3), # 该页音频
|
||||
"-c:v", "libx264", "-tune", "stillimage",
|
||||
"-pix_fmt", "yuv420p",
|
||||
"-r", str(FPS),
|
||||
"-vf", f"scale={WIDTH}:{HEIGHT}",
|
||||
"-c:a", "aac", "-b:a", "192k",
|
||||
"-t", f"{duration:.3f}", # 精确锁定为音频时长
|
||||
str(out),
|
||||
]
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def concat_segments(segments: list, output: Path) -> Path:
|
||||
"""用 concat demuxer 把各分段无损拼接为最终 mp4。"""
|
||||
list_file = SEG_DIR / "concat.txt"
|
||||
list_file.write_text(
|
||||
"".join(f"file '{seg.name}'\n" for seg in segments), encoding="utf-8"
|
||||
)
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
run(
|
||||
[
|
||||
"ffmpeg", "-y",
|
||||
"-f", "concat", "-safe", "0",
|
||||
"-i", str(list_file),
|
||||
"-c", "copy",
|
||||
str(output),
|
||||
]
|
||||
)
|
||||
return output
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 自检（不产生任何 API 调用）：检查外部命令与关键配置是否就绪。
|
||||
# ---------------------------------------------------------------------------
|
||||
def self_check(cfg: Config) -> int:
|
||||
"""快速自检 ffmpeg/ffprobe、中文字体与关键环境变量，返回退出码。"""
|
||||
ok = True
|
||||
print("=== 环境自检（不调用任何 API）===")
|
||||
|
||||
for tool in ("ffmpeg", "ffprobe"):
|
||||
found = shutil.which(tool)
|
||||
print(f" {'[OK]' if found else '[缺失]'} {tool}: {found or '未找到，请安装 ffmpeg'}")
|
||||
ok = ok and bool(found)
|
||||
|
||||
font = next((p for p in FONT_CANDIDATES if os.path.exists(p)), None)
|
||||
print(f" {'[OK]' if font else '[回退]'} 中文字体: {font or '未找到系统中文字体，将回退默认字体'}")
|
||||
|
||||
key_set = bool(os.getenv("OPENAI_API_KEY"))
|
||||
or_set = bool(os.getenv("OPENROUTER_API_KEY"))
|
||||
if cfg.provider == "offline":
|
||||
print(" [OK] 供应商: offline（占位静音音轨，无需 OPENAI_API_KEY）")
|
||||
else:
|
||||
print(f" {'[OK]' if (key_set or or_set) else '[缺失]'} OPENAI_API_KEY: {'已设置' if key_set else '未设置'}"
|
||||
f" OPENROUTER_API_KEY(兜底): {'已设置' if or_set else '未设置'}"
|
||||
+ ("" if key_set else " ← 无直连 key 时讲解词走 OpenRouter、TTS 降级为离线占位"))
|
||||
print(f" [配置] provider={cfg.provider} TEXT_MODEL={cfg.text_model} "
|
||||
f"TTS_MODEL={cfg.tts_model} TTS_VOICE={cfg.tts_voice}")
|
||||
print(f" [配置] OPENAI_BASE_URL={os.getenv('OPENAI_BASE_URL') or '（官方默认）'}")
|
||||
print(f" [配置] 幻灯片页数={len(cfg.slides or SLIDES)} 输出={cfg.output}")
|
||||
|
||||
print("自检" + ("通过。" if ok else "未通过：请先安装缺失的命令行工具。"))
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 主流程
|
||||
# ---------------------------------------------------------------------------
|
||||
def main(cfg: Config) -> None:
|
||||
online = cfg.provider != "offline"
|
||||
need_llm = cfg.script is None and online # 未给脚本且非离线时才调用 LLM 生成讲解词
|
||||
|
||||
# 文本（讲解词）与 TTS 用两个客户端：OpenAI 语音接口不在 OpenRouter 上，
|
||||
# 因此 TTS 必须走直连 OPENAI_API_KEY；讲解词文本则可享受通用 OpenRouter 兜底。
|
||||
client = None # 文本/讲解词客户端
|
||||
tts_client = None # TTS 客户端（仅直连 OpenAI）
|
||||
if online:
|
||||
api_key = os.getenv("OPENAI_API_KEY")
|
||||
base_url = os.getenv("OPENAI_BASE_URL") or None
|
||||
orkey = os.getenv("OPENROUTER_API_KEY")
|
||||
if not (api_key or orkey):
|
||||
sys.exit("[错误] 未设置 OPENAI_API_KEY（或 OPENROUTER_API_KEY 兜底），请复制 env.example 为 .env 并填入；"
|
||||
"或用 --offline 在无 API 时验证合成流水线。")
|
||||
from openai import OpenAI # 延迟导入：--offline 时无需安装/联网 openai
|
||||
|
||||
# 文本客户端：无直连 key，或默认 gpt-5.x（直连需组织实名认证）时改走 OpenRouter。
|
||||
prefer_or = bool(orkey) and (cfg.text_model or "").lower().startswith("gpt-5")
|
||||
if prefer_or or (not api_key and orkey):
|
||||
client = OpenAI(api_key=orkey, base_url=OPENROUTER_BASE_URL, timeout=120.0, max_retries=3)
|
||||
cfg.text_model = map_model_to_openrouter(cfg.text_model)
|
||||
else:
|
||||
client = OpenAI(base_url=base_url, timeout=120.0, max_retries=3)
|
||||
|
||||
# TTS 客户端：只能用直连 OPENAI_API_KEY；缺失则音频降级为离线静音占位
|
||||
#（讲解词仍由文本客户端真实生成）。
|
||||
if api_key:
|
||||
tts_client = OpenAI(base_url=base_url, timeout=120.0, max_retries=3)
|
||||
else:
|
||||
print("[提示] 未配置直连 OPENAI_API_KEY，OpenAI TTS 不在 OpenRouter 上；"
|
||||
"音频改用离线静音占位（讲解词仍由 OpenRouter 真实生成）。\n")
|
||||
cfg.provider = "offline"
|
||||
|
||||
for d in (SLIDES_DIR, AUDIO_DIR, SEG_DIR):
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
all_slides = cfg.slides or SLIDES
|
||||
# --limit / --quick：只处理前 N 页，便于快速冒烟测试（减少 API 调用与耗时）。
|
||||
slides = all_slides[:cfg.limit] if cfg.limit else all_slides
|
||||
total = len(slides)
|
||||
|
||||
if cfg.script is not None and len(cfg.script) < total:
|
||||
sys.exit(f"[错误] --script 提供了 {len(cfg.script)} 段，少于要处理的 {total} 页。")
|
||||
|
||||
segments = []
|
||||
manifest = []
|
||||
|
||||
tag = f"（限 {total}/{len(all_slides)} 页）" if cfg.limit else f"（共 {total} 页）"
|
||||
mode = "离线占位" if not online else f"{cfg.provider}/{cfg.tts_model}"
|
||||
print(f"=== 论文讲解视频自动生成{tag}[{mode}] ===\n")
|
||||
|
||||
for i, slide in enumerate(slides):
|
||||
print(f"[{i + 1}/{total}] {slide['title']}")
|
||||
|
||||
# 1) 渲染幻灯片
|
||||
png = render_slide(slide, i, total)
|
||||
print(f" 幻灯片: {png.relative_to(ROOT)}")
|
||||
|
||||
# 2) 讲解词：优先用传入脚本，其次 LLM 生成，离线则用占位文本
|
||||
if cfg.script is not None:
|
||||
narration = cfg.script[i].strip()
|
||||
elif need_llm:
|
||||
narration = generate_narration(client, cfg, slide, i, total)
|
||||
else:
|
||||
narration = offline_narration(slide)
|
||||
print(f" 讲解词: {narration}")
|
||||
|
||||
# 3) TTS 合成语音（openai 真配音走直连 tts_client / offline 静音占位）
|
||||
mp3 = synthesize_speech(tts_client, cfg, narration, i)
|
||||
dur = ffprobe_duration(mp3)
|
||||
print(f" 音频: {mp3.relative_to(ROOT)} 时长 {dur:.2f}s")
|
||||
|
||||
# 4) 合成分段视频
|
||||
seg = build_segment(png, mp3, i, dur)
|
||||
segments.append(seg)
|
||||
manifest.append(
|
||||
{"page": i + 1, "narration": narration,
|
||||
"audio": str(mp3.relative_to(ROOT)), "audio_seconds": round(dur, 2)}
|
||||
)
|
||||
print()
|
||||
|
||||
# 5) 拼接为最终视频
|
||||
print("=== 拼接为最终视频 ===")
|
||||
concat_segments(segments, cfg.output)
|
||||
|
||||
audio_total = sum(m["audio_seconds"] for m in manifest)
|
||||
video_total = ffprobe_duration(cfg.output)
|
||||
|
||||
# 保存讲解词清单，便于查看
|
||||
(OUTPUT_DIR / "narration.json").write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8"
|
||||
)
|
||||
|
||||
print(f"各页音频总时长: {audio_total:.2f}s")
|
||||
print(f"最终视频时长: {video_total:.2f}s")
|
||||
try:
|
||||
shown = cfg.output.relative_to(ROOT)
|
||||
except ValueError:
|
||||
shown = cfg.output
|
||||
print(f"输出文件: {shown}")
|
||||
print("\n完成。可用以下命令查看视频元信息：")
|
||||
print(f" ffprobe -v error -show_format -show_streams {cfg.output}")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""解析命令行参数。"""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="论文讲解视频自动生成：讲解词生成 -> TTS -> ffmpeg 逐页合成。",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=(
|
||||
"示例：\n"
|
||||
" python demo.py # 生成全部 5 页的完整讲解视频（需 OPENAI_API_KEY）\n"
|
||||
" python demo.py --quick # 只跑第 1 页，快速冒烟测试\n"
|
||||
" python demo.py --limit 2 # 只跑前 2 页\n"
|
||||
" python demo.py --offline # 无需 API：占位静音音轨，验证整条 ffmpeg 流水线\n"
|
||||
" python demo.py --slides my.json # 用外部幻灯片内容替换内置示例\n"
|
||||
" python demo.py --script narr.json # 用现成讲解词脚本，跳过 LLM 生成\n"
|
||||
" python demo.py -o out/talk.mp4 # 指定最终视频输出路径\n"
|
||||
" python demo.py --check # 仅环境自检，不调用任何 API"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--limit", type=int, default=None, metavar="N",
|
||||
help="只处理前 N 页幻灯片（快速测试，显著减少 API 调用与耗时）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quick", action="store_true",
|
||||
help="快速测试：等价于 --limit 1",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--slides", type=Path, default=None, metavar="FILE",
|
||||
help="幻灯片内容 JSON 文件（[{title,subtitle,bullets}, ...]）；默认用内置示例",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--script", type=Path, default=None, metavar="FILE",
|
||||
help="现成讲解词 JSON 文件（字符串列表，每页一段）；提供后跳过 LLM 讲解词生成",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-o", "--output", type=Path, default=FINAL_MP4, metavar="FILE",
|
||||
help=f"最终讲解视频输出路径（默认 {FINAL_MP4.relative_to(ROOT)}）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tts-provider", choices=("openai", "offline"), default="openai",
|
||||
help="TTS 供应商：openai=真实配音（需 API）；offline=ffmpeg 生成占位静音音轨（无需 API）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--offline", action="store_true",
|
||||
help="完全离线：等价于 --tts-provider offline，且用要点占位讲解词（无任何 API 调用）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--text-model", default=DEFAULT_TEXT_MODEL, metavar="NAME",
|
||||
help=f"讲解词生成模型（默认 {DEFAULT_TEXT_MODEL}，或环境变量 TEXT_MODEL）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tts-model", default=DEFAULT_TTS_MODEL, metavar="NAME",
|
||||
help=f"TTS 模型（默认 {DEFAULT_TTS_MODEL}，或环境变量 TTS_MODEL）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tts-voice", default=DEFAULT_TTS_VOICE, metavar="NAME",
|
||||
help=f"TTS 音色（默认 {DEFAULT_TTS_VOICE}，可选 nova/shimmer/echo 等）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--check", action="store_true",
|
||||
help="环境自检（检查 ffmpeg/ffprobe/字体/配置）后退出，不产生任何 API 调用",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def build_config(args: argparse.Namespace) -> Config:
|
||||
"""把命令行参数组装成 Config。"""
|
||||
limit = 1 if args.quick else args.limit
|
||||
if limit is not None and limit < 1:
|
||||
sys.exit("[错误] --limit 必须为正整数。")
|
||||
|
||||
provider = "offline" if args.offline else args.tts_provider
|
||||
slides = load_slides_file(args.slides) if args.slides else None
|
||||
script = load_script_file(args.script) if args.script else None
|
||||
|
||||
return Config(
|
||||
provider=provider,
|
||||
text_model=args.text_model,
|
||||
tts_model=args.tts_model,
|
||||
tts_voice=args.tts_voice,
|
||||
limit=limit,
|
||||
output=args.output,
|
||||
slides=slides,
|
||||
script=script,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
args = parse_args()
|
||||
cfg = build_config(args)
|
||||
if args.check:
|
||||
sys.exit(self_check(cfg))
|
||||
main(cfg)
|
||||
@@ -0,0 +1,18 @@
|
||||
# 必填其一：OpenAI API Key（讲解词生成 + TTS 均可用官方 OpenAI 接口）
|
||||
OPENAI_API_KEY=your_openai_api_key_here
|
||||
|
||||
# 通用兜底：未配置 OPENAI_API_KEY 时，讲解词生成自动改走 OpenRouter；
|
||||
# 但 OpenAI TTS 不在 OpenRouter 上，故此时音频降级为离线静音占位。
|
||||
# 默认讲解词模型 gpt-5.6-luna（gpt-5.x）直连 OpenAI 需组织实名认证，
|
||||
# 故设置了本 key 时讲解词会优先走 OpenRouter（route openai/gpt-5.6-luna）。
|
||||
# OPENROUTER_API_KEY=your_openrouter_api_key_here
|
||||
|
||||
# 可选：兼容 OpenAI 协议的自定义端点（默认官方）
|
||||
# OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
|
||||
# 可选：讲解词生成模型（默认 gpt-5.6-luna）
|
||||
# TEXT_MODEL=gpt-5.6-luna
|
||||
|
||||
# 可选：TTS 模型与音色（默认 tts-1 / alloy）
|
||||
# TTS_MODEL=tts-1
|
||||
# TTS_VOICE=alloy
|
||||
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"experiment_id": "5-5",
|
||||
"protocol_version": "2.0.0",
|
||||
"source": {
|
||||
"paper": "../paper-to-ppt/validation/runs/exp5-4-real-pdf-20260730-v2/source/1706.03762.pdf",
|
||||
"slide_markdown": "../paper-to-ppt/validation/runs/exp5-4-real-pdf-20260730-v2/dual_round1_slides.md",
|
||||
"rendered_slides": "../paper-to-ppt/validation/runs/exp5-4-real-pdf-20260730-v2/rendered/dual_round1",
|
||||
"selected_pages": [1, 2, 3, 5, 8, 9, 11, 12, 15, 18, 19, 21]
|
||||
},
|
||||
"providers": {
|
||||
"narration": {
|
||||
"name": "Moonshot",
|
||||
"endpoint": "https://api.moonshot.cn/v1",
|
||||
"model": "kimi-k3"
|
||||
},
|
||||
"tts": {
|
||||
"name": "Fish Audio",
|
||||
"model": "s1",
|
||||
"authorized_reference_manifest": "../../chapter9/controllable-tts/reference_audio/manifest.json"
|
||||
},
|
||||
"independent_visual_reviewer": {
|
||||
"name": "DashScope",
|
||||
"endpoint": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
|
||||
"model": "qwen-vl-max"
|
||||
}
|
||||
},
|
||||
"acceptance": {
|
||||
"duration_seconds_min": 300,
|
||||
"duration_seconds_max": 900,
|
||||
"pages": 12,
|
||||
"per_page_av_alignment_tolerance_seconds": 0.2,
|
||||
"final_duration_tolerance_seconds": 0.75,
|
||||
"minimum_visual_alignment_score": 4,
|
||||
"maximum_generation_review_rounds": 3,
|
||||
"video_codec": "h264",
|
||||
"audio_codec": "aac"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
openai>=1.30.0
|
||||
Pillow>=10.0
|
||||
python-dotenv>=1.0
|
||||
@@ -0,0 +1,89 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import campaign
|
||||
|
||||
|
||||
def test_protocol_is_exact_twelve_page_five_to_fifteen_minute_contract():
|
||||
protocol = json.loads(campaign.PROTOCOL_PATH.read_text())
|
||||
assert len(protocol["source"]["selected_pages"]) == 12
|
||||
assert protocol["acceptance"]["duration_seconds_min"] == 300
|
||||
assert protocol["acceptance"]["duration_seconds_max"] == 900
|
||||
assert protocol["providers"]["tts"]["name"] == "Fish Audio"
|
||||
|
||||
|
||||
def test_slide_sections_map_to_rendered_pages():
|
||||
protocol = json.loads(campaign.PROTOCOL_PATH.read_text())
|
||||
markdown = (campaign.HERE / protocol["source"]["slide_markdown"]).resolve()
|
||||
sections = campaign.slide_sections(markdown.read_text())
|
||||
assert len(sections) == 22
|
||||
assert "Attention Is All You Need" in sections[0]
|
||||
assert "Long-Distance Dependencies" in sections[17]
|
||||
|
||||
|
||||
def test_parse_json_content_accepts_fenced_provider_result():
|
||||
assert campaign.parse_json_content('```json\n{"visual_alignment": 5}\n```')["visual_alignment"] == 5
|
||||
|
||||
|
||||
def test_selected_source_images_exist_and_are_distinct():
|
||||
protocol = json.loads(campaign.PROTOCOL_PATH.read_text())
|
||||
rendered = (campaign.HERE / protocol["source"]["rendered_slides"]).resolve()
|
||||
paths = [rendered / f"{page}.png" for page in protocol["source"]["selected_pages"]]
|
||||
assert all(path.stat().st_size > 10_000 for path in paths)
|
||||
assert len({campaign.sha256_file(path) for path in paths}) == 12
|
||||
|
||||
|
||||
def test_cached_json_call_retries_and_preserves_malformed_receipt(tmp_path):
|
||||
class Message:
|
||||
def __init__(self, content):
|
||||
self.content = content
|
||||
|
||||
class Choice:
|
||||
def __init__(self, content):
|
||||
self.message = Message(content)
|
||||
|
||||
class Response:
|
||||
def __init__(self, content, request_id):
|
||||
self.choices = [Choice(content)]
|
||||
self.raw = {
|
||||
"id": request_id,
|
||||
"model": "real-model",
|
||||
"usage": {"total_tokens": 4},
|
||||
"choices": [{"message": {"content": content}}],
|
||||
}
|
||||
|
||||
def model_dump(self, mode="json"):
|
||||
return self.raw
|
||||
|
||||
class Completions:
|
||||
def __init__(self):
|
||||
self.calls = 0
|
||||
|
||||
def create(self, **_kwargs):
|
||||
self.calls += 1
|
||||
if self.calls == 1:
|
||||
return Response("not json", "failed-real-call")
|
||||
return Response('{"narration":"valid"}', "accepted-real-call")
|
||||
|
||||
class Client:
|
||||
def __init__(self):
|
||||
self.chat = type("Chat", (), {"completions": Completions()})()
|
||||
|
||||
destination = tmp_path / "receipt.json"
|
||||
client = Client()
|
||||
parsed, receipt = campaign.cached_json_call(
|
||||
destination,
|
||||
client,
|
||||
provider="real-provider",
|
||||
model="real-model",
|
||||
messages=[{"role": "user", "content": "return JSON"}],
|
||||
max_tokens=20,
|
||||
)
|
||||
|
||||
assert parsed == {"narration": "valid"}
|
||||
assert receipt["json_attempt"] == 2
|
||||
assert client.chat.completions.calls == 2
|
||||
failure = json.loads(
|
||||
(tmp_path / "receipt.failed-attempts.json").read_text(encoding="utf-8")
|
||||
)
|
||||
assert failure["attempts"][0]["receipt"]["response"]["id"] == "failed-real-call"
|
||||
@@ -0,0 +1,21 @@
|
||||
"""ffprobe 输出 N/A（无时长元数据）时，ffprobe_duration 应给出清晰报错。"""
|
||||
import pytest
|
||||
|
||||
import demo
|
||||
|
||||
|
||||
def test_ffprobe_duration_na(monkeypatch):
|
||||
monkeypatch.setattr(demo, "run", lambda *a, **k: "N/A\n")
|
||||
with pytest.raises(RuntimeError, match="时长"):
|
||||
demo.ffprobe_duration("no_duration.bin")
|
||||
|
||||
|
||||
def test_ffprobe_duration_empty(monkeypatch):
|
||||
monkeypatch.setattr(demo, "run", lambda *a, **k: "")
|
||||
with pytest.raises(RuntimeError, match="时长"):
|
||||
demo.ffprobe_duration("empty.bin")
|
||||
|
||||
|
||||
def test_ffprobe_duration_normal(monkeypatch):
|
||||
monkeypatch.setattr(demo, "run", lambda *a, **k: "12.345\n")
|
||||
assert demo.ffprobe_duration("a.mp4") == 12.345
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Slide bullets containing null must still render a PNG."""
|
||||
import demo
|
||||
|
||||
|
||||
def test_render_slide_skips_null_bullet(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(demo, "SLIDES_DIR", tmp_path)
|
||||
slide = {
|
||||
"title": "Demo Title",
|
||||
"subtitle": "Demo Subtitle",
|
||||
"bullets": ["ok", None, "also"],
|
||||
}
|
||||
path = demo.render_slide(slide, 0, 1)
|
||||
assert path.exists()
|
||||
assert path.stat().st_size > 0
|
||||
assert path == tmp_path / "slide_01.png"
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"experiment_id": "5-5",
|
||||
"status": "passed",
|
||||
"run_dir": "validation/runs/exp5-5-kimi-fish-qwen-20260730-v1",
|
||||
"manifest_sha256": "93bb69a916a76d12de56270928971f6e39f47755214f7a135817d7effd8b3f09"
|
||||
}
|
||||
+5301
File diff suppressed because one or more lines are too long
+38
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"experiment_id": "5-5",
|
||||
"protocol_version": "2.0.0",
|
||||
"source": {
|
||||
"paper": "../paper-to-ppt/validation/runs/exp5-4-real-pdf-20260730-v2/source/1706.03762.pdf",
|
||||
"slide_markdown": "../paper-to-ppt/validation/runs/exp5-4-real-pdf-20260730-v2/dual_round1_slides.md",
|
||||
"rendered_slides": "../paper-to-ppt/validation/runs/exp5-4-real-pdf-20260730-v2/rendered/dual_round1",
|
||||
"selected_pages": [1, 2, 3, 5, 8, 9, 11, 12, 15, 18, 19, 21]
|
||||
},
|
||||
"providers": {
|
||||
"narration": {
|
||||
"name": "Moonshot",
|
||||
"endpoint": "https://api.moonshot.cn/v1",
|
||||
"model": "kimi-k3"
|
||||
},
|
||||
"tts": {
|
||||
"name": "Fish Audio",
|
||||
"model": "s1",
|
||||
"authorized_reference_manifest": "../../chapter9/controllable-tts/reference_audio/manifest.json"
|
||||
},
|
||||
"independent_visual_reviewer": {
|
||||
"name": "DashScope",
|
||||
"endpoint": "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
|
||||
"model": "qwen-vl-max"
|
||||
}
|
||||
},
|
||||
"acceptance": {
|
||||
"duration_seconds_min": 300,
|
||||
"duration_seconds_max": 900,
|
||||
"pages": 12,
|
||||
"per_page_av_alignment_tolerance_seconds": 0.2,
|
||||
"final_duration_tolerance_seconds": 0.75,
|
||||
"minimum_visual_alignment_score": 4,
|
||||
"maximum_generation_review_rounds": 3,
|
||||
"video_codec": "h264",
|
||||
"audio_codec": "aac"
|
||||
}
|
||||
}
|
||||
+329
@@ -0,0 +1,329 @@
|
||||
{
|
||||
"experiment_id": "5-5",
|
||||
"status": "passed",
|
||||
"acceptance": {
|
||||
"real_paper_and_experiment_5_4_slides": true,
|
||||
"twelve_selected_real_rendered_pages": true,
|
||||
"live_narration_provider_receipts": true,
|
||||
"independent_live_vision_review_receipts": true,
|
||||
"visual_narrative_alignment": true,
|
||||
"real_fish_s1_audio_each_page": true,
|
||||
"per_page_display_matches_audio": true,
|
||||
"duration_5_to_15_minutes": true,
|
||||
"final_duration_matches_page_audio": true,
|
||||
"h264_video_and_aac_audio": true,
|
||||
"raw_artifact_hashes": true,
|
||||
"passed": true
|
||||
},
|
||||
"comparison_sha256": "c33074a0f9197e3c1dae081a35886e2d2c742f6f98c779c8bec427f511a193be",
|
||||
"artifacts": {
|
||||
"audio/page-01.mp3": {
|
||||
"sha256": "e5f87f9156687b60d896dc24e59cc2408eaeaf28d795e0df9d908959ecc54fd1",
|
||||
"bytes": 645746
|
||||
},
|
||||
"audio/page-02.mp3": {
|
||||
"sha256": "2f2dee293de6e3aa07daf2e669f706fd854830dca35fdca1c31496a58697e1de",
|
||||
"bytes": 750653
|
||||
},
|
||||
"audio/page-03.mp3": {
|
||||
"sha256": "8d592cdcaad89c7c2c26cf7a4b62219ed59e4e00646ce8069b702f99d56c2958",
|
||||
"bytes": 680854
|
||||
},
|
||||
"audio/page-04.mp3": {
|
||||
"sha256": "4019a14eec80f2ecda509f29dfcf98d97c69c04af4036deec42ee0e55ffd07de",
|
||||
"bytes": 667061
|
||||
},
|
||||
"audio/page-05.mp3": {
|
||||
"sha256": "3b900913e10d8208f985509424c9aeeb8d57a659725a9d1f278c056ac88989f9",
|
||||
"bytes": 733099
|
||||
},
|
||||
"audio/page-06.mp3": {
|
||||
"sha256": "59eee15356e96e74c2953c7d4b16a142eefbb13c5c76166fb3a338ca069ec9f2",
|
||||
"bytes": 693393
|
||||
},
|
||||
"audio/page-07.mp3": {
|
||||
"sha256": "eb00364e1fbb75906a82421d6ded8d9e3fb49ec48120fd765e3fcd74f2b071df",
|
||||
"bytes": 673749
|
||||
},
|
||||
"audio/page-08.mp3": {
|
||||
"sha256": "88d9f07f22eca3ffdecef543cb4226b7d02b9e251007c63cf55ee2803c32f404",
|
||||
"bytes": 658702
|
||||
},
|
||||
"audio/page-09.mp3": {
|
||||
"sha256": "973c9d12b8c70ba55cb33e5ae0d643b0ed35596bafcc86a2a8fc523c0d7c77f0",
|
||||
"bytes": 667061
|
||||
},
|
||||
"audio/page-10.mp3": {
|
||||
"sha256": "48fc463131c0b1e78bfafa5e0cd48c5c1b252437f5e237fd80daca27113edcf5",
|
||||
"bytes": 685034
|
||||
},
|
||||
"audio/page-11.mp3": {
|
||||
"sha256": "01dca987731afc2baed6810919f5999ad9a912da4e649ee0296164f734f6a216",
|
||||
"bytes": 639894
|
||||
},
|
||||
"audio/page-12.mp3": {
|
||||
"sha256": "d5fa15bb501d23b6f90fc589357b1cbb5154721d9af0b0101704d6f0b7a5dfea",
|
||||
"bytes": 711365
|
||||
},
|
||||
"comparison.json": {
|
||||
"sha256": "c33074a0f9197e3c1dae081a35886e2d2c742f6f98c779c8bec427f511a193be",
|
||||
"bytes": 2026107
|
||||
},
|
||||
"experiment_protocol.json": {
|
||||
"sha256": "9da4f327e378625da9197413dafa43476e2d711d9baef52e6c9b243ada1dd638",
|
||||
"bytes": 1290
|
||||
},
|
||||
"lecture.mp4": {
|
||||
"sha256": "4c848f50fece7bba0d0438d5a9c97b230a43994fc07cd4a143c2644691bb6d1b",
|
||||
"bytes": 15568457
|
||||
},
|
||||
"provider_calls/page-01-fish-tts.json": {
|
||||
"sha256": "793a098d7d71ac138ad413a88a93d4aa5d08cd784f32fe0720ae6b81e26c34d6",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-01-narration-1.json": {
|
||||
"sha256": "9aa640376579d3e625ff5441fc82a37636e5842214242a7116cd7457ec76ecaa",
|
||||
"bytes": 5442
|
||||
},
|
||||
"provider_calls/page-01-vision-1.json": {
|
||||
"sha256": "549dd026f5bfd3f4136a981d42bbd7dc308f5585a382d48abd3007b79e0fb6b4",
|
||||
"bytes": 84643
|
||||
},
|
||||
"provider_calls/page-02-fish-tts.json": {
|
||||
"sha256": "6d3b33a0fb91663f6ee67b6007c7a17b1b4563e416494152fc85798507cf46ce",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-02-narration-1.json": {
|
||||
"sha256": "137099e211dfe2c84b4a636d6ec652f32731e83211a539838b90f15db8aa8a00",
|
||||
"bytes": 6834
|
||||
},
|
||||
"provider_calls/page-02-vision-1.json": {
|
||||
"sha256": "a7329122a45972afb7d1739d41063c67308bc143e3ccd61623d1cee0ab0216f1",
|
||||
"bytes": 137419
|
||||
},
|
||||
"provider_calls/page-03-fish-tts.json": {
|
||||
"sha256": "40921b99d47f3d2e0c52d50ebf85d7c56f25b2b0a51f9fdf221cf48733850a82",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-03-narration-1.json": {
|
||||
"sha256": "99d2756d4f36d2843b1ab7b455c9ca787363f035a4eddbfe5e0d561670ccab48",
|
||||
"bytes": 7978
|
||||
},
|
||||
"provider_calls/page-03-vision-1.json": {
|
||||
"sha256": "5c817ec5446d9f2ec4400b73ea9e72cb01d90ba2a78145c75711e7e2c6601f66",
|
||||
"bytes": 164520
|
||||
},
|
||||
"provider_calls/page-04-fish-tts.json": {
|
||||
"sha256": "4186eaeb5ec9252fa5f8b5909b9b59e57faf9bbda4c466c78c3e85a970f39c48",
|
||||
"bytes": 482
|
||||
},
|
||||
"provider_calls/page-04-narration-1.json": {
|
||||
"sha256": "9e8e2ff53274784a3e5187f8f3fa109d39de53821b160b84a5c1e231078a2656",
|
||||
"bytes": 8219
|
||||
},
|
||||
"provider_calls/page-04-vision-1.json": {
|
||||
"sha256": "d9eecc26adec00e63cd420f7301734608e5b1a8be316637e4496b1c68242603d",
|
||||
"bytes": 162809
|
||||
},
|
||||
"provider_calls/page-05-fish-tts.json": {
|
||||
"sha256": "9d331c8bb2769edacfcfcc1b9941254f575ef4a31dbba217de8ece9d4271d3db",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-05-narration-1.json": {
|
||||
"sha256": "77da502454d7a1ded294f586056111e1475e0980bc344482602d6c0798e7fe2d",
|
||||
"bytes": 8022
|
||||
},
|
||||
"provider_calls/page-05-vision-1.json": {
|
||||
"sha256": "b0c04505a413c12a0aed30ddd04bc0feb15d47e43765fa29859d1ed36d920217",
|
||||
"bytes": 117891
|
||||
},
|
||||
"provider_calls/page-06-fish-tts.json": {
|
||||
"sha256": "e732972d809261475e088e41588984ab490422cdeac666cfcc9ef099ebe694db",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-06-narration-1.json": {
|
||||
"sha256": "8782ef7af9b3955bee10e8c79aaa4b1509143ff19af073a3476674db931bc53f",
|
||||
"bytes": 8607
|
||||
},
|
||||
"provider_calls/page-06-vision-1.json": {
|
||||
"sha256": "96e959d3acb25f966ab59ecfe6f0df5928ee3fc0e5be2c8913ae2bb660788a7b",
|
||||
"bytes": 159793
|
||||
},
|
||||
"provider_calls/page-07-fish-tts.json": {
|
||||
"sha256": "0027c5e3148303cd4334aae0d32d37b91bdfd434fab6f6bef236fdc30a4a54b6",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-07-narration-1.json": {
|
||||
"sha256": "0e66212105ebb1eac3cb5b7f89b5d3bfa961e8752893ec1e47174c4545bfc2e8",
|
||||
"bytes": 7333
|
||||
},
|
||||
"provider_calls/page-07-vision-1.json": {
|
||||
"sha256": "6255ee8e38d85b15bd8ccedc48b9846f4d45a667fc649c0112e2fb1ceb663c30",
|
||||
"bytes": 130435
|
||||
},
|
||||
"provider_calls/page-08-fish-tts.json": {
|
||||
"sha256": "7261f318bfa0868aa21dd1039f5f50770e1ddf77bffa2aac91cc94a15f6b5148",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-08-narration-1.json": {
|
||||
"sha256": "062be3728ed42f92dcf719cc64fbb1e80da5b651aa6f9c0d3853f05d2781f71c",
|
||||
"bytes": 7658
|
||||
},
|
||||
"provider_calls/page-08-vision-1.json": {
|
||||
"sha256": "32fd7e0d5758f23ceb7c8043126e7143e3ab36b9466d459c45db123ad4b569c8",
|
||||
"bytes": 141823
|
||||
},
|
||||
"provider_calls/page-09-fish-tts.json": {
|
||||
"sha256": "2b50c5ee9012337fea926132a8cab366169b666ff7fd4b4be1db0b93c35b82d0",
|
||||
"bytes": 482
|
||||
},
|
||||
"provider_calls/page-09-narration-1.json": {
|
||||
"sha256": "b60051ec69fa7d0957a097a830165a5711fb9525317750a232f5b2907fab4cfc",
|
||||
"bytes": 7235
|
||||
},
|
||||
"provider_calls/page-09-vision-1.json": {
|
||||
"sha256": "06af33d7887773e618c66848a75633f6007d846222188d1cc9bf07b11b5dceb0",
|
||||
"bytes": 154316
|
||||
},
|
||||
"provider_calls/page-10-fish-tts.json": {
|
||||
"sha256": "2835d1e1a52512c30f121f0df51a4b14d1e013de905e209c7e81065f3cb0706d",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-10-narration-1.json": {
|
||||
"sha256": "0264333101cce471c2b5927a90c9acf59b410ab5508cc9e457473be520845d25",
|
||||
"bytes": 5632
|
||||
},
|
||||
"provider_calls/page-10-vision-1.json": {
|
||||
"sha256": "a56ae5cf060897533e94c562e51d7d7a737b1329bde61a7558b424c6ebc0d9e5",
|
||||
"bytes": 167012
|
||||
},
|
||||
"provider_calls/page-11-fish-tts.json": {
|
||||
"sha256": "6da4491170b38cd9202b9d7be4d491a07883d637cb8107f1f077da8d15cbf3e5",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-11-narration-1.json": {
|
||||
"sha256": "282b372097603dd78d80dae322021041c8e8d157b1693363f4d3d6d63e445a1d",
|
||||
"bytes": 8595
|
||||
},
|
||||
"provider_calls/page-11-vision-1.json": {
|
||||
"sha256": "2c2eaa9edae7029ce67b7d217311eedc74dbbbfe933295d15f5aa2416e144f9d",
|
||||
"bytes": 209242
|
||||
},
|
||||
"provider_calls/page-12-fish-tts.json": {
|
||||
"sha256": "883660bd4b34109aa6fbb90ae33a3d7df7457e3b162d3fde132aeedc601e8048",
|
||||
"bytes": 483
|
||||
},
|
||||
"provider_calls/page-12-narration-1.failed-attempts.json": {
|
||||
"sha256": "dfe713072cac4417abfdc0c09a5d12f51ab8533a1ea2f6a32ee8393a7d25dea0",
|
||||
"bytes": 6448
|
||||
},
|
||||
"provider_calls/page-12-narration-1.json": {
|
||||
"sha256": "8d8e67411cb71e15d97e44a03ec13137a7eb05f6d88d4319f204a3fb602fae2b",
|
||||
"bytes": 7549
|
||||
},
|
||||
"provider_calls/page-12-vision-1.json": {
|
||||
"sha256": "5f7f456a7906130c51e3c46ca04f0e33982ea91d9308f9b194b1c22bdfff6766",
|
||||
"bytes": 167789
|
||||
},
|
||||
"segments/concat.txt": {
|
||||
"sha256": "535d98d922f758bc7919856deb1b26077d3aaf9c7aa050ed29bf00ec2ea0ad87",
|
||||
"bytes": 1584
|
||||
},
|
||||
"segments/page-01.mp4": {
|
||||
"sha256": "eb7c11f79d1fffd226224accd1cb7579dd54e8dabc2f5d254899ab931618aa96",
|
||||
"bytes": 1152625
|
||||
},
|
||||
"segments/page-02.mp4": {
|
||||
"sha256": "f574d044727b2600a30922b99ccdf93db92b6704d94ce6bc161256cb7c6175e6",
|
||||
"bytes": 1469146
|
||||
},
|
||||
"segments/page-03.mp4": {
|
||||
"sha256": "e48dcc207cf4d0ae02e901bff812584c237ae255bc253dfcabbe9c1254e16f79",
|
||||
"bytes": 1323421
|
||||
},
|
||||
"segments/page-04.mp4": {
|
||||
"sha256": "d6b99ee9ae2c9c0a528a31da7eed99a9589014e578a51d72ce839bbdadebd1f9",
|
||||
"bytes": 1208322
|
||||
},
|
||||
"segments/page-05.mp4": {
|
||||
"sha256": "d19168d9a5ab82456378833f6169ef963cff29bd96951e8766ab32d9ae0d7eaf",
|
||||
"bytes": 1399238
|
||||
},
|
||||
"segments/page-06.mp4": {
|
||||
"sha256": "3b5be9a6fdd3f080684951da5f29a23bc06e98a492cc75ce4a0b8152110e3dbb",
|
||||
"bytes": 1334873
|
||||
},
|
||||
"segments/page-07.mp4": {
|
||||
"sha256": "08de38a31cf11b268ea281662726d041c73951bfb60568c2d156b947b04e8523",
|
||||
"bytes": 1297259
|
||||
},
|
||||
"segments/page-08.mp4": {
|
||||
"sha256": "3177064d1e8d0810229fa250bc3337d6a8b70f22903fc5311fd9a2c9b3eccb67",
|
||||
"bytes": 1212357
|
||||
},
|
||||
"segments/page-09.mp4": {
|
||||
"sha256": "20e6a397008b8ca34ad02b7fad8e9a593dc5119c33f4fd513bb65181ad7e8d71",
|
||||
"bytes": 1254442
|
||||
},
|
||||
"segments/page-10.mp4": {
|
||||
"sha256": "d492b036701f37caab8ca54a8e7e641c014e88ed176b7bf76e888d07992adfa9",
|
||||
"bytes": 1330115
|
||||
},
|
||||
"segments/page-11.mp4": {
|
||||
"sha256": "52e9a6aae98d0431aa99c0f4e790cdebff3b35e04acc21907c09deec0e03f5a3",
|
||||
"bytes": 1163988
|
||||
},
|
||||
"segments/page-12.mp4": {
|
||||
"sha256": "50606dc71d4d4e1bf3f841d39d5119600f31d9cf7bc44324d61cb798727c3e8a",
|
||||
"bytes": 1435339
|
||||
},
|
||||
"slides/page-01-source-01.png": {
|
||||
"sha256": "e7dcd7f6da756aac3ac9bb0b44b40e3f2b52652434974b3e8e2fb768772b9020",
|
||||
"bytes": 60526
|
||||
},
|
||||
"slides/page-02-source-02.png": {
|
||||
"sha256": "639e08c735d7801c9e3d7be2fcbbd15aa5c03e235d9fb3c6034467e70d92d4e1",
|
||||
"bytes": 99307
|
||||
},
|
||||
"slides/page-03-source-03.png": {
|
||||
"sha256": "45a309955a2306c29f36b54428ba283d00abb5e7bc72ef7359361026d9f3cbfb",
|
||||
"bytes": 119824
|
||||
},
|
||||
"slides/page-04-source-05.png": {
|
||||
"sha256": "4b9275d26c1eb17e3fde09cc81e9f688531d9cef99dfa87f5a264c9b436bafd3",
|
||||
"bytes": 119214
|
||||
},
|
||||
"slides/page-05-source-08.png": {
|
||||
"sha256": "c826a0b0a2f25030163dc03837e391476ca177c44d20675bb767a1117d798734",
|
||||
"bytes": 85065
|
||||
},
|
||||
"slides/page-06-source-09.png": {
|
||||
"sha256": "b7fd553eb05aebaa917d6c9ccc2b1b34e96eacb0c0b8863b10d9bd36ca81b277",
|
||||
"bytes": 115780
|
||||
},
|
||||
"slides/page-07-source-11.png": {
|
||||
"sha256": "2d1fcdc463639bb6be40f1a294c1a7b20f0342b1305cc5ba73df2973bbc1c2ea",
|
||||
"bytes": 94452
|
||||
},
|
||||
"slides/page-08-source-12.png": {
|
||||
"sha256": "0a964cf412060841646cbd8577b99cac4240aedc65c0b8582fc72daf992bf87d",
|
||||
"bytes": 102746
|
||||
},
|
||||
"slides/page-09-source-15.png": {
|
||||
"sha256": "f8f3bc2e71ee9ca33fba11049a6131f232bb9ae2e343f561e5a544ae9597e1bf",
|
||||
"bytes": 112413
|
||||
},
|
||||
"slides/page-10-source-18.png": {
|
||||
"sha256": "046876dd92a84598b8ce70d6257a151a67eec2315c2b64f1806dad67a8e4efad",
|
||||
"bytes": 122064
|
||||
},
|
||||
"slides/page-11-source-19.png": {
|
||||
"sha256": "f21b0c1be0490313f88faa122f6ca50655f5b08cae20d3404d6a2654b55c8f06",
|
||||
"bytes": 153599
|
||||
},
|
||||
"slides/page-12-source-21.png": {
|
||||
"sha256": "fa36349f67cfc69e84821957bc7f3ca3b1bc66e3f25a540bf2ee85d0296c4b70",
|
||||
"bytes": 121947
|
||||
}
|
||||
}
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "ab1a694dada2c304b620004f99b88c321e55d33efd58d60d8a9489eac52cb78f",
|
||||
"text_characters": 265,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-01.mp3",
|
||||
"sha256": "e5f87f9156687b60d896dc24e59cc2408eaeaf28d795e0df9d908959ecc54fd1",
|
||||
"bytes": 645746
|
||||
},
|
||||
"elapsed_ms": 6141.627
|
||||
}
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
{
|
||||
"signature": "889389c29f1a515a166cd4957ebf29c9a103c1d740d3d66dc5279d045cd6a7c1",
|
||||
"parsed": {
|
||||
"narration": "大家好，我们从第一页开始。屏幕上最显眼的是这篇论文的标题——Attention Is All You Need，下面副标题写着“一种用于序列转导的革命性架构”，作者是 Ashish Vaswani 等人，发表于 2017 年的 NIPS 会议。请大家先记住这个标题，因为它本身就是论文的核心主张：只用注意力机制，就足以完成序列到序列的建模，不再需要循环或卷积。为什么这么说、又是怎么做到的？这正是我们接下来十一页要逐步拆解的：先看它要解决什么问题，再看整体架构与关键公式，最后看实验结果是否撑得起这个大胆宣言。现在我们从背景出发。",
|
||||
"visual_references": [
|
||||
"主标题 Attention Is All You Need",
|
||||
"副标题 A Revolutionary Architecture for Sequence Transduction",
|
||||
"作者 Ashish Vaswani et al. 与 NIPS 2017 字样"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 1/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n# Attention Is All You Need\n## A Revolutionary Architecture for Sequence Transduction\n\nAshish Vaswani et al. \nNIPS 2017\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7ebf33277b847199b22e",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"大家好，我们从第一页开始。屏幕上最显眼的是这篇论文的标题——Attention Is All You Need，下面副标题写着“一种用于序列转导的革命性架构”，作者是 Ashish Vaswani 等人，发表于 2017 年的 NIPS 会议。请大家先记住这个标题，因为它本身就是论文的核心主张：只用注意力机制，就足以完成序列到序列的建模，不再需要循环或卷积。为什么这么说、又是怎么做到的？这正是我们接下来十一页要逐步拆解的：先看它要解决什么问题，再看整体架构与关键公式，最后看实验结果是否撑得起这个大胆宣言。现在我们从背景出发。\",\"visual_references\":[\"主标题 Attention Is All You Need\",\"副标题 A Revolutionary Architecture for Sequence Transduction\",\"作者 Ashish Vaswani et al. 与 NIPS 2017 字样\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants a Chinese narration for slide 1/12 of \"Attention Is All You Need\" presentation. This is the title slide. Need 180-240 Chinese characters, oral style, guiding narrative explaining \"why\" and relation to subsequent pages, pointing to visible elements (title, subtitle, authors, NIPS 2017). No fabricated content. Return JSON only.\n\nLet me draft:\n\n\"大家好，我们从这一页开始。屏幕上显示的是今天的主角——论文标题 Attention Is All You Need，副标题写着‘一种用于序列转导的革命性架构’，作者是以 Ashish Vaswani 为首的团队，发表于 2017 年的 NIPS 会议。为什么这篇论文值得我们逐页细读？因为在它之前，序列建模几乎离不开循环和卷积结构，而这个标题本身就在宣告：只靠注意力机制，就够了。接下来的十几页里，我们会先看它要解决什么问题，再拆解模型的整体结构与核心公式，最后看实验结果如何支撑这个大胆的主张。让我们从背景开始。\"\n\nCount characters roughly. Let me count: need 180-240 Chinese characters. The draft seems around 210-230. Good.\n\nVisual references: [\"标题 Attention Is All You Need\", \"副标题 A Revolutionary Architecture for Sequence Transduction\", \"作者 Ashish Vaswani et al. 与 NIPS 2017 字样\"]\n\nReturn JSON only."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364159,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 475,
|
||||
"prompt_tokens": 326,
|
||||
"total_tokens": 801,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 282,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 16795.873
|
||||
}
|
||||
}
|
||||
+87
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "ebb0ffa7d987500273d5a7ba34ad33aefc38fa0165500d2327f7364c8ce03dbe",
|
||||
"text_characters": 250,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-02.mp3",
|
||||
"sha256": "2f2dee293de6e3aa07daf2e669f706fd854830dca35fdca1c31496a58697e1de",
|
||||
"bytes": 750653
|
||||
},
|
||||
"elapsed_ms": 7916.461
|
||||
}
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"signature": "97dc260e58ab3e2bd1fe44bdd97865cb900b51fa78ca03a0d6fd5ad8b88bc7ec",
|
||||
"parsed": {
|
||||
"narration": "上一页我们看了论文标题，现在这一页是摘要，相当于整篇论文的电梯演讲。请大家先看屏幕上加粗的 Transformer 这个词——作者提出这是第一个完全建立在注意力机制上的模型，彻底抛弃了循环和卷积结构。为什么这一点关键？因为没有了循环，训练就可以高度并行，耗时大幅下降。再看屏幕下方两个数字：英德翻译 28.4 BLEU，比此前最好成绩高出 2 个点；英法 41.8 BLEU，直接刷新纪录。最后一行还提到，它能泛化到英语句法分析等其他任务。那么注意力机制究竟是如何运作的？我们下一页就从引言开始拆解。",
|
||||
"visual_references": [
|
||||
"页面标题 Abstract",
|
||||
"加粗的 Transformer 一词",
|
||||
"数值 28.4 BLEU（英德）与 41.8 BLEU（英法）",
|
||||
"最后一条关于 English constituency parsing 的泛化结论"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 2/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Abstract\n\n- Proposes **Transformer** - first model based solely on attention mechanisms\n- Dispenses with recurrence and convolutions entirely\n- Superior quality while being more parallelizable and requiring less training time\n- Achieves 28.4 BLEU on WMT 2014 English-to-German (↑2 BLEU over previous best)\n- Achieves 41.8 BLEU on WMT 2014 English-to-French (new state-of-the-art)\n- Generalizes well to other tasks like English constituency parsing\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6acef17935e4e9b19f0c12",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"上一页我们看了论文标题，现在这一页是摘要，相当于整篇论文的电梯演讲。请大家先看屏幕上加粗的 Transformer 这个词——作者提出这是第一个完全建立在注意力机制上的模型，彻底抛弃了循环和卷积结构。为什么这一点关键？因为没有了循环，训练就可以高度并行，耗时大幅下降。再看屏幕下方两个数字：英德翻译 28.4 BLEU，比此前最好成绩高出 2 个点；英法 41.8 BLEU，直接刷新纪录。最后一行还提到，它能泛化到英语句法分析等其他任务。那么注意力机制究竟是如何运作的？我们下一页就从引言开始拆解。\",\"visual_references\":[\"页面标题 Abstract\",\"加粗的 Transformer 一词\",\"数值 28.4 BLEU（英德）与 41.8 BLEU（英法）\",\"最后一条关于 English constituency parsing 的泛化结论\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants a Chinese narration for slide 2/12 of \"Attention Is All You Need\" — the Abstract slide. Requirements: 180-240 Chinese characters, guiding narrative, point to visible elements, no fabricated numbers, JSON only.\n\nThe slide shows: Abstract with points about Transformer, attention only, no recurrence/convolutions, 28.4 BLEU EN-DE, 41.8 BLEU EN-FR, generalization to parsing.\n\nNeed to connect with previous page (title page presumably) and next page (introduction). Start should match page 2 (following title page). End could lead to page 3.\n\nWrite ~200 characters. Visual references: title \"Abstract\", \"Transformer\" bold text, numbers 28.4 BLEU, 41.8 BLEU.\n\nDraft:\n\n\"上一页我们看了论文标题，现在这一页是摘要——也就是整篇论文的'电梯演讲'。请大家看屏幕上第二条：作者彻底抛弃了循环和卷积结构，只靠注意力机制构建了整个模型，这就是加粗的'Transformer'这个词的由来。为什么这很重要？因为去掉循环意味着训练可以高度并行，速度大幅提升。再看下面两个数字：英德翻译28.4 BLEU，比此前最好成绩高2个点；英法41.8 BLEU，直接刷新纪录。最后一行还提到它能泛化到句法分析等任务。那么注意力机制具体长什么样？我们下一页进入引言部分。\"\n\nCount characters roughly: that's around 200. Let me count more carefully.\n\n\"上一页我们看了论文标题，现在这一页是摘要——也就是整篇论文的'电梯演讲'。\" ~30\n\"请大家看屏幕上第二条：作者彻底抛弃了循环和卷积结构，只靠注意力机制构建了整个模型，这就是加粗的'Transformer'这个词的由来。\" ~55\n\"为什么这很重要？因为去掉循环意味着训练可以高度并行，速度大幅提升。\" ~30\n\"再看下面两个数字：英德翻译28.4 BLEU，比此前最好成绩高2个点；英法41.8 BLEU，直接刷新纪录。\" ~42\n\"最后一行还提到它能泛化到句法分析等任务。\" ~20\n\"那么注意力机制具体长什么样？我们下一页进入引言部分。\" ~25\n\nTotal ~200 characters. Good, within 180-240.\n\nVisual references: [\"标题 Abstract\", \"加粗的 Transformer 一词\", \"28.4 BLEU 和 41.8 BLEU 两个数值\", \"最后一条关于 English constituency parsing 的泛化结论\"]\n\nReturn JSON only."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785384691,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 733,
|
||||
"prompt_tokens": 402,
|
||||
"total_tokens": 1135,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 528,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 26401.716
|
||||
}
|
||||
}
|
||||
+93
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "7a0e6ee892610a65fc570710cf1df3de27f92c92648222478578424175bb9148",
|
||||
"text_characters": 230,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-03.mp3",
|
||||
"sha256": "8d592cdcaad89c7c2c26cf7a4b62219ed59e4e00646ce8069b702f99d56c2958",
|
||||
"bytes": 680854
|
||||
},
|
||||
"elapsed_ms": 6671.605
|
||||
}
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"signature": "40d919072277dbcf0a8bca406c2c16bfc0e73f4f20db060bf993b2d923f77b7c",
|
||||
"parsed": {
|
||||
"narration": "承接上一页对论文的整体介绍，我们先看屏幕上的标题：现有方法到底卡在哪里。上面一栏是RNN家族，包括LSTM和GRU，它们天生要一个词一个词地顺序计算，训练时没法并行，距离很远的词之间也难以建立联系。于是有人转向卷积，也就是下面提到的ByteNet和ConvS2S，并行问题是缓解了，但请注意最后一行：两个位置离得越远，所需操作就越多，ConvS2S是线性增长，ByteNet是对数增长。这两个瓶颈正是作者寻找新方案的动机，下一页我们就看注意力机制如何绕开它们。",
|
||||
"visual_references": [
|
||||
"页面标题“Background: The Problem with Existing Approaches”",
|
||||
"“Recurrent Neural Networks (RNNs/LSTMs/GRUs)”一栏",
|
||||
"“ByteNet, ConvS2S”字样",
|
||||
"“Linear (ConvS2S) or logarithmic (ByteNet) path lengths”一行"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 3/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Background: The Problem with Existing Approaches\n\n### Recurrent Neural Networks (RNNs/LSTMs/GRUs)\n- Inherently sequential computation\n- Cannot parallelize within training examples\n- Difficult to learn long-range dependencies\n\n### Convolutional Approaches\n- ByteNet, ConvS2S use CNNs for parallelization\n- Number of operations grows with distance between positions\n- Linear (ConvS2S) or logarithmic (ByteNet) path lengths\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7edd32ee9d9a48768e53",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"承接上一页对论文的整体介绍，我们先看屏幕上的标题：现有方法到底卡在哪里。上面一栏是RNN家族，包括LSTM和GRU，它们天生要一个词一个词地顺序计算，训练时没法并行，距离很远的词之间也难以建立联系。于是有人转向卷积，也就是下面提到的ByteNet和ConvS2S，并行问题是缓解了，但请注意最后一行：两个位置离得越远，所需操作就越多，ConvS2S是线性增长，ByteNet是对数增长。这两个瓶颈正是作者寻找新方案的动机，下一页我们就看注意力机制如何绕开它们。\",\"visual_references\":[\"页面标题“Background: The Problem with Existing Approaches”\",\"“Recurrent Neural Networks (RNNs/LSTMs/GRUs)”一栏\",\"“ByteNet, ConvS2S”字样\",\"“Linear (ConvS2S) or logarithmic (ByteNet) path lengths”一行\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants a spoken narration for slide 3/12 of \"Attention Is All You Need\" presentation. The slide covers Background: The Problem with Existing Approaches — RNNs (sequential, no parallelization, long-range dependency issues) and Convolutional approaches (ByteNet, ConvS2S, path lengths linear/logarithmic).\n\nRequirements:\n- 180–240 Chinese characters\n- Guiding narrative explaining \"why\" and relation to previous/next pages\n- Point to visible elements on screen\n- No fabricated numbers\n- Match page order (this is page 3, so previous page likely introduced the paper/intro, next pages will introduce the Transformer solution)\n- Return only JSON with narration and visual_references\n\nLet me write natural, spoken Chinese, around 180-240 characters.\n\nDraft:\n\"上一页我们了解了这篇论文的基本背景，现在请看屏幕上的标题——现有方法的问题。为什么需要新架构呢？先看RNN家族，包括LSTM和GRU：它们本质上必须按顺序计算，一个词一个词地处理，训练时无法并行，而且距离远的词之间信息传递困难。那卷积方法呢？比如ByteNet和ConvS2S，它们确实实现了并行，但大家注意幻灯片上这一行：位置之间的距离越远，需要的操作就越多——ConvS2S是线性增长，ByteNet是对数增长。正是这两个痛点，引出了下一页要讲的、完全基于注意力的Transformer方案。\"\n\nCount characters: Let me roughly count. That's probably around 200-220 characters. Good.\n\nVisual references: the title \"Background: The Problem with Existing Approaches\", RNN section, ConvS2S/ByteNet names, linear/logarithmic path lengths line.\n\nLet me refine and count more carefully.\n\nNarration draft:\n\"承接上一页对论文的整体介绍，我们先看屏幕上的标题：现有方法到底卡在哪里。左边是RNN家族，包括LSTM和GRU，它们天生要一个词一个词地顺序计算，训练时没法并行，长距离的词之间也很难建立联系。于是有人转向卷积，也就是下面提到的ByteNet和ConvS2S，并行是解决了，但请注意这行字：两个位置离得越远，所需操作就越多，ConvS2S是线性增长，ByteNet是对数增长。这两个瓶颈，正是作者提出全新方案的动机，下一页我们就看Transformer如何绕开它们。\"\n\nRough count: Let me count in segments:\n\"承接上一页对论文的整体介绍，\" = 14\n\"我们先看屏幕上的标题：\" = 11 (25)\n\"现有方法到底卡在哪里。\" = 11 (36)\n\"左边是RNN家族，\" = ~9 (45)\n\"包括LSTM和GRU，\" ~10 (55)\n\"它们天生要一个词一个词地顺序计算，\" = 17 (72)\n\"训练时没法并行，\" = 8 (80)\n\"长距离的词之间也很难建立联系。\" = 14 (94)\n\"于是有人转向卷积，\" = 9 (103)\n\"也就是下面提到的ByteNet和ConvS2S，\" ~16 (119)\n\"并行是解决了，\" = 7 (126)\n\"但请注意这行字：\" = 8 (134)\n\"两个位置离得越远，\" = 9 (143)\n\"所需操作就越多，\" = 8 (151)\n\"ConvS2S是线性增长，\" ~13 (164)\n\"ByteNet是对数增长。\" ~11 (175)\n\"这两个瓶颈，\" = 6 (181)\n\"正是作者提出全新方案的动机，\" = 13 (194)\n\"下一页我们就看Transformer如何绕开它们。\" ~21 (215)\n\nAround 215 characters. Good, within 180-240.\n\nVisual references: \n- \"标题 'Background: The Problem with Existing Approaches'\"\n- \"RNNs/LSTMs/GRUs 一栏\"\n- \"ConvS2S 与 ByteNet 名称\"\n- \"'Linear (ConvS2S) or logarithmic (ByteNet) path lengths' 一行\"\n\nReturn only JSON."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364190,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 1065,
|
||||
"prompt_tokens": 394,
|
||||
"total_tokens": 1459,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 850,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 31874.737
|
||||
}
|
||||
}
|
||||
+92
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "4e8e7cb817fdcb2980d38da80dd87b4fbd460f9be9c78b97a8cf69315310b447",
|
||||
"text_characters": 225,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-04.mp3",
|
||||
"sha256": "4019a14eec80f2ecda509f29dfcf98d97c69c04af4036deec42ee0e55ffd07de",
|
||||
"bytes": 667061
|
||||
},
|
||||
"elapsed_ms": 8550.04
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"signature": "1c209ba063c562c6da617d2c695f49c7a89e1a5db96ccdfa7955e26f12f15ac4",
|
||||
"parsed": {
|
||||
"narration": "前面我们说过，作者想完全抛弃循环结构，那到底用什么来替代呢？答案就是现在屏幕上的这张图——论文的图1，Transformer的整体架构。请大家先看图的左半边，这是编码器，由多层相同模块堆叠而成，每层包含自注意力和前馈网络两部分；再看右半边，是结构类似、但多了注意力子层的解码器。注意图的最下方，输入在进入网络前要先加上位置编码，因为模型本身没有顺序概念；最上方则是逐步生成输出的部分。接下来几页，我们就从左到右、从下到上，把这张图的每个部件拆开细讲。",
|
||||
"visual_references": [
|
||||
"论文图1：Transformer整体架构图",
|
||||
"图左半边的编码器堆叠（标注Nx）",
|
||||
"图右半边的解码器堆叠",
|
||||
"图底部的输入嵌入与位置编码",
|
||||
"图顶部的输出概率部分"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 4/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Transformer Architecture Overview\n\n<img src=\"/paper_figure_1_transformer.png\" class=\"h-80 mx-auto\" />\n\n*Encoder-decoder structure with stacked self-attention and feed-forward layers*\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6acf2025f3623a799887ee",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"前面我们说过，作者想完全抛弃循环结构，那到底用什么来替代呢？答案就是现在屏幕上的这张图——论文的图1，Transformer的整体架构。请大家先看图的左半边，这是编码器，由多层相同模块堆叠而成，每层包含自注意力和前馈网络两部分；再看右半边，是结构类似、但多了注意力子层的解码器。注意图的最下方，输入在进入网络前要先加上位置编码，因为模型本身没有顺序概念；最上方则是逐步生成输出的部分。接下来几页，我们就从左到右、从下到上，把这张图的每个部件拆开细讲。\",\"visual_references\":[\"论文图1：Transformer整体架构图\",\"图左半边的编码器堆叠（标注Nx）\",\"图右半边的解码器堆叠\",\"图底部的输入嵌入与位置编码\",\"图顶部的输出概率部分\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants a narration for slide 4/12 of \"Attention Is All You Need\" - the Transformer architecture overview slide with the famous Figure 1 image.\n\nRequirements:\n- 180-240 Chinese characters\n- Guiding narrative explaining \"why\" and relationship to previous/next pages\n- Point to visible elements on screen (the image)\n- No fabricated numbers/conclusions\n- Match page order at start/end\n- Return only JSON with \"narration\" and \"visual_references\"\n\nPrevious pages presumably covered background (RNN limitations, attention idea), so this page presents the full architecture. Next pages likely dive into details: attention mechanism, multi-head attention, etc.\n\nI should write natural spoken Chinese, pointing to the image: left side encoder stack, right side decoder stack, Nx repeated layers, multi-head attention blocks, feed-forward, positional encoding at bottom, output probabilities at top.\n\nBut careful not to fabricate things not in source — the source only has title \"Transformer Architecture Overview\", image paper_figure_1_transformer.png, and caption \"Encoder-decoder structure with stacked self-attention and feed-forward layers\". The image is the famous Figure 1 from the paper, so describing what's visible in that figure (encoder on left, decoder on right, Nx stacking, multi-head attention, feed-forward, positional encoding) is fair since it's the visible image. That's fine as visual reference — describing the well-known figure.\n\nCount characters ~180-240 Chinese characters. Let me draft.\n\nDraft:\n\"前面我们说过，论文提出要完全抛弃循环结构，那到底用什么来替代呢？答案就是屏幕上这张图——论文的图1，Transformer的整体架构。请大家先看图的左半边，这是编码器，由若干层相同的模块堆叠而成，每层包含自注意力和前馈网络两部分；再看右半边，是结构类似但多了一个注意力子层的解码器。注意图中最下方，输入在进入网络前要先加上位置编码，因为模型本身没有顺序概念。最上方则是解码器逐步生成输出的部分。接下来几页，我们就按从左到右、从下到上的顺序，把这张图的每个部件拆开细讲。\"\n\nCount: let me estimate. That's roughly 220 characters. Let me count more carefully.\n\n\"前面我们说过，论文提出要完全抛弃循环结构，那到底用什么来替代呢？\" = ~33\n\"答案就是屏幕上这张图——论文的图1，Transformer的整体架构。\" = ~29\n\"请大家先看图的左半边，这是编码器，由若干层相同的模块堆叠而成，每层包含自注意力和前馈网络两部分；\" = ~48\n\"再看右半边，是结构类似但多了一个注意力子层的解码器。\" = ~26\n\"注意图中最下方，输入在进入网络前要先加上位置编码，因为模型本身没有顺序概念。\" = ~38\n\"最上方则是解码器逐步生成输出的部分。\" = ~18\n\"接下来几页，我们就按从左到右、从下到上的顺序，把这张图的每个部件拆开细讲。\" = ~37\n\nTotal ≈ 229. Good, within range.\n\nHmm, \"多了一个注意力子层\" — decoder has masked multi-head attention plus encoder-decoder attention. Saying \"多了一个注意力子层\" is accurate-ish; decoder has two attention sublayers vs encoder's one. Fine. Actually to be safe: \"解码器比编码器多出一层与编码器交互的注意力\" — that's describing the figure accurately (encoder-decoder attention). Also masked attention at bottom. Let me keep it simpler: \"结构类似但多了注意力子层的解码器\".\n\nVisual references: [\"论文图1：Transformer整体架构图\", \"图左侧编码器堆叠（Nx）\", \"图右侧解码器堆叠\", \"图底部输入嵌入与位置编码\", \"图顶部输出概率\"]. These are all visible in the famous figure. The Nx symbol is in the figure. OK.\n\nStart/end matching page order: start references previous discussion, end previews next pages. Good.\n\nReturn only JSON."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785384739,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 1028,
|
||||
"prompt_tokens": 336,
|
||||
"total_tokens": 1364,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 827,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 39888.07
|
||||
}
|
||||
}
|
||||
+90
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "a89790709213d8ae52f50455611267527eeef9dee0e9cef2baee6072675f968f",
|
||||
"text_characters": 297,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-05.mp3",
|
||||
"sha256": "3b900913e10d8208f985509424c9aeeb8d57a659725a9d1f278c056ac88989f9",
|
||||
"bytes": 733099
|
||||
},
|
||||
"elapsed_ms": 6887.651
|
||||
}
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
{
|
||||
"signature": "d4f8155173520ed8b869040026d6118529b486dbaaf1ee52484997c03b51edb9",
|
||||
"parsed": {
|
||||
"narration": "现在来到整篇论文最核心的部分。前面我们说过，Transformer 抛弃了循环和卷积，那它究竟靠什么建模序列之间的关系？答案就是屏幕中央这个公式：Scaled Dot-Product Attention。请大家看这里：Q、K、V 是三个矩阵，先用 Q 乘 K 的转置，算出各位置之间的相似度；再除以根号 d_k 这个缩放因子。为什么要缩放？因为 d_k 较大时点积的数值会很大，softmax 会进入梯度极小的区域，训练就困难了，所以用它把数值压回合理范围。最后再乘上 V，得到加权求和的输出。相比加性注意力，它只做矩阵乘法，更快也更省空间。理解了这个公式，下一页的多头注意力就是它的并行扩展。",
|
||||
"visual_references": [
|
||||
"标题 Scaled Dot-Product Attention",
|
||||
"核心公式 softmax(QK^T/√d_k)V",
|
||||
"公式中的缩放因子 1/√d_k"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 5/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Attention Mechanism\n\n### Scaled Dot-Product Attention\n$$\\text{Attention}(Q, K, V) = \\text{softmax}\\left(\\frac{QK^T}{\\sqrt{d_k}}\\right)V$$\n\n- $Q$ (queries), $K$ (keys), $V$ (values) are matrices\n- Scaling by $\\frac{1}{\\sqrt{d_k}}$ prevents gradients from becoming too small\n- Faster and more space-efficient than additive attention\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7f0e567e9c2404b6b5e9",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"现在来到整篇论文最核心的部分。前面我们说过，Transformer 抛弃了循环和卷积，那它究竟靠什么建模序列之间的关系？答案就是屏幕中央这个公式：Scaled Dot-Product Attention。请大家看这里：Q、K、V 是三个矩阵，先用 Q 乘 K 的转置，算出各位置之间的相似度；再除以根号 d_k 这个缩放因子。为什么要缩放？因为 d_k 较大时点积的数值会很大，softmax 会进入梯度极小的区域，训练就困难了，所以用它把数值压回合理范围。最后再乘上 V，得到加权求和的输出。相比加性注意力，它只做矩阵乘法，更快也更省空间。理解了这个公式，下一页的多头注意力就是它的并行扩展。\",\"visual_references\":[\"标题 Scaled Dot-Product Attention\",\"核心公式 softmax(QK^T/√d_k)V\",\"公式中的缩放因子 1/√d_k\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants me to write a narration for slide 5/12 of \"Attention Is All You Need\" paper explanation. The slide is about Scaled Dot-Product Attention.\n\nRequirements:\n1. 180-240 Chinese characters\n2. Guided narrative explaining \"why\" and relationship to previous/next pages\n3. Point to visible elements on screen (title, formula)\n4. No fabricated numbers/conclusions\n5. Match page order (5/12, so we're in the middle - previous pages likely introduced attention concept, this is the core mechanism)\n6. Return only JSON with narration and visual_references\n\nLet me draft the narration:\n\nWe're at page 5 of 12. Previous pages likely covered the Transformer architecture overview or motivation. This page introduces the core attention mechanism. Next pages likely cover multi-head attention.\n\nDraft:\n\"现在来到整篇论文最核心的一页——注意力机制本身。前面我们说了 Transformer 完全抛弃循环和卷积，那它靠什么建模序列关系？答案就是屏幕中央这个公式：Scaled Dot-Product Attention。请大家看这个公式：Q、K、V 三个矩阵，先把查询 Q 和键 K 的转置做矩阵乘法，得到每对位置之间的相似度；再除以一个缩放因子根号 d_k。为什么要缩放？因为维度 d_k 较大时，点积数值会很大，softmax 会进入梯度极小的区域，训练就困难了，所以用根号 d_k 把数值压回合理范围。最后乘上 V，得到加权的值。相比加性注意力，它只需矩阵乘法，更快也更省空间。掌握这个公式，下一页的多头注意力就是它的并行扩展。\"\n\nLet me count characters... This should be around 220-240. Let me check.\n\nLet me count roughly:\n\"现在来到整篇论文最核心的一页——注意力机制本身。\" ~23\n\"前面我们说了 Transformer 完全抛弃循环和卷积，那它靠什么建模序列关系？答案就是屏幕中央这个公式：Scaled Dot-Product Attention。\" ~55\n\"请大家看这个公式：Q、K、V 三个矩阵，先把查询 Q 和键 K 的转置做矩阵乘法，得到每对位置之间的相似度；\" ~48\n\"再除以一个缩放因子根号 d_k。为什么要缩放？因为维度 d_k 较大时，点积数值会很大，softmax 会进入梯度极小的区域，训练就困难了，所以用根号 d_k 把数值压回合理范围。\" ~73\n\"最后乘上 V，得到加权的值。相比加性注意力，它只需矩阵乘法，更快也更省空间。掌握这个公式，下一页的多头注意力就是它的并行扩展。\" ~57\n\nTotal roughly 256, slightly over. Let me trim.\n\nRevised:\n\"现在来到整篇论文最核心的部分。前面说 Transformer 抛弃循环和卷积，那它靠什么建模序列关系？答案就是屏幕上这个公式：Scaled Dot-Product Attention。请大家看公式：Q、K、V 三个矩阵，先用 Q 乘 K 的转置，算出各位置间的相似度；再除以根号 d_k 这个缩放因子。为什么要缩放？因为 d_k 较大时点积数值会很大，softmax 会进入梯度极小的区域，让训练变难，所以用它把数值压回合理范围。最后再乘 V，得到加权求和的结果。相比加性注意力，它只做矩阵乘法，更快也更省空间。理解了这个公式，下一页的多头注意力就是它的并行扩展。\"\n\nCount: approximately 230 characters. Good.\n\nVisual references: 公式 Attention(Q,K,V) = softmax(QK^T/√d_k)V, 缩放因子 1/√d_k, 标题 \"Scaled Dot-Product Attention\"\n\nJSON format:\n{\"narration\":\"...\",\"visual_references\":[\"标题 Scaled Dot-Product Attention\",\"核心公式 softmax(QK^T/√d_k)V\",\"缩放因子 1/√d_k\"]}"
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364239,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 1055,
|
||||
"prompt_tokens": 392,
|
||||
"total_tokens": 1447,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 835,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 32866.439
|
||||
}
|
||||
}
|
||||
+89
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "b59e996561e12a91147ef542f2d0c20af8e036631fa644d7863e1123620793be",
|
||||
"text_characters": 305,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-06.mp3",
|
||||
"sha256": "59eee15356e96e74c2953c7d4b16a142eefbb13c5c76166fb3a338ca069ec9f2",
|
||||
"bytes": 693393
|
||||
},
|
||||
"elapsed_ms": 6436.549
|
||||
}
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"signature": "944b1cfa89ea65233949079cdf07e0aafd87ff28e3f544430030c62be3757601",
|
||||
"parsed": {
|
||||
"narration": "上一页我们学会了单个注意力怎么算，可只从一个角度看句子，难免片面。所以这一页的标题 Multi-Head Attention，就是把它升级成多头。请看屏幕下方的公式：我们不再只算一次，而是把 Q、K、V 分别乘上各自学出的投影矩阵，投到不同子空间，并行地算出 h 个 head，每个 head 正是上页那个 Attention；然后按公式里的 Concat 把它们拼接，再乘输出矩阵 W 的 O 次方那个 W^O 做融合。上方这张 Transformer 架构图里反复出现的 Multi-Head Attention 方块，说的就是这个模块。这样模型能同时从多种表示角度捕捉依赖，为后面堆叠成完整网络做好准备。",
|
||||
"visual_references": [
|
||||
"标题 Multi-Head Attention",
|
||||
"上方 Transformer 架构图（paper_figure_1_transformer.png）中的 Multi-Head Attention 模块",
|
||||
"公式 MultiHead(Q,K,V)=Concat(head_1,...,head_h)W^O",
|
||||
"公式 head_i=Attention(QW_i^Q, KW_i^K, VW_i^V)"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 6/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Multi-Head Attention\n\n<img src=\"/paper_figure_1_transformer.png\" class=\"h-40 mx-auto\" />\n\n- Projects queries, keys, values $h$ times with different learned projections\n- Performs attention in parallel on projected versions\n- Concatenates results and projects again\n\n$$\\text{MultiHead}(Q, K, V) = \\text{Concat}(\\text{head}_1, ..., \\text{head}_h)W^O$$\nwhere $\\text{head}_i = \\text{Attention}(QW_i^Q, KW_i^K, VW_i^V)$\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7f116dbb98be644ca4d6",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"上一页我们学会了单个注意力怎么算，可只从一个角度看句子，难免片面。所以这一页的标题 Multi-Head Attention，就是把它升级成多头。请看屏幕下方的公式：我们不再只算一次，而是把 Q、K、V 分别乘上各自学出的投影矩阵，投到不同子空间，并行地算出 h 个 head，每个 head 正是上页那个 Attention；然后按公式里的 Concat 把它们拼接，再乘输出矩阵 W 的 O 次方那个 W^O 做融合。上方这张 Transformer 架构图里反复出现的 Multi-Head Attention 方块，说的就是这个模块。这样模型能同时从多种表示角度捕捉依赖，为后面堆叠成完整网络做好准备。\",\"visual_references\":[\"标题 Multi-Head Attention\",\"上方 Transformer 架构图（paper_figure_1_transformer.png）中的 Multi-Head Attention 模块\",\"公式 MultiHead(Q,K,V)=Concat(head_1,...,head_h)W^O\",\"公式 head_i=Attention(QW_i^Q, KW_i^K, VW_i^V)\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants a narration for slide 6/12 about Multi-Head Attention in the Transformer paper. Previous page was likely about Scaled Dot-Product Attention (single attention). This page extends it to multi-head. Need 180-240 Chinese characters, oral style, guiding narrative, pointing at visible elements (title, formula, image of transformer figure), no fabricated numbers... wait, \"h times\" — the paper uses h=8, but the slide doesn't mention 8. So I shouldn't say 8 unless... the slide says \"h times\". Better not fabricate. Actually the figure is the full transformer architecture. I can point at the formula and the image.\n\nMust return only JSON.\n\nLet me draft:\n\n\"上一页我们讲了单个注意力怎么算，但只盯着一个角度看句子，难免片面。所以这一页，标题 Multi-Head Attention，就是把它做'多头'。请大家看屏幕下方的公式：我们不再只算一次注意力，而是把 Q、K、V 用各自学到的投影矩阵 W_i^Q、W_i^K、W_i^V 投到不同子空间，并行地算出 h 个 head，每个 head 就是前面刚讲过的那个 Attention。最后把这 h 个结果拼接起来，再乘一个输出矩阵 W^O 融合信息。上方这张架构图里，编码器、解码器中反复出现的 Multi-Head Attention 模块，就是这一步。它的好处是让模型同时从多种表示角度捕捉关系，为后面堆叠成完整网络打基础。\"\n\nCount characters roughly. Should be 180-240 Chinese chars. Let me count approximately.\n\nLet me count: I'll estimate. The draft seems around 230-250. Let me count more carefully.\n\n\"上一页我们讲了单个注意力怎么算，但只盯着一个角度看句子，难免片面。\"(29)\n\"所以这一页，标题 Multi-Head Attention，就是把它做成'多头'。\"(~28 counting English as chars; Chinese char count rules... just approximate)\nHmm, English words count ambiguously. Aim ~200 Chinese chars.\n\nLet me finalize:\n\n\"上一页我们学会了单个注意力怎么算，可只从一个角度看句子，难免片面。所以这一页的标题 Multi-Head Attention，就是把它升级成'多头'。请看屏幕下方的公式：我们不再只算一次，而是把 Q、K、V 分别乘上各自学出的投影矩阵 W_i^Q、W_i^K、W_i^V，投影 h 次，并行地算出 h 个 head；每个 head 正是上页那个 Attention。然后把它们拼接，再乘输出矩阵 W^O 做融合。上方这张 Transformer 架构图里反复出现的 Multi-Head Attention 方块，说的就是这个模块。这样模型能同时从多种表示角度捕捉依赖，也为下一页把它堆进完整网络做好准备。\"\n\nCount Chinese chars roughly: Let me count by segments:\n上一页我们学会了单个注意力怎么算(15)，可只从一个角度看句子(10)，难免片面(4)。~29+2 punct\n所以这一页的标题(8)Multi-Head Attention，就是把它升级成多头(9+2)。~19\n请看屏幕下方的公式(9)：我们不再只算一次(8)，而是把QKV分别乘上各自学出的投影矩阵(16)，投影h次(4)，并行地算出h个head(8)；每个head正是上页那个Attention(11)。~56\n然后把它们拼接(7)，再乘输出矩阵做融合(10)。~17\n上方这张架构图里反复出现的Multi-Head Attention方块(15)，说的就是这个模块(7)。~22\n这样模型能同时从多种表示角度捕捉依赖(17)，也为下一页把它堆进完整网络做好准备(17)。~34\n\nTotal ≈ 29+19+56+17+22+34 = 177 + punctuation. Slightly low; add a bit. Maybe add \"concat\" reference. Add: \"也就是公式里的 Concat\" etc.\n\nAdd: \"然后把它们按公式里的 Concat 拼接起来\" — adds ~8. And mention \"不同子空间\" — add \"投到不同的子空间\" ~8. That brings to ~195. Good, within 180-240.\n\nvisual_references: [\"标题 Multi-Head Attention\", \"公式 MultiHead(Q,K,V)=Concat(head_1,...,head_h)W^O\", \"head_i = Attention(QW_i^Q, KW_i^K, VW_i^V)\", \"Transformer 架构图（paper_figure_1）\"]\n\nReturn only JSON."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364242,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 1195,
|
||||
"prompt_tokens": 421,
|
||||
"total_tokens": 1616,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 947,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 37204.807
|
||||
}
|
||||
}
|
||||
+93
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "adc1486d26be8f3b851bb68a07001eadce24d1363ee852dbb3dc299ba43ecadd",
|
||||
"text_characters": 257,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-07.mp3",
|
||||
"sha256": "eb00364e1fbb75906a82421d6ded8d9e3fb49ec48120fd765e3fcd74f2b071df",
|
||||
"bytes": 673749
|
||||
},
|
||||
"elapsed_ms": 6915.935
|
||||
}
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"signature": "209b1652aab62b819b59a762e4e3802a932943ffb9ffaa1caeb2171009c73e1a",
|
||||
"parsed": {
|
||||
"narration": "翻到第7页，我们先补一个关键缺口：前面把词元变成向量后，Transformer 没有循环也没有卷积，本身并不知道谁先谁后。请看标题 Positional Encoding 下方这两条公式：偶数维用 sin、奇数维用 cos，频率随维度 i 和 d_model 变化，所以每个位置得到一套可重复计算的位置指纹。屏幕要点还提醒，它和词嵌入同维，便于注入；更重要的是，正弦余弦的相位关系让模型较容易学到相对距离。论文也说，这种手工位置编码几乎不输给可学习位置嵌入。明白这一页，下一页看编码器时，注意力分数才不会丢掉语序。",
|
||||
"visual_references": [
|
||||
"标题：Positional Encoding",
|
||||
"公式：PE_(pos, 2i) = sin(pos / 10000^(2i/d_model))",
|
||||
"公式：PE_(pos, 2i+1) = cos(pos / 10000^(2i/d_model))",
|
||||
"要点：Same dimension as embeddings (d_model)",
|
||||
"要点：Allows model to learn relative position information",
|
||||
"要点：Performed nearly as well as learned positional embeddings"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 7/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Positional Encoding\n\nSince model has no recurrence/convolution, we inject positional information:\n\n$$\\text{PE}_{(pos, 2i)} = \\sin\\left(pos / 10000^{2i/d_{\\text{model}}}\\right)$$\n$$\\text{PE}_{(pos, 2i+1)} = \\cos\\left(pos / 10000^{2i/d_{\\text{model}}}\\right)$$\n\n- Same dimension as embeddings ($d_{\\text{model}}$)\n- Allows model to learn relative position information\n- Performed nearly as well as learned positional embeddings\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7f3f079009943e6a5440",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"翻到第7页，我们先补一个关键缺口：前面把词元变成向量后，Transformer 没有循环也没有卷积，本身并不知道谁先谁后。请看标题 Positional Encoding 下方这两条公式：偶数维用 sin、奇数维用 cos，频率随维度 i 和 d_model 变化，所以每个位置得到一套可重复计算的位置指纹。屏幕要点还提醒，它和词嵌入同维，便于注入；更重要的是，正弦余弦的相位关系让模型较容易学到相对距离。论文也说，这种手工位置编码几乎不输给可学习位置嵌入。明白这一页，下一页看编码器时，注意力分数才不会丢掉语序。\",\"visual_references\":[\"标题：Positional Encoding\",\"公式：PE_(pos, 2i) = sin(pos / 10000^(2i/d_model))\",\"公式：PE_(pos, 2i+1) = cos(pos / 10000^(2i/d_model))\",\"要点：Same dimension as embeddings (d_model)\",\"要点：Allows model to learn relative position information\",\"要点：Performed nearly as well as learned positional embeddings\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "We need answer raw JSON only, with narration 180-240 Chinese characters around. Need satisfy constraints. Need produce final JSON. Need think content.\n\nSlide page 7/12 Positional Encoding. Need mention relation to previous/next: previous likely embedding inputs? Current says since model has no recurrence/convolution, inject positional info. It connects after token embeddings and before encoder/decoder blocks. Need point to visible elements: title and two formulas with sin/cos, bullets d_model same dimension, relative position, learned positional embeddings nearly as well. No invented values. Need opening/ending match page order: start like \"翻到第7页...\" end like \"弄明白这一页，下一页就能进入编码器堆叠\". Count Chinese chars 180-240. Need JSON with narration string and visual_references array.\n\nNeed ensure only JSON no markdown. Let's draft narration around 210 Chinese chars. Count roughly.\n\nNarration: \"翻到第7页，我们先解决一个关键缺口：前面把词元变成向量后，Transformer 没有循环也没有卷积，本身并不知道谁先谁后。请看标题 Positional Encoding 下方这两条公式：偶数维用 sin、奇数维用 cos，频率随维度 i 和 d_model 变化，所以每个位置得到一套可重复计算的位置指纹。屏幕上要点也提醒，它和词嵌入同维，可以直接相加；更重要的是，正弦余弦的相位差让模型较容易学到相对距离。论文还说，这种手工位置编码效果几乎不输给可学习位置嵌入。明白这一页，下一页看编码器时，注意力分数才不会丢失语序。\"\n\nCount Chinese roughly: 翻(1)到2第3 7? Count maybe including digits? Let's approximate length. It is around 198 chars. Good.\n\nNeed visual_references: title \"Positional Encoding\", formulas PE_(pos,2i), PE_(pos,2i+1), bullet same dimension as embeddings d_model, bullet learned positional embeddings. Good.\n\nNeed valid JSON string escape? No quotes inside. Use Chinese punctuation. Ensure JSON only. Need no trailing comma. final only JSON. 180-240 Chinese chars: our narration maybe around 209. Good.\n\nNeed maybe mention page序 ending. Good.\n\nDouble-check no invented conclusion: \"正弦余弦相位差让模型较容易学到相对距离\" slide says allows model to learn relative position information. OK. \"可以直接相加\" implied same dimension as embeddings and injected; paper adds. Slide says same dimension as embeddings. Since source says inject; fine. \"效果几乎不输给\" from performed nearly as well as learned. OK.\n\nFinal JSON only."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364287,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 798,
|
||||
"prompt_tokens": 417,
|
||||
"total_tokens": 1215,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 544,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 28762.403
|
||||
}
|
||||
}
|
||||
+89
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "52144578711becba5e6ed1ff7ed50adda03ac13879df4992853a40920f6a6995",
|
||||
"text_characters": 236,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-08.mp3",
|
||||
"sha256": "88d9f07f22eca3ffdecef543cb4226b7d02b9e251007c63cf55ee2803c32f404",
|
||||
"bytes": 658702
|
||||
},
|
||||
"elapsed_ms": 7183.744
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"signature": "6dd019ceb894c5b82fe8b47a597e2e2037c86807ed5448795a4f43528b47ab2c",
|
||||
"parsed": {
|
||||
"narration": "前面我们把整个编码器解码器架构拆完了，现在回到一个根本问题：为什么偏偏选注意力？请大家看屏幕上这张对比表，三列分别是每层复杂度、顺序操作数和最大路径长度。先看循环层这一行：它必须逐词往后传，顺序操作是 O(n)，路径长度也是 O(n)，长距离依赖很难学。再看自注意力这一行：任意两个位置一步直达，路径长度是常数 O(1)，还能完全并行。卷积居中，要堆 log 层才能连上远端。所以当句子长度 n 小于维度 d 时，自注意力既快又省。下一页我们就看实验如何兑现这些理论优势。",
|
||||
"visual_references": [
|
||||
"标题 Why Self-Attention?",
|
||||
"对比表三列：Complexity、Sequential Operations、Max Path Length",
|
||||
"Recurrent 行的 O(n) 顺序操作与路径长度",
|
||||
"Self-Attention 行的 O(1) 常数路径长度",
|
||||
"Convolutional 行的 O(log_k n)"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 8/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Why Self-Attention?\n\n| Layer Type | Complexity | Sequential Operations | Max Path Length |\n|------------|------------|-----------------------|-----------------|\n| Self-Attention | $O(n^2 \\cdot d)$ | $O(1)$ | $O(1)$ |\n| Recurrent | $O(n \\cdot d^2)$ | $O(n)$ | $O(n)$ |\n| Convolutional | $O(k \\cdot n \\cdot d^2)$ | $O(1)$ | $O(\\log_k n)$ |\n\n- Constant path length between any positions\n- More parallelizable than RNNs\n- Better computational efficiency for typical sentence lengths\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7f47e323e45631c9db51",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"前面我们把整个编码器解码器架构拆完了，现在回到一个根本问题：为什么偏偏选注意力？请大家看屏幕上这张对比表，三列分别是每层复杂度、顺序操作数和最大路径长度。先看循环层这一行：它必须逐词往后传，顺序操作是 O(n)，路径长度也是 O(n)，长距离依赖很难学。再看自注意力这一行：任意两个位置一步直达，路径长度是常数 O(1)，还能完全并行。卷积居中，要堆 log 层才能连上远端。所以当句子长度 n 小于维度 d 时，自注意力既快又省。下一页我们就看实验如何兑现这些理论优势。\",\"visual_references\":[\"标题 Why Self-Attention?\",\"对比表三列：Complexity、Sequential Operations、Max Path Length\",\"Recurrent 行的 O(n) 顺序操作与路径长度\",\"Self-Attention 行的 O(1) 常数路径长度\",\"Convolutional 行的 O(log_k n)\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The user wants a narration for slide 8/12 of \"Attention Is All You Need\" - the \"Why Self-Attention?\" comparison table. Need 180-240 Chinese characters, spoken style, guiding narrative explaining \"why\" and connection to previous/next pages, pointing to visible elements (title, table, formulas, values), no fabricated numbers, matching page order opening/closing.\n\nPrevious pages likely covered the model architecture (encoder-decoder, attention mechanism, multi-head). Next pages might cover experiments/results. This slide justifies why self-attention is chosen — Table 1 in the paper comparing layer types on complexity, sequential operations, max path length.\n\nI should reference the table on screen: three columns — Complexity per layer, Sequential Operations, Maximum Path Length. Compare self-attention O(n²·d), recurrent O(n·d²) with O(n) sequential ops, convolutional O(k·n·d²) with O(log_k n) path length. Key points: constant path length O(1) between any two positions — long-range dependencies learned in one step; parallelizable unlike RNN; when sequence length n smaller than d (typical for sentences), self-attention is cheaper.\n\nOpening should connect: \"看完了模型结构，为什么作者要用自注意力替换掉循环和卷积呢？\" Closing: leads to next page — experiments/results verify this choice.\n\nCount characters: aim 180-240 Chinese characters. Let me draft:\n\n\"前面我们把整个架构拆开了，现在回到一个根本问题：为什么偏偏选注意力？请大家看屏幕上这张对比表，三列分别是每层复杂度、顺序操作数和最大路径长度。先看循环层这一行：它必须一个词一个词往后传，顺序操作是 O(n)，路径长度也是 O(n)，长距离依赖很难学。而自注意力这一行，任意两个位置一步直达，路径长度是常数 O(1)，还能完全并行。卷积居中，需要堆 log 层才能连上远端。所以当句子长度 n 小于维度 d 时，自注意力既快又省。下一页我们就看实验怎么验证这个选择。\"\n\nCount roughly: Let me count characters.\n\n\"前面我们把整个架构拆开了，现在回到一个根本问题：为什么偏偏选注意力？\" ~ 35 chars\n\"请大家看屏幕上这张对比表，三列分别是每层复杂度、顺序操作数和最大路径长度。\" ~ 36\n\"先看循环层这一行：它必须一个词一个词往后传，顺序操作是 O(n)，路径长度也是 O(n)，长距离依赖很难学。\" ~ 45 (letters count as chars roughly)\n\"而自注意力这一行，任意两个位置一步直达，路径长度是常数 O(1)，还能完全并行。\" ~ 35\n\"卷积居中，需要堆 log 层才能连上远端。\" ~ 18\n\"所以当句子长度 n 小于维度 d 时，自注意力既快又省。\" ~ 24\n\"下一页我们就看实验怎么验证这个选择。\" ~ 17\n\nTotal ~210. Good, within 180-240.\n\nVisual references: the comparison table, the sequential operations column, self-attention row O(1), recurrent row O(n), three column headers.\n\nMust return only JSON."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364296,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 894,
|
||||
"prompt_tokens": 431,
|
||||
"total_tokens": 1325,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 666,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 31507.869
|
||||
}
|
||||
}
|
||||
+100
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "dc95c61e282bf006c3d223c877e358abc86afcf95c26cef057435634f36eb1f8",
|
||||
"text_characters": 239,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-09.mp3",
|
||||
"sha256": "973c9d12b8c70ba55cb33e5ae0d643b0ed35596bafcc86a2a8fc523c0d7c77f0",
|
||||
"bytes": 667061
|
||||
},
|
||||
"elapsed_ms": 7264.74
|
||||
}
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"signature": "5db468b21d65ce64103fec1ce935870b4152125c2e637403437f8514521269e6",
|
||||
"parsed": {
|
||||
"narration": "前面几页我们把架构和训练细节都讲完了，那效果到底行不行？这一页就是论文交出的成绩单。请大家看屏幕上这张机器翻译结果表，重点看加粗的第三行：Transformer big 在英德任务上拿到28.4 BLEU，比此前最好的集成模型高出约2个点；英法方向达到41.8，同样是新的最优纪录。更值得关注的是最右边训练成本这一列，它只用了2.3乘10的19次方的浮点运算量，连对手的三分之一都不到。也就是说，纯注意力机制不但质量更高，还更省钱。这样的架构在其他任务上还能打吗？下一页接着看。",
|
||||
"visual_references": [
|
||||
"标题 Machine Translation Results",
|
||||
"结果表格中加粗的 Transformer (big) 行",
|
||||
"EN-DE 列的 28.4 与 EN-FR 列的 41.8",
|
||||
"Training Cost (FLOPs) 列的 2.3×10^19 与上方两行数值对比"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 9/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Machine Translation Results\n\n| Model | EN-DE BLEU | EN-FR BLEU | Training Cost (FLOPs) |\n|-------|------------|------------|-----------------------|\n| GNMT + RL Ensemble | 26.30 | 41.16 | $1.8 \\cdot 10^{20}$ |\n| ConvS2S Ensemble | 26.36 | 41.29 | $7.7 \\cdot 10^{19}$ |\n| **Transformer (big)** | **28.4** | **41.8** | **$2.3 \\cdot 10^{19}$** |\n\n- Transformer outperforms all previous state-of-the-art models\n- Achieves better results with significantly lower training cost\n- 28.4 BLEU on EN-DE (↑2 BLEU over previous best)\n- 41.8 BLEU on EN-FR (new state-of-the-art)\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7f6c89b4c0360c645bb5",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"前面几页我们把架构和训练细节都讲完了，那效果到底行不行？这一页就是论文交出的成绩单。请大家看屏幕上这张机器翻译结果表，重点看加粗的第三行：Transformer big 在英德任务上拿到28.4 BLEU，比此前最好的集成模型高出约2个点；英法方向达到41.8，同样是新的最优纪录。更值得关注的是最右边训练成本这一列，它只用了2.3乘10的19次方的浮点运算量，连对手的三分之一都不到。也就是说，纯注意力机制不但质量更高，还更省钱。这样的架构在其他任务上还能打吗？下一页接着看。\",\"visual_references\":[\"标题 Machine Translation Results\",\"结果表格中加粗的 Transformer (big) 行\",\"EN-DE 列的 28.4 与 EN-FR 列的 41.8\",\"Training Cost (FLOPs) 列的 2.3×10^19 与上方两行数值对比\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The task: write a Chinese oral narration for slide 9/12 of Attention Is All You Need, about Machine Translation Results. Need 180-240 Chinese characters, guiding narrative explaining \"why\" and relation to previous/next pages, pointing to visible elements (table, numbers), no fabrication, JSON only output.\n\nSlide 9 is about translation results — previous pages presumably covered architecture/training details, so this page shows the payoff. Point to the table: EN-DE BLEU 28.4 vs 26.30/26.36, EN-FR 41.8 vs 41.16/41.29, training cost FLOPs 2.3e19 vs 1.8e20/7.7e19. Emphasize better quality at lower cost.\n\nWrite ~200 characters. Include visual_references list: \"Machine Translation Results 标题\", \"BLEU 对比表格\", \"28.4 / 41.8 加粗数值\", \"Training Cost (FLOPs) 列 2.3·10^19\".\n\nCount characters carefully. Let me draft:\n\n\"讲了这么多结构设计，效果到底怎么样？现在看屏幕这张翻译结果表格。这是第9页，也是整篇论文最有说服力的证据。请大家看表格里加粗的那一行：Transformer big 在英德翻译上拿到28.4 BLEU，比此前最好的集成模型高出约2个点；英法方向达到41.8，同样是新纪录。更关键的是最右边一列——训练成本，它只用了2.3×10^19 次浮点运算，不到对手的三分之一。也就是说，纯注意力架构不仅质量更高，还更便宜。那它在其他任务上表现如何？下一页继续看。\"\n\nCount: roughly let me estimate. That's around 200+ characters. Let me count more precisely.\n\n\"讲了这么多结构设计，效果到底怎么样？\" = 19\n\"现在看屏幕这张翻译结果表格。\" = 14\n\"这是第9页，也是整篇论文最有说服力的证据。\" = 21\n\"请大家看表格里加粗的那一行：\" = 14\n\"Transformer big 在英德翻译上拿到28.4 BLEU，比此前最好的集成模型高出约2个点；\" — count Chinese chars plus some Latin. The requirement says 180–240 个中文字符 — likely total characters. Let me just aim ~210 total.\n\n\"英法方向达到41.8，同样是新纪录。\" = ~17\n\"更关键的是最右边一列——训练成本，它只用了2.3×10的19次方次浮点运算，不到对手的三分之一。\" \n\"也就是说，纯注意力架构不仅质量更高，还更便宜。\"\n\"那它在其他任务上表现如何？下一页继续看。\"\n\nTotal likely ~215. Good.\n\nActually the slide says \"↑2 BLEU over previous best\" so \"高出约2个点\" is fine. Cost comparison: 2.3e19 vs 7.7e19 (ConvS2S) — that's less than a third; vs 1.8e20 GNMT it's about 1/8. Saying \"不到对手的三分之一\" is safe for both.\n\nJSON output only."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364333,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 892,
|
||||
"prompt_tokens": 484,
|
||||
"total_tokens": 1376,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 666,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 33233.501
|
||||
}
|
||||
}
|
||||
+92
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "585441cb8fe58d8bb5c0a0e4399e3dcbce56596d45ef66f5d5a60ed08248d21a",
|
||||
"text_characters": 247,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-10.mp3",
|
||||
"sha256": "48fc463131c0b1e78bfafa5e0cd48c5c1b252437f5e237fd80daca27113edcf5",
|
||||
"bytes": 685034
|
||||
},
|
||||
"elapsed_ms": 7867.787
|
||||
}
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"signature": "ca63c262f538d79e6aa5057385782315042b464816023e0f181c47dcb0583655",
|
||||
"parsed": {
|
||||
"narration": "前面我们从机制上讲清了自注意力怎么算，这一页就来看它真正学到的效果。请大家看屏幕中间这张可视化图：这是编码器第五层的自注意力，句子中词与词之间的连线代表注意力强弱。重点请关注动词 making——它没有只盯着相邻的词，而是把注意力投向了距离很远的句首，指向它真正的依赖对象。这就是标题说的长距离依赖：传统循环网络要一步步传递信息，距离一远信号就衰减，而自注意力一步到位，直接建立连接，代价和距离无关。这也回应了我们前面反复强调的并行化与全局感受野。下一页，我们再看一个更有挑战性的指代消歧例子。",
|
||||
"visual_references": [
|
||||
"标题 Attention Visualization: Long-Distance Dependencies",
|
||||
"屏幕中间的注意力连线可视化图",
|
||||
"图中动词 making 的注意力分布",
|
||||
"图注：encoder self-attention in layer 5"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 10/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Attention Visualization: Long-Distance Dependencies\n\n<img src=\"/paper_figure_3_long_distance.png\" class=\"h-70 mx-auto\" />\n\n*Encoder self-attention in layer 5 showing attention to distant dependency of the verb \"making\"*\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6acf696ff171b3d8c3411c",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"前面我们从机制上讲清了自注意力怎么算，这一页就来看它真正学到的效果。请大家看屏幕中间这张可视化图：这是编码器第五层的自注意力，句子中词与词之间的连线代表注意力强弱。重点请关注动词 making——它没有只盯着相邻的词，而是把注意力投向了距离很远的句首，指向它真正的依赖对象。这就是标题说的长距离依赖：传统循环网络要一步步传递信息，距离一远信号就衰减，而自注意力一步到位，直接建立连接，代价和距离无关。这也回应了我们前面反复强调的并行化与全局感受野。下一页，我们再看一个更有挑战性的指代消歧例子。\",\"visual_references\":[\"标题 Attention Visualization: Long-Distance Dependencies\",\"屏幕中间的注意力连线可视化图\",\"图中动词 making 的注意力分布\",\"图注：encoder self-attention in layer 5\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "This is page 10 of 12. Need 180-240 Chinese characters narration, JSON output only. The slide shows attention visualization, encoder self-attention layer 5, long-distance dependency of verb \"making\". Must reference visible image. Connect to previous pages (multi-head attention) and next (remaining visualizations/conclusion). Don't fabricate numbers. Paper figure 3 shows the word \"making\" attending to distant words.\n\nWrite ~180-240 chars. Let me draft:\n\n\"前面我们从机制上讲了自注意力怎么算，现在看看它实际学到了什么。请大家看屏幕中间这张图：这是编码器第五层的自注意力可视化，一句话里每个词与其他词的连线，颜色越深代表注意力越强。重点看动词 making——它并没有只盯着旁边的词，而是把目光投向了句首远处的 it，也就是它真正的主语。这正是长距离依赖：传统 RNN 要一步步传信息，距离远了就容易丢，而自注意力一步直达。下一页我们再看另一个更难的指代例子。\"\n\nCount characters roughly: that's about 200 chars. Good. Visual references: 标题, 中间可视化图, making 的注意力连线, layer 5 caption.\n\nEnsure no fabricated numbers beyond \"第五层\" which is in slide. Good. JSON only."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785384811,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 460,
|
||||
"prompt_tokens": 346,
|
||||
"total_tokens": 806,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 262,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 21099.258
|
||||
}
|
||||
}
|
||||
+90
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "30b3b733b68a98e6dc4ad8cdc64e236fc1b59ff1e615132aa36517033cd165a0",
|
||||
"text_characters": 206,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-11.mp3",
|
||||
"sha256": "01dca987731afc2baed6810919f5999ad9a912da4e649ee0296164f734f6a216",
|
||||
"bytes": 639894
|
||||
},
|
||||
"elapsed_ms": 6750.909
|
||||
}
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
{
|
||||
"signature": "a4f97a7406b904f8788772f54790ffa218c609420559fe034d18a651527daf03",
|
||||
"parsed": {
|
||||
"narration": "上一页我们看到注意力头如何捕捉长距离依赖，这一页再看一个更有意思的现象。请大家看屏幕中间这张论文原图，配合下方的说明文字：注意力头在解析代词 its 时，把最强的连接指向了 The Law。换句话说，没有人专门教过模型语法规则，它却在训练中自己学会了指代消解——判断代词到底指谁。这说明注意力机制能够自发涌现出语言结构，也难怪作者对纯注意力架构如此有信心。到这里，论文的主体内容就讲完了，下一页我们来做整体总结。",
|
||||
"visual_references": [
|
||||
"标题：Attention Visualization: Anaphora Resolution",
|
||||
"屏幕中间的论文原图 paper_figure_4_anaphora.png",
|
||||
"图片下方说明：注意力头将 \"its\" 的指代解析到 \"The Law\""
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 11/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Attention Visualization: Anaphora Resolution\n\n<img src=\"/paper_figure_4_anaphora.png\" class=\"h-70 mx-auto\" />\n\n*Attention heads involved in resolving \"its\" reference to \"The Law\"*\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6a7f9ebcb8b1ca0dc24f49",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"上一页我们看到注意力头如何捕捉长距离依赖，这一页再看一个更有意思的现象。请大家看屏幕中间这张论文原图，配合下方的说明文字：注意力头在解析代词 its 时，把最强的连接指向了 The Law。换句话说，没有人专门教过模型语法规则，它却在训练中自己学会了指代消解——判断代词到底指谁。这说明注意力机制能够自发涌现出语言结构，也难怪作者对纯注意力架构如此有信心。到这里，论文的主体内容就讲完了，下一页我们来做整体总结。\",\"visual_references\":[\"标题：Attention Visualization: Anaphora Resolution\",\"屏幕中间的论文原图 paper_figure_4_anaphora.png\",\"图片下方说明：注意力头将 \\\"its\\\" 的指代解析到 \\\"The Law\\\"\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "Let me analyze this task:\n\n1. This is page 11/12 of a presentation about \"Attention Is All You Need\" paper\n2. The slide is about Attention Visualization for Anaphora Resolution\n3. It shows Figure 4 from the paper - attention heads resolving \"its\" reference to \"The Law\"\n4. I need to write 180-240 Chinese characters of spoken narration\n\nKey requirements:\n- Explain \"why\" and relationship with previous/next pages\n- Point to visible elements on screen (the image, title)\n- Don't fabricate numbers/conclusions not in source\n- Match page order (11/12, so it's near the end, second to last)\n- Return only JSON with narration and visual_references\n\nContext from the paper: This is Figure 4 which shows attention heads performing anaphora resolution - the famous example where the attention head learns to link \"its\" back to \"The Law\" in the sentence \"The Law will never be perfect, but its application should be just\". This demonstrates that the model learned linguistic structure without supervision.\n\nSince this is page 11 of 12, this is part of the visualization/analysis section near the end. Previous pages likely showed other attention visualizations (like Figure 3 showing long-range dependencies), and the next page (12) would be the conclusion/summary.\n\nThe narration should:\n- Connect from previous visualization discussion\n- Explain why this example matters - it shows attention heads learned to do coreference resolution, a linguistic task, without explicit supervision\n- Point to the image on screen, the caption about \"its\" and \"The Law\"\n- Lead toward the conclusion (page 12)\n\nLet me write the narration in natural Chinese:\n\n开头：承接上一页的可视化讨论\n中间：解释这张图——看屏幕中间的图片，注意力头如何把代词\"its\"连回\"The Law\"，这说明模型在无监督情况下学会了指代消解这种语言学现象\n结尾：引向下一页（第12页是最后一页，应该是总结）\n\nDraft:\n\"上一页我们看到注意力头能捕捉长距离依赖，这一页再看一个更有意思的例子。请大家看屏幕中间的这张图，这是论文里的第四张可视化，展示的是指代消解。注意下方的说明文字：注意力头在解析 'its' 这个代词时，把最强的注意力连向了 'The Law'。也就是说，模型并没有人教过它语法，却在训练中自己学会了判断代词指向谁——这正是语言学里很重要的指代消解任务。这也回答了一个问题：为什么注意力机制如此强大？因为它能自发涌现出语言结构。到这里，论文的核心内容就讲完了，下一页我们来做整体总结。\"\n\nLet me count characters... Let me be careful about the count (180-240 Chinese characters).\n\nLet me refine and count:\n\n\"上一页我们看到注意力头如何捕捉长距离依赖，这一页再看一个更有意思的现象。请大家看屏幕中间这张论文原图，以及下方的说明：注意力头在解析代词 'its' 时，把最强的连接指向了 'The Law'。换句话说，没有人教过模型语法规则，它却在训练中自己学会了指代消解——判断代词到底指谁。这说明注意力机制能自发涌现出语言结构，这也是为什么作者对纯注意力架构充满信心。到这里论文主体就讲完了，下一页我们做最后的总结。\"\n\nCount roughly: \n上一页我们看到注意力头如何捕捉长距离依赖，(20) 这一页再看一个更有意思的现象。(14) 请大家看屏幕中间这张论文原图，(15) 以及下方的说明：(6) 注意力头在解析代词 'its' 时，(13) 把最强的连接指向了 'The Law'。(14) 换句话说，(5) 没有人教过模型语法规则，(12) 它却在训练中自己学会了指代消解——(16) 判断代词到底指谁。(9) 这说明注意力机制能自发涌现出语言结构，(19) 这也是为什么作者对纯注意力架构充满信心。(21) 到这里论文主体就讲完了，(12) 下一页我们做最后的总结。(11)\n\nTotal ≈ 187 characters. Good, within range.\n\nVisual references should include: the image (paper_figure_4_anaphora.png), the caption about \"its\" and \"The Law\", the title \"Attention Visualization: Anaphora Resolution\".\n\nvisual_references: [\"标题 'Attention Visualization: Anaphora Resolution'\", \"屏幕中间的论文原图 paper_figure_4_anaphora.png\", \"图片下方说明文字：注意力头将 'its' 解析为指向 'The Law'\"]\n\nMake sure JSON format only, no markdown."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785364383,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 1136,
|
||||
"prompt_tokens": 342,
|
||||
"total_tokens": 1478,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 954,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": null
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 40003.702
|
||||
}
|
||||
}
|
||||
+89
File diff suppressed because one or more lines are too long
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"provider": "Fish Audio",
|
||||
"model": "s1",
|
||||
"request": {
|
||||
"text_sha256": "c21c88e13d882d777c4ba37ea13548851d707a3e6179efb5c42e8606cd6da1da",
|
||||
"text_characters": 302,
|
||||
"reference_id_sha256": "bb2ec197d6276bfad81bbdbcec4473040861e6bc20caf53cf4d1f8184a7416be",
|
||||
"format": "mp3"
|
||||
},
|
||||
"response_artifact": {
|
||||
"path": "page-12.mp3",
|
||||
"sha256": "d5fa15bb501d23b6f90fc589357b1cbb5154721d9af0b0101704d6f0b7a5dfea",
|
||||
"bytes": 711365
|
||||
},
|
||||
"elapsed_ms": 7262.432
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"signature": "e07231c0c9c9918dbbabdd8e7afda92311d0dc08ddc799e7968a9a5d7b9414d8",
|
||||
"attempts": [
|
||||
{
|
||||
"error": "provider response did not contain a JSON object",
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 12/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Conclusion and Future Work\n\n### Key Contributions\n- Introduced Transformer architecture based solely on attention\n- Achieved new state-of-the-art results in machine translation\n- Demonstrated improved parallelization and reduced training time\n- Showed generalization to other tasks like constituency parsing\n\n### Future Directions\n- Apply to other modalities (images, audio, video)\n- Investigate local, restricted attention for large inputs\n- Make generation less sequential\n- Explore interpretability of attention mechanisms\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6ad0eeea5e3f5211347c7f",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "length",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"好，我们来到最后一页。请大家看屏幕上的结论部分：回顾前面十一页，从自注意力公式到编码器解码器结构，再到翻译实验，其实都指向",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "The task: write a narration for page 12/12, the conclusion slide of \"Attention Is All You Need\". 180-240 Chinese characters, guiding narrative, referencing visible elements, no fabricated numbers, matching page order (final page, so should wrap up). Return JSON only.\n\nLet me draft:\n\n\"好，我们来到最后一页。大家看屏幕上的 'Conclusion and Future Work'。回顾前面十一页，从 self-attention 的公式到 Encoder-Decoder 结构，再到翻译任务的实验结果，其实都指向作者在这里总结的一点：完全抛弃循环和卷积、只靠注意力，不但行得通，还能取得当时最好的翻译成绩，而且并行度更高、训练更快。左栏还提到它能泛化到句法分析等其他任务。更重要的是右栏的 Future Directions：作者已经预见到把这套机制推广到图像、音频、视频，用局部注意力处理长输入——后来 Vision Transformer、多模态大模型正是沿着这些方向走出来的。这就是这篇论文改变整个领域的起点。\"\n\nCount characters. Let me count roughly. Need 180-240 Chinese characters. Let me count:\n\n\"好，我们来到最后一页。\" ~ 10\nLet me just estimate. I'll write carefully and count.\n\nDraft:\n\n好(1)，(2)我(3)们(4)来(5)到(6)最(7)后(8)一(9)页(10)。(11)大(12)家(13)看(14)屏(15)幕(16)上(17)的(18)结(19)论(20)部(21)分(22)：(23)回(24)顾(25)前(26)面(27)十(28)一(29)页(30)，(31)从(32)自(33)注(34)意(35)力(36)公(37)式(38)到(39)编(40)码(41)器(42)解(43)码(44)器(45)结(46)构(47)，(48)再(49)到(50)翻(51)译(52)实(53)验(54)，(55)其(56)实(57)都(58)指(59)向(60)作(61)者(62)在(63)这(64)里(65)总(66)结(67)的(68)核(69)心(70)—(71)—(72)完(73)全(74)依(75)靠(76)注(77)意(78)力(79)、(80)抛(81)弃(82)循(83)环(84)与(85)卷(86)积(87)，(88)不(89)仅(90)行(91)得(92)通(93)，(94)还(95)刷(96)新(97)了(98)机(99)器(100)翻(101)译(102)的(103)最(104)好(105)成(106)绩(107)，(108)并(109)行(110)度(111)更(112)高(113)、(114)训(115)练(116)更(117)快(118)。(119)左(120)栏(121)还(122)提(123)到(124)它(125)能(126)泛(127)化(128)到(129)成(130)分(131)句(132)法(133)分(134)析(135)等(136)任(137)务(138)。(139)更(140)有(141)意(142)思(143)的(144)是(145)右(146)栏(147)的(148)未(149)来(150)方(151)向(152)：(153)推(154)广(155)到(156)图(157)像(158)、(159)音(160)频(161)、(162)视(163)频(164)，(165)用(166)局(167)部(168)受(169)限(170)注(171)意(172)力(173)应(174)对(175)超(176)长(177)输(178)入(179)，(180)甚(181)至(182)减(183)少(184)生(185)成(186)的(187)逐(188)词(189)顺(190)序(191)性(192)。(193)后(194)来(195)的(196)视(197)觉(198)和(199)多(200)模(201)态(202)模(203)型(204)，(205)正(206)是(207)沿(208)着(209)这(210)些(211)思(212)路(213)走(214)出(215)来(216)的(217)。(218)\n\nAbout 218 characters including punctuation. Good, within 180-240.\n\nVisual references: [\"标题 'Conclusion and Future Work'\", \"左栏 Key Contributions 列表\", \"右栏 Future Directions 列表\"]\n\nReturn JSON only."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785385199,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 1200,
|
||||
"prompt_tokens": 393,
|
||||
"total_tokens": 1593,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 1150,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": {
|
||||
"audio_tokens": null,
|
||||
"cached_tokens": 256
|
||||
},
|
||||
"cached_tokens": 256
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 33754.516,
|
||||
"json_attempt": 1
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"signature": "e07231c0c9c9918dbbabdd8e7afda92311d0dc08ddc799e7968a9a5d7b9414d8",
|
||||
"parsed": {
|
||||
"narration": "好，我们来到最后一页，给整篇论文收个尾。请看屏幕上半部分的 Key Contributions：还记得我们最初从 RNN 的顺序瓶颈讲起吗？Transformer 干脆只用注意力、完全抛弃循环，这就是第一条 solely on attention 的含义；也正因如此，训练可以大规模并行、时间大幅缩短，还在机器翻译上刷新了最好成绩，并泛化到了成分句法分析这样的任务。再看下方的 Future Directions：作者当时已在展望图像、音频、视频等模态，以及对超长输入做局部注意力、让生成不那么顺序化。这些方向后来几乎都被一一验证。可以说，这一页既是终点，也是今天大模型时代的起点。感谢大家一路听到这里！",
|
||||
"visual_references": [
|
||||
"Conclusion and Future Work 标题",
|
||||
"Key Contributions 小节的项目列表",
|
||||
"solely on attention 与 constituency parsing 条目",
|
||||
"Future Directions 小节的四个展望条目"
|
||||
]
|
||||
},
|
||||
"receipt": {
|
||||
"provider": "Moonshot",
|
||||
"request": {
|
||||
"model": "kimi-k3",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "你是一位严谨而自然的中文技术讲师，正在讲解论文 Attention Is All You Need。\n这是第 12/12 页。下面给出该页真实 Slidev 源码：\n<slide>\n## Conclusion and Future Work\n\n### Key Contributions\n- Introduced Transformer architecture based solely on attention\n- Achieved new state-of-the-art results in machine translation\n- Demonstrated improved parallelization and reduced training time\n- Showed generalization to other tasks like constituency parsing\n\n### Future Directions\n- Apply to other modalities (images, audio, video)\n- Investigate local, restricted attention for large inputs\n- Make generation less sequential\n- Explore interpretability of attention mechanisms\n</slide>\n请写一段 180–240 个中文字符左右的口语讲解。必须满足：\n1. 用引导性叙事解释“为什么”和“它与前后页的关系”，不能逐条照读幻灯片；\n2. 明确指向当前屏幕上的可见元素（标题、公式、表格、数值或图片）；有图片/表格/公式时必须说出观众该看哪里；\n3. 不虚构源码中没有的数值或结论；开头/结尾与页序匹配；\n4. 只返回 JSON：{\"narration\":\"...\",\"visual_references\":[\"...\"]}。\n"
|
||||
}
|
||||
],
|
||||
"temperature": 1,
|
||||
"max_tokens": 1200,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "chatcmpl-6a6ad110ac3274ea8020376b",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"message": {
|
||||
"content": "{\"narration\":\"好，我们来到最后一页，给整篇论文收个尾。请看屏幕上半部分的 Key Contributions：还记得我们最初从 RNN 的顺序瓶颈讲起吗？Transformer 干脆只用注意力、完全抛弃循环，这就是第一条 solely on attention 的含义；也正因如此，训练可以大规模并行、时间大幅缩短，还在机器翻译上刷新了最好成绩，并泛化到了成分句法分析这样的任务。再看下方的 Future Directions：作者当时已在展望图像、音频、视频等模态，以及对超长输入做局部注意力、让生成不那么顺序化。这些方向后来几乎都被一一验证。可以说，这一页既是终点，也是今天大模型时代的起点。感谢大家一路听到这里！\",\"visual_references\":[\"Conclusion and Future Work 标题\",\"Key Contributions 小节的项目列表\",\"solely on attention 与 constituency parsing 条目\",\"Future Directions 小节的四个展望条目\"]}",
|
||||
"refusal": null,
|
||||
"role": "assistant",
|
||||
"annotations": null,
|
||||
"audio": null,
|
||||
"function_call": null,
|
||||
"tool_calls": null,
|
||||
"reasoning_content": "This is page 12/12, the final page of the lecture on \"Attention Is All You Need\". The slide is about Conclusion and Future Work. I need to write a spoken narration of 180-240 Chinese characters that:\n1. Uses guiding narrative explaining \"why\" and relationship to previous pages\n2. Points to visible elements on screen\n3. Doesn't fabricate numbers/conclusions not in source\n4. Matches page sequence at beginning/end (this is the last page, so should wrap up)\n5. Return JSON only\n\nThe previous pages presumably covered: introduction, model architecture (encoder/decoder, attention, multi-head attention, positional encoding), why self-attention, training, results (BLEU scores), etc.\n\nThe narration should reference visible elements: \"Key Contributions\" and \"Future Directions\" sections with bullet points like \"based solely on attention\", \"constituency parsing\", \"local, restricted attention\", \"less sequential generation\".\n\nLet me write ~180-240 Chinese characters, conversational, ending the series.\n\nDraft:\n\"好，我们来到最后一页，一起回顾整个旅程。请看屏幕上半部分的 Key Contributions：还记得我们从 RNN 的顺序瓶颈出发吗？Transformer 用纯注意力机制取代循环，这正是第一条'solely based on attention'的含义。它不仅在机器翻译上刷新了最好成绩，更重要的是训练可以大规模并行，时间大幅缩短；还能泛化到成分句法分析等其他任务。再看下方 Future Directions：作者当时已经在展望图像、音频、视频等模态，以及对超长输入做局部注意力、让生成不那么顺序化——这些方向后来都被一一验证。可以说，这一页既是终点，也是今天大模型时代的起点。感谢大家一路听到这里！\"\n\nCount characters roughly: Let me count. That's likely around 220 characters. Let me count more carefully.\n\n\"好，我们来到最后一页，一起回顾整个旅程。\" = 20\n\"请看屏幕上半部分的 Key Contributions：\" ~ 16\n\"还记得我们从 RNN 的顺序瓶颈出发吗？\" ~ 18\n\"Transformer 用纯注意力机制取代循环，这正是第一条'solely based on attention'的含义。\" ~ 40 (including English)\n...\n\nShould be fine within 180-240. Let me be careful about character counting — the requirement says 180–240 个中文字符左右 (around). English words count as characters too but let's aim for ~200-230 Chinese characters.\n\nvisual_references: [\"Key Contributions 小节列表\", \"solely based on attention 条目\", \"constituency parsing 条目\", \"Future Directions 小节\", \"local, restricted attention 条目\", \"Make generation less sequential 条目\"]\n\nReturn only JSON."
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1785385233,
|
||||
"model": "kimi-k3",
|
||||
"object": "chat.completion",
|
||||
"service_tier": null,
|
||||
"system_fingerprint": null,
|
||||
"usage": {
|
||||
"completion_tokens": 752,
|
||||
"prompt_tokens": 393,
|
||||
"total_tokens": 1145,
|
||||
"completion_tokens_details": {
|
||||
"accepted_prediction_tokens": null,
|
||||
"audio_tokens": null,
|
||||
"reasoning_tokens": 548,
|
||||
"rejected_prediction_tokens": null
|
||||
},
|
||||
"prompt_tokens_details": {
|
||||
"audio_tokens": null,
|
||||
"cached_tokens": 256
|
||||
},
|
||||
"cached_tokens": 256
|
||||
}
|
||||
},
|
||||
"elapsed_ms": 24600.55,
|
||||
"json_attempt": 2
|
||||
}
|
||||
}
|
||||
+94
File diff suppressed because one or more lines are too long
+12
@@ -0,0 +1,12 @@
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-01.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-02.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-03.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-04.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-05.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-06.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-07.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-08.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-09.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-10.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-11.mp4'
|
||||
file '/Users/boj/book/ai-agent-book/chapter5/paper-to-video/validation/runs/exp5-5-kimi-fish-qwen-20260730-v1/segments/page-12.mp4'
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 59 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 97 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 117 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 116 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 83 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 113 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 92 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 100 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 110 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 119 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 150 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 119 KiB |
Reference in New Issue
Block a user