ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,4 @@
|
||||
# 生成的视频、截图、成片——避免仓库膨胀
|
||||
output/
|
||||
__pycache__/
|
||||
.env
|
||||
@@ -0,0 +1,457 @@
|
||||
# Experiment 5-6: API-Driven Smart Video Editing / 实验 5-6：基于 API 的智能视频剪辑
|
||||
|
||||
> Companion lab for *AI Agents in Depth*, Chapter 5 — NL request + multi-scene video → two-step Vision locate → Blender bpy script / ffmpeg cut → Proposer–Reviewer.
|
||||
> 《深入理解 AI Agent》配套：自然语言需求 + 多场景视频 → 两步 Vision 定位 → 生成 Blender bpy / ffmpeg 剪辑 → 提议者-审核者。
|
||||
|
||||
← [Chapter 5 index / 返回第 5 章目录](../README.md)
|
||||
|
||||
---
|
||||
|
||||
## English
|
||||
|
||||
### Purpose
|
||||
|
||||
Given a multi-scene video and one NL request (e.g. “cut out the surfing part”), the Agent locates the target scene, **generates a Blender Python API script**, cuts the clip, and self-reviews.
|
||||
|
||||
Three mechanisms in multimedia processing:
|
||||
|
||||
1. **Two-step Vision locate**: Proposer cannot “watch” video directly; a **video-analysis sub-agent** uses ffmpeg frame extraction + Vision LLM to find time bounds.
|
||||
2. **Code generation (Blender Python API)**: Proposer turns the edit plan into a **bpy** script—import / trim / subtitle / speed / render as API calls, run with `blender --background --python edit.py`. Without Blender, scripts still generate; render falls back to ffmpeg (see “edit backends”).
|
||||
3. **Proposer / Reviewer**: after cut, Reviewer samples key frames with Vision; fail → feedback → iterate.
|
||||
|
||||
### Two-step locate
|
||||
|
||||
Scanning every frame is slow/expensive → coarse then fine:
|
||||
|
||||
- **Coarse**: one frame every **10s**; Vision gets a rough interval (e.g. surfing 20–30s).
|
||||
- **Fine**: expand the coarse window by one coarse step; one frame every **1s**; Vision returns precise bounds (e.g. 15–29s).
|
||||
|
||||
Encapsulated as a **sub-agent**: tens of screenshots live only in the sub-agent’s one-shot context and do not pollute Proposer/Reviewer history. Demo prints token comparison at the end.
|
||||
|
||||
### Proposer–Reviewer flow
|
||||
|
||||
```
|
||||
NL request ──► Proposer parses intent (scene + effects)
|
||||
│
|
||||
▼
|
||||
Video-analysis sub-agent two-step locate ──► [start, end]
|
||||
│
|
||||
▼
|
||||
Proposer emits Blender bpy script (edit.py) ──► render cut (+ subtitle/slow-mo)
|
||||
│ Blender if installed else ffmpeg
|
||||
▼
|
||||
Reviewer samples start/mid/end frames ──► Vision pass/fail + feedback
|
||||
│pass? no → Proposer adjusts bounds, re-cut (max 3 rounds)
|
||||
▼yes
|
||||
final.mp4
|
||||
```
|
||||
|
||||
### Run
|
||||
|
||||
```bash
|
||||
# From the repository root: use the shared Chapter 5 environment
|
||||
uv sync --locked --python 3.12 --extra ch5
|
||||
|
||||
# Activate it before changing directories:
|
||||
# macOS/Linux:
|
||||
source .venv/bin/activate
|
||||
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
|
||||
# Windows cmd: .venv\Scripts\activate.bat
|
||||
|
||||
# pip fallback when uv is not installed:
|
||||
# python -m pip install -e ".[ch5]"
|
||||
|
||||
cd chapter5/video-edit
|
||||
|
||||
# Single-project compatibility path, still supported during migration:
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
cp env.example .env # OPENAI_API_KEY (or OPENROUTER_API_KEY fallback)
|
||||
python demo.py # default: "把冲浪的部分剪出来" (full pipeline)
|
||||
python demo.py "把滑雪部分剪出来，并加上字幕 Winter" # custom request
|
||||
python demo.py -i my.mp4 -o out.mp4 "把演讲开场剪出来" # own video + output
|
||||
python demo.py --backend blender # force Blender headless (Blender required)
|
||||
python demo.py --vision-model gpt-5.6-luna # also --text-model
|
||||
python demo.py --quick # coarse sampling + single review round (cheapest Vision)
|
||||
python demo.py --smoke # smoke: edit path + bpy script only; no API
|
||||
python demo.py --help
|
||||
```
|
||||
|
||||
Common flags (see `--help`): `--input/-i`, `--output/-o`, `--backend {auto,blender,ffmpeg}`, `--text-model` / `--vision-model`.
|
||||
|
||||
One command runs: generate/read video → two-step locate → bpy cut → review → final. Each run clears `output/` (idempotent). Full path calls Vision many times; use `--smoke` (zero API) or `--quick` first.
|
||||
|
||||
### Sample outputs
|
||||
|
||||
#### `--smoke` (zero API, reproducible)
|
||||
|
||||
Real output of `python demo.py --smoke` (no OpenAI key; needs ffmpeg):
|
||||
|
||||
```text
|
||||
==========================================================================
|
||||
冒烟自检 | 剪辑链路 + bpy 脚本生成，不调用任何 API
|
||||
==========================================================================
|
||||
[1/3] 生成测试视频 OK：output/source.mp4（场景真值={'hiking': (0, 15), 'surfing': (15, 30), 'skiing': (30, 42), 'cycling': (42, 54)}）
|
||||
[2/3] 抽帧 OK：output/frames/smoke.png
|
||||
[3/3] 剪辑+字幕 OK（后端=ffmpeg（未装 Blender，回退））：
|
||||
文件: smoke_cut.mp4
|
||||
时长: 5.03s
|
||||
容器: mov,mp4,m4a,3gp,3g2,mj2
|
||||
大小: 121.4 KB
|
||||
视频流: h264 1280x720 @ 30/1 fps
|
||||
音频流: aac 44100Hz 1ch
|
||||
|
||||
已生成 Proposer 的 Blender 脚本：output/edit.py
|
||||
（这正是书中'生成 Blender Python API 代码'的产物；装好 Blender 后可直接
|
||||
`blender --background --python output/edit.py` 无头渲染。）
|
||||
|
||||
✓ 冒烟自检通过：剪辑链路正常 + bpy 脚本已生成（未调用 OpenAI）。
|
||||
```
|
||||
|
||||
`output/edit.py` is an executable Blender bpy script (`new_movie`, `frame_offset_start` / `frame_final_duration`, `new_effect(type='TEXT')`, `bpy.ops.render.render`); syntax-checked with `py_compile`.
|
||||
|
||||
#### `--quick` (full path, needs API)
|
||||
|
||||
Excerpt from `python demo.py --quick` (default surfing request); locate/error/token parts independent of bpy/ffmpeg:
|
||||
|
||||
```text
|
||||
步骤 1 | Proposer 解析自然语言需求
|
||||
解析结果：目标场景='surfing scene' 特效=[]
|
||||
|
||||
步骤 2 | 视频分析子 Agent：两步 Vision 定位（--quick 快速采样）
|
||||
[粗粒度] 每 15s 采样 5 帧 → Vision 得区间 [15, 30]s（依据：The word 'SURFING' appears at t=15s and changes at t=30s.）
|
||||
[细粒度] 窗口 [0.0, 45.0] 内每 2s 采样 23 帧 → 精确边界 [16.0, 28.0]s
|
||||
>>> 最终定位：起 16.0s 止 28.0s
|
||||
真值 [15, 30]s → 起点误差 1.0s，终点误差 2.0s（验收要求 ≤ 3s）
|
||||
|
||||
步骤 3-4 | Proposer 剪辑 + Reviewer 审查（迭代）
|
||||
Proposer 剪出片段 [16.0, 28.0]s，成片时长 12.0s
|
||||
Reviewer：pass=... score=... 检查帧=['0.5', '6.0', '11.5']
|
||||
|
||||
Token 统计（子 Agent 隔离截图，主上下文不被污染）
|
||||
主 Agent（Proposer+Reviewer）：573 tokens
|
||||
子 Agent（两步定位截图） ：2934 tokens
|
||||
```
|
||||
|
||||
Artifacts under `output/`:
|
||||
|
||||
| File | Duration | Note |
|
||||
| --- | --- | --- |
|
||||
| `source.mp4` | 54.0s | Procedural 4-scene test source |
|
||||
| `edit_round1.py` | — | Proposer bpy script (portable) |
|
||||
| `cut_round1.mp4` | 12.0s | Round-1 candidate |
|
||||
| `final.mp4` | 12.0s | Chosen final (H.264 + AAC, 1280x720@30fps) |
|
||||
|
||||
Token stats: tens of screenshots (2934 tok) only in **sub-agent**; main history (~573 tok) almost unpolluted. (Synthetic test shows “SURFING” text, not real surfing—Reviewer may fail on content; real video avoids that.)
|
||||
|
||||
### Dependencies
|
||||
|
||||
- **ffmpeg / ffprobe** for cuts and frames. `brew install ffmpeg` (macOS) / `apt install ffmpeg` (Ubuntu). Validated on ffmpeg 8.0.
|
||||
- **OPENAI_API_KEY**: `gpt-5.6-luna` for vision locate/review and text planning (vision model must accept images); or `OPENROUTER_API_KEY` fallback via OpenRouter.
|
||||
|
||||
### Adapt / extend
|
||||
|
||||
#### Model / provider
|
||||
|
||||
All via env (`env.example`); no code change:
|
||||
|
||||
- `TEXT_MODEL` (default `gpt-5.6-luna`).
|
||||
- `VISION_MODEL` must support images (default `gpt-5.6-luna`).
|
||||
- `OPENAI_BASE_URL` + matching `OPENAI_API_KEY` for compatible endpoints.
|
||||
|
||||
```bash
|
||||
export OPENAI_BASE_URL=https://your-gateway.example.com/v1
|
||||
export VISION_MODEL=gpt-5.6-luna
|
||||
export TEXT_MODEL=gpt-5.6-luna
|
||||
```
|
||||
|
||||
`agents.py` `OpenAI()` client reads these lazily via `client()`.
|
||||
|
||||
#### Own input video
|
||||
|
||||
`make_test_video.py` builds a 54s video with 4 distinct scenes (HIKING green / SURFING blue / SKIING white / CYCLING orange) plus large scene-name and timecode watermarks for Vision reproducibility.
|
||||
|
||||
Own video: `python demo.py -i your.mp4 -o out.mp4 "edit request"` (skips test generation; no ground-truth error print).
|
||||
|
||||
#### Blender vs ffmpeg (edit backends)
|
||||
|
||||
Book uses **Blender Python API (bpy)** on the VSE. First-class here: `blender_editor.generate_bpy_script()` emits real bpy (`new_movie`, frame offsets, `TEXT`/`SPEED` effects, render); `render_with_blender()` runs `blender --background --python edit.py`.
|
||||
|
||||
- `--backend blender`: force Blender (`blender --version` required);
|
||||
- `--backend ffmpeg`: force ffmpeg;
|
||||
- `--backend auto` (default): bpy if Blender installed, else ffmpeg.
|
||||
|
||||
**Always**: Proposer’s bpy script is written to `output/edit_round*.py` (`output/edit.py` under `--smoke`)—core “generate Blender Python API code” artifact. This environment may not have Blender, so render is validated with ffmpeg; bpy is `py_compile`-checked but **not rendered in real Blender** unless you install it and use `--backend blender`.
|
||||
|
||||
| | ffmpeg | Blender (bpy) |
|
||||
| --- | --- | --- |
|
||||
| Fit | 2D cut/join/subtitle/speed pipelines | 3D scenes, compositing, keyframe anim, particles/camera |
|
||||
| Ops | single binary, no GUI, CI-friendly | full Blender install; larger/slower |
|
||||
| When | most “cut a clip + simple effects” | only when 3D/compositing/complex transitions needed |
|
||||
|
||||
Two-step Vision + Proposer–Reviewer is decoupled from the execution layer; same edit plan for both backends.
|
||||
|
||||
### Files
|
||||
|
||||
| File | Role |
|
||||
| --- | --- |
|
||||
| `demo.py` | CLI orchestration, self-check, iteration, token stats |
|
||||
| `agents.py` | `VideoAnalyzerAgent` / `ProposerAgent` / `ReviewerAgent` |
|
||||
| `blender_editor.py` | **bpy script gen + headless render** (book path) |
|
||||
| `video_editor.py` | `apply_edit()`; Blender/ffmpeg backends |
|
||||
| `make_test_video.py` | Procedural 4-scene test video |
|
||||
| `ffmpeg_utils.py` | Thin ffmpeg/ffprobe wrappers |
|
||||
|
||||
`output/` is gitignored.
|
||||
|
||||
### Limitations
|
||||
|
||||
- Locate accuracy depends on visual distinctness; gradual scene transitions increase boundary error vs solid-color test film.
|
||||
- Fine step fixed at 1s → boundary precision about ±1s (book acceptance ±3s).
|
||||
- Slow-mo audio uses `atempo`; extreme ratios hurt quality; complex transitions/multi-track not covered.
|
||||
- Reviewer samples only start/mid/end; mid-clip glitches can be missed (raise sample density if needed).
|
||||
|
||||
---
|
||||
|
||||
## 中文
|
||||
|
||||
### 目的
|
||||
|
||||
用户给一段含多个场景的视频 + 一句自然语言需求（如"把冲浪部分剪出来"），Agent 自动定位目标场景、**生成 Blender Python API 脚本**剪出片段并自我审查。
|
||||
|
||||
验证三个核心机制在多媒体处理中的作用：
|
||||
|
||||
1. **两步 Vision 定位**：Proposer 无法直接"看懂"视频，于是委托一个**视频分析子 Agent**，
|
||||
用 ffmpeg 抽帧 + Vision LLM 读图来定位目标场景的时间边界。
|
||||
2. **代码生成（Blender Python API）**：Proposer 把剪辑计划翻译成一段调用 **Blender
|
||||
Python API（bpy）** 的脚本——导入 / 裁剪 / 字幕 / 变速 / 渲染各对应一个 API 调用，
|
||||
用 `blender --background --python edit.py` 无头执行。这正是书中"把视频编辑重构为
|
||||
API 调用和代码生成问题"的落地。未装 Blender 时脚本照常生成（代码生成产物），
|
||||
实际渲染回退到 ffmpeg（见下文"剪辑后端"）。
|
||||
3. **提议者-审核者（Proposer / Reviewer）**：Proposer 剪辑后无法自证效果，
|
||||
由 Reviewer 抽取成片关键帧、用 Vision LLM 检查是否剪对，不合格则反馈、迭代。
|
||||
|
||||
### 两步定位原理
|
||||
|
||||
Vision LLM 逐帧扫全片既慢又贵，因此采用"先粗后细"：
|
||||
|
||||
- **第一步（粗粒度）**：每 **10 秒**抽一帧，把全片的稀疏截图连同"要找哪个场景"一起
|
||||
交给 Vision，得到大致区间（如"冲浪在 20–30s"）。
|
||||
- **第二步（细粒度）**：在粗区间上下各外扩一个粗间隔，每 **1 秒**抽一帧，
|
||||
再问 Vision 精确边界（如"15–29s"）。
|
||||
|
||||
把这套抽帧-读图封装成**独立子 Agent**：几十张截图只进入子 Agent 的一次性上下文，
|
||||
不会污染主 Agent（Proposer/Reviewer）的对话历史。demo 末尾会打印两者的 token 对比。
|
||||
|
||||
### 提议者-审核者
|
||||
|
||||
```
|
||||
NL 需求 ──► Proposer 解析意图（目标场景 + 特效）
|
||||
│
|
||||
▼
|
||||
视频分析子 Agent 两步定位 ──► [start, end]
|
||||
│
|
||||
▼
|
||||
Proposer 生成 Blender bpy 脚本（edit.py）──► 渲染剪辑（可加字幕/慢动作）
|
||||
│ 装了 Blender 用 bpy，否则回退 ffmpeg
|
||||
▼
|
||||
Reviewer 抽首/中/尾关键帧 ──► Vision 检查 pass/fail + 反馈
|
||||
│pass? 否 → Proposer 据反馈修正边界，重剪（最多 3 轮）
|
||||
▼是
|
||||
输出成片 final.mp4
|
||||
```
|
||||
|
||||
### 运行
|
||||
|
||||
```bash
|
||||
# 在仓库根目录使用统一的第 5 章环境
|
||||
uv sync --locked --python 3.12 --extra ch5
|
||||
|
||||
# 切换目录前先激活环境：
|
||||
# macOS/Linux：
|
||||
source .venv/bin/activate
|
||||
# Windows PowerShell：.\.venv\Scripts\Activate.ps1
|
||||
# Windows cmd：.venv\Scripts\activate.bat
|
||||
|
||||
# 未安装 uv 时可用 pip 兜底：
|
||||
# python -m pip install -e ".[ch5]"
|
||||
|
||||
cd chapter5/video-edit
|
||||
|
||||
# 迁移期间仍支持单项目兼容路径：
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
cp env.example .env # 填入 OPENAI_API_KEY（未配置时设 OPENROUTER_API_KEY 自动改走 OpenRouter）
|
||||
python demo.py # 默认需求"把冲浪的部分剪出来"（完整流程）
|
||||
python demo.py "把滑雪部分剪出来，并加上字幕 Winter" # 自定义需求
|
||||
python demo.py -i my.mp4 -o out.mp4 "把演讲开场剪出来" # 用自己的视频 + 自定义输出
|
||||
python demo.py --backend blender # 强制用 Blender Python API 无头渲染（需装 Blender）
|
||||
python demo.py --vision-model gpt-5.6-luna # 覆盖模型（也可用 --text-model）
|
||||
python demo.py --quick # 快速模式：粗采样 + 单轮审查，Vision 调用最少（省时省钱）
|
||||
python demo.py --smoke # 冒烟自检：仅剪辑链路 + 生成 bpy 脚本，不调用任何 API
|
||||
python demo.py --help # 查看全部参数
|
||||
```
|
||||
|
||||
常用参数（完整见 `--help`）：`--input/-i` 输入视频、`--output/-o` 成片路径、
|
||||
`--backend {auto,blender,ffmpeg}` 剪辑后端、`--text-model`/`--vision-model` 覆盖模型。
|
||||
|
||||
一条命令即可跑通：生成/读取视频 → 两步定位 → 生成 bpy 脚本剪辑 → 审查 → 输出成片。
|
||||
每次运行都会清空 `output/`，从干净状态开始（幂等可重复）。完整流程会多次调用
|
||||
Vision 模型（较慢/耗费额度）；只想验证链路时先跑 `--smoke`（零 API），或用 `--quick`。
|
||||
|
||||
### 预期输出示例
|
||||
|
||||
#### `--smoke`（零 API，可复现）
|
||||
|
||||
以下为 `python demo.py --smoke` 的**真实输出**（无需 OpenAI Key，仅需 ffmpeg）：
|
||||
|
||||
```text
|
||||
==========================================================================
|
||||
冒烟自检 | 剪辑链路 + bpy 脚本生成，不调用任何 API
|
||||
==========================================================================
|
||||
[1/3] 生成测试视频 OK：output/source.mp4（场景真值={'hiking': (0, 15), 'surfing': (15, 30), 'skiing': (30, 42), 'cycling': (42, 54)}）
|
||||
[2/3] 抽帧 OK：output/frames/smoke.png
|
||||
[3/3] 剪辑+字幕 OK（后端=ffmpeg（未装 Blender，回退））：
|
||||
文件: smoke_cut.mp4
|
||||
时长: 5.03s
|
||||
容器: mov,mp4,m4a,3gp,3g2,mj2
|
||||
大小: 121.4 KB
|
||||
视频流: h264 1280x720 @ 30/1 fps
|
||||
音频流: aac 44100Hz 1ch
|
||||
|
||||
已生成 Proposer 的 Blender 脚本：output/edit.py
|
||||
（这正是书中'生成 Blender Python API 代码'的产物；装好 Blender 后可直接
|
||||
`blender --background --python output/edit.py` 无头渲染。）
|
||||
|
||||
✓ 冒烟自检通过：剪辑链路正常 + bpy 脚本已生成（未调用 OpenAI）。
|
||||
```
|
||||
|
||||
生成的 `output/edit.py` 是一段**可执行的 Blender bpy 脚本**（`new_movie` 导入、
|
||||
`frame_offset_start`/`frame_final_duration` 裁剪、`new_effect(type='TEXT')` 字幕、
|
||||
`bpy.ops.render.render` 渲染），本机对其做过 `py_compile` 语法校验。
|
||||
|
||||
#### `--quick`（完整链路，需 API）
|
||||
|
||||
以下为 `python demo.py --quick`（默认需求"把冲浪的部分剪出来"）的真实节选（定位/误差/
|
||||
token 部分与剪辑后端无关，故不受 bpy/ffmpeg 后端切换影响）：
|
||||
|
||||
```text
|
||||
步骤 1 | Proposer 解析自然语言需求
|
||||
解析结果：目标场景='surfing scene' 特效=[]
|
||||
|
||||
步骤 2 | 视频分析子 Agent：两步 Vision 定位（--quick 快速采样）
|
||||
[粗粒度] 每 15s 采样 5 帧 → Vision 得区间 [15, 30]s（依据：The word 'SURFING' appears at t=15s and changes at t=30s.）
|
||||
[细粒度] 窗口 [0.0, 45.0] 内每 2s 采样 23 帧 → 精确边界 [16.0, 28.0]s
|
||||
>>> 最终定位：起 16.0s 止 28.0s
|
||||
真值 [15, 30]s → 起点误差 1.0s，终点误差 2.0s（验收要求 ≤ 3s）
|
||||
|
||||
步骤 3-4 | Proposer 剪辑 + Reviewer 审查（迭代）
|
||||
Proposer 剪出片段 [16.0, 28.0]s，成片时长 12.0s
|
||||
Reviewer：pass=... score=... 检查帧=['0.5', '6.0', '11.5']
|
||||
|
||||
Token 统计（子 Agent 隔离截图，主上下文不被污染）
|
||||
主 Agent（Proposer+Reviewer）：573 tokens
|
||||
子 Agent（两步定位截图） ：2934 tokens
|
||||
```
|
||||
|
||||
产物（`output/` 目录，真实文件）：
|
||||
|
||||
| 文件 | 时长 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `source.mp4` | 54.0s | 程序化生成的 4 场景测试原片 |
|
||||
| `edit_round1.py` | — | Proposer 生成的 Blender bpy 脚本（代码生成产物，可换机执行） |
|
||||
| `cut_round1.mp4` | 12.0s | 第 1 轮剪出的候选片段 |
|
||||
| `final.mp4` | 12.0s | 采用的成片（H.264 + AAC，1280x720@30fps） |
|
||||
|
||||
Token 统计印证了核心结论：几十张截图（2934 tokens）只进入**子 Agent**的一次性
|
||||
上下文，主 Agent 的对话历史（573 tokens）几乎不受截图污染。
|
||||
（注：合成测试片仅显示"SURFING"字样而非真实冲浪画面，Reviewer 有时会据此判为
|
||||
不通过——这正是审核者按画面内容如实反馈的体现；换真实视频即无此现象。）
|
||||
|
||||
### 依赖
|
||||
|
||||
- **ffmpeg / ffprobe**：本机实际剪辑与抽帧。`brew install ffmpeg`（macOS）/
|
||||
`apt install ffmpeg`（Ubuntu）。本项目在 ffmpeg 8.0 上验证通过。
|
||||
- **OPENAI_API_KEY**：用 `gpt-5.6-luna` 做视觉定位/审查与文本规划（视觉模型须支持图像输入）；未配置时用 `OPENROUTER_API_KEY` 兜底，自动改走 OpenRouter。
|
||||
|
||||
### 如何适配 / 扩展
|
||||
|
||||
#### 换模型 / 供应商
|
||||
|
||||
模型与端点全部通过**环境变量**注入（见 `env.example`），无需改代码：
|
||||
|
||||
- `TEXT_MODEL`：规划/边界修正的文本模型（默认 `gpt-5.6-luna`）。
|
||||
- `VISION_MODEL`：定位/审查的视觉模型，**必须支持图像输入**（默认 `gpt-5.6-luna`）。
|
||||
- `OPENAI_BASE_URL`：换成任何兼容 OpenAI 协议的端点（自建代理、Azure OpenAI、
|
||||
或其他厂商网关），配合对应的 `OPENAI_API_KEY` 即可。
|
||||
|
||||
```bash
|
||||
export OPENAI_BASE_URL=https://your-gateway.example.com/v1
|
||||
export VISION_MODEL=gpt-5.6-luna # 例：用当前廉价旗舰视觉模型
|
||||
export TEXT_MODEL=gpt-5.6-luna
|
||||
```
|
||||
|
||||
`agents.py` 里的 `OpenAI()` 客户端会自动读取上述变量（`client()` 惰性初始化）。
|
||||
|
||||
#### 换输入视频
|
||||
|
||||
`make_test_video.py` 用 ffmpeg **程序化生成**一段 54s 的视频，含 4 个明显不同的场景
|
||||
（HIKING 绿 / SURFING 蓝 / SKIING 白 / CYCLING 橙），每段都叠加大号场景名与时间码水印，
|
||||
让 Vision 仅凭画面就能准确定位——便于复现验收。
|
||||
|
||||
换成**你自己的真实视频**：直接 `python demo.py -i 你的.mp4 -o 输出.mp4 "剪辑需求"`
|
||||
即可（无需改代码）。此时跳过测试片生成，也不再打印定位误差（外部视频无真值）。
|
||||
|
||||
#### Blender vs. ffmpeg（剪辑后端）
|
||||
|
||||
书中原方案用 **Blender Python API（bpy）** 驱动视频序列编辑器（VSE）完成剪辑。
|
||||
本项目把它实现为**一等后端**：`blender_editor.generate_bpy_script()` 把剪辑计划翻译成
|
||||
一段真实可执行的 bpy 脚本（`new_movie` 导入、`frame_offset_start`/`frame_final_duration`
|
||||
裁剪、`new_effect(type='TEXT'/'SPEED')` 字幕/变速、`bpy.ops.render.render` 渲染），
|
||||
`render_with_blender()` 再用 `blender --background --python edit.py` 无头执行。
|
||||
|
||||
- `--backend blender`：强制走 Blender（需 `blender --version` 可用）；
|
||||
- `--backend ffmpeg`：强制走 ffmpeg；
|
||||
- `--backend auto`（默认）：装了 Blender 用 bpy，否则回退 ffmpeg。
|
||||
|
||||
**关键点：无论哪个后端，Proposer 生成的 bpy 脚本都会落盘到 `output/edit_round*.py`**
|
||||
（`--smoke` 下为 `output/edit.py`）——即"生成 Blender Python API 代码"这一核心产物，
|
||||
可人工核对、也可拷到装了 Blender 的机器上执行。本机未安装 Blender，故本仓库的实际
|
||||
渲染由 ffmpeg 完成并验证；bpy 脚本已通过 `py_compile` 语法校验，但**未在真实 Blender
|
||||
上跑过渲染**（装好 Blender 后即可用 `--backend blender` 端到端执行）。两种后端的取舍：
|
||||
|
||||
| | ffmpeg | Blender（bpy） |
|
||||
| --- | --- | --- |
|
||||
| 定位 | 裁剪/拼接/字幕/变速等 2D 流水线 | 3D 场景、合成、关键帧动画、粒子/摄像机 |
|
||||
| 上手 | 单二进制、无 GUI、CI 友好 | 需装完整 Blender，体积大、渲染慢 |
|
||||
| 适用 | 绝大多数"剪一段 + 简单特效"需求 | 需要 3D 合成/复杂转场/图层混合时才值得 |
|
||||
|
||||
核心的"两步 Vision 定位 + 提议者-审核者"与执行层解耦，两个后端共用同一份剪辑计划，
|
||||
`agents.py`/`demo.py` 无需为切换后端改动逻辑。
|
||||
|
||||
### 文件
|
||||
|
||||
| 文件 | 作用 |
|
||||
| --- | --- |
|
||||
| `demo.py` | 一条命令跑通的编排入口（CLI、启动自检、迭代循环、token 统计） |
|
||||
| `agents.py` | `VideoAnalyzerAgent`（两步定位）/ `ProposerAgent` / `ReviewerAgent` |
|
||||
| `blender_editor.py` | **Blender bpy 脚本生成 + 无头渲染**（书中原方案，核心实验点） |
|
||||
| `video_editor.py` | 剪辑执行层：`apply_edit()` 统一入口，调度 Blender/ffmpeg 双后端 |
|
||||
| `make_test_video.py` | 程序化生成含 4 个场景的测试视频 |
|
||||
| `ffmpeg_utils.py` | ffmpeg/ffprobe 薄封装（统一错误检查、抽帧、探测时长/流） |
|
||||
|
||||
`output/`（生成的视频、截图、成片）已被 `.gitignore` 忽略，避免仓库膨胀。
|
||||
|
||||
### 局限
|
||||
|
||||
- 定位精度取决于场景在画面上的可辨识度；真实视频若场景过渡渐变，边界误差会大于纯色测试片。
|
||||
- 细粒度步长固定 1s，边界精度上限即 ±1s 量级（满足书中 ±3s 验收）。
|
||||
- 慢动作音频用 `atempo` 变速，倍率过大时音质下降；转场/多轨混音等复杂特效未覆盖。
|
||||
- Reviewer 仅抽首/中/尾三帧，长片段中段的偶发错误可能漏检（可调高抽帧密度）。
|
||||
|
||||
---
|
||||
|
||||
## Notes / 说明
|
||||
|
||||
- Prefer `--smoke` then `--quick` before a full Vision run. / 完整 Vision 前先 `--smoke`，再 `--quick`。
|
||||
- Commands/code/paths/env vars are identical in both language sections. / 命令、代码、路径与环境变量在中英文两侧保持一致。
|
||||
@@ -0,0 +1,325 @@
|
||||
"""
|
||||
实验 5-6 的三个 Agent：
|
||||
|
||||
VideoAnalyzerAgent —— 视频分析子 Agent，用"两步 Vision 定位"找目标场景边界。
|
||||
ProposerAgent —— 把自然语言需求解析成剪辑计划，调用子 Agent 定位并执行剪辑。
|
||||
ReviewerAgent —— 抽取成片关键帧，用 Vision 检查是否剪对，给出结构化反馈。
|
||||
|
||||
把视频分析封装为独立子 Agent 的意义：大量截图只进入子 Agent 的一次性上下文，
|
||||
不会污染主 Agent（Proposer/Reviewer）的对话历史——见 demo.py 打印的 token 统计。
|
||||
"""
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
from ffmpeg_utils import extract_frame, probe_duration
|
||||
|
||||
TEXT_MODEL = os.getenv("TEXT_MODEL", "gpt-5.6-luna")
|
||||
VISION_MODEL = os.getenv("VISION_MODEL", "gpt-5.6-luna") # 必须支持图像输入
|
||||
|
||||
OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"
|
||||
|
||||
_client = None
|
||||
|
||||
|
||||
def map_model_to_openrouter(model: str) -> str:
|
||||
"""把直连模型名映射为 OpenRouter 上的 id（非可映射 id 统一兜底到当前廉价旗舰）。"""
|
||||
if not model or "/" in model:
|
||||
return model or "openai/gpt-5.6-luna"
|
||||
m = model.lower()
|
||||
if m.startswith(("gpt-", "o1", "o3", "o4")):
|
||||
return "openai/" + model
|
||||
if m.startswith("claude"):
|
||||
if "haiku" in m:
|
||||
return "anthropic/claude-haiku-4.5"
|
||||
if "sonnet" in m:
|
||||
return "anthropic/claude-sonnet-4.6"
|
||||
return "anthropic/claude-opus-4.8"
|
||||
if m.startswith("gemini"):
|
||||
return "google/" + model
|
||||
return "openai/gpt-5.6-luna"
|
||||
|
||||
|
||||
def _temp_for(model):
|
||||
"""推理模型（gpt-5 / o 系列等）不接受 temperature=0。"""
|
||||
return (1 if any(k in (model or "").lower()
|
||||
for k in ("gpt-5", "o1", "o3", "o4", "thinking", "reasoner", "kimi-k3"))
|
||||
else 0)
|
||||
|
||||
|
||||
def client() -> OpenAI:
|
||||
"""构造（并缓存）OpenAI 客户端，含通用 OpenRouter 兜底。
|
||||
|
||||
- 有 OPENAI_API_KEY：直连；但默认模型 gpt-5.x（直连需组织实名认证）且设置了
|
||||
OPENROUTER_API_KEY 时优先走 OpenRouter。
|
||||
- 无 OPENAI_API_KEY 但有 OPENROUTER_API_KEY：改走 OpenRouter（模型名自动映射）。
|
||||
"""
|
||||
global _client, TEXT_MODEL, VISION_MODEL
|
||||
if _client is None:
|
||||
api_key = os.getenv("OPENAI_API_KEY")
|
||||
base_url = os.getenv("OPENAI_BASE_URL")
|
||||
orkey = os.getenv("OPENROUTER_API_KEY")
|
||||
prefer_or = bool(orkey) and (
|
||||
(TEXT_MODEL or "").lower().startswith("gpt-5") or (VISION_MODEL or "").lower().startswith("gpt-5")
|
||||
)
|
||||
if prefer_or or (not api_key and orkey):
|
||||
api_key, base_url = orkey, OPENROUTER_BASE_URL
|
||||
TEXT_MODEL = map_model_to_openrouter(TEXT_MODEL)
|
||||
VISION_MODEL = map_model_to_openrouter(VISION_MODEL)
|
||||
kw = {}
|
||||
if api_key:
|
||||
kw["api_key"] = api_key
|
||||
if base_url:
|
||||
kw["base_url"] = base_url
|
||||
_client = OpenAI(**kw)
|
||||
return _client
|
||||
|
||||
|
||||
def _img_part(path: str) -> dict:
|
||||
with open(path, "rb") as f:
|
||||
b64 = base64.b64encode(f.read()).decode()
|
||||
return {"type": "image_url",
|
||||
"image_url": {"url": f"data:image/png;base64,{b64}", "detail": "low"}}
|
||||
|
||||
|
||||
def _extract_json(text: str) -> dict:
|
||||
"""从 LLM 回复里稳健地抠出第一个 JSON 对象。"""
|
||||
start = text.find("{")
|
||||
if start < 0:
|
||||
raise ValueError(f"未能从回复中解析 JSON：{text[:200]}")
|
||||
try:
|
||||
obj, _ = json.JSONDecoder().raw_decode(text, start)
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError(f"未能从回复中解析 JSON：{text[:200]}") from e
|
||||
if not isinstance(obj, dict):
|
||||
raise ValueError(f"未能从回复中解析 JSON：{text[:200]}")
|
||||
return obj
|
||||
|
||||
|
||||
def _num(value, default: float) -> float:
|
||||
"""把 LLM 返回的数值字段转成 float；字段缺失、为 null 或非法时回退 default。"""
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
class TokenMeter:
|
||||
"""累计 token，用于对比'子 Agent 隔离截图'带来的主上下文节省。"""
|
||||
|
||||
def __init__(self):
|
||||
self.prompt = 0
|
||||
self.completion = 0
|
||||
|
||||
def add(self, resp):
|
||||
u = getattr(resp, "usage", None)
|
||||
if u:
|
||||
self.prompt += u.prompt_tokens
|
||||
self.completion += u.completion_tokens
|
||||
|
||||
def total(self):
|
||||
return self.prompt + self.completion
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 视频分析子 Agent：两步 Vision 定位
|
||||
# --------------------------------------------------------------------------- #
|
||||
class VideoAnalyzerAgent:
|
||||
def __init__(self, meter: TokenMeter = None):
|
||||
self.meter = meter or TokenMeter()
|
||||
|
||||
def _vision_locate(self, video, timestamps, question, frame_dir):
|
||||
"""抽取给定时间点的帧，连同问题交给 Vision LLM，返回 {start,end}。"""
|
||||
content = [{
|
||||
"type": "text",
|
||||
"text": (
|
||||
f"下面是同一段视频在不同时间点的截图（每张图前标注了该帧的时间，单位秒）。\n"
|
||||
f"目标问题：{question}\n"
|
||||
f"请判断'目标场景'在视频中出现的时间区间。只依据画面内容判断。\n"
|
||||
f"严格输出 JSON：{{\"start\": <起点秒>, \"end\": <终点秒>, "
|
||||
f"\"reason\": \"<简要依据>\"}}。若所有截图都看不到目标场景，"
|
||||
f"令 start=end=-1。"
|
||||
),
|
||||
}]
|
||||
for t in timestamps:
|
||||
png = os.path.join(frame_dir, f"f_{t:.1f}.png")
|
||||
extract_frame(video, t, png)
|
||||
content.append({"type": "text", "text": f"[时间 t={t:.1f}s]"})
|
||||
content.append(_img_part(png))
|
||||
|
||||
resp = client().chat.completions.create(
|
||||
model=VISION_MODEL,
|
||||
messages=[{"role": "user", "content": content}],
|
||||
temperature=_temp_for(VISION_MODEL),
|
||||
max_tokens=300,
|
||||
)
|
||||
self.meter.add(resp)
|
||||
data = _extract_json(resp.choices[0].message.content)
|
||||
# 模型可能省略 start/end 或返回 null——按约定的 -1 哨兵处理，走兜底逻辑。
|
||||
return _num(data.get("start"), -1.0), _num(data.get("end"), -1.0), data.get("reason", "")
|
||||
|
||||
def locate(self, video, question, coarse_interval=10.0, fine_interval=1.0,
|
||||
frame_dir="output/frames"):
|
||||
"""
|
||||
两步定位：
|
||||
第一步（粗）：每 coarse_interval 秒一帧，Vision 给出大致场景区间。
|
||||
第二步（细）：在粗区间上下各扩一个粗间隔，每 fine_interval 秒一帧，
|
||||
Vision 精确定位边界。
|
||||
返回 (start, end, trace)。
|
||||
"""
|
||||
os.makedirs(frame_dir, exist_ok=True)
|
||||
duration = probe_duration(video)
|
||||
trace = {}
|
||||
|
||||
# ---- 第一步：粗粒度 ----
|
||||
coarse_ts = [t for t in _frange(0, duration, coarse_interval)]
|
||||
cs, ce, creason = self._vision_locate(video, coarse_ts, question, frame_dir)
|
||||
trace["coarse"] = {"timestamps": coarse_ts, "start": cs, "end": ce,
|
||||
"reason": creason}
|
||||
|
||||
if cs < 0 or ce < 0:
|
||||
# 兜底：粗定位失败——退化为全视频精扫（步长放大以控制成本）。
|
||||
trace["coarse_fallback"] = True
|
||||
step = max(fine_interval, duration / 20.0)
|
||||
scan_ts = list(_frange(0, duration, step))
|
||||
cs, ce, creason = self._vision_locate(video, scan_ts, question, frame_dir)
|
||||
trace["coarse"]["fallback_scan"] = {"start": cs, "end": ce}
|
||||
if cs < 0:
|
||||
raise RuntimeError(
|
||||
"Vision 定位失败：在整段视频里都没找到匹配'{}'的场景。\n"
|
||||
"请检查需求描述是否与视频内容相符，或更换视频。".format(question)
|
||||
)
|
||||
|
||||
# ---- 第二步：细粒度（在粗区间外扩一个粗间隔）----
|
||||
lo = max(0.0, cs - coarse_interval)
|
||||
hi = min(duration, ce + coarse_interval)
|
||||
fine_ts = list(_frange(lo, hi, fine_interval))
|
||||
fs, fe, freason = self._vision_locate(video, fine_ts, question, frame_dir)
|
||||
trace["fine"] = {"window": [lo, hi], "timestamps_count": len(fine_ts),
|
||||
"start": fs, "end": fe, "reason": freason}
|
||||
|
||||
if fs < 0 or fe < 0 or fe <= fs:
|
||||
# 兜底：细定位失败——采用粗定位结果，保证流程可继续。
|
||||
trace["fine_fallback"] = True
|
||||
fs, fe = cs, ce
|
||||
|
||||
# 收敛到视频范围内。
|
||||
fs = max(0.0, fs)
|
||||
fe = min(duration, fe)
|
||||
return fs, fe, trace
|
||||
|
||||
|
||||
def _frange(start, stop, step):
|
||||
"""浮点 range（含首、含接近末尾的采样点）。"""
|
||||
out = []
|
||||
t = start
|
||||
while t < stop - 1e-6:
|
||||
out.append(round(t, 3))
|
||||
t += step
|
||||
# 补一个接近末尾的采样点，确保末段场景被覆盖。
|
||||
last = round(max(start, stop - 0.5), 3)
|
||||
if not out or abs(out[-1] - last) > step / 2:
|
||||
out.append(last)
|
||||
return out
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Proposer Agent
|
||||
# --------------------------------------------------------------------------- #
|
||||
class ProposerAgent:
|
||||
def __init__(self, meter: TokenMeter = None):
|
||||
self.meter = meter or TokenMeter()
|
||||
|
||||
def parse_request(self, nl_request: str) -> dict:
|
||||
"""把自然语言需求解析成结构化意图：目标场景描述 + 特效列表。"""
|
||||
resp = client().chat.completions.create(
|
||||
model=TEXT_MODEL,
|
||||
temperature=_temp_for(TEXT_MODEL),
|
||||
max_tokens=400,
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": (
|
||||
"你是视频剪辑规划器。把用户的中文剪辑需求解析成 JSON。\n"
|
||||
"字段：\n"
|
||||
" target_query: 用于视觉定位的一句话描述（英文更利于匹配画面文字），"
|
||||
"说明要剪出哪个场景；\n"
|
||||
" effects: 特效数组，元素形如 "
|
||||
"{\"type\":\"subtitle\",\"text\":\"...\"} 或 "
|
||||
"{\"type\":\"slowmo\",\"factor\":2.0}，无特效则为 []。\n"
|
||||
f"用户需求：{nl_request}\n"
|
||||
"只输出 JSON。"
|
||||
),
|
||||
}],
|
||||
)
|
||||
self.meter.add(resp)
|
||||
return _extract_json(resp.choices[0].message.content)
|
||||
|
||||
def revise_bounds(self, start, end, feedback, duration):
|
||||
"""根据 Reviewer 反馈微调边界（保守外扩/内收）。"""
|
||||
resp = client().chat.completions.create(
|
||||
model=TEXT_MODEL,
|
||||
temperature=_temp_for(TEXT_MODEL),
|
||||
max_tokens=200,
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": (
|
||||
f"当前剪辑区间 start={start:.1f}s end={end:.1f}s，视频总长 {duration:.1f}s。\n"
|
||||
f"审核反馈：{feedback}\n"
|
||||
"请给出修正后的区间，输出 JSON {\"start\":..,\"end\":..}。"
|
||||
"若反馈指出包含了无关片段则内收，若指出遗漏内容则外扩，幅度 1~5 秒。"
|
||||
),
|
||||
}],
|
||||
)
|
||||
self.meter.add(resp)
|
||||
d = _extract_json(resp.choices[0].message.content)
|
||||
# 模型可能省略 start/end 或返回 null——缺失时维持当前区间不变。
|
||||
return max(0.0, _num(d.get("start"), start)), min(duration, _num(d.get("end"), end))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Reviewer Agent
|
||||
# --------------------------------------------------------------------------- #
|
||||
class ReviewerAgent:
|
||||
def __init__(self, meter: TokenMeter = None):
|
||||
self.meter = meter or TokenMeter()
|
||||
|
||||
def review(self, clip_path, target_query, frame_dir="output/review_frames"):
|
||||
"""
|
||||
抽取成片的首/中/尾关键帧，用 Vision 检查：
|
||||
- 是否完整包含目标场景（无遗漏）；
|
||||
- 是否夹带了无关场景（无多余）。
|
||||
返回结构化结果 {pass, score, feedback, frames_checked}。
|
||||
"""
|
||||
os.makedirs(frame_dir, exist_ok=True)
|
||||
dur = probe_duration(clip_path)
|
||||
# 取首/中/尾，并在首尾稍微内缩避开黑帧。
|
||||
keyts = [min(0.5, dur * 0.1), dur / 2.0, max(0.0, dur - 0.5)]
|
||||
|
||||
content = [{
|
||||
"type": "text",
|
||||
"text": (
|
||||
f"这是剪辑成片的几个关键帧（首/中/尾）。剪辑目标是：{target_query}。\n"
|
||||
"请检查：(1) 成片是否完整呈现了目标场景；(2) 是否夹带了不该出现的其他场景。\n"
|
||||
"严格输出 JSON：{\"pass\": true/false, \"score\": 0-10, "
|
||||
"\"feedback\": \"<发现的问题或确认无误>\"}。"
|
||||
),
|
||||
}]
|
||||
for t in keyts:
|
||||
png = os.path.join(frame_dir, f"r_{t:.1f}.png")
|
||||
extract_frame(clip_path, t, png)
|
||||
content.append({"type": "text", "text": f"[成片内 t={t:.1f}s]"})
|
||||
content.append(_img_part(png))
|
||||
|
||||
resp = client().chat.completions.create(
|
||||
model=VISION_MODEL,
|
||||
temperature=_temp_for(VISION_MODEL),
|
||||
max_tokens=300,
|
||||
messages=[{"role": "user", "content": content}],
|
||||
)
|
||||
self.meter.add(resp)
|
||||
data = _extract_json(resp.choices[0].message.content)
|
||||
data["frames_checked"] = keyts
|
||||
return data
|
||||
@@ -0,0 +1,210 @@
|
||||
"""
|
||||
Blender Python API（bpy）剪辑执行层 —— 实验 5-6 的核心。
|
||||
|
||||
书中方案强调"代码生成"：Proposer Agent 不去点 GUI，而是**生成一段调用
|
||||
Blender Python API 的脚本**，每个编辑操作（导入 / 裁剪 / 字幕 / 变速 / 渲染）
|
||||
对应一个清晰的函数调用，再用 `blender --background --python edit.py` 无头执行。
|
||||
|
||||
本模块两个出口：
|
||||
generate_bpy_script(source, plan, out_video) -> str
|
||||
纯字符串生成，**不依赖 bpy**，任何机器都能产出这段脚本（体现代码生成能力，
|
||||
可人工核对，或拷到装了 Blender 的机器上执行）。
|
||||
render_with_blender(source, plan, out_video, script_path) -> str
|
||||
若本机 `blender` 可执行，写出脚本并无头渲染产出成片；否则抛错由调用方回退。
|
||||
|
||||
裁剪基于 Blender 视频序列编辑器（VSE）：new_movie / new_sound 导入素材，
|
||||
frame_offset_start + frame_final_duration 完成裁剪，TEXT / SPEED 特效条叠加，
|
||||
FFMPEG(H.264+AAC) 容器渲染。API 面向 Blender 3.x / 4.x。
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
|
||||
def blender_available() -> bool:
|
||||
"""本机是否有 blender 可执行文件（决定 backend=auto 时走 Blender 还是 ffmpeg）。"""
|
||||
return shutil.which("blender") is not None
|
||||
|
||||
|
||||
# 生成的 bpy 脚本模板。占位符全部通过 repr() 注入，保证是合法的 Python 字面量。
|
||||
_BPY_TEMPLATE = '''"""
|
||||
本文件由 blender_editor.generate_bpy_script() 自动生成（实验 5-6）。
|
||||
执行：blender --background --python edit.py
|
||||
它把一条剪辑计划翻译成 Blender 视频序列编辑器（VSE）的 API 调用序列。
|
||||
"""
|
||||
import os
|
||||
import bpy
|
||||
|
||||
SRC = {src}
|
||||
OUT = {out}
|
||||
FPS = {fps}
|
||||
START = {start} # 目标片段起点（秒）
|
||||
END = {end} # 目标片段终点（秒）
|
||||
SUBTITLE = {subtitle} # None 或字幕文本
|
||||
SLOWMO = {slowmo} # None 或放慢倍率（factor>1 表示放慢 factor 倍）
|
||||
|
||||
scene = bpy.context.scene
|
||||
scene.render.fps = FPS
|
||||
scene.render.fps_base = 1.0
|
||||
|
||||
# 清掉可能存在的旧序列，保证幂等
|
||||
if scene.sequence_editor:
|
||||
bpy.ops.sequencer.select_all(action='SELECT')
|
||||
bpy.ops.sequencer.delete()
|
||||
se = scene.sequence_editor_create()
|
||||
|
||||
start_frame = int(round(START * FPS))
|
||||
dur_frames = max(1, int(round((END - START) * FPS)))
|
||||
|
||||
# 1) 导入影片 + 音轨（new_sound 在无音轨素材上会抛 RuntimeError，忽略即可）
|
||||
# frame_offset_start trims the strip's visible left edge as well as advancing
|
||||
# into the source. Start the raw strip earlier by the same amount so the
|
||||
# trimmed clip's final visible start remains at output frame 1.
|
||||
movie = se.sequences.new_movie(
|
||||
name="clip", filepath=SRC, channel=1, frame_start=1 - start_frame
|
||||
)
|
||||
try:
|
||||
sound = se.sequences.new_sound(
|
||||
name="audio", filepath=SRC, channel=2, frame_start=1 - start_frame
|
||||
)
|
||||
except RuntimeError:
|
||||
sound = None
|
||||
|
||||
# 2) 裁剪 [START, END]：偏移掉片头，再固定成片时长
|
||||
for strip in (movie, sound):
|
||||
if strip is None:
|
||||
continue
|
||||
strip.frame_offset_start = start_frame
|
||||
strip.frame_final_duration = dur_frames
|
||||
|
||||
top_channel = 3
|
||||
|
||||
# 3) 慢动作：SPEED 特效条（MULTIPLY 模式，speed_factor = 1/倍率）
|
||||
if SLOWMO:
|
||||
speed = se.sequences.new_effect(
|
||||
name="slowmo", type='SPEED', channel=top_channel,
|
||||
frame_start=1, frame_end=1 + dur_frames, seq1=movie,
|
||||
)
|
||||
speed.use_default_fade = False
|
||||
speed.speed_control = 'MULTIPLY'
|
||||
speed.speed_factor = 1.0 / SLOWMO
|
||||
top_channel += 1
|
||||
# 放慢后成片总帧数按倍率拉长
|
||||
render_dur = int(round(dur_frames * SLOWMO))
|
||||
movie.frame_final_duration = render_dur
|
||||
else:
|
||||
render_dur = dur_frames
|
||||
|
||||
# 4) 字幕：TEXT 特效条，底部居中带半透明底框
|
||||
if SUBTITLE:
|
||||
txt = se.sequences.new_effect(
|
||||
name="subtitle", type='TEXT', channel=top_channel,
|
||||
frame_start=1, frame_end=1 + render_dur,
|
||||
)
|
||||
txt.text = SUBTITLE
|
||||
txt.font_size = 100
|
||||
txt.location = (0.5, 0.12)
|
||||
txt.align_x = 'CENTER'
|
||||
txt.align_y = 'BOTTOM'
|
||||
txt.use_box = True
|
||||
txt.box_color = (0.0, 0.0, 0.0, 0.6)
|
||||
|
||||
# 5) 渲染范围 + 输出为 mp4(H.264+AAC)
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = render_dur
|
||||
|
||||
r = scene.render
|
||||
r.image_settings.file_format = 'FFMPEG'
|
||||
r.ffmpeg.format = 'MPEG4'
|
||||
r.ffmpeg.codec = 'H264'
|
||||
r.ffmpeg.audio_codec = 'AAC'
|
||||
r.filepath = OUT
|
||||
os.makedirs(os.path.dirname(OUT) or ".", exist_ok=True)
|
||||
|
||||
bpy.ops.render.render(animation=True)
|
||||
print("BLENDER_RENDER_DONE", OUT)
|
||||
'''
|
||||
|
||||
|
||||
def _plan_fields(plan: dict):
|
||||
"""从剪辑计划里抽出 bpy 脚本需要的字段。"""
|
||||
start, end = float(plan["start"]), float(plan["end"])
|
||||
if end <= start:
|
||||
raise ValueError(f"剪辑区间非法：start={start} >= end={end}")
|
||||
effects = plan.get("effects", []) or []
|
||||
subtitle = None
|
||||
slowmo = None
|
||||
for eff in effects:
|
||||
etype = eff.get("type")
|
||||
if etype == "subtitle":
|
||||
subtitle = eff.get("text", "")
|
||||
elif etype == "slowmo":
|
||||
# Skip null factor like non-positive.
|
||||
raw = eff.get("factor", 2.0)
|
||||
if raw is None:
|
||||
continue
|
||||
factor = float(raw)
|
||||
if factor <= 0:
|
||||
continue
|
||||
slowmo = factor
|
||||
return start, end, subtitle, slowmo
|
||||
|
||||
|
||||
def generate_bpy_script(source: str, plan: dict, out_video: str, fps: int = 30) -> str:
|
||||
"""把剪辑计划渲染成一段可执行的 Blender Python(bpy) 脚本文本（不依赖 bpy）。"""
|
||||
start, end, subtitle, slowmo = _plan_fields(plan)
|
||||
return _BPY_TEMPLATE.format(
|
||||
src=repr(os.path.abspath(source)),
|
||||
out=repr(os.path.abspath(out_video)),
|
||||
fps=int(fps),
|
||||
start=repr(start),
|
||||
end=repr(end),
|
||||
subtitle=repr(subtitle),
|
||||
slowmo=repr(slowmo),
|
||||
)
|
||||
|
||||
|
||||
def write_bpy_script(source: str, plan: dict, out_video: str,
|
||||
script_path: str, fps: int = 30) -> str:
|
||||
"""生成 bpy 脚本并落盘（无论用哪个后端都会产出，作为代码生成产物）。"""
|
||||
script = generate_bpy_script(source, plan, out_video, fps=fps)
|
||||
os.makedirs(os.path.dirname(script_path) or ".", exist_ok=True)
|
||||
with open(script_path, "w") as f:
|
||||
f.write(script)
|
||||
return script_path
|
||||
|
||||
|
||||
def render_with_blender(source: str, plan: dict, out_video: str,
|
||||
script_path: str, fps: int = 30) -> str:
|
||||
"""写出 bpy 脚本并用 `blender --background --python` 无头执行，产出成片。"""
|
||||
if not blender_available():
|
||||
raise RuntimeError(
|
||||
"指定用 Blender 后端，但未找到 blender 可执行文件。\n"
|
||||
" 安装后确保 `blender --version` 可用：https://www.blender.org/download/\n"
|
||||
" 或改用 --backend ffmpeg。"
|
||||
)
|
||||
write_bpy_script(source, plan, out_video, script_path, fps=fps)
|
||||
os.makedirs(os.path.dirname(out_video) or ".", exist_ok=True)
|
||||
proc = subprocess.run(
|
||||
["blender", "--background", "--python", script_path],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
if proc.returncode != 0 or not os.path.exists(out_video):
|
||||
tail = "\n".join(proc.stderr.strip().splitlines()[-12:])
|
||||
raise RuntimeError(
|
||||
f"Blender 渲染失败（exit={proc.returncode}）：\n{tail}\n"
|
||||
f" 可人工检查生成的脚本：{script_path}"
|
||||
)
|
||||
return out_video
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# 零依赖自检：打印一段"含裁剪 + 字幕"的示例 bpy 脚本（可 py_compile 校验其语法）。
|
||||
demo_plan = {
|
||||
"start": 16.0,
|
||||
"end": 28.0,
|
||||
"effects": [{"type": "subtitle", "text": "SURFING"}],
|
||||
}
|
||||
print("# blender available:", blender_available())
|
||||
print("# ---- generated edit.py ----")
|
||||
print(generate_bpy_script("output/source.mp4", demo_plan, "output/final.mp4"))
|
||||
@@ -0,0 +1,792 @@
|
||||
"""Real-media, real-Vision, real-Blender acceptance campaign for Experiment 5-6."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
VALIDATION = ROOT / "validation"
|
||||
SOURCE_CACHE = VALIDATION / "source_cache" / "big-buck-bunny-trailer-480p.mov"
|
||||
SOURCE_URL = "https://download.blender.org/peach/trailer/trailer_480p.mov"
|
||||
SOURCE_SHA256 = "36801b74638c12be9aa587e93cd18edfc9bc51a1c089ab2a19ee42beed9f497d"
|
||||
BLENDER = Path("/Applications/Blender.app/Contents/MacOS/Blender")
|
||||
GROUND_TRUTH = {"start": 9.08, "end": 11.20}
|
||||
TARGET = (
|
||||
"the single continuous shot showing the large white rabbit walking alone in a sunny green "
|
||||
"meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title"
|
||||
)
|
||||
REQUEST = (
|
||||
"Cut out the shot of the large white rabbit walking alone in the sunny meadow, slow it to "
|
||||
"1.5x duration, and add the subtitle BIG BUNNY along the bottom."
|
||||
)
|
||||
|
||||
|
||||
def _utc() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def _sha(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def _write_json(path: Path, value: Any) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
text = json.dumps(value, ensure_ascii=False, indent=2) + "\n"
|
||||
if re.search(r"\b(?:sk|gh[opusr])-[A-Za-z0-9_-]{12,}\b", text):
|
||||
raise ValueError(f"credential-shaped value in {path}")
|
||||
path.write_text(text, encoding="utf-8")
|
||||
|
||||
|
||||
def _probe(path: Path) -> dict[str, Any]:
|
||||
process = subprocess.run(
|
||||
[
|
||||
"ffprobe", "-v", "error", "-show_entries",
|
||||
"format=duration,size,format_name:stream=index,codec_type,codec_name,width,height,r_frame_rate,sample_rate,channels",
|
||||
"-of", "json", str(path),
|
||||
],
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
return json.loads(process.stdout)
|
||||
|
||||
|
||||
def _extract(source: Path, timestamp: float, output: Path) -> None:
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg", "-hide_banner", "-loglevel", "error", "-ss", f"{timestamp:.3f}",
|
||||
"-i", str(source), "-frames:v", "1", "-vf", "scale=640:-2", "-y", str(output),
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
if not output.is_file():
|
||||
raise RuntimeError(f"ffmpeg produced no frame at {timestamp:.3f}s: {source}")
|
||||
|
||||
|
||||
def _frame_set(source: Path, timestamps: list[float], directory: Path) -> list[dict[str, Any]]:
|
||||
result = []
|
||||
for timestamp in timestamps:
|
||||
# PNG avoids the local ffmpeg build's strict MJPEG full-range rejection
|
||||
# on the trailer's final credits frame while preserving the exact pixels.
|
||||
path = directory / f"frame-{timestamp:06.2f}.png"
|
||||
_extract(source, timestamp, path)
|
||||
result.append(
|
||||
{
|
||||
"timestamp_s": timestamp,
|
||||
"path": str(path),
|
||||
"sha256": _sha(path),
|
||||
"bytes": path.stat().st_size,
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
class Backend:
|
||||
def __init__(self, provider: str):
|
||||
if provider == "ark":
|
||||
key = os.environ.get("ARK_API_KEY")
|
||||
self.endpoint = os.environ.get("ARK_BASE_URL") or "https://ark.cn-beijing.volces.com/api/v3"
|
||||
self.model = os.environ.get("ARK_MODEL") or "doubao-seed-1-6-250615"
|
||||
elif provider == "moonshot":
|
||||
key = os.environ.get("MOONSHOT_API_KEY") or os.environ.get("KIMI_API_KEY")
|
||||
self.endpoint = os.environ.get("MOONSHOT_BASE_URL") or "https://api.moonshot.cn/v1"
|
||||
self.model = os.environ.get("KIMI_MODEL") or "kimi-k3"
|
||||
else:
|
||||
key = os.environ.get("OPENAI_API_KEY")
|
||||
self.endpoint = os.environ.get("OPENAI_BASE_URL") or "https://api.openai.com/v1"
|
||||
self.model = os.environ.get("OPENAI_MODEL") or "gpt-5.6-luna"
|
||||
if not key:
|
||||
raise RuntimeError(f"missing credential for {provider}")
|
||||
self.provider = provider
|
||||
self.receipt_checkpoint: Path | None = None
|
||||
self.client = OpenAI(api_key=key, base_url=self.endpoint, timeout=240, max_retries=0)
|
||||
|
||||
|
||||
def _json_object(text: str) -> dict[str, Any]:
|
||||
text = text.strip()
|
||||
if text.startswith("```"):
|
||||
text = text.split("\n", 1)[1].rsplit("```", 1)[0]
|
||||
value = json.loads(text)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("model response must be an object")
|
||||
return value
|
||||
|
||||
|
||||
def _call(
|
||||
backend: Backend,
|
||||
*,
|
||||
purpose: str,
|
||||
messages: list[dict[str, Any]],
|
||||
receipt_messages: list[dict[str, Any]],
|
||||
receipts: list[dict[str, Any]],
|
||||
max_tokens: int,
|
||||
) -> dict[str, Any]:
|
||||
request: dict[str, Any] = {
|
||||
"model": backend.model,
|
||||
"messages": messages,
|
||||
"temperature": 0,
|
||||
"max_tokens": max_tokens,
|
||||
"response_format": {"type": "json_object"},
|
||||
}
|
||||
started = time.perf_counter()
|
||||
response = backend.client.chat.completions.create(**request)
|
||||
latency = round(time.perf_counter() - started, 3)
|
||||
content = response.choices[0].message.content or ""
|
||||
usage = response.usage
|
||||
receipts.append(
|
||||
{
|
||||
"purpose": purpose,
|
||||
"provider": backend.provider,
|
||||
"endpoint": backend.endpoint,
|
||||
"request": {
|
||||
"model": backend.model,
|
||||
"messages": receipt_messages,
|
||||
"temperature": 0,
|
||||
"max_tokens": max_tokens,
|
||||
"response_format": {"type": "json_object"},
|
||||
},
|
||||
"response": {
|
||||
"id": response.id,
|
||||
"model": response.model,
|
||||
"finish_reason": response.choices[0].finish_reason,
|
||||
"content": content,
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": getattr(usage, "prompt_tokens", None),
|
||||
"completion_tokens": getattr(usage, "completion_tokens", None),
|
||||
"total_tokens": getattr(usage, "total_tokens", None),
|
||||
},
|
||||
"latency_s": latency,
|
||||
}
|
||||
)
|
||||
if backend.receipt_checkpoint is not None:
|
||||
_write_json(backend.receipt_checkpoint, {"calls": receipts})
|
||||
return _json_object(content)
|
||||
|
||||
|
||||
def _vision(
|
||||
backend: Backend,
|
||||
*,
|
||||
purpose: str,
|
||||
prompt: str,
|
||||
frames: list[dict[str, Any]],
|
||||
receipts: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
content: list[dict[str, Any]] = [{"type": "text", "text": prompt}]
|
||||
summarized: list[dict[str, Any]] = [{"type": "text", "text": prompt}]
|
||||
for frame in frames:
|
||||
path = Path(frame["path"])
|
||||
content.append({"type": "text", "text": f"timestamp={frame['timestamp_s']:.2f}s"})
|
||||
content.append(
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "data:image/png;base64," + base64.b64encode(path.read_bytes()).decode("ascii"),
|
||||
"detail": "low",
|
||||
},
|
||||
}
|
||||
)
|
||||
summarized.append({"type": "text", "text": f"timestamp={frame['timestamp_s']:.2f}s"})
|
||||
summarized.append(
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": frame["path"],
|
||||
"sha256": frame["sha256"],
|
||||
"bytes": frame["bytes"],
|
||||
}
|
||||
)
|
||||
return _call(
|
||||
backend,
|
||||
purpose=purpose,
|
||||
messages=[{"role": "user", "content": content}],
|
||||
receipt_messages=[{"role": "user", "content": summarized}],
|
||||
receipts=receipts,
|
||||
max_tokens=700,
|
||||
)
|
||||
|
||||
|
||||
def _bounds(value: dict[str, Any], duration: float) -> tuple[float, float]:
|
||||
start = float(value["start"])
|
||||
end = float(value["end"])
|
||||
if not (0 <= start < end <= duration):
|
||||
raise ValueError(f"invalid model interval [{start}, {end}] for duration {duration}")
|
||||
return start, end
|
||||
|
||||
|
||||
def _script_is_safe(code: str, source: Path, output: Path) -> None:
|
||||
tree = ast.parse(code)
|
||||
imports = {
|
||||
alias.name
|
||||
for node in ast.walk(tree)
|
||||
if isinstance(node, (ast.Import, ast.ImportFrom))
|
||||
for alias in node.names
|
||||
}
|
||||
if "bpy" not in imports:
|
||||
raise ValueError("generated script does not import bpy")
|
||||
forbidden_imports = {"subprocess", "socket", "requests", "urllib", "http", "ftplib"}
|
||||
if imports & forbidden_imports:
|
||||
raise ValueError(f"generated script imports forbidden modules: {sorted(imports & forbidden_imports)}")
|
||||
forbidden_calls = {"eval", "exec", "compile", "__import__", "system", "popen", "remove", "unlink", "rmtree"}
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Call):
|
||||
name = ""
|
||||
if isinstance(node.func, ast.Name):
|
||||
name = node.func.id
|
||||
elif isinstance(node.func, ast.Attribute):
|
||||
name = node.func.attr
|
||||
if name.casefold() in forbidden_calls:
|
||||
raise ValueError(f"generated script calls forbidden function: {name}")
|
||||
if str(source.resolve()) not in code or str(output.resolve()) not in code:
|
||||
raise ValueError("generated script does not pin the supplied input and output paths")
|
||||
required_markers = ("new_movie", "frame_offset_start", "new_effect", "TEXT", "SPEED", "render")
|
||||
missing = [marker for marker in required_markers if marker not in code]
|
||||
if missing:
|
||||
raise ValueError(f"generated script omits requested Blender API operations: {missing}")
|
||||
|
||||
|
||||
def _render_model_script(
|
||||
backend: Backend,
|
||||
*,
|
||||
label: str,
|
||||
source: Path,
|
||||
output: Path,
|
||||
start: float,
|
||||
end: float,
|
||||
scripts: Path,
|
||||
logs: Path,
|
||||
receipts: list[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
prompt = f"""Generate a complete Blender 4.3 Python script for this video edit.
|
||||
|
||||
Input movie: {source.resolve()}
|
||||
Output MP4: {output.resolve()}
|
||||
Source FPS: 25
|
||||
Source size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)
|
||||
Trim interval: [{start:.3f}, {end:.3f}] seconds
|
||||
Effects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.
|
||||
|
||||
Requirements:
|
||||
- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.
|
||||
- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.
|
||||
- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).
|
||||
- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.
|
||||
- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).
|
||||
- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.
|
||||
- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.
|
||||
|
||||
Use this real-Blender-4.3-validated timing pattern, substituting the supplied paths and interval but preserving the timing relationships exactly:
|
||||
```
|
||||
FPS = 25
|
||||
start_frame = int(round(START_SECONDS * FPS))
|
||||
dur_frames = max(1, int(round((END_SECONDS - START_SECONDS) * FPS)))
|
||||
render_dur = int(round(dur_frames * 1.5))
|
||||
se = scene.sequence_editor_create()
|
||||
movie = se.sequences.new_movie(name='clip', filepath=INPUT_PATH, channel=1,
|
||||
frame_start=1 - start_frame)
|
||||
sound = se.sequences.new_sound(name='audio', filepath=INPUT_PATH, channel=2,
|
||||
frame_start=1 - start_frame)
|
||||
for strip in (movie, sound):
|
||||
strip.frame_offset_start = start_frame
|
||||
strip.frame_final_duration = dur_frames
|
||||
speed = se.sequences.new_effect(name='slowmo', type='SPEED', channel=3,
|
||||
frame_start=1, frame_end=1 + dur_frames, seq1=movie)
|
||||
speed.use_default_fade = False
|
||||
speed.speed_control = 'MULTIPLY'
|
||||
speed.speed_factor = 1.0 / 1.5
|
||||
movie.frame_final_duration = render_dur
|
||||
text = se.sequences.new_effect(name='subtitle', type='TEXT', channel=4,
|
||||
frame_start=1, frame_end=1 + render_dur)
|
||||
text.text = 'BIG BUNNY'; text.location = (0.5, 0.12)
|
||||
text.align_x = 'CENTER'; text.align_y = 'BOTTOM'
|
||||
text.use_box = True; text.box_color = (0.0, 0.0, 0.0, 0.6)
|
||||
scene.frame_start = 1; scene.frame_end = render_dur
|
||||
```
|
||||
This pattern was executed against Blender 4.3.2 and visibly retained the requested source frames through the whole slowed output. Do not replace its negative source-strip frame_start with frame_start=1: frame_offset_start would then move the visible strip later and render black frames.
|
||||
|
||||
Return one JSON object only: {{"code":"complete executable Python source"}}.
|
||||
"""
|
||||
feedback = ""
|
||||
attempts: list[dict[str, Any]] = []
|
||||
for attempt in range(1, 7):
|
||||
payload = _call(
|
||||
backend,
|
||||
purpose=f"blender_script_{label}_attempt_{attempt}",
|
||||
messages=[
|
||||
{"role": "system", "content": "You are a Blender VSE engineer. Return one JSON object only."},
|
||||
{"role": "user", "content": prompt + (f"\nPrior executable feedback:\n{feedback}" if feedback else "")},
|
||||
],
|
||||
receipt_messages=[
|
||||
{"role": "system", "content": "You are a Blender VSE engineer. Return one JSON object only."},
|
||||
{"role": "user", "content": prompt + (f"\nPrior executable feedback:\n{feedback}" if feedback else "")},
|
||||
],
|
||||
receipts=receipts,
|
||||
max_tokens=7000,
|
||||
)
|
||||
code = payload.get("code")
|
||||
script_path = scripts / f"{label}-attempt-{attempt}.py"
|
||||
if not isinstance(code, str):
|
||||
feedback = "Response lacked a string code field."
|
||||
attempts.append({"attempt": attempt, "accepted": False, "error": feedback})
|
||||
continue
|
||||
script_path.write_text(code, encoding="utf-8")
|
||||
try:
|
||||
_script_is_safe(code, source, output)
|
||||
except (SyntaxError, ValueError) as exc:
|
||||
feedback = f"Static safety/API validation failed: {type(exc).__name__}: {exc}"
|
||||
attempts.append({"attempt": attempt, "script": str(script_path), "accepted": False, "error": feedback})
|
||||
continue
|
||||
if output.exists():
|
||||
output.unlink()
|
||||
started = time.perf_counter()
|
||||
process = subprocess.run(
|
||||
[str(BLENDER), "--background", "--python", str(script_path)],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
timeout=300,
|
||||
)
|
||||
latency = round(time.perf_counter() - started, 3)
|
||||
log_path = logs / f"{label}-attempt-{attempt}.log"
|
||||
log_path.write_text(process.stdout, encoding="utf-8")
|
||||
accepted = (
|
||||
process.returncode == 0
|
||||
and "Traceback (most recent call last)" not in process.stdout
|
||||
and output.is_file()
|
||||
and output.stat().st_size > 1000
|
||||
)
|
||||
attempt_record = {
|
||||
"attempt": attempt,
|
||||
"script": str(script_path),
|
||||
"script_sha256": _sha(script_path),
|
||||
"blender_log": str(log_path),
|
||||
"blender_log_sha256": _sha(log_path),
|
||||
"blender_exit_code": process.returncode,
|
||||
"blender_latency_s": latency,
|
||||
"accepted": accepted,
|
||||
}
|
||||
if not accepted and output.is_file():
|
||||
partial_output = logs / f"{label}-attempt-{attempt}.partial.mp4"
|
||||
output.replace(partial_output)
|
||||
attempt_record["partial_output"] = str(partial_output)
|
||||
attempt_record["partial_output_sha256"] = _sha(partial_output)
|
||||
attempts.append(attempt_record)
|
||||
if accepted:
|
||||
return {"attempts": attempts, "accepted_script": str(script_path), "output": str(output)}
|
||||
feedback = (
|
||||
f"Blender exit={process.returncode}; output_exists={output.exists()}. "
|
||||
"Tail of real Blender log:\n" + "\n".join(process.stdout.splitlines()[-30:])
|
||||
)
|
||||
raise RuntimeError(f"model never generated an executable Blender script for {label}: {attempts}")
|
||||
|
||||
|
||||
def _review(
|
||||
backend: Backend,
|
||||
*,
|
||||
purpose: str,
|
||||
clip: Path,
|
||||
directory: Path,
|
||||
receipts: list[dict[str, Any]],
|
||||
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
duration = float(_probe(clip)["format"]["duration"])
|
||||
timestamps = sorted({round(min(0.35, duration / 5), 3), round(duration / 2, 3), round(max(0.0, duration - 0.35), 3)})
|
||||
frames = _frame_set(clip, timestamps, directory)
|
||||
prompt = f"""Independently review these actual rendered keyframes against this edit request:
|
||||
{REQUEST}
|
||||
|
||||
The target source shot is: {TARGET}.
|
||||
Judge the pixels, not filenames or claims. Return JSON exactly with:
|
||||
{{"pass":boolean,"target_present":boolean,"irrelevant_content_present":boolean,
|
||||
"subtitle_visible":boolean,"feedback":"specific evidence from the frames"}}.
|
||||
Pass only if all frames show the intended large white rabbit meadow shot without title cards or unrelated scenes and the BIG BUNNY subtitle is visibly overlaid near the bottom.
|
||||
"""
|
||||
result = _vision(backend, purpose=purpose, prompt=prompt, frames=frames, receipts=receipts)
|
||||
for key in ("pass", "target_present", "irrelevant_content_present", "subtitle_visible"):
|
||||
if not isinstance(result.get(key), bool):
|
||||
raise ValueError(f"review response lacks boolean {key}")
|
||||
return result, frames
|
||||
|
||||
|
||||
def _refine_boundaries(
|
||||
backend: Backend,
|
||||
*,
|
||||
source: Path,
|
||||
duration: float,
|
||||
start: float,
|
||||
end: float,
|
||||
reviewer_feedback: str,
|
||||
round_number: int,
|
||||
directory: Path,
|
||||
receipts: list[dict[str, Any]],
|
||||
) -> tuple[float, float, dict[str, Any], list[dict[str, Any]]]:
|
||||
"""Ask Vision to tighten a rejected edit using dense source-frame evidence.
|
||||
|
||||
Ground-truth annotations are deliberately not supplied here: the next edit
|
||||
must be driven by the independent review and actual source pixels.
|
||||
"""
|
||||
lo = max(0.0, start - 1.0)
|
||||
hi = min(duration, end + 1.0)
|
||||
first_tick = math.ceil(lo * 4)
|
||||
last_tick = math.floor(hi * 4)
|
||||
timestamps = [tick / 4 for tick in range(first_tick, last_tick + 1)]
|
||||
frames = _frame_set(source, timestamps, directory)
|
||||
prompt = f"""A rendered video edit was rejected by an independent pixel reviewer.
|
||||
Target shot: {TARGET}
|
||||
Prior attempted source interval: [{start:.3f}, {end:.3f}] seconds
|
||||
Reviewer feedback: {reviewer_feedback}
|
||||
|
||||
These are dense 0.25-second frames from the source around both attempted boundaries.
|
||||
Using only the timestamped pixels and the reviewer feedback, return the maximal safe
|
||||
continuous interval containing the intended rabbit-meadow shot while excluding every
|
||||
neighboring title card or unrelated shot. It is better to trim a fraction of a second
|
||||
of the target than include even one title-card frame.
|
||||
Return JSON exactly as {{"start":seconds,"end":seconds,"reason":"visual boundary evidence"}}.
|
||||
"""
|
||||
result = _vision(
|
||||
backend,
|
||||
purpose=f"reviewer_driven_quarter_second_boundary_refinement_{round_number}",
|
||||
prompt=prompt,
|
||||
frames=frames,
|
||||
receipts=receipts,
|
||||
)
|
||||
refined_start, refined_end = _bounds(result, duration)
|
||||
return refined_start, refined_end, result, frames
|
||||
|
||||
|
||||
def run(provider: str, run_id: str) -> dict[str, Any]:
|
||||
run_dir = VALIDATION / "runs" / run_id
|
||||
if run_dir.exists():
|
||||
raise FileExistsError(run_dir)
|
||||
run_dir.mkdir(parents=True)
|
||||
if not SOURCE_CACHE.is_file() or _sha(SOURCE_CACHE) != SOURCE_SHA256:
|
||||
raise RuntimeError("pinned source cache missing or hash mismatch")
|
||||
if not BLENDER.is_file():
|
||||
raise RuntimeError(f"real Blender binary not found: {BLENDER}")
|
||||
source = run_dir / "source.mov"
|
||||
shutil.copy2(SOURCE_CACHE, source)
|
||||
source_probe = _probe(source)
|
||||
duration = float(source_probe["format"]["duration"])
|
||||
backend = Backend(provider)
|
||||
receipts: list[dict[str, Any]] = []
|
||||
backend.receipt_checkpoint = run_dir / "provider_receipts.checkpoint.json"
|
||||
frames_dir = run_dir / "frames"
|
||||
scripts_dir = run_dir / "scripts"
|
||||
logs_dir = run_dir / "blender_logs"
|
||||
scripts_dir.mkdir()
|
||||
logs_dir.mkdir()
|
||||
|
||||
source_evidence = {
|
||||
"url": SOURCE_URL,
|
||||
"title": "Big Buck Bunny trailer (Blender Foundation Peach Open Movie Project)",
|
||||
"license": "Creative Commons Attribution 3.0",
|
||||
"license_url": "https://creativecommons.org/licenses/by/3.0/",
|
||||
"sha256": _sha(source),
|
||||
"probe": source_probe,
|
||||
"naturally_occurring_scenes": True,
|
||||
"ground_truth": {
|
||||
**GROUND_TRUTH,
|
||||
"target": TARGET,
|
||||
"method": "Human-labeled target shot bounded by ffmpeg scene-change frames; source inspection found transitions at 9.08 and 11.20 seconds.",
|
||||
"ffmpeg_scene_threshold": 0.35,
|
||||
},
|
||||
}
|
||||
_write_json(run_dir / "source_evidence.json", source_evidence)
|
||||
|
||||
# Container audio extends slightly beyond the last decodable video frame;
|
||||
# stay 1.5 s inside format duration for the final sparse sample.
|
||||
coarse_times = [0.0, 10.0, 20.0, 30.0, round(duration - 1.5, 3)]
|
||||
coarse_frames = _frame_set(source, coarse_times, frames_dir / "coarse")
|
||||
coarse_prompt = f"""These are frames from one contiguous real trailer sampled about every 10 seconds.
|
||||
Locate this target: {TARGET}.
|
||||
Return JSON {{"start":seconds,"end":seconds,"reason":"visual evidence"}} with a rough continuous interval. Use timestamps and visual content only."""
|
||||
coarse = _vision(
|
||||
backend,
|
||||
purpose="coarse_10_second_visual_localization",
|
||||
prompt=coarse_prompt,
|
||||
frames=coarse_frames,
|
||||
receipts=receipts,
|
||||
)
|
||||
coarse_start, coarse_end = _bounds(coarse, duration)
|
||||
midpoint = (coarse_start + coarse_end) / 2
|
||||
fine_lo = max(0, math.floor(midpoint - 10))
|
||||
fine_hi = min(duration, math.ceil(midpoint + 10))
|
||||
fine_times = [float(value) for value in range(int(fine_lo), int(math.floor(fine_hi)) + 1)]
|
||||
fine_frames = _frame_set(source, fine_times, frames_dir / "fine")
|
||||
fine_prompt = f"""These are one-second samples from the narrowed window [{fine_lo:.1f}, {fine_hi:.1f}] seconds of the same real trailer.
|
||||
Precisely locate the boundaries of this one continuous target shot: {TARGET}.
|
||||
Return JSON {{"start":seconds,"end":seconds,"reason":"visual boundary evidence"}}. The answer may interpolate between adjacent one-second samples; do not include either neighboring title card."""
|
||||
fine = _vision(
|
||||
backend,
|
||||
purpose="fine_1_second_visual_localization",
|
||||
prompt=fine_prompt,
|
||||
frames=fine_frames,
|
||||
receipts=receipts,
|
||||
)
|
||||
start, end = _bounds(fine, duration)
|
||||
localization = {
|
||||
"request": REQUEST,
|
||||
"target": TARGET,
|
||||
"coarse_interval_s": 10,
|
||||
"coarse_frames": coarse_frames,
|
||||
"coarse_result": coarse,
|
||||
"fine_interval_s": 1,
|
||||
"fine_window": [fine_lo, fine_hi],
|
||||
"fine_frames": fine_frames,
|
||||
"fine_result": fine,
|
||||
"ground_truth": GROUND_TRUTH,
|
||||
"start_error_s": round(abs(start - GROUND_TRUTH["start"]), 3),
|
||||
"end_error_s": round(abs(end - GROUND_TRUTH["end"]), 3),
|
||||
}
|
||||
_write_json(run_dir / "localization.json", localization)
|
||||
|
||||
negative_output = run_dir / "negative_control.mp4"
|
||||
negative_render = _render_model_script(
|
||||
backend,
|
||||
label="negative-control",
|
||||
source=source,
|
||||
output=negative_output,
|
||||
start=0.0,
|
||||
end=3.0,
|
||||
scripts=scripts_dir,
|
||||
logs=logs_dir,
|
||||
receipts=receipts,
|
||||
)
|
||||
negative_review, negative_frames = _review(
|
||||
backend,
|
||||
purpose="review_negative_control_rendered_pixels",
|
||||
clip=negative_output,
|
||||
directory=frames_dir / "negative-review",
|
||||
receipts=receipts,
|
||||
)
|
||||
correction_triggered = negative_review["pass"] is False
|
||||
if not correction_triggered:
|
||||
raise RuntimeError("independent reviewer failed to reject the obvious wrong-shot negative control")
|
||||
|
||||
final_output = run_dir / "final.mp4"
|
||||
final_render = _render_model_script(
|
||||
backend,
|
||||
label="corrected-final",
|
||||
source=source,
|
||||
output=final_output,
|
||||
start=start,
|
||||
end=end,
|
||||
scripts=scripts_dir,
|
||||
logs=logs_dir,
|
||||
receipts=receipts,
|
||||
)
|
||||
final_review, final_frames = _review(
|
||||
backend,
|
||||
purpose="review_corrected_final_rendered_pixels",
|
||||
clip=final_output,
|
||||
directory=frames_dir / "final-review",
|
||||
receipts=receipts,
|
||||
)
|
||||
correction_rounds: list[dict[str, Any]] = [
|
||||
{
|
||||
"round": 0,
|
||||
"kind": "initial_fine_localization",
|
||||
"plan": {"start": start, "end": end, "effects": ["slowmo:1.5", "subtitle:BIG BUNNY"]},
|
||||
"render": final_render,
|
||||
"review": final_review,
|
||||
"frames": final_frames,
|
||||
}
|
||||
]
|
||||
# A reviewer rejection is actionable evidence, not merely a failed gate.
|
||||
# Densely inspect the real source pixels, ask Vision to tighten the interval,
|
||||
# then generate and execute a fresh Blender script before reviewing again.
|
||||
for refinement_round in range(1, 4):
|
||||
if final_review["pass"]:
|
||||
break
|
||||
prior_start, prior_end = start, end
|
||||
start, end, refinement, refinement_frames = _refine_boundaries(
|
||||
backend,
|
||||
source=source,
|
||||
duration=duration,
|
||||
start=prior_start,
|
||||
end=prior_end,
|
||||
reviewer_feedback=str(final_review.get("feedback") or ""),
|
||||
round_number=refinement_round,
|
||||
directory=frames_dir / f"boundary-refinement-{refinement_round}",
|
||||
receipts=receipts,
|
||||
)
|
||||
final_render = _render_model_script(
|
||||
backend,
|
||||
label=f"corrected-final-refinement-{refinement_round}",
|
||||
source=source,
|
||||
output=final_output,
|
||||
start=start,
|
||||
end=end,
|
||||
scripts=scripts_dir,
|
||||
logs=logs_dir,
|
||||
receipts=receipts,
|
||||
)
|
||||
final_review, final_frames = _review(
|
||||
backend,
|
||||
purpose=f"review_corrected_final_refinement_{refinement_round}_rendered_pixels",
|
||||
clip=final_output,
|
||||
directory=frames_dir / f"final-review-refinement-{refinement_round}",
|
||||
receipts=receipts,
|
||||
)
|
||||
correction_rounds.append(
|
||||
{
|
||||
"round": refinement_round,
|
||||
"kind": "reviewer_driven_quarter_second_refinement",
|
||||
"prior_interval": [prior_start, prior_end],
|
||||
"source_frames": refinement_frames,
|
||||
"refinement": refinement,
|
||||
"plan": {"start": start, "end": end, "effects": ["slowmo:1.5", "subtitle:BIG BUNNY"]},
|
||||
"render": final_render,
|
||||
"review": final_review,
|
||||
"frames": final_frames,
|
||||
}
|
||||
)
|
||||
localization["reviewer_driven_refinements"] = [
|
||||
{
|
||||
"round": item["round"],
|
||||
"prior_interval": item["prior_interval"],
|
||||
"source_frames": item["source_frames"],
|
||||
"result": item["refinement"],
|
||||
}
|
||||
for item in correction_rounds[1:]
|
||||
]
|
||||
localization["final_interval"] = [start, end]
|
||||
localization["final_start_error_s"] = round(abs(start - GROUND_TRUTH["start"]), 3)
|
||||
localization["final_end_error_s"] = round(abs(end - GROUND_TRUTH["end"]), 3)
|
||||
_write_json(run_dir / "localization.json", localization)
|
||||
final_probe = _probe(final_output)
|
||||
output_duration = float(final_probe["format"]["duration"])
|
||||
expected_duration = (end - start) * 1.5
|
||||
blender_version = subprocess.run(
|
||||
[str(BLENDER), "--version"], check=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
|
||||
).stdout
|
||||
(run_dir / "blender_version.txt").write_text(blender_version, encoding="utf-8")
|
||||
_write_json(
|
||||
run_dir / "review.json",
|
||||
{
|
||||
"negative_control": {
|
||||
"plan": {"start": 0.0, "end": 3.0, "effects": ["slowmo:1.5", "subtitle:BIG BUNNY"]},
|
||||
"render": negative_render,
|
||||
"review": negative_review,
|
||||
"frames": negative_frames,
|
||||
},
|
||||
"correction_triggered": correction_triggered,
|
||||
"correction_rounds": correction_rounds,
|
||||
"corrected_final": {
|
||||
"plan": {"start": start, "end": end, "effects": ["slowmo:1.5", "subtitle:BIG BUNNY"]},
|
||||
"render": final_render,
|
||||
"review": final_review,
|
||||
"frames": final_frames,
|
||||
"probe": final_probe,
|
||||
"expected_slowed_duration_s": expected_duration,
|
||||
"observed_duration_s": output_duration,
|
||||
},
|
||||
},
|
||||
)
|
||||
_write_json(run_dir / "provider_receipts.json", {"calls": receipts})
|
||||
|
||||
video_streams = [stream for stream in final_probe["streams"] if stream.get("codec_type") == "video"]
|
||||
gates = {
|
||||
"pinned_real_public_multiscene_source": source_evidence["sha256"] == SOURCE_SHA256,
|
||||
"coarse_real_vision_about_10_seconds": len(coarse_frames) >= 4 and bool(coarse.get("reason")),
|
||||
"fine_real_vision_every_1_second": len(fine_frames) >= 10 and all(
|
||||
abs(b["timestamp_s"] - a["timestamp_s"] - 1.0) < 1e-6 for a, b in zip(fine_frames, fine_frames[1:])
|
||||
),
|
||||
"start_boundary_error_le_3s": localization["final_start_error_s"] <= 3.0,
|
||||
"end_boundary_error_le_3s": localization["final_end_error_s"] <= 3.0,
|
||||
"model_generated_blender_python": all(
|
||||
Path(item["accepted_script"]).is_file()
|
||||
for item in [negative_render, *[round_["render"] for round_ in correction_rounds]]
|
||||
),
|
||||
"actual_blender_execution": all(
|
||||
any(attempt.get("accepted") and attempt.get("blender_exit_code") == 0 for attempt in item["attempts"])
|
||||
for item in [negative_render, *[round_["render"] for round_ in correction_rounds]]
|
||||
),
|
||||
"reviewer_rejected_bad_edit": negative_review["pass"] is False and negative_review["target_present"] is False,
|
||||
"reviewer_triggered_correction": correction_triggered,
|
||||
"reviewer_accepted_corrected_pixels": final_review["pass"] is True and final_review["target_present"] is True,
|
||||
"subtitle_visually_verified": final_review["subtitle_visible"] is True,
|
||||
"slow_motion_duration_verified": abs(output_duration - expected_duration) <= 0.8,
|
||||
"final_format_and_quality": bool(video_streams)
|
||||
and video_streams[0].get("codec_name") == "h264"
|
||||
and int(video_streams[0].get("width") or 0) == 854
|
||||
and int(video_streams[0].get("height") or 0) == 480,
|
||||
"raw_receipts_usage_latency_complete": all(
|
||||
call.get("response")
|
||||
and call.get("latency_s") is not None
|
||||
and call.get("usage", {}).get("prompt_tokens") is not None
|
||||
and call.get("usage", {}).get("completion_tokens") is not None
|
||||
for call in receipts
|
||||
),
|
||||
}
|
||||
official_complete = all(gates.values())
|
||||
artifacts = {
|
||||
str(path.relative_to(run_dir)): {"sha256": _sha(path), "bytes": path.stat().st_size}
|
||||
for path in sorted(run_dir.rglob("*"))
|
||||
if path.is_file()
|
||||
}
|
||||
manifest = {
|
||||
"schema_version": "1.0",
|
||||
"experiment": "5-6",
|
||||
"run_id": run_id,
|
||||
"generated_at_utc": _utc(),
|
||||
"provider": backend.provider,
|
||||
"model": backend.model,
|
||||
"source_url": SOURCE_URL,
|
||||
"source_sha256": SOURCE_SHA256,
|
||||
"blender_binary": str(BLENDER),
|
||||
"blender_version": blender_version.splitlines()[0],
|
||||
"localization": {
|
||||
"predicted": [start, end],
|
||||
"ground_truth": [GROUND_TRUTH["start"], GROUND_TRUTH["end"]],
|
||||
"errors_s": [localization["final_start_error_s"], localization["final_end_error_s"]],
|
||||
"reviewer_driven_refinement_rounds": len(correction_rounds) - 1,
|
||||
},
|
||||
"final_video": {"path": "final.mp4", "sha256": _sha(final_output), "probe": final_probe},
|
||||
"model_call_count": len(receipts),
|
||||
"gates": gates,
|
||||
"artifacts": artifacts,
|
||||
"official_complete": official_complete,
|
||||
}
|
||||
_write_json(run_dir / "manifest.json", manifest)
|
||||
if not official_complete:
|
||||
raise RuntimeError("Experiment 5-6 acceptance gates failed: " + json.dumps(gates))
|
||||
_write_json(
|
||||
VALIDATION / "latest.json",
|
||||
{
|
||||
"experiment": "5-6",
|
||||
"run_id": run_id,
|
||||
"manifest": str((run_dir / "manifest.json").relative_to(ROOT)),
|
||||
"manifest_sha256": _sha(run_dir / "manifest.json"),
|
||||
"official_complete": True,
|
||||
},
|
||||
)
|
||||
return manifest
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--provider", choices=("ark", "moonshot", "openai"), default="ark")
|
||||
parser.add_argument("--run-id", default=f"exp5-6-real-blender-{datetime.now().strftime('%Y%m%d-%H%M%S')}")
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(run(args.provider, args.run_id), ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,293 @@
|
||||
"""
|
||||
实验 5-6：基于 API 的智能视频剪辑（两步 Vision 定位 + 提议者-审核者）
|
||||
|
||||
一条命令跑通：
|
||||
python demo.py # 默认需求"把冲浪的部分剪出来"
|
||||
python demo.py "把滑雪部分剪出来，并加上字幕 Winter" # 自定义需求
|
||||
|
||||
流程：
|
||||
1. 程序化生成含 4 个明显不同场景的测试视频（HIKING/SURFING/SKIING/CYCLING）；
|
||||
2. Proposer 解析自然语言需求 → 目标场景 + 特效；
|
||||
3. 视频分析子 Agent 两步定位（粗粒度每 10s → 细粒度每 1s）找到精确边界；
|
||||
4. Proposer 生成 Blender Python API（bpy）脚本剪出片段（可含字幕/慢动作）；
|
||||
装了 Blender 则无头渲染，否则回退 ffmpeg——但 bpy 脚本始终生成（代码生成产物）；
|
||||
5. Reviewer 检查成片关键帧，给出反馈；不合格则 Proposer 修正边界重剪，迭代。
|
||||
|
||||
依赖：ffmpeg/ffprobe（回退后端 + 抽帧）、OPENAI_API_KEY（gpt-5.6-luna 视觉 + 文本；未配置时可用 OPENROUTER_API_KEY 兜底）；
|
||||
可选 Blender（书中原方案，`--backend blender`）。
|
||||
|
||||
常用命令（完整用法见 `python demo.py --help`）：
|
||||
python demo.py # 默认需求，完整流程
|
||||
python demo.py --quick # 快速模式：粗采样 + 单轮审查，省时省钱
|
||||
python demo.py --smoke # 冒烟自检：仅剪辑链路 + 生成 bpy 脚本，不调用任何 API
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
OUT_DIR = os.path.join(HERE, "output")
|
||||
SOURCE_VIDEO = os.path.join(OUT_DIR, "source.mp4") # 测试片输出位置
|
||||
FINAL_VIDEO = os.path.join(OUT_DIR, "final.mp4")
|
||||
MAX_ROUNDS = 3 # Reviewer 反馈后最多重剪次数（默认，可用 --max-rounds 覆盖）
|
||||
|
||||
DEFAULT_REQUEST = "把冲浪的部分剪出来"
|
||||
|
||||
|
||||
def banner(title):
|
||||
print("\n" + "=" * 74)
|
||||
print(f" {title}")
|
||||
print("=" * 74)
|
||||
|
||||
|
||||
def build_arg_parser() -> argparse.ArgumentParser:
|
||||
"""命令行参数：位置参数为中文剪辑需求，另有输入/输出/后端/模型/快速等开关。"""
|
||||
p = argparse.ArgumentParser(
|
||||
prog="demo.py",
|
||||
description="实验 5-6：基于 API 的智能视频剪辑（两步 Vision 定位 + 提议者-审核者）",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=(
|
||||
"示例：\n"
|
||||
" python demo.py\n"
|
||||
" python demo.py \"把滑雪部分剪出来，并加上字幕 Winter\"\n"
|
||||
" python demo.py -i my.mp4 -o out.mp4 \"把演讲开场剪出来\"\n"
|
||||
" python demo.py --backend blender # 强制用 Blender Python API 渲染\n"
|
||||
" python demo.py --quick # 更少 Vision 调用，快速验证链路\n"
|
||||
" python demo.py --smoke # 只跑剪辑链路 + 生成 bpy 脚本，不调用任何 API\n"
|
||||
),
|
||||
)
|
||||
p.add_argument("request", nargs="?", default=DEFAULT_REQUEST,
|
||||
help="中文剪辑需求（默认：%(default)s）")
|
||||
p.add_argument("--input", "-i", metavar="VIDEO", default=None,
|
||||
help="输入视频路径（不指定则程序化生成 4 场景测试片）")
|
||||
p.add_argument("--output", "-o", metavar="VIDEO", default=FINAL_VIDEO,
|
||||
help="成片输出路径（默认 output/final.mp4）")
|
||||
p.add_argument("--backend", choices=["auto", "blender", "ffmpeg"], default="auto",
|
||||
help="剪辑后端：auto=装了 Blender 用 bpy 否则 ffmpeg；"
|
||||
"blender=强制 Blender Python API；ffmpeg=强制 ffmpeg（默认 auto）")
|
||||
p.add_argument("--text-model", metavar="NAME", default=None,
|
||||
help="覆盖文本模型（否则用 $TEXT_MODEL，默认 gpt-5.6-luna）")
|
||||
p.add_argument("--vision-model", metavar="NAME", default=None,
|
||||
help="覆盖视觉模型，须支持图像输入（否则用 $VISION_MODEL，默认 gpt-5.6-luna）")
|
||||
p.add_argument("--quick", action="store_true",
|
||||
help="快速模式：粗采样（15s/2s）+ 单轮审查，减少 Vision API 调用")
|
||||
p.add_argument("--max-rounds", type=int, default=MAX_ROUNDS, metavar="N",
|
||||
help="Reviewer 反馈后最多重剪轮数（默认 %(default)s；--quick 时强制为 1）")
|
||||
p.add_argument("--smoke", action="store_true",
|
||||
help="冒烟自检：仅剪辑链路 + 生成 bpy 脚本，不调用任何 API")
|
||||
return p
|
||||
|
||||
|
||||
def smoke_check():
|
||||
"""冒烟自检：不触碰 OpenAI，验证剪辑链路可用并生成 Proposer 的 bpy 脚本。"""
|
||||
from blender_editor import blender_available
|
||||
from ffmpeg_utils import ensure_ffmpeg, extract_frame, format_probe
|
||||
from make_test_video import GROUND_TRUTH, make as make_test_video
|
||||
from video_editor import apply_edit
|
||||
|
||||
banner("冒烟自检 | 剪辑链路 + bpy 脚本生成，不调用任何 API")
|
||||
try:
|
||||
ensure_ffmpeg()
|
||||
except RuntimeError as e:
|
||||
print(f"\n[错误] {e}")
|
||||
sys.exit(1)
|
||||
if os.path.isdir(OUT_DIR):
|
||||
shutil.rmtree(OUT_DIR)
|
||||
os.makedirs(OUT_DIR, exist_ok=True)
|
||||
|
||||
make_test_video(SOURCE_VIDEO)
|
||||
print(f"[1/3] 生成测试视频 OK：{SOURCE_VIDEO}（场景真值={GROUND_TRUTH}）")
|
||||
frame_dir = os.path.join(OUT_DIR, "frames")
|
||||
os.makedirs(frame_dir, exist_ok=True) # extract_frame 要求目录已存在
|
||||
frame = extract_frame(SOURCE_VIDEO, 20.0, os.path.join(frame_dir, "smoke.png"))
|
||||
print(f"[2/3] 抽帧 OK：{frame}")
|
||||
clip = os.path.join(OUT_DIR, "smoke_cut.mp4")
|
||||
script_path = os.path.join(OUT_DIR, "edit.py")
|
||||
# backend="auto"：未装 Blender 则用 ffmpeg 实际渲染，但仍生成 bpy 脚本（代码生成产物）。
|
||||
apply_edit(SOURCE_VIDEO, {"start": 15.0, "end": 20.0,
|
||||
"effects": [{"type": "subtitle", "text": "SMOKE"}]},
|
||||
clip, backend="auto", script_path=script_path)
|
||||
used = "Blender bpy" if blender_available() else "ffmpeg（未装 Blender，回退）"
|
||||
print(f"[3/3] 剪辑+字幕 OK（后端={used}）：\n{format_probe(clip)}")
|
||||
print(f"\n已生成 Proposer 的 Blender 脚本：{script_path}")
|
||||
print("（这正是书中'生成 Blender Python API 代码'的产物；装好 Blender 后可直接")
|
||||
print(f" `blender --background --python {script_path}` 无头渲染。）")
|
||||
print("\n✓ 冒烟自检通过：剪辑链路正常 + bpy 脚本已生成（未调用 OpenAI）。")
|
||||
|
||||
|
||||
def preflight():
|
||||
"""启动自检：给出清晰中文报错，而非 traceback。"""
|
||||
from ffmpeg_utils import ensure_ffmpeg
|
||||
if not (os.getenv("OPENAI_API_KEY") or os.getenv("OPENROUTER_API_KEY")):
|
||||
print("\n[错误] 未检测到 OPENAI_API_KEY（或 OPENROUTER_API_KEY 兜底）。\n"
|
||||
" 请复制 env.example 为 .env 并填入有效的 OpenAI Key，或执行：\n"
|
||||
" export OPENAI_API_KEY=your-openai-api-key # 或 export OPENROUTER_API_KEY=your-openrouter-api-key\n"
|
||||
" 本实验用 gpt-5.6-luna 做视觉定位与审查，必须提供有效 Key。")
|
||||
sys.exit(1)
|
||||
try:
|
||||
ensure_ffmpeg()
|
||||
except RuntimeError as e:
|
||||
print(f"\n[错误] {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
args = build_arg_parser().parse_args()
|
||||
if args.smoke: # 仅剪辑链路，不需要 API Key，提前返回。
|
||||
smoke_check()
|
||||
return
|
||||
|
||||
nl_request = args.request
|
||||
# --quick：粗化采样步长并只审查一轮，把 Vision 调用降到最少（用于快速验证链路）。
|
||||
coarse_interval = 15.0 if args.quick else 10.0
|
||||
fine_interval = 2.0 if args.quick else 1.0
|
||||
max_rounds = 1 if args.quick else max(1, args.max_rounds)
|
||||
|
||||
# 模型覆盖：写回环境变量，供 agents 模块（惰性初始化）读取。须在导入 agents 前设置。
|
||||
if args.text_model:
|
||||
os.environ["TEXT_MODEL"] = args.text_model
|
||||
if args.vision_model:
|
||||
os.environ["VISION_MODEL"] = args.vision_model
|
||||
preflight()
|
||||
|
||||
# 延迟导入：确保 preflight 的报错优先于任何 SDK 初始化。
|
||||
from agents import (ProposerAgent, ReviewerAgent, VideoAnalyzerAgent,
|
||||
TokenMeter, TEXT_MODEL, VISION_MODEL)
|
||||
from blender_editor import blender_available
|
||||
from ffmpeg_utils import format_probe, probe_duration
|
||||
from make_test_video import make as make_test_video, GROUND_TRUTH
|
||||
from video_editor import apply_edit
|
||||
|
||||
# 幂等：每次从干净的 output/ 开始。
|
||||
if os.path.isdir(OUT_DIR):
|
||||
shutil.rmtree(OUT_DIR)
|
||||
os.makedirs(OUT_DIR, exist_ok=True)
|
||||
|
||||
ground_truth = None
|
||||
if args.input:
|
||||
banner("步骤 0 | 使用外部输入视频")
|
||||
source_video = os.path.abspath(args.input)
|
||||
if not os.path.isfile(source_video):
|
||||
print(f"\n[错误] 输入视频不存在：{source_video}")
|
||||
sys.exit(1)
|
||||
print(f"输入视频：{source_video}")
|
||||
else:
|
||||
banner("步骤 0 | 生成测试视频（4 个明显不同的场景）")
|
||||
source_video = SOURCE_VIDEO
|
||||
make_test_video(source_video)
|
||||
ground_truth = GROUND_TRUTH
|
||||
print(f"已生成 {source_video}")
|
||||
print(f"场景真值（用于核对定位误差）：{ground_truth}")
|
||||
total_dur = probe_duration(source_video)
|
||||
print(f"时长 {total_dur:.1f}s")
|
||||
print(f"文本模型={TEXT_MODEL} 视觉模型={VISION_MODEL} 剪辑后端={args.backend}")
|
||||
|
||||
# 分离的 token 计量：主 Agent（Proposer+Reviewer）vs 子 Agent（截图定位）。
|
||||
main_meter = TokenMeter()
|
||||
sub_meter = TokenMeter()
|
||||
proposer = ProposerAgent(main_meter)
|
||||
reviewer = ReviewerAgent(main_meter)
|
||||
analyzer = VideoAnalyzerAgent(sub_meter)
|
||||
|
||||
banner("步骤 1 | Proposer 解析自然语言需求")
|
||||
print(f"用户需求：{nl_request}")
|
||||
intent = proposer.parse_request(nl_request)
|
||||
# 模型可能省略 target_query 或返回 null——退化为用原始需求文本做视觉定位。
|
||||
target_query = intent.get("target_query") or nl_request
|
||||
effects = intent.get("effects", [])
|
||||
print(f"解析结果：目标场景='{target_query}' 特效={effects}")
|
||||
|
||||
banner("步骤 2 | 视频分析子 Agent：两步 Vision 定位"
|
||||
+ ("（--quick 快速采样）" if args.quick else ""))
|
||||
start, end, trace = analyzer.locate(
|
||||
source_video, target_query,
|
||||
coarse_interval=coarse_interval, fine_interval=fine_interval,
|
||||
frame_dir=os.path.join(OUT_DIR, "frames"),
|
||||
)
|
||||
c = trace["coarse"]
|
||||
print(f" [粗粒度] 每 {coarse_interval:.0f}s 采样 {len(c['timestamps'])} 帧 → Vision 得区间 "
|
||||
f"[{c['start']:.0f}, {c['end']:.0f}]s（依据：{c['reason']}）")
|
||||
f = trace["fine"]
|
||||
print(f" [细粒度] 窗口 {f['window']} 内每 {fine_interval:.0f}s 采样 {f['timestamps_count']} 帧 → "
|
||||
f"精确边界 [{f['start']:.1f}, {f['end']:.1f}]s（依据：{f['reason']}）")
|
||||
print(f" >>> 最终定位：起 {start:.1f}s 止 {end:.1f}s")
|
||||
|
||||
# 与真值对比，打印定位误差（验收：误差 ≤ ±3s）。仅测试片有真值。
|
||||
key = _match_ground_truth(target_query, ground_truth) if ground_truth else None
|
||||
if key:
|
||||
gs, ge = ground_truth[key]
|
||||
print(f" 真值 [{gs}, {ge}]s → 起点误差 {abs(start - gs):.1f}s，"
|
||||
f"终点误差 {abs(end - ge):.1f}s（验收要求 ≤ 3s）")
|
||||
|
||||
banner("步骤 3-4 | Proposer 生成 bpy 脚本剪辑 + Reviewer 审查（迭代）")
|
||||
plan = {"start": start, "end": end, "effects": effects}
|
||||
final_path = None
|
||||
for rnd in range(1, max_rounds + 1):
|
||||
print(f"\n--- 第 {rnd} 轮 ---")
|
||||
clip = os.path.join(OUT_DIR, f"cut_round{rnd}.mp4")
|
||||
script_path = os.path.join(OUT_DIR, f"edit_round{rnd}.py")
|
||||
apply_edit(source_video, plan, clip, backend=args.backend,
|
||||
script_path=script_path)
|
||||
cdur = probe_duration(clip)
|
||||
used = "Blender bpy" if (args.backend == "blender" or
|
||||
(args.backend == "auto" and blender_available())) else "ffmpeg"
|
||||
print(f" Proposer 生成 Blender 脚本 → {script_path}")
|
||||
print(f" 剪出片段 [{plan['start']:.1f}, {plan['end']:.1f}]s（后端={used}），"
|
||||
f"成片时长 {cdur:.1f}s")
|
||||
|
||||
review = reviewer.review(clip, target_query,
|
||||
frame_dir=os.path.join(OUT_DIR, "review_frames"))
|
||||
print(f" Reviewer：pass={review.get('pass')} score={review.get('score')} "
|
||||
f"检查帧={['%.1f' % t for t in review.get('frames_checked', [])]}")
|
||||
print(f" Reviewer 反馈：{review.get('feedback', '（无）')}")
|
||||
|
||||
if review.get("pass"):
|
||||
final_path = clip
|
||||
print(" ✓ 审核通过。")
|
||||
break
|
||||
if rnd == max_rounds:
|
||||
final_path = clip
|
||||
print(" 达到最大轮数，采用当前成片。")
|
||||
break
|
||||
# 未通过：Proposer 据反馈修正边界后重剪。
|
||||
ns, ne = proposer.revise_bounds(plan["start"], plan["end"],
|
||||
review.get("feedback", ""), total_dur)
|
||||
print(f" Proposer 据反馈修正边界：[{ns:.1f}, {ne:.1f}]s")
|
||||
plan["start"], plan["end"] = ns, ne
|
||||
|
||||
output_path = os.path.abspath(args.output)
|
||||
os.makedirs(os.path.dirname(output_path) or ".", exist_ok=True)
|
||||
shutil.copy(final_path, output_path)
|
||||
|
||||
banner("步骤 5 | 成片信息（ffprobe）")
|
||||
print(format_probe(output_path))
|
||||
|
||||
banner("Token 统计（子 Agent 隔离截图，主上下文不被污染）")
|
||||
print(f" 主 Agent（Proposer+Reviewer）：{main_meter.total()} tokens "
|
||||
f"(prompt={main_meter.prompt}, completion={main_meter.completion})")
|
||||
print(f" 子 Agent（两步定位截图） ：{sub_meter.total()} tokens "
|
||||
f"(prompt={sub_meter.prompt}, completion={sub_meter.completion})")
|
||||
print(f"\n完成：{output_path}")
|
||||
|
||||
|
||||
def _match_ground_truth(query, gt):
|
||||
q = query.lower()
|
||||
for key in gt:
|
||||
if key in q:
|
||||
return key
|
||||
# 中文关键词兜底映射。
|
||||
zh = {"冲浪": "surfing", "徒步": "hiking", "滑雪": "skiing", "骑": "cycling",
|
||||
"hik": "hiking", "surf": "surfing", "ski": "skiing", "cycl": "cycling"}
|
||||
for k, v in zh.items():
|
||||
if k in q:
|
||||
return v
|
||||
return None
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,14 @@
|
||||
# 必填其一：OpenAI API Key（本实验用 gpt-5.6-luna 做视觉定位/审查 + 文本规划）
|
||||
OPENAI_API_KEY=your_openai_api_key_here
|
||||
|
||||
# 通用兜底：未配置 OPENAI_API_KEY 时自动改走 OpenRouter；
|
||||
# 默认模型 gpt-5.6-luna（gpt-5.x）直连 OpenAI 需组织实名认证，
|
||||
# 故设置了本 key 时会优先走 OpenRouter（route openai/gpt-5.6-luna）。
|
||||
# OPENROUTER_API_KEY=your_openrouter_api_key_here
|
||||
|
||||
# 可选：兼容 OpenAI 协议的自定义端点
|
||||
# OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
|
||||
# 可选：指定模型（默认均为 gpt-5.6-luna；视觉模型必须支持图像输入）
|
||||
# TEXT_MODEL=gpt-5.6-luna
|
||||
# VISION_MODEL=gpt-5.6-luna
|
||||
@@ -0,0 +1,103 @@
|
||||
"""
|
||||
ffmpeg / ffprobe 薄封装：所有对外部进程的调用都集中在这里，统一做错误检查。
|
||||
|
||||
设计要点：
|
||||
- run() 捕获非零退出码并抛出带 stderr 的清晰异常（而非让 traceback 泄漏）；
|
||||
- 提供 probe_duration / probe_streams，供 Reviewer 与验证环节读取成片信息；
|
||||
- extract_frame 把某一时间点抽成一张 PNG（缩放到 512 宽以节省 Vision token）。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
# macOS 自带字体；换平台时改这里即可（Linux 常见 DejaVuSans.ttf）。
|
||||
FONT_CANDIDATES = [
|
||||
"/System/Library/Fonts/Supplemental/Arial.ttf",
|
||||
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
|
||||
"/Library/Fonts/Arial.ttf",
|
||||
]
|
||||
|
||||
|
||||
def find_font() -> str:
|
||||
for p in FONT_CANDIDATES:
|
||||
if os.path.exists(p):
|
||||
return p
|
||||
return "" # drawtext 会退化为默认字体
|
||||
|
||||
|
||||
def ensure_ffmpeg():
|
||||
"""启动前自检：ffmpeg / ffprobe 是否可用，给出清晰中文报错。"""
|
||||
for tool in ("ffmpeg", "ffprobe"):
|
||||
if shutil.which(tool) is None:
|
||||
raise RuntimeError(
|
||||
f"未找到 {tool}，本项目用 ffmpeg 完成实际剪辑。\n"
|
||||
f" macOS: brew install ffmpeg\n"
|
||||
f" Ubuntu: sudo apt install ffmpeg"
|
||||
)
|
||||
|
||||
|
||||
def run(cmd, desc="ffmpeg 命令"):
|
||||
"""执行命令，失败时抛出带 stderr 尾部的异常。"""
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
tail = "\n".join(proc.stderr.strip().splitlines()[-8:])
|
||||
raise RuntimeError(f"{desc} 执行失败（exit={proc.returncode}）：\n{tail}")
|
||||
return proc
|
||||
|
||||
|
||||
def probe_duration(path: str) -> float:
|
||||
"""返回视频时长（秒）。文件缺少时长元数据时 ffprobe 输出 N/A，给出清晰报错。"""
|
||||
proc = run(
|
||||
["ffprobe", "-v", "error", "-show_entries", "format=duration",
|
||||
"-of", "default=noprint_wrappers=1:nokey=1", path],
|
||||
desc="ffprobe 读取时长",
|
||||
)
|
||||
out = proc.stdout.strip()
|
||||
if not out or out == "N/A":
|
||||
raise RuntimeError(f"ffprobe 无法读取时长（文件缺少时长元数据或不是音视频文件）：{path}")
|
||||
return float(out)
|
||||
|
||||
|
||||
def probe_streams(path: str) -> dict:
|
||||
"""返回 ffprobe 的 JSON（format + streams），用于打印成片信息。"""
|
||||
proc = run(
|
||||
["ffprobe", "-v", "error", "-show_format", "-show_streams",
|
||||
"-of", "json", path],
|
||||
desc="ffprobe 读取流信息",
|
||||
)
|
||||
return json.loads(proc.stdout)
|
||||
|
||||
|
||||
def format_probe(path: str) -> str:
|
||||
"""把成片信息格式化成一行行的人类可读文本（用于验证输出）。"""
|
||||
info = probe_streams(path)
|
||||
fmt = info.get("format", {})
|
||||
lines = [
|
||||
f" 文件: {os.path.basename(path)}",
|
||||
f" 时长: {float(fmt.get('duration', 0)):.2f}s",
|
||||
f" 容器: {fmt.get('format_name', '?')}",
|
||||
f" 大小: {int(fmt.get('size', 0)) / 1024:.1f} KB",
|
||||
]
|
||||
for s in info.get("streams", []):
|
||||
if s.get("codec_type") == "video":
|
||||
lines.append(
|
||||
f" 视频流: {s.get('codec_name')} {s.get('width')}x{s.get('height')} "
|
||||
f"@ {s.get('r_frame_rate')} fps"
|
||||
)
|
||||
elif s.get("codec_type") == "audio":
|
||||
lines.append(
|
||||
f" 音频流: {s.get('codec_name')} {s.get('sample_rate')}Hz "
|
||||
f"{s.get('channels')}ch"
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def extract_frame(video: str, t: float, out_png: str, width: int = 512):
|
||||
"""抽取 t 秒处的一帧，缩放到 width 宽存为 PNG。"""
|
||||
run(
|
||||
["ffmpeg", "-y", "-ss", f"{t:.3f}", "-i", video,
|
||||
"-frames:v", "1", "-vf", f"scale={width}:-1", out_png],
|
||||
desc=f"抽帧 t={t:.1f}s",
|
||||
)
|
||||
return out_png
|
||||
@@ -0,0 +1,81 @@
|
||||
"""
|
||||
程序化生成一段"含多个明显不同场景"的测试视频（无需任何素材文件）。
|
||||
|
||||
每个场景 = 一种纯色背景 + 一个大号运动标题（场景英文名）+ 时间码水印。
|
||||
标题让 Vision LLM 能仅凭画面就准确判断"这是哪个场景"，从而验证两步定位。
|
||||
换成真实视频时：把 demo.py 里的 SOURCE_VIDEO 指向你自己的 mp4 即可（见 README）。
|
||||
"""
|
||||
import os
|
||||
|
||||
from ffmpeg_utils import find_font, run
|
||||
|
||||
# 每个场景：(名称, 背景色, 起始秒, 时长秒)。刻意让每段 > 10s，
|
||||
# 使"每 10s 一张"的粗粒度采样必然命中每个场景。
|
||||
SCENES = [
|
||||
("HIKING", "0x1E6B3A", 0, 15), # 森林绿
|
||||
("SURFING", "0x1565C0", 15, 15), # 海洋蓝
|
||||
("SKIING", "0xE0E0E0", 30, 12), # 雪地白
|
||||
("CYCLING", "0xE65100", 42, 12), # 落日橙
|
||||
]
|
||||
TOTAL = SCENES[-1][2] + SCENES[-1][3] # 54s
|
||||
W, H, FPS = 1280, 720, 30
|
||||
|
||||
|
||||
def _drawtext(text, size, y_expr, color="white", box=False):
|
||||
font = find_font()
|
||||
parts = [f"text='{text}'", f"fontsize={size}", f"fontcolor={color}",
|
||||
"x=(w-text_w)/2", f"y={y_expr}"]
|
||||
if font:
|
||||
parts.insert(0, f"fontfile={font}")
|
||||
if box:
|
||||
parts += ["box=1", "boxcolor=black@0.4", "boxborderw=20"]
|
||||
return "drawtext=" + ":".join(parts)
|
||||
|
||||
|
||||
def make(out_path: str) -> str:
|
||||
"""生成测试视频，返回路径。幂等：每次覆盖，保证从干净状态开始。"""
|
||||
out_dir = os.path.dirname(out_path)
|
||||
if out_dir:
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
clip_paths = []
|
||||
tmp_dir = out_dir or "."
|
||||
|
||||
for i, (name, color, start, dur) in enumerate(SCENES):
|
||||
clip = os.path.join(tmp_dir, f"_scene_{i}.mp4")
|
||||
# 让标题上下缓慢漂移，制造真实"运动画面"，避免纯静止帧。
|
||||
title = _drawtext(name, 140, "(h-text_h)/2 + 60*sin(t)", box=True)
|
||||
# 左上角时间码：t 为片段内相对时间，加 start 得到全局时间。
|
||||
# drawtext 里表达式含冒号，必须转义为 \: 否则被当成选项分隔符。
|
||||
clock = _drawtext(rf"t=%{{eif\:t+{start}\:d}}s", 48,
|
||||
"40", color="yellow")
|
||||
vf = f"{title},{clock}"
|
||||
run(
|
||||
["ffmpeg", "-y",
|
||||
"-f", "lavfi", "-i", f"color=c={color}:s={W}x{H}:d={dur}:r={FPS}",
|
||||
"-f", "lavfi", "-i", f"sine=frequency={220 + i * 110}:duration={dur}",
|
||||
"-vf", vf, "-pix_fmt", "yuv420p",
|
||||
"-c:v", "libx264", "-c:a", "aac", "-shortest", clip],
|
||||
desc=f"生成场景 {name}",
|
||||
)
|
||||
clip_paths.append(clip)
|
||||
|
||||
# 用 concat demuxer 无缝拼接成完整原始素材。
|
||||
list_file = os.path.join(tmp_dir, "_concat_list.txt")
|
||||
with open(list_file, "w") as f:
|
||||
for c in clip_paths:
|
||||
f.write(f"file '{os.path.abspath(c)}'\n")
|
||||
run(
|
||||
["ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", list_file,
|
||||
"-c", "copy", out_path],
|
||||
desc="拼接测试视频",
|
||||
)
|
||||
|
||||
# 清理中间片段。
|
||||
for c in clip_paths:
|
||||
os.remove(c)
|
||||
os.remove(list_file)
|
||||
return out_path
|
||||
|
||||
|
||||
# 供 demo / README 引用：场景真值表，用于验证定位误差。
|
||||
GROUND_TRUTH = {name.lower(): (start, start + dur) for name, _, start, dur in SCENES}
|
||||
@@ -0,0 +1,2 @@
|
||||
openai>=1.30.0
|
||||
python-dotenv>=1.0
|
||||
@@ -0,0 +1,63 @@
|
||||
"""LLM 返回的 JSON 缺字段/为 null 时，Agent 解析应按约定哨兵处理，不应崩溃。"""
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
import agents
|
||||
import ffmpeg_utils
|
||||
|
||||
|
||||
def _fake_client(content):
|
||||
resp = types.SimpleNamespace(
|
||||
choices=[types.SimpleNamespace(
|
||||
message=types.SimpleNamespace(content=content))],
|
||||
usage=None)
|
||||
completions = types.SimpleNamespace(create=lambda **kw: resp)
|
||||
return types.SimpleNamespace(chat=types.SimpleNamespace(completions=completions))
|
||||
|
||||
|
||||
def _stub_io(monkeypatch, content):
|
||||
"""替换掉网络与帧抽取 IO，让 Agent 直接吃到给定的 LLM 回复文本。"""
|
||||
monkeypatch.setattr(agents, "client", lambda: _fake_client(content))
|
||||
monkeypatch.setattr(agents, "extract_frame", lambda *a, **k: None)
|
||||
monkeypatch.setattr(agents, "_img_part",
|
||||
lambda p: {"type": "image_url", "image_url": {"url": "data:,"}})
|
||||
|
||||
|
||||
def test_vision_locate_missing_keys(monkeypatch):
|
||||
"""模型省略 start/end → 按 -1 哨兵返回（走兜底逻辑），不抛 KeyError。"""
|
||||
_stub_io(monkeypatch, '{"reason": "画面里看不到目标场景"}')
|
||||
start, end, reason = agents.VideoAnalyzerAgent()._vision_locate(
|
||||
"fake.mp4", [0.0], "目标", "frames")
|
||||
assert (start, end) == (-1.0, -1.0)
|
||||
assert reason == "画面里看不到目标场景"
|
||||
|
||||
|
||||
def test_vision_locate_null_fields(monkeypatch):
|
||||
"""模型返回显式 null → 同样按 -1 哨兵返回，不抛 TypeError。"""
|
||||
_stub_io(monkeypatch, '{"start": null, "end": null, "reason": "not visible"}')
|
||||
start, end, _ = agents.VideoAnalyzerAgent()._vision_locate(
|
||||
"fake.mp4", [0.0], "目标", "frames")
|
||||
assert (start, end) == (-1.0, -1.0)
|
||||
|
||||
|
||||
def test_revise_bounds_null_start_keeps_current(monkeypatch):
|
||||
"""修正区间为 null/缺失时维持当前值，正常数值仍生效。"""
|
||||
_stub_io(monkeypatch, '{"start": null, "end": 5}')
|
||||
ns, ne = agents.ProposerAgent().revise_bounds(1.0, 3.0, "反馈", 10.0)
|
||||
assert ns == 1.0
|
||||
assert ne == 5.0
|
||||
|
||||
|
||||
def test_probe_duration_na(monkeypatch):
|
||||
"""ffprobe 输出 N/A（无时长元数据）→ 清晰的 RuntimeError，而非 ValueError。"""
|
||||
fake_proc = types.SimpleNamespace(stdout="N/A\n")
|
||||
monkeypatch.setattr(ffmpeg_utils, "run", lambda *a, **k: fake_proc)
|
||||
with pytest.raises(RuntimeError, match="时长"):
|
||||
ffmpeg_utils.probe_duration("no_duration.bin")
|
||||
|
||||
|
||||
def test_probe_duration_normal(monkeypatch):
|
||||
fake_proc = types.SimpleNamespace(stdout="12.5\n")
|
||||
monkeypatch.setattr(ffmpeg_utils, "run", lambda *a, **k: fake_proc)
|
||||
assert ffmpeg_utils.probe_duration("a.mp4") == 12.5
|
||||
@@ -0,0 +1,45 @@
|
||||
import os
|
||||
import make_test_video
|
||||
from video_editor import apply_edit
|
||||
|
||||
|
||||
def test_make_test_video_bare_filename():
|
||||
out_name = "test_temp_bare_video.mp4"
|
||||
if os.path.exists(out_name):
|
||||
os.remove(out_name)
|
||||
try:
|
||||
path = make_test_video.make(out_name)
|
||||
assert os.path.exists(path)
|
||||
assert path == out_name
|
||||
finally:
|
||||
if os.path.exists(out_name):
|
||||
os.remove(out_name)
|
||||
for i in range(len(make_test_video.SCENES)):
|
||||
scene_file = f"_scene_{i}.mp4"
|
||||
if os.path.exists(scene_file):
|
||||
os.remove(scene_file)
|
||||
|
||||
|
||||
def test_apply_edit_bare_filename():
|
||||
source = "test_source.mp4"
|
||||
make_test_video.make(source)
|
||||
out_name = "test_output_bare_video.mp4"
|
||||
if os.path.exists(out_name):
|
||||
os.remove(out_name)
|
||||
plan = {
|
||||
"start": 0.0,
|
||||
"end": 2.0,
|
||||
"effects": [{"type": "subtitle", "text": "hello"}],
|
||||
}
|
||||
try:
|
||||
path = apply_edit(source, plan, out_name, backend="ffmpeg")
|
||||
assert os.path.exists(path)
|
||||
assert path == out_name
|
||||
finally:
|
||||
for name in (out_name, source, "edit.py"):
|
||||
if os.path.exists(name):
|
||||
os.remove(name)
|
||||
for i in range(len(make_test_video.SCENES)):
|
||||
scene_file = f"_scene_{i}.mp4"
|
||||
if os.path.exists(scene_file):
|
||||
os.remove(scene_file)
|
||||
@@ -0,0 +1,22 @@
|
||||
"""_extract_json must accept the first object when another object follows."""
|
||||
|
||||
import pytest
|
||||
|
||||
from agents import _extract_json
|
||||
|
||||
|
||||
def test_adjacent_json_objects_returns_first():
|
||||
assert _extract_json('{"a":1}{"b":2}') == {"a": 1}
|
||||
|
||||
|
||||
def test_prose_with_two_objects_returns_first():
|
||||
assert _extract_json('note {"a":1} mid {"b":2}') == {"a": 1}
|
||||
|
||||
|
||||
def test_single_object_unchanged():
|
||||
assert _extract_json('prefix {"ok": true, "n": 3} suffix') == {"ok": True, "n": 3}
|
||||
|
||||
|
||||
def test_no_object_raises():
|
||||
with pytest.raises(ValueError, match="未能从回复中解析 JSON"):
|
||||
_extract_json("no braces here")
|
||||
@@ -0,0 +1,18 @@
|
||||
"""Regression: slowmo factor<=0 must not ZeroDivisionError."""
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def test_source_skips_nonpositive_factor():
|
||||
src = Path(__file__).with_name("video_editor.py").read_text()
|
||||
assert "if factor <= 0:" in src
|
||||
assert "continue" in src.split("if factor <= 0:")[1][:80]
|
||||
|
||||
|
||||
def test_division_guard_math():
|
||||
factor = 0.0
|
||||
if factor <= 0:
|
||||
skipped = True
|
||||
else:
|
||||
_ = 1.0 / factor
|
||||
skipped = False
|
||||
assert skipped is True
|
||||
@@ -0,0 +1,49 @@
|
||||
"""slowmo factor:null must not TypeError (skip like non-positive)."""
|
||||
from unittest.mock import patch, MagicMock
|
||||
from blender_editor import _plan_fields, generate_bpy_script
|
||||
from video_editor import _apply_edit_ffmpeg
|
||||
|
||||
|
||||
def test_null_slowmo_factor_skipped_in_plan_fields():
|
||||
start, end, subtitle, slowmo = _plan_fields(
|
||||
{"start": 0.0, "end": 2.0, "effects": [{"type": "slowmo", "factor": None}]}
|
||||
)
|
||||
assert start == 0.0 and end == 2.0
|
||||
assert subtitle is None
|
||||
assert slowmo is None
|
||||
|
||||
|
||||
def test_missing_slowmo_factor_still_defaults():
|
||||
_, _, _, slowmo = _plan_fields(
|
||||
{"start": 0.0, "end": 2.0, "effects": [{"type": "slowmo"}]}
|
||||
)
|
||||
assert slowmo == 2.0
|
||||
|
||||
|
||||
def test_positive_slowmo_factor_kept():
|
||||
_, _, _, slowmo = _plan_fields(
|
||||
{"start": 0.0, "end": 2.0, "effects": [{"type": "slowmo", "factor": 3.0}]}
|
||||
)
|
||||
assert slowmo == 3.0
|
||||
|
||||
|
||||
def test_generate_bpy_script_with_null_factor():
|
||||
script = generate_bpy_script(
|
||||
"/tmp/in.mp4",
|
||||
{"start": 0.0, "end": 1.0, "effects": [{"type": "slowmo", "factor": None}]},
|
||||
"/tmp/out.mp4",
|
||||
)
|
||||
assert "SLOWMO = None" in script
|
||||
|
||||
|
||||
def test_apply_edit_ffmpeg_factor_speedup_and_bare_filename():
|
||||
plan = {
|
||||
"start": 0.0,
|
||||
"end": 2.0,
|
||||
"effects": [{"type": "slowmo", "factor": 0.5}]
|
||||
}
|
||||
with patch("video_editor.run") as mock_run:
|
||||
_apply_edit_ffmpeg("input.mp4", plan, "output.mp4")
|
||||
mock_run.assert_called_once()
|
||||
cmd = mock_run.call_args[0][0]
|
||||
assert "atempo=2.0" in cmd[cmd.index("-af") + 1]
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"experiment": "5-6",
|
||||
"run_id": "exp5-6-real-blender-20260730-055102",
|
||||
"manifest": "validation/runs/exp5-6-real-blender-20260730-055102/manifest.json",
|
||||
"manifest_sha256": "fd044738e812faa832863f4c112880a9583c268afb7d08f388c00623498ab7a8",
|
||||
"official_complete": true
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
"""
|
||||
本文件由 blender_editor.generate_bpy_script() 自动生成（实验 5-6）。
|
||||
执行：blender --background --python edit.py
|
||||
它把一条剪辑计划翻译成 Blender 视频序列编辑器（VSE）的 API 调用序列。
|
||||
"""
|
||||
import os
|
||||
import bpy
|
||||
|
||||
SRC = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/source_cache/big-buck-bunny-trailer-480p.mov'
|
||||
OUT = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/preflight-nospeed.mp4'
|
||||
FPS = 25
|
||||
START = 9.0 # 目标片段起点（秒）
|
||||
END = 11.0 # 目标片段终点（秒）
|
||||
SUBTITLE = 'BIG BUNNY' # None 或字幕文本
|
||||
SLOWMO = None # None 或放慢倍率（factor>1 表示放慢 factor 倍）
|
||||
|
||||
scene = bpy.context.scene
|
||||
scene.render.fps = FPS
|
||||
scene.render.fps_base = 1.0
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
|
||||
# 清掉可能存在的旧序列，保证幂等
|
||||
if scene.sequence_editor:
|
||||
bpy.ops.sequencer.select_all(action='SELECT')
|
||||
bpy.ops.sequencer.delete()
|
||||
se = scene.sequence_editor_create()
|
||||
|
||||
start_frame = int(round(START * FPS))
|
||||
dur_frames = max(1, int(round((END - START) * FPS)))
|
||||
|
||||
# 1) 导入影片 + 音轨（new_sound 在无音轨素材上会抛 RuntimeError，忽略即可）
|
||||
movie = se.sequences.new_movie(name="clip", filepath=SRC, channel=1, frame_start=1 - start_frame)
|
||||
try:
|
||||
sound = se.sequences.new_sound(name="audio", filepath=SRC, channel=2, frame_start=1 - start_frame)
|
||||
except RuntimeError:
|
||||
sound = None
|
||||
|
||||
# 2) 裁剪 [START, END]：偏移掉片头，再固定成片时长
|
||||
for strip in (movie, sound):
|
||||
if strip is None:
|
||||
continue
|
||||
strip.frame_offset_start = start_frame
|
||||
strip.frame_final_duration = dur_frames
|
||||
|
||||
top_channel = 3
|
||||
|
||||
# 3) 慢动作：SPEED 特效条（MULTIPLY 模式，speed_factor = 1/倍率）
|
||||
if SLOWMO:
|
||||
speed = se.sequences.new_effect(
|
||||
name="slowmo", type='SPEED', channel=top_channel,
|
||||
frame_start=1, frame_end=1 + dur_frames, seq1=movie,
|
||||
)
|
||||
speed.use_default_fade = False
|
||||
speed.speed_control = 'MULTIPLY'
|
||||
speed.speed_factor = 1.0 / SLOWMO
|
||||
top_channel += 1
|
||||
# 放慢后成片总帧数按倍率拉长
|
||||
render_dur = int(round(dur_frames * SLOWMO))
|
||||
movie.frame_final_duration = render_dur
|
||||
else:
|
||||
render_dur = dur_frames
|
||||
|
||||
# 4) 字幕：TEXT 特效条，底部居中带半透明底框
|
||||
if SUBTITLE:
|
||||
txt = se.sequences.new_effect(
|
||||
name="subtitle", type='TEXT', channel=top_channel,
|
||||
frame_start=1, frame_end=1 + render_dur,
|
||||
)
|
||||
txt.text = SUBTITLE
|
||||
txt.font_size = 100
|
||||
txt.location = (0.5, 0.12)
|
||||
txt.align_x = 'CENTER'
|
||||
txt.align_y = 'BOTTOM'
|
||||
txt.use_box = True
|
||||
txt.box_color = (0.0, 0.0, 0.0, 0.6)
|
||||
|
||||
# 5) 渲染范围 + 输出为 mp4(H.264+AAC)
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = render_dur
|
||||
|
||||
r = scene.render
|
||||
r.image_settings.file_format = 'FFMPEG'
|
||||
r.ffmpeg.format = 'MPEG4'
|
||||
r.ffmpeg.codec = 'H264'
|
||||
r.ffmpeg.audio_codec = 'AAC'
|
||||
r.filepath = OUT
|
||||
os.makedirs(os.path.dirname(OUT) or ".", exist_ok=True)
|
||||
|
||||
bpy.ops.render.render(animation=True)
|
||||
print("BLENDER_RENDER_DONE", OUT)
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 371 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 364 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 366 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 342 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
@@ -0,0 +1,92 @@
|
||||
"""
|
||||
本文件由 blender_editor.generate_bpy_script() 自动生成（实验 5-6）。
|
||||
执行：blender --background --python edit.py
|
||||
它把一条剪辑计划翻译成 Blender 视频序列编辑器（VSE）的 API 调用序列。
|
||||
"""
|
||||
import os
|
||||
import bpy
|
||||
|
||||
SRC = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/source_cache/big-buck-bunny-trailer-480p.mov'
|
||||
OUT = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/preflight-reference.mp4'
|
||||
FPS = 25
|
||||
START = 9.0 # 目标片段起点（秒）
|
||||
END = 11.0 # 目标片段终点（秒）
|
||||
SUBTITLE = 'BIG BUNNY' # None 或字幕文本
|
||||
SLOWMO = 1.5 # None 或放慢倍率（factor>1 表示放慢 factor 倍）
|
||||
|
||||
scene = bpy.context.scene
|
||||
scene.render.fps = FPS
|
||||
scene.render.fps_base = 1.0
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
|
||||
# 清掉可能存在的旧序列，保证幂等
|
||||
if scene.sequence_editor:
|
||||
bpy.ops.sequencer.select_all(action='SELECT')
|
||||
bpy.ops.sequencer.delete()
|
||||
se = scene.sequence_editor_create()
|
||||
|
||||
start_frame = int(round(START * FPS))
|
||||
dur_frames = max(1, int(round((END - START) * FPS)))
|
||||
|
||||
# 1) 导入影片 + 音轨（new_sound 在无音轨素材上会抛 RuntimeError，忽略即可）
|
||||
movie = se.sequences.new_movie(name="clip", filepath=SRC, channel=1, frame_start=1 - start_frame)
|
||||
try:
|
||||
sound = se.sequences.new_sound(name="audio", filepath=SRC, channel=2, frame_start=1 - start_frame)
|
||||
except RuntimeError:
|
||||
sound = None
|
||||
|
||||
# 2) 裁剪 [START, END]：偏移掉片头，再固定成片时长
|
||||
for strip in (movie, sound):
|
||||
if strip is None:
|
||||
continue
|
||||
strip.frame_offset_start = start_frame
|
||||
strip.frame_final_duration = dur_frames
|
||||
|
||||
top_channel = 3
|
||||
|
||||
# 3) 慢动作：SPEED 特效条（MULTIPLY 模式，speed_factor = 1/倍率）
|
||||
if SLOWMO:
|
||||
speed = se.sequences.new_effect(
|
||||
name="slowmo", type='SPEED', channel=top_channel,
|
||||
frame_start=1, frame_end=1 + dur_frames, seq1=movie,
|
||||
)
|
||||
speed.use_default_fade = False
|
||||
speed.speed_control = 'MULTIPLY'
|
||||
speed.speed_factor = 1.0 / SLOWMO
|
||||
top_channel += 1
|
||||
# 放慢后成片总帧数按倍率拉长
|
||||
render_dur = int(round(dur_frames * SLOWMO))
|
||||
movie.frame_final_duration = render_dur
|
||||
else:
|
||||
render_dur = dur_frames
|
||||
|
||||
# 4) 字幕：TEXT 特效条，底部居中带半透明底框
|
||||
if SUBTITLE:
|
||||
txt = se.sequences.new_effect(
|
||||
name="subtitle", type='TEXT', channel=top_channel,
|
||||
frame_start=1, frame_end=1 + render_dur,
|
||||
)
|
||||
txt.text = SUBTITLE
|
||||
txt.font_size = 100
|
||||
txt.location = (0.5, 0.12)
|
||||
txt.align_x = 'CENTER'
|
||||
txt.align_y = 'BOTTOM'
|
||||
txt.use_box = True
|
||||
txt.box_color = (0.0, 0.0, 0.0, 0.6)
|
||||
|
||||
# 5) 渲染范围 + 输出为 mp4(H.264+AAC)
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = render_dur
|
||||
|
||||
r = scene.render
|
||||
r.image_settings.file_format = 'FFMPEG'
|
||||
r.ffmpeg.format = 'MPEG4'
|
||||
r.ffmpeg.codec = 'H264'
|
||||
r.ffmpeg.audio_codec = 'AAC'
|
||||
r.filepath = OUT
|
||||
os.makedirs(os.path.dirname(OUT) or ".", exist_ok=True)
|
||||
|
||||
bpy.ops.render.render(animation=True)
|
||||
print("BLENDER_RENDER_DONE", OUT)
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
+42
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"url": "https://download.blender.org/peach/trailer/trailer_480p.mov",
|
||||
"title": "Big Buck Bunny trailer (Blender Foundation Peach Open Movie Project)",
|
||||
"license": "Creative Commons Attribution 3.0",
|
||||
"license_url": "https://creativecommons.org/licenses/by/3.0/",
|
||||
"sha256": "36801b74638c12be9aa587e93cd18edfc9bc51a1c089ab2a19ee42beed9f497d",
|
||||
"probe": {
|
||||
"programs": [],
|
||||
"stream_groups": [],
|
||||
"streams": [
|
||||
{
|
||||
"index": 0,
|
||||
"codec_name": "h264",
|
||||
"codec_type": "video",
|
||||
"width": 853,
|
||||
"height": 480,
|
||||
"r_frame_rate": "25/1"
|
||||
},
|
||||
{
|
||||
"index": 1,
|
||||
"codec_name": "aac",
|
||||
"codec_type": "audio",
|
||||
"sample_rate": "48000",
|
||||
"channels": 6,
|
||||
"r_frame_rate": "0/0"
|
||||
}
|
||||
],
|
||||
"format": {
|
||||
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
|
||||
"duration": "32.995000",
|
||||
"size": "11061011"
|
||||
}
|
||||
},
|
||||
"naturally_occurring_scenes": true,
|
||||
"ground_truth": {
|
||||
"start": 9.08,
|
||||
"end": 11.2,
|
||||
"target": "the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title",
|
||||
"method": "Human-labeled target shot bounded by ffmpeg scene-change frames; source inspection found transitions at 9.08 and 11.20 seconds.",
|
||||
"ffmpeg_scene_threshold": 0.35
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 179 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 258 KiB |
+42
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"url": "https://download.blender.org/peach/trailer/trailer_480p.mov",
|
||||
"title": "Big Buck Bunny trailer (Blender Foundation Peach Open Movie Project)",
|
||||
"license": "Creative Commons Attribution 3.0",
|
||||
"license_url": "https://creativecommons.org/licenses/by/3.0/",
|
||||
"sha256": "36801b74638c12be9aa587e93cd18edfc9bc51a1c089ab2a19ee42beed9f497d",
|
||||
"probe": {
|
||||
"programs": [],
|
||||
"stream_groups": [],
|
||||
"streams": [
|
||||
{
|
||||
"index": 0,
|
||||
"codec_name": "h264",
|
||||
"codec_type": "video",
|
||||
"width": 853,
|
||||
"height": 480,
|
||||
"r_frame_rate": "25/1"
|
||||
},
|
||||
{
|
||||
"index": 1,
|
||||
"codec_name": "aac",
|
||||
"codec_type": "audio",
|
||||
"sample_rate": "48000",
|
||||
"channels": 6,
|
||||
"r_frame_rate": "0/0"
|
||||
}
|
||||
],
|
||||
"format": {
|
||||
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
|
||||
"duration": "32.995000",
|
||||
"size": "11061011"
|
||||
}
|
||||
},
|
||||
"naturally_occurring_scenes": true,
|
||||
"ground_truth": {
|
||||
"start": 9.08,
|
||||
"end": 11.2,
|
||||
"target": "the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title",
|
||||
"method": "Human-labeled target shot bounded by ffmpeg scene-change frames; source inspection found transitions at 9.08 and 11.20 seconds.",
|
||||
"ffmpeg_scene_threshold": 0.35
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 179 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 258 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 30 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 30 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 382 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 440 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 440 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 192 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 284 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 23 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 284 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 330 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 116 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 179 KiB |
+186
@@ -0,0 +1,186 @@
|
||||
{
|
||||
"request": "Cut out the shot of the large white rabbit walking alone in the sunny meadow, slow it to 1.5x duration, and add the subtitle BIG BUNNY along the bottom.",
|
||||
"target": "the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title",
|
||||
"coarse_interval_s": 10,
|
||||
"coarse_frames": [
|
||||
{
|
||||
"timestamp_s": 0.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/coarse/frame-000.00.png",
|
||||
"sha256": "cdd5c7f27f84347a40eb467c356c5a2fda3facdad9057f246034e355b89dcfdd",
|
||||
"bytes": 1580
|
||||
},
|
||||
{
|
||||
"timestamp_s": 10.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/coarse/frame-010.00.png",
|
||||
"sha256": "bdf01683fab23e1e98099d0f6e19596eabcd96ddb1c643b3d0c6749b5f2995a9",
|
||||
"bytes": 288771
|
||||
},
|
||||
{
|
||||
"timestamp_s": 20.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/coarse/frame-020.00.png",
|
||||
"sha256": "713c2f7a5db41b01640dbae71ef4d5057c382dd1480d9e31bcc3f6d05c9da52b",
|
||||
"bytes": 183618
|
||||
},
|
||||
{
|
||||
"timestamp_s": 30.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/coarse/frame-030.00.png",
|
||||
"sha256": "ce5d5733e716607000f9d6aef75dbb9b31279dd50e7db7c6c1e714819624224a",
|
||||
"bytes": 264372
|
||||
},
|
||||
{
|
||||
"timestamp_s": 31.495,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/coarse/frame-031.50.png",
|
||||
"sha256": "57e555d305ac18228155454935f9a411a9e21ccc092348997f8e3299d4011b89",
|
||||
"bytes": 40146
|
||||
}
|
||||
],
|
||||
"coarse_result": {
|
||||
"start": 5.0,
|
||||
"end": 15.0,
|
||||
"reason": "The 10.00s frame shows the large white rabbit alone in a sunny green meadow with green grass, mountains, and clouds. The next frame at 20.00s features three rodents, indicating the rabbit shot ends before 20.00s. It starts after the 'ONE BIG RABBIT' title (before 10.00s), so the rough interval is 5.0-15.0s."
|
||||
},
|
||||
"fine_interval_s": 1,
|
||||
"fine_window": [
|
||||
0,
|
||||
20
|
||||
],
|
||||
"fine_frames": [
|
||||
{
|
||||
"timestamp_s": 0.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-000.00.png",
|
||||
"sha256": "cdd5c7f27f84347a40eb467c356c5a2fda3facdad9057f246034e355b89dcfdd",
|
||||
"bytes": 1580
|
||||
},
|
||||
{
|
||||
"timestamp_s": 1.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-001.00.png",
|
||||
"sha256": "702f6c8f994090d5f989709bea4372d9cfccb64ea4e09e8245639be85436ddc0",
|
||||
"bytes": 30336
|
||||
},
|
||||
{
|
||||
"timestamp_s": 2.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-002.00.png",
|
||||
"sha256": "0ad215be811389364b7616996f62082234c5759d2f5e489b50c2c794933dcb11",
|
||||
"bytes": 30702
|
||||
},
|
||||
{
|
||||
"timestamp_s": 3.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-003.00.png",
|
||||
"sha256": "ac7283cb0e9b36e87fffa5ef702b4ebc839c2cae9d68c2486ad6b7c6137281de",
|
||||
"bytes": 31566
|
||||
},
|
||||
{
|
||||
"timestamp_s": 4.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-004.00.png",
|
||||
"sha256": "83e61400d490cb6d008465e31f72466b14b8efbc25695003e571715a5ecf2609",
|
||||
"bytes": 391627
|
||||
},
|
||||
{
|
||||
"timestamp_s": 5.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-005.00.png",
|
||||
"sha256": "7bf0f2f437e8781ac96e6a27b41f975189c3784dbd3368672d43fd4b2d92813a",
|
||||
"bytes": 451029
|
||||
},
|
||||
{
|
||||
"timestamp_s": 6.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-006.00.png",
|
||||
"sha256": "618ee855deb607995c0be98d0a64db5614c4f851837af105e0672822947fe361",
|
||||
"bytes": 451002
|
||||
},
|
||||
{
|
||||
"timestamp_s": 7.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-007.00.png",
|
||||
"sha256": "40e9724ee1834abee7d12002cd83a4120738b0ffd0a294741c25c2135edb02b8",
|
||||
"bytes": 196294
|
||||
},
|
||||
{
|
||||
"timestamp_s": 8.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-008.00.png",
|
||||
"sha256": "1da6191f1a0c6ccc987a8e5d0682d7e664aa4ce8b8ce31d232bec5ae695b0c4e",
|
||||
"bytes": 25229
|
||||
},
|
||||
{
|
||||
"timestamp_s": 9.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-009.00.png",
|
||||
"sha256": "c0358569f2defb8cfdcd0c73393fc2e75cab6168920819a7c0d28395389132b7",
|
||||
"bytes": 26259
|
||||
},
|
||||
{
|
||||
"timestamp_s": 10.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-010.00.png",
|
||||
"sha256": "bdf01683fab23e1e98099d0f6e19596eabcd96ddb1c643b3d0c6749b5f2995a9",
|
||||
"bytes": 288771
|
||||
},
|
||||
{
|
||||
"timestamp_s": 11.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-011.00.png",
|
||||
"sha256": "c1abe47a8bd0e10472b75a472cea78ff24613c64ccc74ffba9b566d71336c4f4",
|
||||
"bytes": 290263
|
||||
},
|
||||
{
|
||||
"timestamp_s": 12.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-012.00.png",
|
||||
"sha256": "5833475f37ad6ccc7f98c953193368b11c4f3390d1633e224b0a43514c7aa2ec",
|
||||
"bytes": 23585
|
||||
},
|
||||
{
|
||||
"timestamp_s": 13.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-013.00.png",
|
||||
"sha256": "f14dca5370861a1b1f1ee95d1e00172314f27e60ec9f3451a16e08efb46cabf4",
|
||||
"bytes": 24310
|
||||
},
|
||||
{
|
||||
"timestamp_s": 14.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-014.00.png",
|
||||
"sha256": "ab5ed49e0a1267d80e53bc524db96a521aedf419d4a2e766463bd58fc2cbd96a",
|
||||
"bytes": 290328
|
||||
},
|
||||
{
|
||||
"timestamp_s": 15.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-015.00.png",
|
||||
"sha256": "101c7a220045bdad559ffabcf2345a307c4b2a8ace71c1f289f861cb5ac8dfcf",
|
||||
"bytes": 337454
|
||||
},
|
||||
{
|
||||
"timestamp_s": 16.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-016.00.png",
|
||||
"sha256": "6b674c7689ae86485ce112daa9a14cbd4334d3c41cbc12bcc8a06b0c34c3d0bd",
|
||||
"bytes": 118937
|
||||
},
|
||||
{
|
||||
"timestamp_s": 17.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-017.00.png",
|
||||
"sha256": "44134f20bffcd9fb57c3ca421a7be03fa2596962423a955140bf44e3d5c2582b",
|
||||
"bytes": 24201
|
||||
},
|
||||
{
|
||||
"timestamp_s": 18.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-018.00.png",
|
||||
"sha256": "bf44d8b974343063293c92d2933edd0e9a711802d44753f5c0c8b4f4cf3406aa",
|
||||
"bytes": 25412
|
||||
},
|
||||
{
|
||||
"timestamp_s": 19.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-019.00.png",
|
||||
"sha256": "7d2fdecd1cff9d025880e069d111c9e8aa0c488dd1e570b62f5147a136543271",
|
||||
"bytes": 78776
|
||||
},
|
||||
{
|
||||
"timestamp_s": 20.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/frames/fine/frame-020.00.png",
|
||||
"sha256": "713c2f7a5db41b01640dbae71ef4d5057c382dd1480d9e31bcc3f6d05c9da52b",
|
||||
"bytes": 183618
|
||||
}
|
||||
],
|
||||
"fine_result": {
|
||||
"start": 9.0,
|
||||
"end": 11.0,
|
||||
"reason": "Continuous shot of large white rabbit in sunny green meadow between 'ONE BIG RABBIT' title (ends at 8.00s) and 'THREE RODENTS' title (starts at 12.00s), visible in 10.00s and 11.00s samples."
|
||||
},
|
||||
"ground_truth": {
|
||||
"start": 9.08,
|
||||
"end": 11.2
|
||||
},
|
||||
"start_error_s": 0.08,
|
||||
"end_error_s": 0.2
|
||||
}
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
import bpy
|
||||
|
||||
# Clear default objects
|
||||
bpy.ops.object.select_all(action='SELECT')
|
||||
bpy.ops.object.delete()
|
||||
|
||||
# Set up scene and sequence editor
|
||||
scene = bpy.context.scene
|
||||
scene.sequence_editor = scene.sequence_editor_create()
|
||||
se = scene.sequence_editor
|
||||
|
||||
# File paths
|
||||
input_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/source.mov'
|
||||
output_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/negative_control.mp4'
|
||||
|
||||
# Video parameters
|
||||
source_fps = 25
|
||||
trim_start_sec = 0.0
|
||||
trim_end_sec = 3.0
|
||||
trim_duration_sec = trim_end_sec - trim_start_sec
|
||||
trim_duration_frames = int(trim_duration_sec * source_fps) # 75 frames
|
||||
output_duration_sec = trim_duration_sec * 1.5 # 4.5 seconds
|
||||
output_duration_frames = int(output_duration_sec * source_fps) # 113 frames (rounded up from 112.5)
|
||||
|
||||
speed_factor = trim_duration_sec / output_duration_sec # 0.666...
|
||||
|
||||
# Add video strip
|
||||
video_strip = se.sequences.new_movie(
|
||||
name='Video',
|
||||
filepath=input_path,
|
||||
channel=2,
|
||||
frame_start=1
|
||||
)
|
||||
video_strip.frame_offset_start = 0 # Start at 0s of source
|
||||
video_strip.frame_final_duration = trim_duration_frames # Trim to 3 seconds
|
||||
|
||||
# Add audio strip
|
||||
audio_strip = se.sequences.new_sound(
|
||||
name='Audio',
|
||||
filepath=input_path,
|
||||
channel=1,
|
||||
frame_start=1
|
||||
)
|
||||
audio_strip.frame_offset_start = 0
|
||||
audio_strip.frame_final_duration = trim_duration_frames
|
||||
|
||||
# Add speed effect to video
|
||||
speed_video = se.sequences.new_effect(
|
||||
name='Speed_Video',
|
||||
type='SPEED',
|
||||
channel=3,
|
||||
frame_start=1,
|
||||
seq1=video_strip
|
||||
)
|
||||
speed_video.speed_factor = speed_factor
|
||||
speed_video.frame_final_duration = output_duration_frames
|
||||
|
||||
# Add speed effect to audio
|
||||
speed_audio = se.sequences.new_effect(
|
||||
name='Speed_Audio',
|
||||
type='SPEED',
|
||||
channel=2,
|
||||
frame_start=1,
|
||||
seq1=audio_strip
|
||||
)
|
||||
speed_audio.speed_factor = speed_factor
|
||||
speed_audio.frame_final_duration = output_duration_frames
|
||||
|
||||
# Add subtitle text strip
|
||||
text_strip = se.sequences.new_effect(
|
||||
name='Subtitle',
|
||||
type='TEXT',
|
||||
channel=4,
|
||||
frame_start=1,
|
||||
frame_end=1 + output_duration_frames - 1
|
||||
)
|
||||
text_strip.text = 'BIG BUNNY'
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.font_size = 48
|
||||
text_strip.color = (1.0, 1.0, 1.0, 1.0) # White text
|
||||
|
||||
# Subtitle background box
|
||||
text_strip.use_box = True
|
||||
text_strip.box_color = (0.0, 0.0, 0.0, 0.5) # Semi-transparent black
|
||||
text_strip.box_margin_left = 10
|
||||
text_strip.box_margin_right = 10
|
||||
text_strip.box_margin_top = 10
|
||||
text_strip.box_margin_bottom = 10
|
||||
|
||||
# Render settings
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
scene.render.fps = source_fps
|
||||
scene.render.fps_base = 1.0
|
||||
|
||||
# Output format settings
|
||||
scene.render.image_settings.file_format = 'MPEG4'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H.264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.ffmpeg.audio_bitrate = 192
|
||||
scene.render.ffmpeg.constant_rate_factor = 'MEDIUM'
|
||||
scene.render.filepath = output_path
|
||||
|
||||
# Set scene frame range
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = output_duration_frames
|
||||
|
||||
# Render animation
|
||||
bpy.ops.render.render(animation=True)
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
import bpy
|
||||
import math
|
||||
|
||||
# Clear default objects
|
||||
bpy.ops.object.select_all(action='SELECT')
|
||||
bpy.ops.object.delete()
|
||||
|
||||
# Set up scene
|
||||
scene = bpy.context.scene
|
||||
|
||||
# Render settings
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
scene.render.fps = 25
|
||||
scene.render.fps_base = 1.0
|
||||
scene.render.image_settings.file_format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.filepath = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/negative_control.mp4'
|
||||
|
||||
# Set up Video Sequence Editor
|
||||
if not scene.sequence_editor:
|
||||
scene.sequence_editor_create()
|
||||
scene.sequence_editor.sequences.clear()
|
||||
|
||||
# Import source movie
|
||||
input_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/source.mov'
|
||||
movie_strip = scene.sequence_editor.sequences.new_movie(
|
||||
name='Source',
|
||||
filepath=input_path,
|
||||
channel=1,
|
||||
frame_start=1
|
||||
)
|
||||
# Trim to 3 seconds (75 frames at 25fps)
|
||||
movie_strip.frame_final_duration = 75
|
||||
|
||||
# Add speed effect (1.5x duration = 2/3 speed)
|
||||
speed_strip = scene.sequence_editor.sequences.new_effect(
|
||||
name='Slow',
|
||||
type='SPEED',
|
||||
channel=2,
|
||||
frame_start=1,
|
||||
frame_end=movie_strip.frame_final_end,
|
||||
seq1=movie_strip
|
||||
)
|
||||
speed_strip.speed_factor = 2/3 # 3s * 1.5 = 4.5s duration
|
||||
|
||||
# Calculate scene frame range
|
||||
slowed_frames = movie_strip.frame_final_duration / speed_strip.speed_factor
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = math.ceil(scene.frame_start + slowed_frames - 1)
|
||||
|
||||
# Create sequence editor if not exists
|
||||
if not scene.sequence_editor:
|
||||
scene.sequence_editor_create()
|
||||
|
||||
# Subtitle background (semi-transparent dark box)
|
||||
color_strip = scene.sequence_editor.sequences.new_effect(
|
||||
name='SubtitleBG',
|
||||
type='COLOR',
|
||||
channel=3,
|
||||
frame_start=scene.frame_start,
|
||||
frame_end=scene.frame_end
|
||||
)
|
||||
color_strip.color = (0.0, 0.0, 0.0) # Black
|
||||
color_strip.alpha = 0.5 # Semi-transparent
|
||||
color_strip.transform.scale_x = 0.4 # Box width
|
||||
color_strip.transform.scale_y = 0.15 # Box height
|
||||
color_strip.transform.translate_y = -0.4 # Bottom position
|
||||
color_strip.transform.align_x = 'CENTER' # Center horizontally
|
||||
|
||||
# Subtitle text
|
||||
text_strip = scene.sequence_editor.sequences.new_effect(
|
||||
name='SubtitleText',
|
||||
type='TEXT',
|
||||
channel=4,
|
||||
frame_start=scene.frame_start,
|
||||
frame_end=scene.frame_end
|
||||
)
|
||||
text_strip.text = 'BIG BUNNY'
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'CENTER'
|
||||
text_strip.font_size = 40
|
||||
text_strip.color = (1.0, 1.0, 1.0) # White text
|
||||
text_strip.transform.translate_y = -0.4 # Match background Y position
|
||||
text_strip.transform.align_x = 'CENTER'
|
||||
|
||||
# Render animation
|
||||
bpy.ops.render.render(animation=True)
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
import bpy
|
||||
|
||||
# Clear existing data to start fresh
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
# Get the current scene
|
||||
scene = bpy.context.scene
|
||||
|
||||
# Set render resolution and FPS
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
scene.render.fps = 25
|
||||
scene.render.fps_base = 1.0
|
||||
|
||||
# Set render output settings
|
||||
scene.render.image_settings.file_format = 'FFMPEG'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.ffmpeg.constant_rate_factor = 'MEDIUM'
|
||||
scene.render.filepath = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/negative_control.mp4'
|
||||
|
||||
# Create sequence editor
|
||||
scene.sequence_editor_create()
|
||||
seq_ed = scene.sequence_editor
|
||||
|
||||
# Input movie path
|
||||
input_movie = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-050817/source.mov'
|
||||
|
||||
# Add video strip with trim (0.000-3.000s = 0-75 frames at 25fps)
|
||||
video_strip = seq_ed.sequences.new_movie(name='Video', filepath=input_movie, channel=1, frame_start=0)
|
||||
video_strip.frame_offset_start = 0 # Start at source frame 0
|
||||
video_strip.frame_final_duration = 75 # 3s * 25fps = 75 frames
|
||||
|
||||
# Add speed effect to slow down (1.5x duration: 3s → 4.5s = 112.5 frames → 113 frames)
|
||||
speed_effect = seq_ed.sequences.new_effect(name='Speed', type='SPEED', channel=2, frame_start=0, frame_end=113, seq1=video_strip)
|
||||
speed_effect.speed_factor = 2/3 # Slows to 1.5x original duration
|
||||
|
||||
# Add audio strip from movie
|
||||
audio_strip = seq_ed.sequences.new_sound(name='Audio', filepath=input_movie, channel=1, frame_start=0)
|
||||
audio_strip.frame_offset_start = 0 # Match video trim start
|
||||
audio_strip.frame_final_duration = 75 # Match video trim duration
|
||||
|
||||
# Add speed effect to audio
|
||||
audio_speed = seq_ed.sequences.new_effect(name='AudioSpeed', type='SPEED', channel=2, frame_start=0, frame_end=113, seq1=audio_strip)
|
||||
audio_speed.speed_factor = 2/3
|
||||
audio_speed.use_audio = True
|
||||
|
||||
# Create subtitle background (semi-transparent dark box)
|
||||
bg_strip = seq_ed.sequences.new_effect(name='SubtitleBG', type='COLOR', channel=3, frame_start=0, frame_end=113)
|
||||
bg_strip.color = (0.1, 0.1, 0.1, 0.7) # Dark gray with 70% transparency
|
||||
bg_strip.align_x = 'CENTER'
|
||||
bg_strip.align_y = 'BOTTOM'
|
||||
bg_strip.transform.scale_x = 0.5 # Width of background box
|
||||
bg_strip.transform.scale_y = 0.15 # Height of background box
|
||||
bg_strip.transform.location_y = 0.03 # Position slightly above bottom edge
|
||||
|
||||
# Create subtitle text
|
||||
text_strip = seq_ed.sequences.new_effect(name='SubtitleText', type='TEXT', channel=4, frame_start=0, frame_end=113)
|
||||
text_strip.text = 'BIG BUNNY'
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.font_size = 48
|
||||
text_strip.color = (1.0, 1.0, 1.0) # White text
|
||||
text_strip.transform.location_y = 0.05 # Align with background box
|
||||
|
||||
# Set scene frame range to cover slowed duration (4.5 seconds = 112.5 frames → 113 frames)
|
||||
scene.frame_start = 0
|
||||
scene.frame_end = 113
|
||||
|
||||
# Render the animation
|
||||
bpy.ops.render.render(animation=True)
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"url": "https://download.blender.org/peach/trailer/trailer_480p.mov",
|
||||
"title": "Big Buck Bunny trailer (Blender Foundation Peach Open Movie Project)",
|
||||
"license": "Creative Commons Attribution 3.0",
|
||||
"license_url": "https://creativecommons.org/licenses/by/3.0/",
|
||||
"sha256": "36801b74638c12be9aa587e93cd18edfc9bc51a1c089ab2a19ee42beed9f497d",
|
||||
"probe": {
|
||||
"programs": [],
|
||||
"stream_groups": [],
|
||||
"streams": [
|
||||
{
|
||||
"index": 0,
|
||||
"codec_name": "h264",
|
||||
"codec_type": "video",
|
||||
"width": 853,
|
||||
"height": 480,
|
||||
"r_frame_rate": "25/1"
|
||||
},
|
||||
{
|
||||
"index": 1,
|
||||
"codec_name": "aac",
|
||||
"codec_type": "audio",
|
||||
"sample_rate": "48000",
|
||||
"channels": 6,
|
||||
"r_frame_rate": "0/0"
|
||||
}
|
||||
],
|
||||
"format": {
|
||||
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
|
||||
"duration": "32.995000",
|
||||
"size": "11061011"
|
||||
}
|
||||
},
|
||||
"naturally_occurring_scenes": true,
|
||||
"ground_truth": {
|
||||
"start": 9.08,
|
||||
"end": 11.2,
|
||||
"target": "the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title",
|
||||
"method": "Human-labeled target shot bounded by ffmpeg scene-change frames; source inspection found transitions at 9.08 and 11.20 seconds.",
|
||||
"ffmpeg_scene_threshold": 0.35
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 179 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 258 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 440 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 440 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 192 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 282 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 284 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 23 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 284 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 330 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 116 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 77 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 179 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 94 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 188 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 206 KiB |
+186
@@ -0,0 +1,186 @@
|
||||
{
|
||||
"request": "Cut out the shot of the large white rabbit walking alone in the sunny meadow, slow it to 1.5x duration, and add the subtitle BIG BUNNY along the bottom.",
|
||||
"target": "the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title",
|
||||
"coarse_interval_s": 10,
|
||||
"coarse_frames": [
|
||||
{
|
||||
"timestamp_s": 0.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-000.00.png",
|
||||
"sha256": "cdd5c7f27f84347a40eb467c356c5a2fda3facdad9057f246034e355b89dcfdd",
|
||||
"bytes": 1580
|
||||
},
|
||||
{
|
||||
"timestamp_s": 10.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-010.00.png",
|
||||
"sha256": "bdf01683fab23e1e98099d0f6e19596eabcd96ddb1c643b3d0c6749b5f2995a9",
|
||||
"bytes": 288771
|
||||
},
|
||||
{
|
||||
"timestamp_s": 20.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-020.00.png",
|
||||
"sha256": "713c2f7a5db41b01640dbae71ef4d5057c382dd1480d9e31bcc3f6d05c9da52b",
|
||||
"bytes": 183618
|
||||
},
|
||||
{
|
||||
"timestamp_s": 30.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-030.00.png",
|
||||
"sha256": "ce5d5733e716607000f9d6aef75dbb9b31279dd50e7db7c6c1e714819624224a",
|
||||
"bytes": 264372
|
||||
},
|
||||
{
|
||||
"timestamp_s": 31.495,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-031.50.png",
|
||||
"sha256": "57e555d305ac18228155454935f9a411a9e21ccc092348997f8e3299d4011b89",
|
||||
"bytes": 40146
|
||||
}
|
||||
],
|
||||
"coarse_result": {
|
||||
"start": 10.0,
|
||||
"end": 20.0,
|
||||
"reason": "The 10.00s frame shows the large white rabbit standing alone in a sunny green meadow with rolling hills and a clear sky, matching the target description. The next frame at 20.00s transitions to three rodents in a darker forested area (likely the 'THREE RODENTS' title content), indicating the rabbit shot is continuous between these timestamps."
|
||||
},
|
||||
"fine_interval_s": 1,
|
||||
"fine_window": [
|
||||
5,
|
||||
25
|
||||
],
|
||||
"fine_frames": [
|
||||
{
|
||||
"timestamp_s": 5.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-005.00.png",
|
||||
"sha256": "7bf0f2f437e8781ac96e6a27b41f975189c3784dbd3368672d43fd4b2d92813a",
|
||||
"bytes": 451029
|
||||
},
|
||||
{
|
||||
"timestamp_s": 6.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-006.00.png",
|
||||
"sha256": "618ee855deb607995c0be98d0a64db5614c4f851837af105e0672822947fe361",
|
||||
"bytes": 451002
|
||||
},
|
||||
{
|
||||
"timestamp_s": 7.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-007.00.png",
|
||||
"sha256": "40e9724ee1834abee7d12002cd83a4120738b0ffd0a294741c25c2135edb02b8",
|
||||
"bytes": 196294
|
||||
},
|
||||
{
|
||||
"timestamp_s": 8.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-008.00.png",
|
||||
"sha256": "1da6191f1a0c6ccc987a8e5d0682d7e664aa4ce8b8ce31d232bec5ae695b0c4e",
|
||||
"bytes": 25229
|
||||
},
|
||||
{
|
||||
"timestamp_s": 9.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-009.00.png",
|
||||
"sha256": "c0358569f2defb8cfdcd0c73393fc2e75cab6168920819a7c0d28395389132b7",
|
||||
"bytes": 26259
|
||||
},
|
||||
{
|
||||
"timestamp_s": 10.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-010.00.png",
|
||||
"sha256": "bdf01683fab23e1e98099d0f6e19596eabcd96ddb1c643b3d0c6749b5f2995a9",
|
||||
"bytes": 288771
|
||||
},
|
||||
{
|
||||
"timestamp_s": 11.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-011.00.png",
|
||||
"sha256": "c1abe47a8bd0e10472b75a472cea78ff24613c64ccc74ffba9b566d71336c4f4",
|
||||
"bytes": 290263
|
||||
},
|
||||
{
|
||||
"timestamp_s": 12.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-012.00.png",
|
||||
"sha256": "5833475f37ad6ccc7f98c953193368b11c4f3390d1633e224b0a43514c7aa2ec",
|
||||
"bytes": 23585
|
||||
},
|
||||
{
|
||||
"timestamp_s": 13.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-013.00.png",
|
||||
"sha256": "f14dca5370861a1b1f1ee95d1e00172314f27e60ec9f3451a16e08efb46cabf4",
|
||||
"bytes": 24310
|
||||
},
|
||||
{
|
||||
"timestamp_s": 14.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-014.00.png",
|
||||
"sha256": "ab5ed49e0a1267d80e53bc524db96a521aedf419d4a2e766463bd58fc2cbd96a",
|
||||
"bytes": 290328
|
||||
},
|
||||
{
|
||||
"timestamp_s": 15.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-015.00.png",
|
||||
"sha256": "101c7a220045bdad559ffabcf2345a307c4b2a8ace71c1f289f861cb5ac8dfcf",
|
||||
"bytes": 337454
|
||||
},
|
||||
{
|
||||
"timestamp_s": 16.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-016.00.png",
|
||||
"sha256": "6b674c7689ae86485ce112daa9a14cbd4334d3c41cbc12bcc8a06b0c34c3d0bd",
|
||||
"bytes": 118937
|
||||
},
|
||||
{
|
||||
"timestamp_s": 17.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-017.00.png",
|
||||
"sha256": "44134f20bffcd9fb57c3ca421a7be03fa2596962423a955140bf44e3d5c2582b",
|
||||
"bytes": 24201
|
||||
},
|
||||
{
|
||||
"timestamp_s": 18.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-018.00.png",
|
||||
"sha256": "bf44d8b974343063293c92d2933edd0e9a711802d44753f5c0c8b4f4cf3406aa",
|
||||
"bytes": 25412
|
||||
},
|
||||
{
|
||||
"timestamp_s": 19.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-019.00.png",
|
||||
"sha256": "7d2fdecd1cff9d025880e069d111c9e8aa0c488dd1e570b62f5147a136543271",
|
||||
"bytes": 78776
|
||||
},
|
||||
{
|
||||
"timestamp_s": 20.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-020.00.png",
|
||||
"sha256": "713c2f7a5db41b01640dbae71ef4d5057c382dd1480d9e31bcc3f6d05c9da52b",
|
||||
"bytes": 183618
|
||||
},
|
||||
{
|
||||
"timestamp_s": 21.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-021.00.png",
|
||||
"sha256": "c9758159919eb5f8612e3af01258d1c7792d446ca5c08add2ff3dcd3b97e7617",
|
||||
"bytes": 96371
|
||||
},
|
||||
{
|
||||
"timestamp_s": 22.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-022.00.png",
|
||||
"sha256": "54b41278b256654341515bee8f7adb3166bdb15c90f3da9f7f47eb72cad77d33",
|
||||
"bytes": 192864
|
||||
},
|
||||
{
|
||||
"timestamp_s": 23.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-023.00.png",
|
||||
"sha256": "3e49ae19ac287a5acdcff0db46d93d3ca77f98d039d9c53f07210466a323b5b3",
|
||||
"bytes": 22428
|
||||
},
|
||||
{
|
||||
"timestamp_s": 24.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-024.00.png",
|
||||
"sha256": "6eb1ae362db441052be6685922270fcd86572c8b223e99a41a519424ac113c0d",
|
||||
"bytes": 24410
|
||||
},
|
||||
{
|
||||
"timestamp_s": 25.0,
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-025.00.png",
|
||||
"sha256": "354595be0073fcaed614da1ff41177f01e54f74b63f5614ed5033a9c00e0a773",
|
||||
"bytes": 211411
|
||||
}
|
||||
],
|
||||
"fine_result": {
|
||||
"start": 9.0,
|
||||
"end": 12.0,
|
||||
"reason": "Starts immediately after the 'ONE BIG RABBIT' title (ends at 9.00s) and ends just before the 'THREE RODENTS' title (starts at 12.00s), with continuous visuals of the large white rabbit in the sunny green meadow at 10.00s and 11.00s."
|
||||
},
|
||||
"ground_truth": {
|
||||
"start": 9.08,
|
||||
"end": 11.2
|
||||
},
|
||||
"start_error_s": 0.08,
|
||||
"end_error_s": 0.8
|
||||
}
|
||||
+546
@@ -0,0 +1,546 @@
|
||||
{
|
||||
"calls": [
|
||||
{
|
||||
"purpose": "coarse_10_second_visual_localization",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "These are frames from one contiguous real trailer sampled about every 10 seconds.\nLocate this target: the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title.\nReturn JSON {\"start\":seconds,\"end\":seconds,\"reason\":\"visual evidence\"} with a rough continuous interval. Use timestamps and visual content only."
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=0.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-000.00.png",
|
||||
"sha256": "cdd5c7f27f84347a40eb467c356c5a2fda3facdad9057f246034e355b89dcfdd",
|
||||
"bytes": 1580
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=10.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-010.00.png",
|
||||
"sha256": "bdf01683fab23e1e98099d0f6e19596eabcd96ddb1c643b3d0c6749b5f2995a9",
|
||||
"bytes": 288771
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=20.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-020.00.png",
|
||||
"sha256": "713c2f7a5db41b01640dbae71ef4d5057c382dd1480d9e31bcc3f6d05c9da52b",
|
||||
"bytes": 183618
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=30.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-030.00.png",
|
||||
"sha256": "ce5d5733e716607000f9d6aef75dbb9b31279dd50e7db7c6c1e714819624224a",
|
||||
"bytes": 264372
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=31.50s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/coarse/frame-031.50.png",
|
||||
"sha256": "57e555d305ac18228155454935f9a411a9e21ccc092348997f8e3299d4011b89",
|
||||
"bytes": 40146
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 700,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536012758415b00b15f7b15a122bee0bef9a71075862cfbb",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"start\":10.00,\"end\":20.00,\"reason\":\"The 10.00s frame shows the large white rabbit standing alone in a sunny green meadow with rolling hills and a clear sky, matching the target description. The next frame at 20.00s transitions to three rodents in a darker forested area (likely the 'THREE RODENTS' title content), indicating the rabbit shot is continuous between these timestamps.\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 1686,
|
||||
"completion_tokens": 734,
|
||||
"total_tokens": 2420
|
||||
},
|
||||
"latency_s": 20.057
|
||||
},
|
||||
{
|
||||
"purpose": "fine_1_second_visual_localization",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "These are one-second samples from the narrowed window [5.0, 25.0] seconds of the same real trailer.\nPrecisely locate the boundaries of this one continuous target shot: the single continuous shot showing the large white rabbit walking alone in a sunny green meadow, after the ONE BIG RABBIT title and before the THREE RODENTS title.\nReturn JSON {\"start\":seconds,\"end\":seconds,\"reason\":\"visual boundary evidence\"}. The answer may interpolate between adjacent one-second samples; do not include either neighboring title card."
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=5.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-005.00.png",
|
||||
"sha256": "7bf0f2f437e8781ac96e6a27b41f975189c3784dbd3368672d43fd4b2d92813a",
|
||||
"bytes": 451029
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=6.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-006.00.png",
|
||||
"sha256": "618ee855deb607995c0be98d0a64db5614c4f851837af105e0672822947fe361",
|
||||
"bytes": 451002
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=7.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-007.00.png",
|
||||
"sha256": "40e9724ee1834abee7d12002cd83a4120738b0ffd0a294741c25c2135edb02b8",
|
||||
"bytes": 196294
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=8.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-008.00.png",
|
||||
"sha256": "1da6191f1a0c6ccc987a8e5d0682d7e664aa4ce8b8ce31d232bec5ae695b0c4e",
|
||||
"bytes": 25229
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=9.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-009.00.png",
|
||||
"sha256": "c0358569f2defb8cfdcd0c73393fc2e75cab6168920819a7c0d28395389132b7",
|
||||
"bytes": 26259
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=10.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-010.00.png",
|
||||
"sha256": "bdf01683fab23e1e98099d0f6e19596eabcd96ddb1c643b3d0c6749b5f2995a9",
|
||||
"bytes": 288771
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=11.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-011.00.png",
|
||||
"sha256": "c1abe47a8bd0e10472b75a472cea78ff24613c64ccc74ffba9b566d71336c4f4",
|
||||
"bytes": 290263
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=12.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-012.00.png",
|
||||
"sha256": "5833475f37ad6ccc7f98c953193368b11c4f3390d1633e224b0a43514c7aa2ec",
|
||||
"bytes": 23585
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=13.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-013.00.png",
|
||||
"sha256": "f14dca5370861a1b1f1ee95d1e00172314f27e60ec9f3451a16e08efb46cabf4",
|
||||
"bytes": 24310
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=14.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-014.00.png",
|
||||
"sha256": "ab5ed49e0a1267d80e53bc524db96a521aedf419d4a2e766463bd58fc2cbd96a",
|
||||
"bytes": 290328
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=15.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-015.00.png",
|
||||
"sha256": "101c7a220045bdad559ffabcf2345a307c4b2a8ace71c1f289f861cb5ac8dfcf",
|
||||
"bytes": 337454
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=16.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-016.00.png",
|
||||
"sha256": "6b674c7689ae86485ce112daa9a14cbd4334d3c41cbc12bcc8a06b0c34c3d0bd",
|
||||
"bytes": 118937
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=17.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-017.00.png",
|
||||
"sha256": "44134f20bffcd9fb57c3ca421a7be03fa2596962423a955140bf44e3d5c2582b",
|
||||
"bytes": 24201
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=18.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-018.00.png",
|
||||
"sha256": "bf44d8b974343063293c92d2933edd0e9a711802d44753f5c0c8b4f4cf3406aa",
|
||||
"bytes": 25412
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=19.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-019.00.png",
|
||||
"sha256": "7d2fdecd1cff9d025880e069d111c9e8aa0c488dd1e570b62f5147a136543271",
|
||||
"bytes": 78776
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=20.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-020.00.png",
|
||||
"sha256": "713c2f7a5db41b01640dbae71ef4d5057c382dd1480d9e31bcc3f6d05c9da52b",
|
||||
"bytes": 183618
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=21.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-021.00.png",
|
||||
"sha256": "c9758159919eb5f8612e3af01258d1c7792d446ca5c08add2ff3dcd3b97e7617",
|
||||
"bytes": 96371
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=22.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-022.00.png",
|
||||
"sha256": "54b41278b256654341515bee8f7adb3166bdb15c90f3da9f7f47eb72cad77d33",
|
||||
"bytes": 192864
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=23.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-023.00.png",
|
||||
"sha256": "3e49ae19ac287a5acdcff0db46d93d3ca77f98d039d9c53f07210466a323b5b3",
|
||||
"bytes": 22428
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=24.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-024.00.png",
|
||||
"sha256": "6eb1ae362db441052be6685922270fcd86572c8b223e99a41a519424ac113c0d",
|
||||
"bytes": 24410
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "timestamp=25.00s"
|
||||
},
|
||||
{
|
||||
"type": "image_artifact",
|
||||
"path": "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/frames/fine/frame-025.00.png",
|
||||
"sha256": "354595be0073fcaed614da1ff41177f01e54f74b63f5614ed5033a9c00e0a773",
|
||||
"bytes": 211411
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 700,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536014931215b00b15f7b15a122bee0bef9a7107589f12c6",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"start\":9.0,\"end\":12.0,\"reason\":\"Starts immediately after the 'ONE BIG RABBIT' title (ends at 9.00s) and ends just before the 'THREE RODENTS' title (starts at 12.00s), with continuous visuals of the large white rabbit in the sunny green meadow at 10.00s and 11.00s.\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 6555,
|
||||
"completion_tokens": 1158,
|
||||
"total_tokens": 7713
|
||||
},
|
||||
"latency_s": 31.727
|
||||
},
|
||||
{
|
||||
"purpose": "blender_script_negative-control_attempt_1",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Blender VSE engineer. Return one JSON object only."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Generate a complete Blender 4.3 Python script for this video edit.\n\nInput movie: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\nOutput MP4: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\nSource FPS: 25\nSource size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)\nTrim interval: [0.000, 3.000] seconds\nEffects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.\n\nRequirements:\n- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.\n- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.\n- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).\n- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.\n- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).\n- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.\n- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.\n\nReturn one JSON object only: {\"code\":\"complete executable Python source\"}.\n"
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 7000,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536018101615b00b15f7b15a122bee0bef9a710758b49a93",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"code\":\"import bpy\\n\\n# Clear existing objects to start fresh\\nbpy.ops.object.select_all(action='SELECT')\\nbpy.ops.object.delete()\\n\\n# Configuration paths and parameters\\nINPUT_MOVIE = \\\"/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\\\"\\nOUTPUT_MP4 = \\\"/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\\\"\\nSOURCE_FPS = 25\\nTRIM_DURATION_SECONDS = 3.0\\nSLOW_FACTOR = 1.5\\nRENDER_RESOLUTION = (854, 480)\\n\\n# Calculate frame values\\ntrim_frames = int(TRIM_DURATION_SECONDS * SOURCE_FPS)\\nslowed_frames = int(trim_frames * SLOW_FACTOR)\\nscene = bpy.context.scene\\n\\n# Create sequence editor\\nseq_ed = scene.sequence_editor_create()\\n\\n# Add movie strip with trim\\nmovie_strip = seq_ed.sequences.new_movie(\\n name=\\\"SourceVideo\\\",\\n filepath=INPUT_MOVIE,\\n channel=1,\\n frame_start=1\\n)\\nmovie_strip.frame_offset_start = 0 # Start at beginning of source\\nmovie_strip.frame_final_duration = trim_frames # Trim to 3 seconds\\n\\n# Locate and trim audio strip (auto-created with movie)\\nsound_strip = None\\nfor strip in seq_ed.sequences:\\n if strip.type == 'SOUND' and strip.channel == 2 and strip.frame_start == 1:\\n sound_strip = strip\\nif sound_strip:\\n sound_strip.frame_final_duration = trim_frames # Keep audio at original speed\\n\\n# Add speed effect to video (slow motion)\\nspeed_strip = seq_ed.sequences.new_effect(\\n name=\\\"SlowMotion\\\",\\n type='SPEED',\\n channel=3,\\n frame_start=1,\\n frame_end=1 + slowed_frames - 1,\\n target=movie_strip\\n)\\nspeed_strip.speed_control = 'MULTIPLY'\\nspeed_strip.speed_factor = 1 / SLOW_FACTOR # 1/1.5 speed factor\\n\\n# Add subtitle text strip with background box\\ntext_strip = seq_ed.sequences.new_text(\\n name=\\\"Subtitle\\\",\\n text=\\\"BIG BUNNY\\\",\\n channel=4,\\n frame_start=1,\\n frame_end=1 + slowed_frames - 1\\n)\\ntext_strip.align_x = 'CENTER'\\ntext_strip.align_y = 'BOTTOM'\\ntext_strip.use_box = True\\ntext_strip.box_color = (0.0, 0.0, 0.0, 0.6) # Black semi-transparent box\\ntext_strip.font_size = 48 # Visible size for 854x480 resolution\\n\\n# Configure render settings\\nscene.render.resolution_x = RENDER_RESOLUTION[0]\\nscene.render.resolution_y = RENDER_RESOLUTION[1]\\nscene.render.resolution_percentage = 100\\nscene.render.fps = SOURCE_FPS\\nscene.render.fps_base = 1.0\\nscene.render.image_settings.file_format = 'FFMPEG'\\nscene.render.ffmpeg.format = 'MPEG4'\\nscene.render.ffmpeg.codec = 'H264'\\nscene.render.ffmpeg.audio_codec = 'AAC'\\nscene.render.filepath = OUTPUT_MP4\\n\\n# Set render frame range\\nscene.frame_start = 1\\nscene.frame_end = 1 + slowed_frames - 1\\n\\n# Execute render\\nbpy.ops.render.render(animation=True)\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 614,
|
||||
"completion_tokens": 6404,
|
||||
"total_tokens": 7018
|
||||
},
|
||||
"latency_s": 153.914
|
||||
},
|
||||
{
|
||||
"purpose": "blender_script_negative-control_attempt_2",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Blender VSE engineer. Return one JSON object only."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Generate a complete Blender 4.3 Python script for this video edit.\n\nInput movie: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\nOutput MP4: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\nSource FPS: 25\nSource size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)\nTrim interval: [0.000, 3.000] seconds\nEffects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.\n\nRequirements:\n- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.\n- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.\n- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).\n- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.\n- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).\n- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.\n- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.\n\nReturn one JSON object only: {\"code\":\"complete executable Python source\"}.\n\nPrior executable feedback:\nStatic safety/API validation failed: ValueError: generated script omits requested Blender API operations: ['TEXT']"
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 7000,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536033493315b00b15f7b15a122bee0bef9a710758de0884",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"code\":\"import bpy\\n\\n# Clear existing data\\nbpy.ops.wm.read_factory_settings(use_empty=True)\\nscene = bpy.context.scene\\n\\n# Scene settings\\nscene.render.resolution_x = 854\\nscene.render.resolution_y = 480\\nscene.render.resolution_percentage = 100\\nscene.render.fps = 25\\nscene.render.fps_base = 1.0\\n\\n# Create sequence editor\\nif not scene.sequence_editor:\\n scene.sequence_editor_create()\\nseq_ed = scene.sequence_editor\\nsequences = seq_ed.sequences\\n\\n# File paths\\ninput_movie = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'\\noutput_mp4 = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'\\n\\n# Add movie strip (3 seconds = 75 frames at 25fps)\\nmov_strip = sequences.new_movie(\\n name='SourceMovie',\\n filepath=input_movie,\\n frame_start=1,\\n channel=1\\n)\\nmov_strip.frame_offset_start = 0\\nmov_strip.frame_final_duration = 75 # 3s * 25fps\\n\\n# Locate and trim sound strip\\nsound_strip = next((s for s in sequences if s.type == 'SOUND' and s.frame_start == mov_strip.frame_start), None)\\nif sound_strip:\\n sound_strip.frame_final_duration = 75 # Keep original audio duration\\n\\n# Add speed effect (1.5x slowdown)\\nspeed_strip = sequences.new_effect(\\n name='SlowMotion',\\n type='SPEED',\\n frame_start=1,\\n channel=2,\\n input_1=mov_strip\\n)\\nspeed_strip.speed_control = 'MULTIPLY'\\nspeed_strip.speed_factor = 1 / 1.5\\n\\n# Calculate slowed duration (4.5s = 112.5 frames → 113 frames)\\nslowed_frames = int(4.5 * 25) + 1\\nscene.frame_start = 1\\nscene.frame_end = slowed_frames\\n\\n# Add subtitle text strip\\ntext_strip = sequences.new_effect(\\n name='Subtitle',\\n type='TEXT',\\n frame_start=1,\\n frame_end=scene.frame_end,\\n channel=3\\n)\\ntext_strip.text = 'BIG BUNNY'\\ntext_strip.align_x = 'CENTER'\\ntext_strip.align_y = 'BOTTOM'\\ntext_strip.use_box = True\\ntext_strip.box_color = (0, 0, 0, 0.6)\\ntext_strip.font_size = 48\\n\\n# Render settings\\nscene.render.filepath = output_mp4\\nscene.render.image_settings.file_format = 'FFMPEG'\\nscene.render.ffmpeg.format = 'MPEG4'\\nscene.render.ffmpeg.codec = 'H264'\\nscene.render.ffmpeg.audio_codec = 'AAC'\\nscene.render.ffmpeg.constant_rate_factor = 'MEDIUM'\\n\\n# Render animation\\nbpy.ops.render.render(animation=True)\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 641,
|
||||
"completion_tokens": 4682,
|
||||
"total_tokens": 5323
|
||||
},
|
||||
"latency_s": 103.993
|
||||
},
|
||||
{
|
||||
"purpose": "blender_script_negative-control_attempt_3",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Blender VSE engineer. Return one JSON object only."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Generate a complete Blender 4.3 Python script for this video edit.\n\nInput movie: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\nOutput MP4: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\nSource FPS: 25\nSource size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)\nTrim interval: [0.000, 3.000] seconds\nEffects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.\n\nRequirements:\n- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.\n- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.\n- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).\n- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.\n- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).\n- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.\n- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.\n\nReturn one JSON object only: {\"code\":\"complete executable Python source\"}.\n\nPrior executable feedback:\nBlender exit=0; output_exists=False. Tail of real Blender log:\nBlender 4.3.2 (hash 32f5fdce0a0a built 2024-12-17 02:14:25)\nTraceback (most recent call last):\n File \"/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/scripts/negative-control-attempt-2.py\", line 40, in <module>\n speed_strip = sequences.new_effect(\n ^^^^^^^^^^^^^^^^^^^^^\nTypeError: SequencesTopLevel.new_effect(): was called with invalid keyword argument(s) (input_1), expected (name, type, channel, frame_start, frame_end, seq1, seq2)\n\nBlender quit"
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 7000,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536043966315b00b15f7b15a122bee0bef9a710758ef6add",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"code\":\"import bpy\\nimport os\\n\\n# Clear default objects\\nbpy.ops.object.select_all(action='SELECT')\\nbpy.ops.object.delete()\\n\\n# Set up scene\\nscene = bpy.context.scene\\nscene.render.fps = 25\\nscene.render.fps_base = 1.0\\nscene.render.resolution_x = 854\\nscene.render.resolution_y = 480\\nscene.render.resolution_percentage = 100\\n\\n# Configure output settings\\nscene.render.image_settings.file_format = 'FFMPEG'\\nscene.render.ffmpeg.format = 'MPEG4'\\nscene.render.ffmpeg.codec = 'H264'\\nscene.render.ffmpeg.audio_codec = 'AAC'\\nscene.render.ffmpeg.audio_bitrate = 192\\noutput_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'\\nscene.render.filepath = output_path\\n\\n# Create sequence editor\\nscene.sequence_editor_create()\\nsequences = scene.sequence_editor.sequences\\n\\n# Import source movie and sound\\nsource_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'\\nmovie_strip = sequences.new_movie(name='SourceMovie', filepath=source_path, channel=1, frame_start=0)\\n\\nsound_strip = None\\nfor strip in sequences:\\n if strip.type == 'SOUND' and strip.name.startswith('SourceMovie'):\\n sound_strip = strip\\n break\\nif not sound_strip:\\n sound_strip = sequences.new_sound(name='SourceAudio', filepath=source_path, channel=2, frame_start=0)\\n\\n# Trim to 3-second interval (75 frames at 25fps)\\ntrim_frames = 75 # 3.0s * 25fps\\nmovie_strip.frame_start = 0\\nmovie_strip.frame_final_duration = trim_frames\\nsound_strip.frame_start = 0\\nsound_strip.frame_final_duration = trim_frames\\n\\n# Apply speed effect (1.5x slowdown = 4.5s output)\\nspeed_strip = sequences.new_effect(\\n name='SpeedControl',\\n type='SPEED',\\n channel=2,\\n frame_start=0,\\n frame_end=113, # 75 frames * 1.5 = 112.5 → 113 frames\\n seq1=movie_strip,\\n seq2=None\\n)\\nspeed_strip.speed_control = 'MULTIPLY'\\nspeed_strip.speed_factor = 1/1.5\\n\\n# Set scene frame range\\nduration_frames = 113 # 4.5s * 25fps = 112.5 → 113 frames\\nscene.frame_start = 0\\nscene.frame_end = duration_frames\\n\\n# Add subtitle with background box\\ntext_strip = sequences.new_effect(\\n name='Subtitle',\\n type='TEXT',\\n channel=3,\\n frame_start=0,\\n frame_end=duration_frames,\\n seq1=None,\\n seq2=None\\n)\\ntext_strip.text = 'BIG BUNNY'\\ntext_strip.align_x = 'CENTER'\\ntext_strip.align_y = 'BOTTOM'\\ntext_strip.use_box = True\\ntext_strip.box_color = (0.0, 0.0, 0.0, 0.6)\\ntext_strip.font_size = 48\\n\\n# Render animation\\nbpy.ops.render.render(animation=True)\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 821,
|
||||
"completion_tokens": 5369,
|
||||
"total_tokens": 6190
|
||||
},
|
||||
"latency_s": 111.15
|
||||
},
|
||||
{
|
||||
"purpose": "blender_script_negative-control_attempt_4",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Blender VSE engineer. Return one JSON object only."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Generate a complete Blender 4.3 Python script for this video edit.\n\nInput movie: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\nOutput MP4: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\nSource FPS: 25\nSource size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)\nTrim interval: [0.000, 3.000] seconds\nEffects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.\n\nRequirements:\n- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.\n- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.\n- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).\n- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.\n- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).\n- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.\n- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.\n\nReturn one JSON object only: {\"code\":\"complete executable Python source\"}.\n\nPrior executable feedback:\nStatic safety/API validation failed: ValueError: generated script omits requested Blender API operations: ['frame_offset_start']"
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 7000,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536055081715b00b15f7b15a122bee0bef9a71075821cc7c",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"code\":\"import bpy\\nimport os\\n\\n# Clear existing data\\nbpy.ops.wm.read_factory_settings(use_empty=True)\\n\\n# Get the current scene\\nscene = bpy.context.scene\\n\\n# Set render resolution (854x480 for H.264 even width)\\nscene.render.resolution_x = 854\\nscene.render.resolution_y = 480\\nscene.render.resolution_percentage = 100\\n\\n# Set frame rate (25 FPS)\\nscene.render.fps = 25\\nscene.render.fps_base = 1.0\\n\\n# Output settings\\noutput_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'\\nscene.render.filepath = output_path\\nscene.render.image_settings.file_format = 'FFMPEG'\\nscene.render.ffmpeg.format = 'MPEG4'\\nscene.render.ffmpeg.codec = 'H.264'\\nscene.render.ffmpeg.audio_codec = 'AAC'\\nscene.render.ffmpeg.audio_mixrate = 44100\\nscene.render.ffmpeg.audio_bitrate = 192\\nscene.render.use_audio = True\\n\\n# Create sequence editor\\nscene.sequence_editor_create()\\nseq_ed = scene.sequence_editor\\n\\n# Source movie path\\nsource_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'\\n\\n# Add video strip with trim (0.000-3.000s = 75 frames @25FPS)\\nvideo_strip = seq_ed.sequences.new_movie(\\n name='Video',\\n filepath=source_path,\\n channel=1,\\n frame_start=1\\n)\\nvideo_strip.frame_offset_start = 0 # Start at 0s of source\\nvideo_strip.frame_final_duration = 75 # Trim to 3s (75 frames)\\n\\n# Add SPEED effect for slow motion (1.5x duration)\\n speed_effect = seq_ed.sequences.new_effect(\\n name='SpeedControl',\\n type='SPEED',\\n channel=2,\\n frame_start=1,\\n seq1=video_strip\\n)\\nspeed_effect.speed_control = 'MULTIPLY'\\nspeed_effect.speed_factor = 1/1.5 # Slow playback speed\\n\\n# Extend video duration to 1.5x trimmed interval (4.5s = 112.5 frames → 113 frames)\\nvideo_strip.frame_final_duration = 113\\n\\n# Configure audio strip (trim to original 3s duration)\\nsound_strip = None\\nfor strip in seq_ed.sequences:\\n if strip.type == 'SOUND' and strip.name.startswith('Audio'):\\n sound_strip = strip\\nif sound_strip:\\n sound_strip.frame_offset_start = 0 # Start at 0s of source audio\\n sound_strip.frame_final_duration = 75 # Keep original 3s duration\\n\\n# Add subtitle text strip with background box\\ntext_strip = seq_ed.sequences.new_effect(\\n name='Subtitle',\\n type='TEXT',\\n channel=3,\\n frame_start=1,\\n frame_final_duration=113\\n)\\ntext_strip.text = 'BIG BUNNY'\\ntext_strip.align_x = 'CENTER'\\ntext_strip.align_y = 'BOTTOM'\\ntext_strip.use_box = True\\ntext_strip.box_color = (0, 0, 0, 0.6) # Semi-transparent black box\\ntext_strip.font_size = 48\\n\\n# Set scene frame range to match slowed video duration\\nscene.frame_start = 1\\nscene.frame_end = 113\\n\\n# Render the animation\\nbpy.ops.render.render(animation=True)\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 643,
|
||||
"completion_tokens": 4515,
|
||||
"total_tokens": 5158
|
||||
},
|
||||
"latency_s": 97.508
|
||||
},
|
||||
{
|
||||
"purpose": "blender_script_negative-control_attempt_5",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Blender VSE engineer. Return one JSON object only."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Generate a complete Blender 4.3 Python script for this video edit.\n\nInput movie: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\nOutput MP4: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\nSource FPS: 25\nSource size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)\nTrim interval: [0.000, 3.000] seconds\nEffects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.\n\nRequirements:\n- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.\n- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.\n- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).\n- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.\n- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).\n- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.\n- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.\n\nReturn one JSON object only: {\"code\":\"complete executable Python source\"}.\n\nPrior executable feedback:\nStatic safety/API validation failed: IndentationError: unexpected indent (<unknown>, line 48)"
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 7000,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536064833115b00b15f7b15a122bee0bef9a710758d5fc02",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"code\":\"import bpy\\n\\n# Clear existing data\\nbpy.ops.wm.read_factory_settings(use_empty=True)\\n\\n# Get the scene\\nscene = bpy.context.scene\\n\\n# Set render resolution and FPS\\nscene.render.resolution_x = 854\\nscene.render.resolution_y = 480\\nscene.render.resolution_percentage = 100\\nscene.render.fps = 25\\nscene.render.fps_base = 1.0\\n\\n# Set output settings\\noutput_path = \\\"/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\\\"\\nscene.render.filepath = output_path\\nscene.render.image_settings.file_format = 'FFMPEG'\\nscene.render.ffmpeg.format = 'MPEG4'\\nscene.render.ffmpeg.codec = 'H264'\\nscene.render.ffmpeg.audio_codec = 'AAC'\\nscene.render.ffmpeg.constant_rate_factor = 'MEDIUM'\\n\\n# Create sequence editor\\nseq_ed = scene.sequence_editor_create()\\n\\n# Import movie strip\\nmovie_path = \\\"/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\\\"\\nmovie_strip = seq_ed.sequences.new_movie(\\n name=\\\"SourceMovie\\\",\\n filepath=movie_path,\\n channel=1,\\n frame_start=0\\n)\\n\\n# Trim movie to [0.000, 3.000] seconds (75 frames at 25 FPS)\\nmovie_strip.frame_offset_start = 0\\nmovie_strip.frame_final_duration = 75\\n\\n# Add speed control effect\\nspeed_strip = seq_ed.sequences.new_effect(\\n name=\\\"SpeedControl\\\",\\n type='SPEED',\\n channel=2,\\n frame_start=0,\\n frame_end=movie_strip.frame_start + movie_strip.frame_final_duration,\\n seq1=movie_strip\\n)\\nspeed_strip.speed_control = 'MULTIPLY'\\nspeed_strip.speed_factor = 1 / 1.5 # Slow down by 1.5x\\n\\n# Find sound strip and trim it\\nsound_strip = None\\nfor strip in seq_ed.sequences:\\n if strip.type == 'SOUND' and strip.frame_start == movie_strip.frame_start:\\n sound_strip = strip\\n break\\nif sound_strip:\\n sound_strip.frame_offset_start = 0\\n sound_strip.frame_final_duration = 75 # Keep sound at normal duration\\n\\n# Calculate render duration: 3s * 1.5 = 4.5s = 112.5 frames → 113 frames (0-112)\\nscene.frame_start = 0\\nscene.frame_end = 113 # 113 frames total\\n\\n# Add subtitle text strip\\ntext_strip = seq_ed.sequences.new_effect(\\n name=\\\"Subtitle\\\",\\n type='TEXT',\\n channel=3,\\n frame_start=0,\\n frame_end=scene.frame_end\\n)\\ntext_strip.text = \\\"BIG BUNNY\\\"\\ntext_strip.align_x = 'CENTER'\\ntext_strip.align_y = 'BOTTOM'\\ntext_strip.font_size = 48\\ntext_strip.color = (1, 1, 1, 1) # White text\\ntext_strip.use_box = True\\ntext_strip.box_color = (0, 0, 0, 0.6) # Dark semi-transparent box\\n\\n# Render animation\\nscene.render.engine = 'BLENDER_EEVEE'\\nbpy.ops.render.render(animation=True)\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 641,
|
||||
"completion_tokens": 6960,
|
||||
"total_tokens": 7601
|
||||
},
|
||||
"latency_s": 139.053
|
||||
},
|
||||
{
|
||||
"purpose": "blender_script_negative-control_attempt_6",
|
||||
"provider": "ark",
|
||||
"endpoint": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
"request": {
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a Blender VSE engineer. Return one JSON object only."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Generate a complete Blender 4.3 Python script for this video edit.\n\nInput movie: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov\nOutput MP4: /Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4\nSource FPS: 25\nSource size: 853x480; render at 854x480 (one-pixel even-width pad required by H.264)\nTrim interval: [0.000, 3.000] seconds\nEffects: slow playback so output duration is 1.5 times the trimmed interval; add bottom-centered subtitle BIG BUNNY with a visible semi-transparent dark box.\n\nRequirements:\n- Use bpy and Blender's Video Sequence Editor, including new_movie, new_sound when available, frame_offset_start/frame_final_duration, a SPEED effect, and a TEXT effect.\n- Render exactly 854x480 at 25 fps through Blender to MPEG-4 H.264 with AAC audio.\n- Set the scene frame range to the slowed duration and call bpy.ops.render.render(animation=True).\n- Create only the requested output. Do not invoke ffmpeg, subprocesses, a shell, the network, or read credentials.\n- Use APIs available in Blender 4.3 (scene.sequence_editor_create().sequences).\n- In Blender 4.3, SpeedControlSequence has no use_audio property. Never read or assign use_audio, and do not apply a SPEED effect to the sound strip. A valid movie slow-motion pattern is speed_control='MULTIPLY', speed_factor=1/1.5, and extending the movie/render duration; the sound strip may remain normally trimmed.\n- For the subtitle background, prefer the TEXT strip's use_box=True and box_color=(0,0,0,0.6); do not assume a COLOR strip has text-layout properties.\n\nReturn one JSON object only: {\"code\":\"complete executable Python source\"}.\n\nPrior executable feedback:\nBlender exit=0; output_exists=False. Tail of real Blender log:\nBlender 4.3.2 (hash 32f5fdce0a0a built 2024-12-17 02:14:25)\nTraceback (most recent call last):\n File \"/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/scripts/negative-control-attempt-5.py\", line 42, in <module>\n speed_strip = seq_ed.sequences.new_effect(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nTypeError: SequencesTopLevel.new_effect(): error with keyword argument \"frame_end\" - Function.frame_end expected an int type, not float\n\nBlender quit"
|
||||
}
|
||||
],
|
||||
"temperature": 0,
|
||||
"max_tokens": 7000,
|
||||
"response_format": {
|
||||
"type": "json_object"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
"id": "02178536078809715b00b15f7b15a122bee0bef9a710758656f73",
|
||||
"model": "doubao-seed-1-6-250615",
|
||||
"finish_reason": "stop",
|
||||
"content": "{\"code\":\"import bpy\\nimport math\\n\\n# Clear existing objects\\nbpy.ops.object.select_all(action='SELECT')\\nbpy.ops.object.delete()\\n\\n# File paths\\ninput_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'\\noutput_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'\\n\\n# Scene setup\\nscene = bpy.context.scene\\n\\n# Render settings\\nscene.render.fps = 25\\nscene.render.fps_base = 1.0\\nscene.render.resolution_x = 854\\nscene.render.resolution_y = 480\\nscene.render.resolution_percentage = 100\\n\\n# Output format settings\\nscene.render.image_settings.file_format = 'MPEG4'\\nscene.render.ffmpeg.format = 'MPEG4'\\nscene.render.ffmpeg.codec = 'H264'\\nscene.render.ffmpeg.audio_codec = 'AAC'\\nscene.render.filepath = output_path\\n\\n# VSE setup\\nscene.sequence_editor_create()\\nseq_ed = scene.sequence_editor\\n\\n# Add video strip\\nvideo_strip = seq_ed.sequences.new_movie(\\n name='Video',\\n filepath=input_path,\\n frame_start=0,\\n channel=1\\n)\\nvideo_strip.frame_final_duration = 75 # 3s * 25fps\\n\\n# Add speed effect (1.5x slowdown)\\noriginal_duration = 75\\nnew_duration = math.ceil(original_duration * 1.5) # 113 frames (4.52s)\\nspeed_strip = seq_ed.sequences.new_effect(\\n name='SpeedControl',\\n type='SPEED',\\n channel=2,\\n frame_start=0,\\n frame_end=new_duration,\\n seq1=video_strip\\n)\\nspeed_strip.speed_control = 'MULTIPLY'\\nspeed_strip.speed_factor = 1/1.5\\n\\n# Add audio strip\\naudio_strip = seq_ed.sequences.new_sound(\\n name='Audio',\\n filepath=input_path,\\n frame_start=0,\\n channel=3\\n)\\naudio_strip.frame_final_duration = 75 # Keep original audio duration\\n\\n# Add subtitle text strip\\ntext_strip = seq_ed.sequences.new_effect(\\n name='Subtitle',\\n type='TEXT',\\n channel=4,\\n frame_start=0,\\n frame_end=new_duration\\n)\\ntext_strip.text = 'BIG BUNNY'\\ntext_strip.align_x = 'CENTER'\\ntext_strip.align_y = 'BOTTOM'\\ntext_strip.font_size = 48\\ntext_strip.use_box = True\\ntext_strip.box_color = (0, 0, 0, 0.6) # Semi-transparent black box\\n\\n# Set render frame range\\nscene.frame_start = 1\\nscene.frame_end = new_duration\\n\\n# Render animation\\nbpy.ops.render.render(animation=True)\"}"
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 812,
|
||||
"completion_tokens": 6780,
|
||||
"total_tokens": 7592
|
||||
},
|
||||
"latency_s": 129.381
|
||||
}
|
||||
]
|
||||
}
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
import bpy
|
||||
|
||||
# Clear existing objects to start fresh
|
||||
bpy.ops.object.select_all(action='SELECT')
|
||||
bpy.ops.object.delete()
|
||||
|
||||
# Configuration paths and parameters
|
||||
INPUT_MOVIE = "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov"
|
||||
OUTPUT_MP4 = "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4"
|
||||
SOURCE_FPS = 25
|
||||
TRIM_DURATION_SECONDS = 3.0
|
||||
SLOW_FACTOR = 1.5
|
||||
RENDER_RESOLUTION = (854, 480)
|
||||
|
||||
# Calculate frame values
|
||||
trim_frames = int(TRIM_DURATION_SECONDS * SOURCE_FPS)
|
||||
slowed_frames = int(trim_frames * SLOW_FACTOR)
|
||||
scene = bpy.context.scene
|
||||
|
||||
# Create sequence editor
|
||||
seq_ed = scene.sequence_editor_create()
|
||||
|
||||
# Add movie strip with trim
|
||||
movie_strip = seq_ed.sequences.new_movie(
|
||||
name="SourceVideo",
|
||||
filepath=INPUT_MOVIE,
|
||||
channel=1,
|
||||
frame_start=1
|
||||
)
|
||||
movie_strip.frame_offset_start = 0 # Start at beginning of source
|
||||
movie_strip.frame_final_duration = trim_frames # Trim to 3 seconds
|
||||
|
||||
# Locate and trim audio strip (auto-created with movie)
|
||||
sound_strip = None
|
||||
for strip in seq_ed.sequences:
|
||||
if strip.type == 'SOUND' and strip.channel == 2 and strip.frame_start == 1:
|
||||
sound_strip = strip
|
||||
if sound_strip:
|
||||
sound_strip.frame_final_duration = trim_frames # Keep audio at original speed
|
||||
|
||||
# Add speed effect to video (slow motion)
|
||||
speed_strip = seq_ed.sequences.new_effect(
|
||||
name="SlowMotion",
|
||||
type='SPEED',
|
||||
channel=3,
|
||||
frame_start=1,
|
||||
frame_end=1 + slowed_frames - 1,
|
||||
target=movie_strip
|
||||
)
|
||||
speed_strip.speed_control = 'MULTIPLY'
|
||||
speed_strip.speed_factor = 1 / SLOW_FACTOR # 1/1.5 speed factor
|
||||
|
||||
# Add subtitle text strip with background box
|
||||
text_strip = seq_ed.sequences.new_text(
|
||||
name="Subtitle",
|
||||
text="BIG BUNNY",
|
||||
channel=4,
|
||||
frame_start=1,
|
||||
frame_end=1 + slowed_frames - 1
|
||||
)
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.use_box = True
|
||||
text_strip.box_color = (0.0, 0.0, 0.0, 0.6) # Black semi-transparent box
|
||||
text_strip.font_size = 48 # Visible size for 854x480 resolution
|
||||
|
||||
# Configure render settings
|
||||
scene.render.resolution_x = RENDER_RESOLUTION[0]
|
||||
scene.render.resolution_y = RENDER_RESOLUTION[1]
|
||||
scene.render.resolution_percentage = 100
|
||||
scene.render.fps = SOURCE_FPS
|
||||
scene.render.fps_base = 1.0
|
||||
scene.render.image_settings.file_format = 'FFMPEG'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.filepath = OUTPUT_MP4
|
||||
|
||||
# Set render frame range
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = 1 + slowed_frames - 1
|
||||
|
||||
# Execute render
|
||||
bpy.ops.render.render(animation=True)
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
import bpy
|
||||
|
||||
# Clear existing data
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
scene = bpy.context.scene
|
||||
|
||||
# Scene settings
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
scene.render.fps = 25
|
||||
scene.render.fps_base = 1.0
|
||||
|
||||
# Create sequence editor
|
||||
if not scene.sequence_editor:
|
||||
scene.sequence_editor_create()
|
||||
seq_ed = scene.sequence_editor
|
||||
sequences = seq_ed.sequences
|
||||
|
||||
# File paths
|
||||
input_movie = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'
|
||||
output_mp4 = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'
|
||||
|
||||
# Add movie strip (3 seconds = 75 frames at 25fps)
|
||||
mov_strip = sequences.new_movie(
|
||||
name='SourceMovie',
|
||||
filepath=input_movie,
|
||||
frame_start=1,
|
||||
channel=1
|
||||
)
|
||||
mov_strip.frame_offset_start = 0
|
||||
mov_strip.frame_final_duration = 75 # 3s * 25fps
|
||||
|
||||
# Locate and trim sound strip
|
||||
sound_strip = next((s for s in sequences if s.type == 'SOUND' and s.frame_start == mov_strip.frame_start), None)
|
||||
if sound_strip:
|
||||
sound_strip.frame_final_duration = 75 # Keep original audio duration
|
||||
|
||||
# Add speed effect (1.5x slowdown)
|
||||
speed_strip = sequences.new_effect(
|
||||
name='SlowMotion',
|
||||
type='SPEED',
|
||||
frame_start=1,
|
||||
channel=2,
|
||||
input_1=mov_strip
|
||||
)
|
||||
speed_strip.speed_control = 'MULTIPLY'
|
||||
speed_strip.speed_factor = 1 / 1.5
|
||||
|
||||
# Calculate slowed duration (4.5s = 112.5 frames → 113 frames)
|
||||
slowed_frames = int(4.5 * 25) + 1
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = slowed_frames
|
||||
|
||||
# Add subtitle text strip
|
||||
text_strip = sequences.new_effect(
|
||||
name='Subtitle',
|
||||
type='TEXT',
|
||||
frame_start=1,
|
||||
frame_end=scene.frame_end,
|
||||
channel=3
|
||||
)
|
||||
text_strip.text = 'BIG BUNNY'
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.use_box = True
|
||||
text_strip.box_color = (0, 0, 0, 0.6)
|
||||
text_strip.font_size = 48
|
||||
|
||||
# Render settings
|
||||
scene.render.filepath = output_mp4
|
||||
scene.render.image_settings.file_format = 'FFMPEG'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.ffmpeg.constant_rate_factor = 'MEDIUM'
|
||||
|
||||
# Render animation
|
||||
bpy.ops.render.render(animation=True)
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
import bpy
|
||||
import os
|
||||
|
||||
# Clear default objects
|
||||
bpy.ops.object.select_all(action='SELECT')
|
||||
bpy.ops.object.delete()
|
||||
|
||||
# Set up scene
|
||||
scene = bpy.context.scene
|
||||
scene.render.fps = 25
|
||||
scene.render.fps_base = 1.0
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
|
||||
# Configure output settings
|
||||
scene.render.image_settings.file_format = 'FFMPEG'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.ffmpeg.audio_bitrate = 192
|
||||
output_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'
|
||||
scene.render.filepath = output_path
|
||||
|
||||
# Create sequence editor
|
||||
scene.sequence_editor_create()
|
||||
sequences = scene.sequence_editor.sequences
|
||||
|
||||
# Import source movie and sound
|
||||
source_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'
|
||||
movie_strip = sequences.new_movie(name='SourceMovie', filepath=source_path, channel=1, frame_start=0)
|
||||
|
||||
sound_strip = None
|
||||
for strip in sequences:
|
||||
if strip.type == 'SOUND' and strip.name.startswith('SourceMovie'):
|
||||
sound_strip = strip
|
||||
break
|
||||
if not sound_strip:
|
||||
sound_strip = sequences.new_sound(name='SourceAudio', filepath=source_path, channel=2, frame_start=0)
|
||||
|
||||
# Trim to 3-second interval (75 frames at 25fps)
|
||||
trim_frames = 75 # 3.0s * 25fps
|
||||
movie_strip.frame_start = 0
|
||||
movie_strip.frame_final_duration = trim_frames
|
||||
sound_strip.frame_start = 0
|
||||
sound_strip.frame_final_duration = trim_frames
|
||||
|
||||
# Apply speed effect (1.5x slowdown = 4.5s output)
|
||||
speed_strip = sequences.new_effect(
|
||||
name='SpeedControl',
|
||||
type='SPEED',
|
||||
channel=2,
|
||||
frame_start=0,
|
||||
frame_end=113, # 75 frames * 1.5 = 112.5 → 113 frames
|
||||
seq1=movie_strip,
|
||||
seq2=None
|
||||
)
|
||||
speed_strip.speed_control = 'MULTIPLY'
|
||||
speed_strip.speed_factor = 1/1.5
|
||||
|
||||
# Set scene frame range
|
||||
duration_frames = 113 # 4.5s * 25fps = 112.5 → 113 frames
|
||||
scene.frame_start = 0
|
||||
scene.frame_end = duration_frames
|
||||
|
||||
# Add subtitle with background box
|
||||
text_strip = sequences.new_effect(
|
||||
name='Subtitle',
|
||||
type='TEXT',
|
||||
channel=3,
|
||||
frame_start=0,
|
||||
frame_end=duration_frames,
|
||||
seq1=None,
|
||||
seq2=None
|
||||
)
|
||||
text_strip.text = 'BIG BUNNY'
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.use_box = True
|
||||
text_strip.box_color = (0.0, 0.0, 0.0, 0.6)
|
||||
text_strip.font_size = 48
|
||||
|
||||
# Render animation
|
||||
bpy.ops.render.render(animation=True)
|
||||
+90
@@ -0,0 +1,90 @@
|
||||
import bpy
|
||||
import os
|
||||
|
||||
# Clear existing data
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
# Get the current scene
|
||||
scene = bpy.context.scene
|
||||
|
||||
# Set render resolution (854x480 for H.264 even width)
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
|
||||
# Set frame rate (25 FPS)
|
||||
scene.render.fps = 25
|
||||
scene.render.fps_base = 1.0
|
||||
|
||||
# Output settings
|
||||
output_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4'
|
||||
scene.render.filepath = output_path
|
||||
scene.render.image_settings.file_format = 'FFMPEG'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H.264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.ffmpeg.audio_mixrate = 44100
|
||||
scene.render.ffmpeg.audio_bitrate = 192
|
||||
scene.render.use_audio = True
|
||||
|
||||
# Create sequence editor
|
||||
scene.sequence_editor_create()
|
||||
seq_ed = scene.sequence_editor
|
||||
|
||||
# Source movie path
|
||||
source_path = '/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov'
|
||||
|
||||
# Add video strip with trim (0.000-3.000s = 75 frames @25FPS)
|
||||
video_strip = seq_ed.sequences.new_movie(
|
||||
name='Video',
|
||||
filepath=source_path,
|
||||
channel=1,
|
||||
frame_start=1
|
||||
)
|
||||
video_strip.frame_offset_start = 0 # Start at 0s of source
|
||||
video_strip.frame_final_duration = 75 # Trim to 3s (75 frames)
|
||||
|
||||
# Add SPEED effect for slow motion (1.5x duration)
|
||||
speed_effect = seq_ed.sequences.new_effect(
|
||||
name='SpeedControl',
|
||||
type='SPEED',
|
||||
channel=2,
|
||||
frame_start=1,
|
||||
seq1=video_strip
|
||||
)
|
||||
speed_effect.speed_control = 'MULTIPLY'
|
||||
speed_effect.speed_factor = 1/1.5 # Slow playback speed
|
||||
|
||||
# Extend video duration to 1.5x trimmed interval (4.5s = 112.5 frames → 113 frames)
|
||||
video_strip.frame_final_duration = 113
|
||||
|
||||
# Configure audio strip (trim to original 3s duration)
|
||||
sound_strip = None
|
||||
for strip in seq_ed.sequences:
|
||||
if strip.type == 'SOUND' and strip.name.startswith('Audio'):
|
||||
sound_strip = strip
|
||||
if sound_strip:
|
||||
sound_strip.frame_offset_start = 0 # Start at 0s of source audio
|
||||
sound_strip.frame_final_duration = 75 # Keep original 3s duration
|
||||
|
||||
# Add subtitle text strip with background box
|
||||
text_strip = seq_ed.sequences.new_effect(
|
||||
name='Subtitle',
|
||||
type='TEXT',
|
||||
channel=3,
|
||||
frame_start=1,
|
||||
frame_final_duration=113
|
||||
)
|
||||
text_strip.text = 'BIG BUNNY'
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.use_box = True
|
||||
text_strip.box_color = (0, 0, 0, 0.6) # Semi-transparent black box
|
||||
text_strip.font_size = 48
|
||||
|
||||
# Set scene frame range to match slowed video duration
|
||||
scene.frame_start = 1
|
||||
scene.frame_end = 113
|
||||
|
||||
# Render the animation
|
||||
bpy.ops.render.render(animation=True)
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
import bpy
|
||||
|
||||
# Clear existing data
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
# Get the scene
|
||||
scene = bpy.context.scene
|
||||
|
||||
# Set render resolution and FPS
|
||||
scene.render.resolution_x = 854
|
||||
scene.render.resolution_y = 480
|
||||
scene.render.resolution_percentage = 100
|
||||
scene.render.fps = 25
|
||||
scene.render.fps_base = 1.0
|
||||
|
||||
# Set output settings
|
||||
output_path = "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/negative_control.mp4"
|
||||
scene.render.filepath = output_path
|
||||
scene.render.image_settings.file_format = 'FFMPEG'
|
||||
scene.render.ffmpeg.format = 'MPEG4'
|
||||
scene.render.ffmpeg.codec = 'H264'
|
||||
scene.render.ffmpeg.audio_codec = 'AAC'
|
||||
scene.render.ffmpeg.constant_rate_factor = 'MEDIUM'
|
||||
|
||||
# Create sequence editor
|
||||
seq_ed = scene.sequence_editor_create()
|
||||
|
||||
# Import movie strip
|
||||
movie_path = "/Users/boj/book/ai-agent-book/chapter5/video-edit/validation/runs/exp5-6-real-blender-20260730-052206/source.mov"
|
||||
movie_strip = seq_ed.sequences.new_movie(
|
||||
name="SourceMovie",
|
||||
filepath=movie_path,
|
||||
channel=1,
|
||||
frame_start=0
|
||||
)
|
||||
|
||||
# Trim movie to [0.000, 3.000] seconds (75 frames at 25 FPS)
|
||||
movie_strip.frame_offset_start = 0
|
||||
movie_strip.frame_final_duration = 75
|
||||
|
||||
# Add speed control effect
|
||||
speed_strip = seq_ed.sequences.new_effect(
|
||||
name="SpeedControl",
|
||||
type='SPEED',
|
||||
channel=2,
|
||||
frame_start=0,
|
||||
frame_end=movie_strip.frame_start + movie_strip.frame_final_duration,
|
||||
seq1=movie_strip
|
||||
)
|
||||
speed_strip.speed_control = 'MULTIPLY'
|
||||
speed_strip.speed_factor = 1 / 1.5 # Slow down by 1.5x
|
||||
|
||||
# Find sound strip and trim it
|
||||
sound_strip = None
|
||||
for strip in seq_ed.sequences:
|
||||
if strip.type == 'SOUND' and strip.frame_start == movie_strip.frame_start:
|
||||
sound_strip = strip
|
||||
break
|
||||
if sound_strip:
|
||||
sound_strip.frame_offset_start = 0
|
||||
sound_strip.frame_final_duration = 75 # Keep sound at normal duration
|
||||
|
||||
# Calculate render duration: 3s * 1.5 = 4.5s = 112.5 frames → 113 frames (0-112)
|
||||
scene.frame_start = 0
|
||||
scene.frame_end = 113 # 113 frames total
|
||||
|
||||
# Add subtitle text strip
|
||||
text_strip = seq_ed.sequences.new_effect(
|
||||
name="Subtitle",
|
||||
type='TEXT',
|
||||
channel=3,
|
||||
frame_start=0,
|
||||
frame_end=scene.frame_end
|
||||
)
|
||||
text_strip.text = "BIG BUNNY"
|
||||
text_strip.align_x = 'CENTER'
|
||||
text_strip.align_y = 'BOTTOM'
|
||||
text_strip.font_size = 48
|
||||
text_strip.color = (1, 1, 1, 1) # White text
|
||||
text_strip.use_box = True
|
||||
text_strip.box_color = (0, 0, 0, 0.6) # Dark semi-transparent box
|
||||
|
||||
# Render animation
|
||||
scene.render.engine = 'BLENDER_EEVEE'
|
||||
bpy.ops.render.render(animation=True)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user