ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round1_review.json

@@ -0,0 +1,66 @@
+{
+  "overall_score": 65,
+  "pass": false,
+  "issues": [
+    {
+      "page": 2,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将要点拆分为2页，例如将State-of-the-art results单独作为一页展示"
+    },
+    {
+      "page": 3,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将三个部分拆分为独立页面，每个部分单独一页详细说明"
+    },
+    {
+      "page": 4,
+      "issue_type": "image_size",
+      "severity": "high",
+      "suggestion": "放大架构图至占页面2/3空间，删除图片下方的文字说明或将其移至下一页"
+    },
+    {
+      "page": 4,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "删除图片下方的长段文字，仅保留标题和简短说明"
+    },
+    {
+      "page": 5,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将Encoder和Decoder分拆为两页，每页单独详细说明其结构"
+    },
+    {
+      "page": 7,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将Multi-Head Attention原理和Applications分拆为两页"
+    },
+    {
+      "page": 10,
+      "issue_type": "overcrowded",
+      "severity": "high",
+      "suggestion": "将Data & Batching、Hardware & Schedule、Optimizer & Regularization三个部分拆分为3页"
+    },
+    {
+      "page": 14,
+      "issue_type": "image_size",
+      "severity": "high",
+      "suggestion": "放大注意力可视化图，使其占满页面，确保文字和连线清晰可见"
+    },
+    {
+      "page": 15,
+      "issue_type": "image_size",
+      "severity": "high",
+      "suggestion": "放大注意力可视化图，增加对比度，确保不同颜色的注意力头可区分"
+    },
+    {
+      "page": 16,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将Contributions和Future Work分拆为两页，每部分保留3-4个要点"
+    }
+  ]
+}

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round1_slides.md

@@ -0,0 +1,221 @@
+---
+theme: default
+title: 'Attention Is All You Need'
+author: 'Ashish Vaswani et al.'
+date: 'NIPS 2017'
+---
+
+# Attention Is All You Need
+
+**Authors:** Ashish Vaswani et al.  
+**Conference:** NIPS 2017
+
+---
+
+## Abstract
+
+- Proposes **Transformer** - a new network architecture based solely on attention
+- Dispenses with recurrence and convolutions entirely
+- More parallelizable and requires significantly less training time
+- State-of-the-art results:
+  - 28.4 BLEU on WMT 2014 English-to-German
+  - 41.8 BLEU on WMT 2014 English-to-French
+- Generalizes well to other tasks like English constituency parsing
+
+---
+
+## Background & Key Idea
+
+### Traditional Sequence Models
+- **Recurrent models** (LSTM, GRU): Sequential computation, limited parallelization
+- **Convolutional models**: Limited long-range dependencies
+
+### Attention Mechanism
+- Already used with recurrent networks
+- Allows modeling dependencies without regard to distance
+
+### The Transformer
+- **Key innovation**: Replace recurrence and convolution with self-attention
+- Enables more parallelization and better long-range dependencies
+
+---
+
+## Transformer Architecture
+
+<img src="/paper_figure_1_transformer.png" class="h-80 mx-auto" />
+
+*Encoder (left) and Decoder (right) stacks with self-attention and feed-forward layers*
+
+---
+
+## Encoder & Decoder Stacks
+
+### Encoder
+- **6 identical layers** each with:
+  1. Multi-head self-attention mechanism
+  2. Position-wise fully connected feed-forward network
+- Residual connections + layer normalization
+- Output dimension: `d_model = 512`
+
+### Decoder
+- **6 identical layers** each with:
+  1. Masked multi-head self-attention (prevents leftward flow)
+  2. Multi-head attention over encoder output
+  3. Position-wise feed-forward network
+- Same residual connections and layer normalization
+
+---
+
+## Attention Mechanism
+
+### Scaled Dot-Product Attention
+$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$
+
+- $Q$ (queries), $K$ (keys), $V$ (values) are matrices
+- Scaling by $\sqrt{d_k}$ prevents gradients from becoming too small
+- More efficient than additive attention
+
+---
+
+## Multi-Head Attention
+
+- Projects queries, keys, values $h$ times with different linear projections
+- Performs attention in parallel on each projected version
+- Concatenates results and projects again:
+
+$$\text{MultiHead}(Q, K, V) = \text{Concat}(\text{head}_1, ..., \text{head}_h)W^O$$
+
+- $h = 8$ heads, $d_k = d_v = d_{\text{model}}/h = 64$
+
+### Applications
+1. **Encoder-decoder attention**: Queries from decoder, keys/values from encoder
+2. **Encoder self-attention**: All positions attend to all positions
+3. **Decoder self-attention**: Positions attend to previous positions (masked)
+
+---
+
+## Positional Encoding
+
+Since there's no recurrence/convolution, we add positional information:
+
+$$\text{PE}_{(pos, 2i)} = \sin\left(pos / 10000^{2i/d_{\text{model}}}\right)$$
+$$\text{PE}_{(pos, 2i+1)} = \cos\left(pos / 10000^{2i/d_{\text{model}}}\right)$$
+
+- Same dimension as embeddings ($d_{\text{model}}$)
+- Allows model to learn relative position information
+- Performed similarly to learned positional embeddings
+
+---
+
+## Why Self-Attention?
+
+| Layer Type | Complexity | Sequential Operations | Max Path Length |
+|------------|------------|-----------------------|-----------------|
+| Self-Attention | $O(n^2 \cdot d)$ | $O(1)$ | $O(1)$ |
+| Recurrent | $O(n \cdot d^2)$ | $O(n)$ | $O(n)$ |
+| Convolutional | $O(k \cdot n \cdot d^2)$ | $O(1)$ | $O(\log_k n)$ |
+
+- Better parallelization than RNNs
+- Shorter path length than CNNs/RNNs
+- More interpretable attention distributions
+
+---
+
+## Training Details
+
+### Data & Batching
+- WMT 2014 English-German (4.5M pairs), English-French (36M pairs)
+- Byte-pair encoding with shared vocabulary
+- Batches with ~25,000 source and target tokens
+
+### Hardware & Schedule
+- 8 NVIDIA P100 GPUs
+- Base model: 100,000 steps (12 hours)
+- Big model: 300,000 steps (3.5 days)
+
+### Optimizer & Regularization
+- Adam with $\beta_1=0.9$, $\beta_2=0.98$, $\epsilon=10^{-9}$
+- Learning rate schedule with warmup steps=4000
+- Residual dropout (P_drop=0.1) and label smoothing ($\epsilon_{ls}=0.1$)
+
+---
+
+## Machine Translation Results
+
+| Model | EN-DE BLEU | EN-FR BLEU | Training Cost (FLOPs) |
+|-------|------------|------------|-----------------------|
+| GNMT + RL Ensemble | 26.30 | 41.16 | $1.8 \cdot 10^{20}$ |
+| ConvS2S Ensemble | 26.36 | 41.29 | $7.7 \cdot 10^{19}$ |
+| **Transformer (big)** | **28.4** | **41.8** | **$2.3 \cdot 10^{19}$** |
+
+- Transformer outperforms all previous state-of-the-art models
+- Achieves better results with significantly lower training cost
+- 2+ BLEU improvement on English-to-German
+
+---
+
+## Model Architecture Ablations
+
+| Variation | Dev PPL | Dev BLEU |
+|-----------|---------|----------|
+| Base model | 4.92 | 25.8 |
+| (A) 1 attention head | 5.29 | 24.9 |
+| (B) d_k = 16 | 5.75 | 24.5 |
+| (C) 2 layers | 6.11 | 23.7 |
+| (D) No dropout | 5.77 | 24.6 |
+| (E) Learned positional embeddings | 4.92 | 25.7 |
+
+- Multi-head attention improves performance
+- Dropout is crucial for avoiding overfitting
+- Sinusoidal and learned positional encodings perform similarly
+
+---
+
+## English Constituency Parsing
+
+| Parser | Training | WSJ 23 F1 |
+|--------|----------|-----------|
+| Petrov et al. (2006) | WSJ only | 90.4 |
+| Dyer et al. (2016) | WSJ only | 91.7 |
+| **Transformer (4 layers)** | **WSJ only** | **91.3** |
+| Vinyals & Kaiser et al. (2014) | Semi-supervised | 92.1 |
+| **Transformer (4 layers)** | **Semi-supervised** | **92.7** |
+
+- Transformer generalizes well to other sequence tasks
+- Performs comparably to state-of-the-art parsers
+- Better than RNN models in small-data regimes
+
+---
+
+## Attention Visualization: Long-Distance Dependencies
+
+<img src="/paper_figure_3_long_distance.png" class="h-70 mx-auto" />
+
+*Encoder self-attention in layer 5 showing attention to distant dependency "making...more difficult"*
+
+---
+
+## Attention Visualization: Anaphora Resolution
+
+<img src="/paper_figure_4_anaphora.png" class="h-70 mx-auto" />
+
+*Attention heads focusing on resolving "its" to "The Law"*
+
+---
+
+## Conclusion & Future Work
+
+### Contributions
+- Introduced **Transformer**, first transduction model based entirely on attention
+- Eliminated recurrence and convolution
+- Achieved new state-of-the-art in machine translation
+- More parallelizable and faster to train
+- Generalizes well to other tasks
+
+### Future Work
+- Apply to other modalities (images, audio, video)
+- Investigate local, restricted attention mechanisms
+- Make generation less sequential
+- Explore interpretability of attention patterns
+
+**Code available at:** https://github.com/tensorflow/tensor2tensor

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round2_review.json

@@ -0,0 +1,60 @@
+{
+  "overall_score": 75,
+  "pass": false,
+  "issues": [
+    {
+      "page": 4,
+      "issue_type": "image_size",
+      "severity": "medium",
+      "suggestion": "放大Transformer架构图，确保图中模块名称和连接关系清晰可见"
+    },
+    {
+      "page": 4,
+      "issue_type": "readability",
+      "severity": "medium",
+      "suggestion": "移除图片下方的长段文字说明，仅保留简洁标题或关键标注"
+    },
+    {
+      "page": 5,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将Encoder和Decoder内容拆分到两页，每页不超过4个要点"
+    },
+    {
+      "page": 10,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将Training Setup的三个部分拆分到单独页面，每个部分保留2-3个要点"
+    },
+    {
+      "page": 12,
+      "issue_type": "image_size",
+      "severity": "high",
+      "suggestion": "大幅放大注意力可视化图，确保文字标签和连接线清晰可辨"
+    },
+    {
+      "page": 12,
+      "issue_type": "readability",
+      "severity": "high",
+      "suggestion": "移除图片下方的详细说明文字，仅保留标题和简短解释"
+    },
+    {
+      "page": 13,
+      "issue_type": "image_size",
+      "severity": "high",
+      "suggestion": "大幅放大指代消解可视化图，确保文字和注意力权重分布清晰可见"
+    },
+    {
+      "page": 13,
+      "issue_type": "readability",
+      "severity": "high",
+      "suggestion": "移除图片下方的说明文字，改用简洁标题突出核心发现"
+    },
+    {
+      "page": 14,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将Key Contributions和Future Work拆分到两页，每部分保留3-4个要点"
+    }
+  ]
+}

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round2_slides.md

@@ -0,0 +1,173 @@
+---
+theme: default
+title: 'Attention Is All You Need'
+author: 'Ashish Vaswani et al.'
+date: 'NIPS 2017'
+---
+
+# Attention Is All You Need
+
+**Authors:** Ashish Vaswani et al.  
+**Conference:** NIPS 2017
+
+---
+
+## Abstract
+
+### Key Innovation
+- First sequence transduction model based **solely on attention**
+- Dispenses with recurrence and convolutions entirely
+- More parallelizable with significantly less training time
+
+### Performance Highlights
+- **WMT 2014 EN-DE**: 28.4 BLEU (+2+ over previous SOTA)
+- **WMT 2014 EN-FR**: 41.8 BLEU (new single-model SOTA)
+- Trained in 3.5 days on 8 GPUs (small fraction of previous costs)
+
+---
+
+## Background & Key Idea
+
+### Traditional Sequence Models
+- **Recurrent (LSTM/GRU)**: Sequential computation limits parallelization
+- **Convolutional**: Requires multiple layers for long-range dependencies
+
+### The Transformer
+- Replaces recurrence/convolution with **self-attention**
+- Enables direct modeling of long-range dependencies
+- Massive parallelization → faster training
+
+---
+
+## Transformer Architecture
+
+<img src="/paper_figure_1_transformer.png" class="h-96 mx-auto" />
+
+*Encoder (left) and Decoder (right) with self-attention and feed-forward layers*
+
+---
+
+## Encoder & Decoder Stacks
+
+### Encoder (6 identical layers)
+- **Sub-layer 1**: Multi-head self-attention
+- **Sub-layer 2**: Position-wise feed-forward network
+- Residual connections + layer normalization
+- Output dimension: `d_model = 512`
+
+### Decoder (6 identical layers)
+- **Sub-layer 1**: Masked multi-head self-attention (prevents leftward flow)
+- **Sub-layer 2**: Multi-head attention over encoder output
+- **Sub-layer 3**: Position-wise feed-forward network
+- Same residual connections and normalization
+
+---
+
+## Attention Mechanism
+
+### Scaled Dot-Product Attention
+$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$
+- Scaling by $\sqrt{d_k}$ prevents small gradients
+- Efficient with matrix multiplication
+
+### Multi-Head Attention
+$$\text{MultiHead}(Q, K, V) = \text{Concat}(\text{head}_1,...,\text{head}_h)W^O$$
+- $h=8$ parallel heads, $d_k=d_v=64$
+- Captures diverse dependency patterns
+
+---
+
+## Attention Applications
+
+1. **Encoder-decoder attention**: Queries from decoder, keys/values from encoder  
+2. **Encoder self-attention**: All positions attend to all input positions  
+3. **Decoder self-attention**: Positions attend to previous positions (masked)  
+
+---
+
+## Positional Encoding
+
+Adds sequence position information via sinusoidal functions:
+$$\text{PE}_{(pos,2i)}=\sin(pos/10000^{2i/d_{\text{model}}})$$
+$$\text{PE}_{(pos,2i+1)}=\cos(pos/10000^{2i/d_{\text{model}}})$$
+
+- Same dimension as embeddings ($d_{\text{model}}$)
+- Enables learning of relative positions
+- Performs similarly to learned embeddings
+
+---
+
+## Why Self-Attention?
+
+| Layer Type       | Complexity       | Sequential Ops | Max Path Length |
+|------------------|------------------|----------------|-----------------|
+| Self-Attention   | $O(n^2 \cdot d)$ | $O(1)$         | $O(1)$          |
+| Recurrent        | $O(n \cdot d^2)$ | $O(n)$         | $O(n)$          |
+| Convolutional    | $O(k \cdot n \cdot d^2)$ | $O(1)$ | $O(\log_k n)$   |
+
+- Better parallelization than RNNs
+- Shorter path length than CNNs
+- More interpretable attention patterns
+
+---
+
+## Training Setup
+
+### Data & Batching
+- WMT 2014 EN-DE (4.5M pairs), EN-FR (36M pairs)
+- Byte-pair encoding (37K/32K vocab)
+- Batches with ~25K source/target tokens
+
+### Hardware & Schedule
+- 8 NVIDIA P100 GPUs
+- Base model: 100K steps (12h), Big model: 300K steps (3.5d)
+
+### Optimization
+- Adam ($\beta_1=0.9$, $\beta_2=0.98$), learning rate warmup
+- Dropout (0.1), label smoothing (0.1)
+
+---
+
+## Machine Translation Results
+
+| Model                | EN-DE BLEU | EN-FR BLEU | Training Cost (FLOPs) |
+|----------------------|------------|------------|-----------------------|
+| GNMT + RL Ensemble   | 26.30      | 41.16      | $1.8 \cdot 10^{20}$   |
+| ConvS2S Ensemble     | 26.36      | 41.29      | $7.7 \cdot 10^{19}$   |
+| **Transformer (big)**| **28.4**   | **41.8**   | **$2.3 \cdot 10^{19}$**|
+
+- Outperforms all previous SOTA with lower training cost
+
+---
+
+## Attention Visualization: Long-Distance Dependencies
+
+<img src="/paper_figure_3_long_distance.png" class="h-96 w-full object-contain" />
+
+*Encoder self-attention (layer 5) tracking "making...more difficult" dependency*
+
+---
+
+## Attention Visualization: Anaphora Resolution
+
+<img src="/paper_figure_4_anaphora.png" class="h-96 w-full object-contain" />
+
+*Attention heads resolving "its" to "The Law"*
+
+---
+
+## Conclusion & Future Work
+
+### Key Contributions
+- Introduced Transformer, first attention-only transduction model
+- Eliminated recurrence/convolution → better parallelization
+- Set new SOTA in machine translation with lower training cost
+- Generalizes to other tasks (e.g., constituency parsing)
+
+### Future Work
+- Apply to other modalities (images, audio)
+- Explore local attention for large sequences
+- Improve generation sequentiality
+- Enhance attention interpretability
+
+**Code:** https://github.com/tensorflow/tensor2tensor

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round3_review.json

@@ -0,0 +1,30 @@
+{
+  "overall_score": 85,
+  "pass": false,
+  "issues": [
+    {
+      "page": 2,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将6个要点拆分为2页，Key Innovation和Performance Highlights各占一页，或删减次要信息使每页不超过5个要点"
+    },
+    {
+      "page": 11,
+      "issue_type": "overcrowded",
+      "severity": "medium",
+      "suggestion": "将6个子要点拆分为2页，例如将Datasets和Tokenization作为一页，Batching单独作为一页"
+    },
+    {
+      "page": 15,
+      "issue_type": "readability",
+      "severity": "medium",
+      "suggestion": "放大可视化图中的文字，或截取局部放大展示关键区域，确保观众能看清注意力模式和词语对应关系"
+    },
+    {
+      "page": 16,
+      "issue_type": "readability",
+      "severity": "medium",
+      "suggestion": "放大可视化图中的文字，或选择更清晰的可视化案例，确保代词解析的注意力关系清晰可见"
+    }
+  ]
+}

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round3_slides.md

@@ -0,0 +1,201 @@
+---
+theme: default
+title: 'Attention Is All You Need'
+author: 'Ashish Vaswani et al.'
+date: 'NIPS 2017'
+---
+
+# Attention Is All You Need
+
+**Authors:** Ashish Vaswani et al.  
+**Conference:** NIPS 2017
+
+---
+
+## Abstract
+
+### Key Innovation
+- First sequence transduction model based **solely on attention**
+- Dispenses with recurrence and convolutions entirely
+- More parallelizable with significantly less training time
+
+### Performance Highlights
+- **WMT 2014 EN-DE**: 28.4 BLEU (+2+ over previous SOTA)
+- **WMT 2014 EN-FR**: 41.8 BLEU (new single-model SOTA)
+- Trained in 3.5 days on 8 GPUs (small fraction of previous costs)
+
+---
+
+## Background & Key Idea
+
+### Traditional Sequence Models
+- **Recurrent (LSTM/GRU)**: Sequential computation limits parallelization
+- **Convolutional**: Requires multiple layers for long-range dependencies
+
+### The Transformer
+- Replaces recurrence/convolution with **self-attention**
+- Enables direct modeling of long-range dependencies
+- Massive parallelization → faster training
+
+---
+
+## Transformer Architecture
+
+<img src="/paper_figure_1_transformer.png" class="h-[600px] w-full object-contain" />
+
+---
+
+## Encoder Stack
+
+- **6 identical layers** with two sub-layers:
+  1. Multi-head self-attention mechanism
+  2. Position-wise feed-forward network
+- Residual connections around each sub-layer
+- Layer normalization after each sub-layer
+- Output dimension: `d_model = 512`
+
+---
+
+## Decoder Stack
+
+- **6 identical layers** with three sub-layers:
+  1. Masked multi-head self-attention (prevents leftward flow)
+  2. Multi-head attention over encoder output
+  3. Position-wise feed-forward network
+- Same residual connections and normalization as encoder
+- Output embeddings offset by one position (auto-regressive property)
+
+---
+
+## Attention Mechanism
+
+### Scaled Dot-Product Attention
+$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$
+- Scaling by $\sqrt{d_k}$ prevents small gradients
+- Efficient with matrix multiplication
+
+### Multi-Head Attention
+$$\text{MultiHead}(Q, K, V) = \text{Concat}(\text{head}_1,...,\text{head}_h)W^O$$
+- $h=8$ parallel heads, $d_k=d_v=64$
+- Captures diverse dependency patterns
+
+---
+
+## Attention Applications
+
+1. **Encoder-decoder attention**: Queries from decoder, keys/values from encoder  
+2. **Encoder self-attention**: All positions attend to all input positions  
+3. **Decoder self-attention**: Positions attend to previous positions (masked)  
+
+---
+
+## Positional Encoding
+
+Adds sequence position information via sinusoidal functions:
+$$\text{PE}_{(pos,2i)}=\sin(pos/10000^{2i/d_{\text{model}}})$$
+$$\text{PE}_{(pos,2i+1)}=\cos(pos/10000^{2i/d_{\text{model}}})$$
+
+- Same dimension as embeddings ($d_{\text{model}}$)
+- Enables learning of relative positions
+- Performs similarly to learned embeddings
+
+---
+
+## Why Self-Attention?
+
+| Layer Type       | Complexity       | Sequential Ops | Max Path Length |
+|------------------|------------------|----------------|-----------------|
+| Self-Attention   | $O(n^2 \cdot d)$ | $O(1)$         | $O(1)$          |
+| Recurrent        | $O(n \cdot d^2)$ | $O(n)$         | $O(n)$          |
+| Convolutional    | $O(k \cdot n \cdot d^2)$ | $O(1)$ | $O(\log_k n)$   |
+
+- Better parallelization than RNNs
+- Shorter path length than CNNs
+- More interpretable attention patterns
+
+---
+
+## Training Data & Batching
+
+- **Datasets**:
+  - WMT 2014 EN-DE (4.5M sentence pairs)
+  - WMT 2014 EN-FR (36M sentence pairs)
+- **Tokenization**:
+  - Byte-pair encoding with shared vocabulary
+  - 37K tokens (EN-DE), 32K tokens (EN-FR)
+- **Batching**:
+  - Sentences grouped by approximate length
+  - Each batch contains ~25K source and target tokens
+
+---
+
+## Training Hardware & Schedule
+
+- **Hardware**: 8 NVIDIA P100 GPUs
+- **Base model**:
+  - 100,000 training steps
+  - ~12 hours total training time
+- **Big model**:
+  - 300,000 training steps
+  - ~3.5 days total training time
+
+---
+
+## Training Optimization
+
+- **Optimizer**: Adam
+  - $\beta_1 = 0.9$, $\beta_2 = 0.98$, $\epsilon = 10^{-9}$
+  - Learning rate schedule with warmup steps=4000
+
+- **Regularization**:
+  - Residual dropout (P_drop = 0.1)
+  - Label smoothing ($\epsilon_{ls} = 0.1$)
+  - Dropout on embeddings + positional encodings
+
+---
+
+## Machine Translation Results
+
+| Model                | EN-DE BLEU | EN-FR BLEU | Training Cost (FLOPs) |
+|----------------------|------------|------------|-----------------------|
+| GNMT + RL Ensemble   | 26.30      | 41.16      | $1.8 \cdot 10^{20}$   |
+| ConvS2S Ensemble     | 26.36      | 41.29      | $7.7 \cdot 10^{19}$   |
+| **Transformer (big)**| **28.4**   | **41.8**   | **$2.3 \cdot 10^{19}$**|
+
+- Outperforms all previous SOTA with lower training cost
+
+---
+
+## Attention Visualization: Long-Distance Dependencies
+
+<img src="/paper_figure_3_long_distance.png" class="h-[600px] w-full object-contain" />
+
+*Tracking "making...more difficult" dependency across distant positions*
+
+---
+
+## Attention Visualization: Anaphora Resolution
+
+<img src="/paper_figure_4_anaphora.png" class="h-[600px] w-full object-contain" />
+
+*Resolving "its" to antecedent "The Law"*
+
+---
+
+## Key Contributions
+
+- Introduced Transformer, first attention-only transduction model
+- Eliminated recurrence/convolution → better parallelization
+- Set new SOTA in machine translation with lower training cost
+- Generalizes to other tasks (e.g., constituency parsing)
+
+---
+
+## Future Work
+
+- Apply to other modalities (images, audio, video)
+- Explore local attention for large sequences
+- Improve generation sequentiality
+- Enhance attention interpretability
+
+**Code:** https://github.com/tensorflow/tensor2tensor

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round4_review.json

@@ -0,0 +1,30 @@
+{
+  "overall_score": 85,
+  "pass": false,
+  "issues": [
+    {
+      "page": 16,
+      "issue_type": "readability",
+      "severity": "medium",
+      "suggestion": "放大注意力可视化图中的文字，确保观众能清晰辨认词语和连接线；可适当简化图例说明文字"
+    },
+    {
+      "page": 16,
+      "issue_type": "image_size",
+      "severity": "medium",
+      "suggestion": "增大可视化图表尺寸，占据更多页面空间以提升细节可见度，避免文字挤压"
+    },
+    {
+      "page": 17,
+      "issue_type": "readability",
+      "severity": "high",
+      "suggestion": "重新生成高分辨率的注意力可视化图，确保所有词语清晰可辨；考虑使用更大字号的文本标注"
+    },
+    {
+      "page": 17,
+      "issue_type": "image_size",
+      "severity": "high",
+      "suggestion": "显著放大图表至页面70%以上空间，或拆分图表为单独页面展示；添加简洁标题说明图表内容"
+    }
+  ]
+}

chapter5/paper-to-ppt/validation/runs/exp5-4-real-pdf-both-20260730-v4/dual_round4_slides.md

@@ -0,0 +1,208 @@
+---
+theme: default
+title: 'Attention Is All You Need'
+author: 'Ashish Vaswani et al.'
+date: 'NIPS 2017'
+---
+
+# Attention Is All You Need
+
+**Authors:** Ashish Vaswani et al.  
+**Conference:** NIPS 2017
+
+---
+
+## Key Innovation
+
+- First sequence transduction model based **solely on attention**
+- Dispenses with recurrence and convolutions entirely
+- Enables significantly more parallelization
+- Requires substantially less training time
+- Maintains or improves model quality
+
+---
+
+## Performance Highlights
+
+- **WMT 2014 English-to-German**: 28.4 BLEU
+  - Improves over existing best results by 2+ BLEU
+- **WMT 2014 English-to-French**: 41.8 BLEU
+  - New single-model state-of-the-art
+  - Trained for 3.5 days on 8 GPUs (small fraction of previous costs)
+
+---
+
+## Background & Motivation
+
+### Traditional Sequence Models
+- **Recurrent (LSTM/GRU)**: Sequential computation limits parallelization
+- **Convolutional**: Requires multiple layers for long-range dependencies
+
+### The Transformer Solution
+- Replaces recurrence/convolution with **self-attention**
+- Directly models long-range dependencies
+- Massive parallelization enables faster training
+
+---
+
+## Transformer Architecture
+
+<img src="/paper_figure_1_transformer.png" class="h-[600px] w-full object-contain" />
+
+---
+
+## Encoder Stack
+
+- **6 identical layers** with two sub-layers:
+  1. Multi-head self-attention mechanism
+  2. Position-wise feed-forward network
+- Residual connections around each sub-layer
+- Layer normalization after each sub-layer
+- Output dimension: `d_model = 512`
+
+---
+
+## Decoder Stack
+
+- **6 identical layers** with three sub-layers:
+  1. Masked multi-head self-attention (prevents leftward flow)
+  2. Multi-head attention over encoder output
+  3. Position-wise feed-forward network
+- Same residual connections and normalization as encoder
+- Output embeddings offset by one position (auto-regressive property)
+
+---
+
+## Attention Mechanism
+
+### Scaled Dot-Product Attention
+$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$
+- Scaling by $\sqrt{d_k}$ prevents small gradients
+- Efficient with matrix multiplication
+
+### Multi-Head Attention
+$$\text{MultiHead}(Q, K, V) = \text{Concat}(\text{head}_1,...,\text{head}_h)W^O$$
+- $h=8$ parallel heads, $d_k=d_v=64$
+- Captures diverse dependency patterns
+
+---
+
+## Attention Applications & Positional Encoding
+
+### Attention Applications
+1. **Encoder-decoder attention**: Queries from decoder, keys/values from encoder  
+2. **Encoder self-attention**: All positions attend to all input positions  
+3. **Decoder self-attention**: Positions attend to previous positions (masked)  
+
+### Positional Encoding
+Adds sequence position information via sinusoidal functions:
+$$\text{PE}_{(pos,2i)}=\sin(pos/10000^{2i/d_{\text{model}}})$$
+$$\text{PE}_{(pos,2i+1)}=\cos(pos/10000^{2i/d_{\text{model}}})$$
+
+---
+
+## Why Self-Attention?
+
+| Layer Type       | Complexity       | Sequential Ops | Max Path Length |
+|------------------|------------------|----------------|-----------------|
+| Self-Attention   | $O(n^2 \cdot d)$ | $O(1)$         | $O(1)$          |
+| Recurrent        | $O(n \cdot d^2)$ | $O(n)$         | $O(n)$          |
+| Convolutional    | $O(k \cdot n \cdot d^2)$ | $O(1)$ | $O(\log_k n)$   |
+
+- Better parallelization than RNNs
+- Shorter path length than CNNs
+- More interpretable attention patterns
+
+---
+
+## Training Datasets & Tokenization
+
+- **Datasets**:
+  - WMT 2014 English-German (4.5M sentence pairs)
+  - WMT 2014 English-French (36M sentence pairs)
+
+- **Tokenization**:
+  - Byte-pair encoding with shared vocabulary
+  - 37,000 tokens (EN-DE), 32,000 tokens (EN-FR)
+
+---
+
+## Training Batching Strategy
+
+- Sentences grouped by approximate length
+- Each batch contains ~25,000 source tokens
+- Each batch contains ~25,000 target tokens
+- Balances computational efficiency and sequence length variation
+
+---
+
+## Training Hardware & Schedule
+
+- **Hardware**: 8 NVIDIA P100 GPUs
+- **Base model**:
+  - 100,000 training steps
+  - ~12 hours total training time
+- **Big model**:
+  - 300,000 training steps
+  - ~3.5 days total training time
+
+---
+
+## Training Optimization
+
+- **Optimizer**: Adam
+  - $\beta_1 = 0.9$, $\beta_2 = 0.98$, $\epsilon = 10^{-9}$
+  - Learning rate schedule with warmup steps=4000
+
+- **Regularization**:
+  - Residual dropout (P_drop = 0.1)
+  - Label smoothing ($\epsilon_{ls} = 0.1$)
+  - Dropout on embeddings + positional encodings
+
+---
+
+## Machine Translation Results
+
+| Model                | EN-DE BLEU | EN-FR BLEU | Training Cost (FLOPs) |
+|----------------------|------------|------------|-----------------------|
+| GNMT + RL Ensemble   | 26.30      | 41.16      | $1.8 \cdot 10^{20}$   |
+| ConvS2S Ensemble     | 26.36      | 41.29      | $7.7 \cdot 10^{19}$   |
+| **Transformer (big)**| **28.4**   | **41.8**   | **$2.3 \cdot 10^{19}$**|
+
+- Outperforms all previous SOTA with lower training cost
+
+---
+
+## Attention Visualization: Long-Distance Dependencies
+
+<img src="/paper_figure_3_long_distance.png" class="h-[600px] w-full object-contain" />
+
+*Tracking "making...more difficult" dependency across distant positions*
+
+---
+
+## Attention Visualization: Anaphora Resolution
+
+<img src="/paper_figure_4_anaphora.png" class="h-[600px] w-full object-contain" />
+
+*Resolving "its" to antecedent "The Law"*
+
+---
+
+## Key Contributions
+
+- Introduced Transformer, first attention-only transduction model
+- Eliminated recurrence/convolution → better parallelization
+- Set new SOTA in machine translation with lower training cost
+- Generalizes to other tasks (e.g., constituency parsing)
+
+---
+
+## Future Work
+
+- Apply to other modalities (images, audio, video)
+- Explore local attention for large sequences
+- Improve generation sequentiality
+- Enhance attention interpretability
+
+**Code:** https://github.com/tensorflow/tensor2tensor