liqiang
b119135836
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
35 lines
8.2 KiB
XML
35 lines
8.2 KiB
XML
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 40 780 450" width="780" height="450" style="background:#ffffff">
|
||
<defs><marker id="ah" markerWidth="12" markerHeight="8" refX="12" refY="4" orient="auto"><polygon points="0 0, 12 4, 0 8" fill="#333333"/></marker><marker id="ah-light" markerWidth="12" markerHeight="8" refX="12" refY="4" orient="auto"><polygon points="0 0, 12 4, 0 8" fill="#999999"/></marker></defs>
|
||
|
||
<rect x="300" y="80" width="180" height="120" rx="6" fill="#f0f0f0" stroke="#333333" stroke-width="2"/>
|
||
<text x="390" y="105" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="18.5" fill="#333333" text-anchor="middle" dominant-baseline="central" font-weight="bold">Исходные веса W</text>
|
||
<text x="390" y="130" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">d × d</text>
|
||
<text x="390" y="152" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">Пример: 4096 × 4096</text>
|
||
<text x="390" y="172" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="7.5" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">Параметры зафиксированы (не обучаются)</text>
|
||
<text x="520" y="140" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="24" fill="#333333" text-anchor="middle" dominant-baseline="central" font-weight="bold">+</text>
|
||
<rect x="560" y="70" width="100" height="65" rx="6" fill="#d0d0d0" stroke="#333333" stroke-width="2"/>
|
||
<text x="610" y="88" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="middle" dominant-baseline="central" font-weight="bold">Матрица B</text>
|
||
<text x="610" y="108" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">d × r</text>
|
||
<text x="610" y="122" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">4096×8</text>
|
||
<rect x="560" y="145" width="100" height="65" rx="6" fill="#d0d0d0" stroke="#333333" stroke-width="2"/>
|
||
<text x="610" y="163" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="middle" dominant-baseline="central" font-weight="bold">Матрица A</text>
|
||
<text x="610" y="183" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">r × d</text>
|
||
<text x="610" y="197" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">8×4096</text>
|
||
<path d="M 665,70 C 685,70 685,135.0 690,140.0 C 685,145.0 685,210 665,210" fill="none" stroke="#333333" stroke-width="2"/>
|
||
<text x="695" y="140" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="bold">ΔW = αBA</text>
|
||
<text x="280" y="140" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="20" fill="#333333" text-anchor="middle" dominant-baseline="central" font-weight="normal">x →</text>
|
||
<text x="695" y="165" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#666666" text-anchor="start" dominant-baseline="central" font-weight="normal">ранг r = 8</text>
|
||
<rect x="30" y="230" width="720" height="228" rx="4" fill="#f5f5f5" stroke="#333333" stroke-width="2"/>
|
||
<text x="390" y="252" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="20" fill="#333333" text-anchor="middle" dominant-baseline="central" font-weight="bold">Ключевые практические выводы (Thinking Machines Lab)</text>
|
||
<text x="50" y="282" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="bold">▸ Оптимальная скорость обучения</text>
|
||
<text x="50" y="300" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="14" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="normal">LoRA LR = LR полной тонкой настройки × 10 (примерно ×15 для короткого обучения)</text>
|
||
<text x="50" y="317" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="bold">▸ Область применения</text>
|
||
<text x="50" y="335" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="9.5" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="normal">Должно применяться ко всем слоям (особенно MLP/MoE; применение только к слоям внимания значительно снижает производительность)</text>
|
||
<text x="50" y="352" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="bold">▸ Сценарий RL</text>
|
||
<text x="50" y="370" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="11.5" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="normal">Даже rank=1 может достичь производительности полной тонкой настройки (усваивается только O(1) бит на эпизод)</text>
|
||
<text x="50" y="387" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="bold">▸ Вычислительная эффективность</text>
|
||
<text x="50" y="405" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="13" fill="#666666" text-anchor="start" dominant-baseline="central" font-weight="normal">FLOPs ≈ 2/3 полного дообучения (2N² vs 3N²); LoRA лучше при сравнении производительности по FLOPs</text>
|
||
<text x="50" y="422" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="16" fill="#333333" text-anchor="start" dominant-baseline="central" font-weight="bold">▸ Преимущества вывода</text>
|
||
<text x="50" y="440" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="13" fill="#666666" text-anchor="start" dominant-baseline="central" font-weight="normal">Мультитенантный сервис: vLLM/SGLang поддерживают одновременную загрузку нескольких адаптеров</text>
|
||
<text x="390" y="472" font-family="Arial, 'Helvetica Neue', Helvetica, 'PingFang SC', 'Microsoft YaHei', sans-serif" font-size="11" fill="#666666" text-anchor="middle" dominant-baseline="central" font-weight="normal">Рекомендуемый ранг для SFT 64-256 | Рекомендуемый ранг для RL 8-32 | Масштабирование 1/r делает LR примерно независимым от ранга</text>
|
||
</svg> |
