ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）

chapter10/book-translation/sample_book/chapter1.md

@@ -0,0 +1,28 @@
+# Chapter 1: Foundations of LLM Inference
+
+A large language model turns text into numbers before it can reason about
+anything. Each chunk of text is first split into a **token**, the smallest unit
+the model consumes. Every token is then mapped to an **embedding**, a dense
+vector that captures its meaning in a high-dimensional space.
+
+When a user sends a request, the text they write is called a **prompt**. The
+process of running the model over that prompt to produce an answer is called
+**inference**. The time between sending the prompt and receiving the first
+response is the **latency** that users feel directly.
+
+A minimal inference call looks like this:
+
+```python
+def generate(prompt: str, model) -> str:
+    tokens = model.tokenize(prompt)      # split prompt into tokens
+    embeddings = model.embed(tokens)     # map each token to an embedding
+    output = model.forward(embeddings)   # run inference
+    return model.detokenize(output)
+```
+
+Two numbers dominate the user experience. First, the number of tokens in the
+prompt, because a longer prompt costs more compute. Second, the latency of the
+first token, because a slow first token makes the whole system feel sluggish.
+Throughout this book we keep returning to these ideas: token, embedding, prompt,
+inference, and latency. Getting their definitions right now will save confusion
+later.

chapter10/book-translation/sample_book/chapter2.md

@@ -0,0 +1,24 @@
+# Chapter 2: The Transformer and Attention
+
+Modern language models are built on the **transformer** architecture. Its
+central idea is **attention**: instead of reading a sequence strictly left to
+right, the model lets every token look at every other token and decide which
+ones matter. This is why a transformer can connect a pronoun to a noun that
+appeared many tokens earlier.
+
+Attention works on the **embedding** of each token. For every token the model
+computes three vectors — a query, a key, and a value — and uses them to weigh
+how much each token should attend to the others.
+
+```python
+def attention(query, key, value):
+    scores = query @ key.T          # similarity between tokens
+    weights = softmax(scores)       # attention weights
+    return weights @ value          # weighted embedding
+```
+
+Because attention compares every token with every other token, its cost grows
+quickly as the prompt gets longer. This is the root cause of the latency
+problems we will attack in the next chapter. Still, attention is what gives the
+transformer its power: during inference, it lets the model route information
+flexibly across the whole prompt rather than through a fixed pipeline.

chapter10/book-translation/sample_book/chapter3.md

@@ -0,0 +1,25 @@
+# Chapter 3: Optimizing Inference Latency
+
+Once a model works, the next battle is speed. The goal is to lower **latency**
+while raising **throughput**, the number of requests the system finishes per
+second. These two often trade off against each other.
+
+The most important trick is the **KV cache**. During inference the model would
+otherwise recompute attention over every previous token at each step. By caching
+the key and value vectors of past tokens, the model only processes the newest
+token, which cuts latency dramatically for long prompts.
+
+```python
+def decode_step(new_token, kv_cache):
+    q, k, v = project(new_token)         # only the new token
+    kv_cache.append(k, v)                # reuse past keys and values
+    return attention(q, kv_cache.keys, kv_cache.values)
+```
+
+A second trick is **batching**: grouping several prompts together so the
+hardware stays busy. Larger batches raise throughput but can hurt the latency of
+any single request, so serving systems tune the batch size carefully.
+
+The lesson is that inference performance is a balance. Every token we avoid
+recomputing, and every prompt we batch well, moves the system toward lower
+latency and higher throughput at the same time.

chapter10/book-translation/sample_book/chapter4.md

@@ -0,0 +1,27 @@
+# Chapter 4: Fine-tuning and Deployment
+
+A general model rarely fits a specific product out of the box. The usual fix is
+**fine-tuning**: continuing to train the model on a smaller, task-specific
+dataset so it adapts to your domain while keeping its general ability.
+
+Fine-tuning changes how the model turns a **prompt** into an answer, but it does
+not change the basic pipeline: text becomes a **token**, each token becomes an
+**embedding**, and **inference** produces the result. What changes is the
+weights the model learned.
+
+```python
+def fine_tune(model, dataset):
+    for prompt, target in dataset:
+        loss = model.loss(prompt, target)   # compare output to target
+        model.update(loss)                  # adjust weights
+    return model
+```
+
+After fine-tuning comes **deployment**: packaging the model behind an API so real
+users can send a prompt and get an answer. Here the earlier concerns return with
+full force. Latency must stay low, throughput must stay high, and the KV cache
+and batching from the previous chapter do the heavy lifting.
+
+The full journey — token, embedding, prompt, inference, latency, fine-tuning,
+and deployment — is now complete. A model that was once a research artifact has
+become a service that people can actually use.