ai-agent-book 精选快照（<2MB 代码与文档，来自 github.com/bojieli/ai-agent-book）

chapter10/book-translation/sample_book/chapter1.md

@@ -0,0 +1,28 @@
+# Chapter 1: Foundations of LLM Inference
+
+A large language model turns text into numbers before it can reason about
+anything. Each chunk of text is first split into a **token**, the smallest unit
+the model consumes. Every token is then mapped to an **embedding**, a dense
+vector that captures its meaning in a high-dimensional space.
+
+When a user sends a request, the text they write is called a **prompt**. The
+process of running the model over that prompt to produce an answer is called
+**inference**. The time between sending the prompt and receiving the first
+response is the **latency** that users feel directly.
+
+A minimal inference call looks like this:
+
+```python
+def generate(prompt: str, model) -> str:
+    tokens = model.tokenize(prompt)      # split prompt into tokens
+    embeddings = model.embed(tokens)     # map each token to an embedding
+    output = model.forward(embeddings)   # run inference
+    return model.detokenize(output)
+```
+
+Two numbers dominate the user experience. First, the number of tokens in the
+prompt, because a longer prompt costs more compute. Second, the latency of the
+first token, because a slow first token makes the whole system feel sluggish.
+Throughout this book we keep returning to these ideas: token, embedding, prompt,
+inference, and latency. Getting their definitions right now will save confusion
+later.