liqiang
b119135836
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
494 lines
17 KiB
Markdown
494 lines
17 KiB
Markdown
# Cursor Chat: ai-agent-book
|
|
|
|
## Metadata
|
|
- **Project**: ai-agent-book
|
|
- **Path**: `/Users/boj`
|
|
- **Date**: 2025-09-14 16:24:22
|
|
- **Session ID**: `78b81b21-23c5-4c3c-bb21-e526fa9a2979`
|
|
|
|
## Conversation
|
|
|
|
### 👤 You
|
|
|
|
% python main.py
|
|
============================================================
|
|
Tool-Calling Agent Demo with Attention Tracking
|
|
============================================================
|
|
INFO:agent:Initializing Qwen/Qwen3-0.6B on mps
|
|
INFO:agent:Model has 28 layers
|
|
|
|
--- Sample 1 ---
|
|
User: What is the current time in Vancouver?
|
|
INFO:agent:Input: 858 tokens
|
|
`generation_config` default values have been modified to match model-specific defaults: {'top_k': 20, 'pad_token_id': 151643, 'bos_token_id': 151643, 'eos_token_id': [151645, 151643]}. If this is not desired, please set these values explicitly.
|
|
|
|
|
|
|
|
INFO:agent:Generated 100 tokens with 99 attention steps
|
|
|
|
Assistant: <think>
|
|
Okay, the user is asking for the current time in Vancouver. Let me check which tool I can use here. The available tools include get_current_time, which requires a timezone parameter. The default timezone is UTC, but the user mentioned Vancouver. I need to make sure the timezone is set correctly for Vancouver. Since the example uses IANA timezone names, I should specify "Vancouver" as the location. Wait, does the get_current_time function accept a location parameter? Looking back at
|
|
|
|
📊 Attention tracking:
|
|
• Total steps: 99
|
|
• Input tokens: 858
|
|
• Output tokens: 100
|
|
----------------------------------------
|
|
|
|
--- Sample 2 ---
|
|
User: What's the weather like in Vancouver right now?
|
|
INFO:agent:Input: 860 tokens
|
|
INFO:agent:Generated 100 tokens with 99 attention steps
|
|
|
|
|
|
--
|
|
main.py is a simple LLM, rather than agent at all. It does not have any tool calls. Update it to an agent with ReAct loop, and visualize each of the steps. Check local_llm_serving on how to build the context of tool calls and how to parse the responses.
|
|
|
|
Do not limit the number of input and output tokens. Agents need a lot of tokens!
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
I'll help you transform the simple LLM in `main.py` into a proper ReAct agent with tool calling capabilities. Let me first examine the current implementation and then check the reference implementation in `local_llm_serving`.
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
I'll help you transform the simple LLM in `main.py` into a proper ReAct agent with tool calling capabilities. Let me first examine the current implementation and the reference code from local_llm_serving.
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
Now I'll transform the `main.py` file into a proper ReAct agent with tool calling capabilities, removing token limits and adding step-by-step visualization.
|
|
|
|
---
|
|
|
|
### 👤 You
|
|
|
|
log the output tokens by streaming to console. and stop output at EOS
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
I'll modify the ReAct agent to stream tokens to the console and stop at EOS token. Let me update the implementation to add streaming capabilities.
|
|
|
|
---
|
|
|
|
### 👤 You
|
|
|
|
--- Step 5 ---
|
|
|
|
📊 Input: 1135 tokens | Max new: 500
|
|
🔤 Streaming output:
|
|
------------------------------------------------------------
|
|
The current time in Vancouver is 08:38:50 UTC. I can now present this information to the user.
|
|
|
|
Final Answer:
|
|
🛑 [Stop string detected: Final Answer:]
|
|
📈 Generated 30 tokens
|
|
|
|
--- Step 6 ---
|
|
|
|
📊 Input: 1174 tokens | Max new: 500
|
|
🔤 Streaming output:
|
|
------------------------------------------------------------
|
|
The current time in Vancouver is 08:38:50 UTC. I can now present this information to the user.
|
|
|
|
Final Answer:
|
|
🛑 [Stop string detected: Final Answer:]
|
|
📈 Generated 30 tokens
|
|
|
|
--- Step 7 ---
|
|
|
|
📊 Input: 1213 tokens | Max new: 500
|
|
🔤 Streaming output:
|
|
------------------------------------------------------------
|
|
The current time in Vancouver is 08:38:50 UTC. I can now present this information to the user.
|
|
Thought: The current time in Vancouver is 08:38:50 UTC. I can now present this information to the user.
|
|
Final Answer:
|
|
🛑 [Stop string detected: Final Answer:]
|
|
📈 Generated 60 tokens
|
|
|
|
|
|
--
|
|
You should not stop based on final answer. You should stop the react loop based on no tool calls. (No tool calls means stop). In addition, you must save the attention map and visualize it for each LLM call (i.e. each step).
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
I'll fix the ReAct loop to properly handle the final answer and add attention visualization for each step.
|
|
|
|
---
|
|
|
|
### 👤 You
|
|
|
|
# Add observation to conversation
|
|
observation = f"\nObservation: {tool_result}\nThought: "
|
|
conversation_context += response_text + observation
|
|
|
|
|
|
--
|
|
This part is ad-hoc. You should check the prompt template of chapter 2, and use the standard chat template in tokenizer_config.json
|
|
|
|
Hugging Face's logo
|
|
Hugging Face
|
|
Models
|
|
Datasets
|
|
Spaces
|
|
Community
|
|
Docs
|
|
Enterprise
|
|
Pricing
|
|
|
|
|
|
|
|
|
|
Qwen
|
|
/
|
|
Qwen3-0.6B
|
|
|
|
like
|
|
614
|
|
|
|
Follow
|
|
|
|
Qwen
|
|
48.9k
|
|
Text Generation
|
|
Transformers
|
|
Safetensors
|
|
qwen3
|
|
conversational
|
|
text-generation-inference
|
|
|
|
arxiv:
|
|
2505.09388
|
|
|
|
License:
|
|
apache-2.0
|
|
Model card
|
|
Files and versions
|
|
xet
|
|
Community
|
|
22
|
|
Qwen3-0.6B
|
|
/
|
|
tokenizer_config.json
|
|
|
|
feihu.hf
|
|
update tokenizer_config.json
|
|
7e4ae26
|
|
4 months ago
|
|
raw
|
|
|
|
Copy download link
|
|
history
|
|
blame
|
|
contribute
|
|
delete
|
|
|
|
9.73 kB
|
|
{
|
|
"add_bos_token": false,
|
|
"add_prefix_space": false,
|
|
"added_tokens_decoder": {
|
|
"151643": {
|
|
"content": "<|endoftext|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151644": {
|
|
"content": "<|im_start|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151645": {
|
|
"content": "<|im_end|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151646": {
|
|
"content": "<|object_ref_start|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151647": {
|
|
"content": "<|object_ref_end|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151648": {
|
|
"content": "<|box_start|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151649": {
|
|
"content": "<|box_end|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151650": {
|
|
"content": "<|quad_start|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151651": {
|
|
"content": "<|quad_end|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151652": {
|
|
"content": "<|vision_start|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151653": {
|
|
"content": "<|vision_end|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151654": {
|
|
"content": "<|vision_pad|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151655": {
|
|
"content": "<|image_pad|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151656": {
|
|
"content": "<|video_pad|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": true
|
|
},
|
|
"151657": {
|
|
"content": "<tool_call>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151658": {
|
|
"content": "</tool_call>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151659": {
|
|
"content": "<|fim_prefix|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151660": {
|
|
"content": "<|fim_middle|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151661": {
|
|
"content": "<|fim_suffix|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151662": {
|
|
"content": "<|fim_pad|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151663": {
|
|
"content": "<|repo_name|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151664": {
|
|
"content": "<|file_sep|>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151665": {
|
|
"content": "<tool_response>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151666": {
|
|
"content": "</tool_response>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151667": {
|
|
"content": "<think>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
},
|
|
"151668": {
|
|
"content": "</think>",
|
|
"lstrip": false,
|
|
"normalized": false,
|
|
"rstrip": false,
|
|
"single_word": false,
|
|
"special": false
|
|
}
|
|
},
|
|
"additional_special_tokens": [
|
|
"<|im_start|>",
|
|
"<|im_end|>",
|
|
"<|object_ref_start|>",
|
|
"<|object_ref_end|>",
|
|
"<|box_start|>",
|
|
"<|box_end|>",
|
|
"<|quad_start|>",
|
|
"<|quad_end|>",
|
|
"<|vision_start|>",
|
|
"<|vision_end|>",
|
|
"<|vision_pad|>",
|
|
"<|image_pad|>",
|
|
"<|video_pad|>"
|
|
],
|
|
"bos_token": null,
|
|
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if message.content is string %}\n {%- set content = message.content %}\n {%- else %}\n {%- set content = '' %}\n {%- endif %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in content %}\n {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if loop.last or (not loop.last and reasoning_content) %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n {%- if enable_thinking is defined and enable_thinking is false %}\n {{- '<think>\\n\\n</think>\\n\\n' }}\n {%- endif %}\n{%- endif %}",
|
|
"clean_up_tokenization_spaces": false,
|
|
"eos_token": "<|im_end|>",
|
|
"errors": "replace",
|
|
"model_max_length": 131072,
|
|
"pad_token": "<|endoftext|>",
|
|
"split_special_tokens": false,
|
|
"tokenizer_class": "Qwen2Tokenizer",
|
|
"unk_token": null
|
|
}
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
I'll update the code to use the proper Qwen3 chat template format from the tokenizer_config.json instead of the ad-hoc approach.
|
|
|
|
---
|
|
|
|
### 👤 You
|
|
|
|
Query: What is the current time in Vancouver?
|
|
|
|
|
|
--- Step 1 ---
|
|
|
|
📊 Input: 703 tokens | Max new: 800
|
|
🔤 Streaming output:
|
|
------------------------------------------------------------
|
|
<think>
|
|
Okay, the user is asking for the current time in Vancouver. Let me think about how to approach this.
|
|
|
|
First, I remember that there's a function called get_current_time which can get the current date and time in a specific timezone. The parameters for that function include the timezone name. The default is UTC, but the user specified Vancouver. I need to check if Vancouver is correctly mapped to a timezone.
|
|
|
|
Wait, the function's parameter for timezone expects a standard IANA timezone name. Vancouver is in the Pacific Time Zone, which is UTC-8. So the correct timezone for Vancouver is America/Vancouver. I should use that in the function call.
|
|
|
|
So the function call should be get_current_time with the timezone set to America/Vancouver. Let me make sure there are no typos. Also, the user didn't mention a specific unit, so I don't need to worry about that. Just need to call the function with the right timezone parameter.
|
|
</think>
|
|
|
|
<tool_call>
|
|
{"name": "get_current_time", "arguments": {"timezone": "America/Vancouver"}}
|
|
</tool_call>!
|
|
🛑 [EOS token detected: 151645]
|
|
📈 Generated 220 tokens total
|
|
|
|
------------------------------------------------------------
|
|
📈 Total generated: 220 tokens
|
|
Traceback (most recent call last):
|
|
File "/Users/boj/ai-agent-book/projects/week2/attention_visualization/main.py", line 490, in <module>
|
|
demonstrate_react_agent()
|
|
File "/Users/boj/ai-agent-book/projects/week2/attention_visualization/main.py", line 436, in demonstrate_react_agent
|
|
steps = agent.execute_react_loop(
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "/Users/boj/ai-agent-book/projects/week2/attention_visualization/main.py", line 280, in execute_react_loop
|
|
self.visualize_step_attention(
|
|
File "/Users/boj/ai-agent-book/projects/week2/attention_visualization/main.py", line 385, in visualize_step_attention
|
|
attention_matrix = np.array(attention_weights)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (221,) + inhomogeneous part.
|
|
|
|
---
|
|
|
|
### 🤖 Cursor Assistant
|
|
|
|
I see the issue - the attention weights have varying lengths as tokens are generated (each new token attends to all previous tokens). Let me fix the visualization function to handle this properly.
|
|
|
|
---
|
|
|
|
|
|
*Exported from [Cursor View](https://github.com/saharmor/cursor-view)* |