ai-agent-book 精选快照(<2MB 代码与文档,来自 github.com/bojieli/ai-agent-book)
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
# Environment files
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
|
||||
# Data and logs
|
||||
data/
|
||||
logs/
|
||||
*.log
|
||||
results/
|
||||
state.json
|
||||
*.pkl
|
||||
|
||||
# Memory databases
|
||||
chroma_db/
|
||||
qdrant_db/
|
||||
*.db
|
||||
*.sqlite
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
|
||||
# Jupyter
|
||||
.ipynb_checkpoints/
|
||||
*.ipynb
|
||||
|
||||
# Test artifacts
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.tox/
|
||||
.hypothesis/
|
||||
|
||||
# Benchmark outputs
|
||||
benchmark_results/
|
||||
report_*/
|
||||
*.png
|
||||
*.pdf
|
||||
@@ -0,0 +1,296 @@
|
||||
# Mem0 Agent with Kimi K3 for LOCOMO Benchmark / Mem0 Agent 与 LOCOMO 评测
|
||||
|
||||
> Companion material for *AI Agents in Depth*, Chapter 3 — Mem0 memory framework + Kimi for long-context multi-session memory (Experiment 3-2 comparison track).
|
||||
> 配套《深入理解 AI Agent》第 3 章——Mem0 记忆框架 + Kimi,长上下文多会话记忆(实验 3-2 对照实现之一)。
|
||||
|
||||
← [Chapter 3 index / 返回第 3 章目录](../README.md)
|
||||
|
||||
---
|
||||
|
||||
## English
|
||||
|
||||
### Overview
|
||||
|
||||
An agent that combines the **Mem0** memory framework with the **Kimi** language model for LOCOMO-style long-context multi-agent / multi-session tasks:
|
||||
|
||||
- **Persistent memory** via Mem0 across sessions
|
||||
- **Kimi** integration (experiment caps context budget below the model’s full window)
|
||||
- **LOCOMO benchmark** scenarios
|
||||
- Multi-session and multi-agent collaboration with shared memory
|
||||
|
||||
### Features
|
||||
|
||||
**Core:** Mem0 v3 ADD-only extraction and hybrid retrieval; context preservation; metrics (consistency, coherence, latency, memory use); local or cloud memory backend.
|
||||
|
||||
**LOCOMO scenarios:** collaborative planning; information sharing; multi-step problem solving; negotiation; teaching & learning.
|
||||
|
||||
### Installation
|
||||
|
||||
Prerequisites: Python 3.12 with the root `ch3` extra (including Mem0's NLP support for entity/BM25 signals), Kimi API key; optional Mem0 cloud key.
|
||||
|
||||
```bash
|
||||
# From the repository root: use the shared Chapter 3 environment
|
||||
uv sync --locked --python 3.12 --extra ch3
|
||||
|
||||
# Activate it before changing directories:
|
||||
# macOS/Linux:
|
||||
source .venv/bin/activate
|
||||
# Windows PowerShell: .venv\Scripts\Activate.ps1
|
||||
# Windows cmd: .venv\Scripts\activate.bat
|
||||
|
||||
# pip fallback when uv is not installed:
|
||||
# python -m pip install -e ".[ch3]"
|
||||
|
||||
cd chapter3/mem0
|
||||
|
||||
# Single-project compatibility path, still supported during migration:
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
cp env.example .env
|
||||
# Edit .env with API keys
|
||||
```
|
||||
|
||||
Required env:
|
||||
|
||||
- `KIMI_API_KEY`
|
||||
- `MODEL_NAME` (default `kimi-k3`) — **raw Moonshot model id** (e.g. `kimi-k3`, `kimi-k2.5`); do **not** use `provider/model` slash form; Mem0 uses OpenAI-compatible provider pointed at Moonshot `base_url` and forwards the string verbatim (`kimi/k3` → “Not found the model”)
|
||||
- `MEMORY_BACKEND`: `local` / `cloud`
|
||||
- `MAX_TOKENS` (default 128000)
|
||||
|
||||
### Quick start
|
||||
|
||||
```bash
|
||||
python quickstart.py
|
||||
```
|
||||
|
||||
Shows basic chat with memory, multi-session persistence, multi-agent collaboration.
|
||||
|
||||
#### Memory pipeline demo (ADD-only extraction + hybrid retrieval)
|
||||
|
||||
Demonstrates Mem0 v3's append-only history and cross-session recall:
|
||||
|
||||
```bash
|
||||
python main.py --mode demo --user-id demo_user
|
||||
```
|
||||
|
||||
Book example: a user lives in Beijing and later moves to Shanghai. Mem0 preserves both dated facts, while hybrid, time-aware retrieval ranks the current one. Same routine: `memory_pipeline_example()` in `quickstart.py`.
|
||||
|
||||
#### Direct memory operations CLI
|
||||
|
||||
```bash
|
||||
python main.py --help # Chinese descriptions
|
||||
|
||||
python main.py --mode memory --op add --text "我住在北京,是一名后端工程师" --user-id u1
|
||||
python main.py --mode memory --op search --query "这个用户住在哪里?" --user-id u1
|
||||
python main.py --mode memory --op get-all --user-id u1 --output mem.json
|
||||
python main.py --mode memory --op history --memory-id <id>
|
||||
python main.py --mode memory --op delete --memory-id <id>
|
||||
```
|
||||
|
||||
Flags: `--op {add,search,get-all,history,delete}`, `--text`, `--query`, `--memory-id`, `--user-id`, `--agent-id`, `--model`, `--output`. `--text` may be a raw string or path to a JSON message list.
|
||||
|
||||
> Demo, memory ops, and chat modes need a working LLM key (`KIMI_API_KEY`) and vector store. Without a key the CLI parses args then reports the missing key—no fabricated memory output.
|
||||
|
||||
#### Interactive / batch
|
||||
|
||||
```bash
|
||||
python main.py --mode interactive
|
||||
# commands: help, memories, metrics, save, load, new, exit
|
||||
|
||||
python main.py --mode batch --input conversations.json --output results.json
|
||||
```
|
||||
|
||||
Batch input format:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"session_id": "session_001",
|
||||
"user_id": "user_001",
|
||||
"agent_id": "agent_001",
|
||||
"turns": ["First user message", "Second user message"]
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### LOCOMO benchmark
|
||||
|
||||
```bash
|
||||
python experiment.py --scenarios 10 --output results/
|
||||
```
|
||||
|
||||
Metrics: consistency, coherence, memory retention, response time, context utilization. Results JSON under `results/` with per-scenario and overall metrics.
|
||||
|
||||
### Architecture
|
||||
|
||||
- `agent.py`: `Mem0Agent`, `KimiK3Client`, `AgentContext`
|
||||
- `config.py`: Kimi / Mem0 / LOCOMO config
|
||||
- `experiment.py`: `LOCOMOBenchmark`
|
||||
|
||||
Mem0 provides append-only extraction, hybrid retrieval, and multi-level (user/agent/session) organization.
|
||||
|
||||
### Memory backends
|
||||
|
||||
```python
|
||||
# Local Chroma
|
||||
config.mem0.backend = "local"
|
||||
config.mem0.vector_store_config = {
|
||||
"provider": "chroma",
|
||||
"config": {"collection_name": "my_collection", "path": "./data/chroma_db"}
|
||||
}
|
||||
|
||||
# Cloud
|
||||
config.mem0.backend = "cloud"
|
||||
config.mem0.api_key = "your_mem0_api_key"
|
||||
```
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
1. API key: set valid `KIMI_API_KEY` in `.env`
|
||||
2. Local backend: write permission under `./data/`
|
||||
3. Cloud: valid `MEM0_API_KEY`
|
||||
4. Debug: `export LOG_LEVEL=DEBUG`
|
||||
|
||||
### Project structure
|
||||
|
||||
```
|
||||
mem0/
|
||||
├── agent.py, config.py, experiment.py, main.py, quickstart.py
|
||||
├── requirements.txt, env.example, README.md
|
||||
```
|
||||
|
||||
### Limitations
|
||||
|
||||
Needs network for APIs; memory grows with use; context capped in experiment config; quality depends on model availability.
|
||||
|
||||
### License / acknowledgments
|
||||
|
||||
Part of AI Agent Book materials. Mem0 by Mem0 AI; Kimi by Moonshot AI.
|
||||
|
||||
---
|
||||
|
||||
## 中文
|
||||
|
||||
### 概述
|
||||
|
||||
将 **Mem0** 记忆框架与 **Kimi** 语言模型结合,面向 LOCOMO 风格长上下文、多会话 / 多 Agent 任务:
|
||||
|
||||
- 跨会话**持久记忆**
|
||||
- Kimi 集成(实验中会限制上下文预算)
|
||||
- LOCOMO 场景评测
|
||||
- 多会话、多 Agent 共享记忆协作
|
||||
|
||||
### 功能
|
||||
|
||||
**核心:** Mem0 v3 的 ADD-only 抽取与混合检索;跨会话上下文保持;一致性、连贯性、时延、记忆利用率等指标;本地或云端记忆后端。
|
||||
|
||||
**LOCOMO 场景:** 协作规划、信息共享、多步解题、谈判、教与学。
|
||||
|
||||
### 安装
|
||||
|
||||
Python 3.12 与根目录 `ch3` extra(包含实体 / BM25 信号所需的 Mem0 NLP 支持)、Kimi API Key;可选 Mem0 云端 Key。
|
||||
|
||||
```bash
|
||||
# 在仓库根目录使用统一的第 3 章环境
|
||||
uv sync --locked --python 3.12 --extra ch3
|
||||
|
||||
# 切换目录前先激活环境:
|
||||
# macOS/Linux:
|
||||
source .venv/bin/activate
|
||||
# Windows PowerShell:.venv\Scripts\Activate.ps1
|
||||
# Windows cmd:.venv\Scripts\activate.bat
|
||||
|
||||
# 未安装 uv 时可用 pip 兜底:
|
||||
# python -m pip install -e ".[ch3]"
|
||||
|
||||
cd chapter3/mem0
|
||||
|
||||
# 迁移期间仍支持单项目兼容路径:
|
||||
# python -m pip install -r requirements.txt
|
||||
|
||||
cp env.example .env
|
||||
# 编辑 .env 填入 API Key
|
||||
```
|
||||
|
||||
环境变量:
|
||||
|
||||
- `KIMI_API_KEY`
|
||||
- `MODEL_NAME`(默认 `kimi-k3`)——**原始 Moonshot 模型 id**,不要用 `provider/model` 斜杠形式
|
||||
- `MEMORY_BACKEND`:`local` / `cloud`
|
||||
- `MAX_TOKENS`(默认 128000)
|
||||
|
||||
### 快速开始
|
||||
|
||||
```bash
|
||||
python quickstart.py
|
||||
```
|
||||
|
||||
#### 记忆管线演示(仅追加提取 + 混合检索)
|
||||
|
||||
```bash
|
||||
python main.py --mode demo --user-id demo_user
|
||||
```
|
||||
|
||||
书中示例:先说住在北京,后来说搬到上海。Mem0 保留两条带时间的事实,由混合、时间感知检索优先返回当前事实。
|
||||
|
||||
#### 直接记忆操作 CLI
|
||||
|
||||
```bash
|
||||
python main.py --help
|
||||
|
||||
python main.py --mode memory --op add --text "我住在北京,是一名后端工程师" --user-id u1
|
||||
python main.py --mode memory --op search --query "这个用户住在哪里?" --user-id u1
|
||||
python main.py --mode memory --op get-all --user-id u1 --output mem.json
|
||||
python main.py --mode memory --op history --memory-id <id>
|
||||
python main.py --mode memory --op delete --memory-id <id>
|
||||
```
|
||||
|
||||
无 Key 时 CLI 会解析参数后明确报错,**不会伪造**记忆输出。
|
||||
|
||||
#### 交互 / 批处理
|
||||
|
||||
```bash
|
||||
python main.py --mode interactive
|
||||
python main.py --mode batch --input conversations.json --output results.json
|
||||
```
|
||||
|
||||
### LOCOMO 基准
|
||||
|
||||
```bash
|
||||
python experiment.py --scenarios 10 --output results/
|
||||
```
|
||||
|
||||
指标:一致性、连贯性、记忆保持、响应时间、上下文利用等。
|
||||
|
||||
### 架构与后端
|
||||
|
||||
- `agent.py` / `config.py` / `experiment.py`
|
||||
- 本地 Chroma 或 Mem0 Cloud(配置见 English 节代码块)
|
||||
|
||||
### 故障排查
|
||||
|
||||
检查 `KIMI_API_KEY`、`./data/` 写权限、`MEM0_API_KEY`;`LOG_LEVEL=DEBUG`。
|
||||
|
||||
### 项目结构
|
||||
|
||||
```
|
||||
mem0/
|
||||
├── agent.py, config.py, experiment.py, main.py, quickstart.py
|
||||
├── requirements.txt, env.example, README.md
|
||||
```
|
||||
|
||||
### 局限与许可
|
||||
|
||||
需联网调用 API;记忆随使用增长;实验中上下文有上限。教学材料许可。
|
||||
|
||||
---
|
||||
|
||||
## Notes / 说明
|
||||
|
||||
### OpenRouter 通用回退 / Universal OpenRouter fallback
|
||||
|
||||
- Primary provider keys unchanged if set.
|
||||
- Else `OPENROUTER_API_KEY` routes chat LLM via `https://openrouter.ai/api/v1` with automatic model id mapping; `OPENROUTER_MODEL` forces a specific id.
|
||||
- **Note:** Mem0’s embedder still uses OpenAI embeddings (OpenRouter has no embeddings endpoint), so `OPENAI_API_KEY` is still required for store/retrieve. OpenRouter only covers the chat LLM (ADD-only fact extraction and answering).
|
||||
|
||||
Add `OPENROUTER_API_KEY=...` to `.env` (see `env.example`).
|
||||
@@ -0,0 +1,524 @@
|
||||
"""Mem0-powered agent with Kimi K3 integration for LOCOMO benchmark."""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import Dict, List, Any, Optional, Tuple
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
import asyncio
|
||||
from collections import defaultdict
|
||||
|
||||
from mem0 import Memory, MemoryClient
|
||||
from openai import OpenAI
|
||||
from tenacity import retry, stop_after_attempt, wait_exponential
|
||||
import numpy as np
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
from rich.progress import track
|
||||
|
||||
from config import Config, config as default_config
|
||||
|
||||
|
||||
def _reasoning_safe_temperature(model, requested=1.0):
|
||||
"""Reasoning models (Kimi K3, GPT-5, ...) only accept temperature=1.
|
||||
Return 1 for those; otherwise the requested value so non-reasoning
|
||||
providers (Doubao, DeepSeek, older Moonshot) are unchanged."""
|
||||
m = str(model or "").lower().replace("/", "-")
|
||||
return 1 if ("kimi-k3" in m or "gpt-5" in m) else requested
|
||||
|
||||
|
||||
def _as_memory_list(result: Any) -> List[Dict[str, Any]]:
|
||||
"""Normalize a mem0 return value to a plain list of memory dicts.
|
||||
|
||||
Current mem0 OSS returns ``{"results": [...]}``; accepting a bare list as
|
||||
well keeps the helper useful for simple test doubles.
|
||||
"""
|
||||
if isinstance(result, dict):
|
||||
return result.get("results", []) or []
|
||||
if isinstance(result, list):
|
||||
return result
|
||||
return []
|
||||
|
||||
|
||||
def _extract_added_memories(add_result: Any) -> List[Dict[str, str]]:
|
||||
"""Return facts appended by mem0's v3 ADD-only extraction pass."""
|
||||
added = []
|
||||
for item in _as_memory_list(add_result):
|
||||
added.append({
|
||||
"memory": item.get("memory", item.get("text", "")),
|
||||
"id": item.get("id", ""),
|
||||
})
|
||||
return added
|
||||
|
||||
|
||||
def _memory_filters(user_id: str, agent_id: Optional[str] = None) -> Dict[str, str]:
|
||||
"""Build the entity filter required by mem0 v3 search/get_all."""
|
||||
filters = {"user_id": user_id}
|
||||
if agent_id:
|
||||
filters["agent_id"] = agent_id
|
||||
return filters
|
||||
|
||||
|
||||
# Set up logging
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, default_config.logging.level),
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
console = Console()
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentContext:
|
||||
"""Context information for an agent in the LOCOMO benchmark."""
|
||||
|
||||
agent_id: str
|
||||
user_id: str
|
||||
session_id: str
|
||||
turn_count: int = 0
|
||||
conversation_history: List[Dict[str, str]] = field(default_factory=list)
|
||||
metadata: Dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
def add_turn(self, role: str, content: str) -> None:
|
||||
"""Add a turn to the conversation history."""
|
||||
self.conversation_history.append({
|
||||
"role": role,
|
||||
"content": content,
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"turn": self.turn_count
|
||||
})
|
||||
self.turn_count += 1
|
||||
|
||||
|
||||
class KimiK3Client:
|
||||
"""Client for interacting with Kimi K3 model."""
|
||||
|
||||
def __init__(self, config: Config):
|
||||
self.config = config
|
||||
self.client = OpenAI(
|
||||
api_key=config.kimi.api_key,
|
||||
base_url=config.kimi.api_base
|
||||
)
|
||||
|
||||
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
|
||||
def generate(self, messages: List[Dict[str, str]], **kwargs) -> str:
|
||||
"""Generate response using Kimi K3 model."""
|
||||
try:
|
||||
response = self.client.chat.completions.create(
|
||||
model=self.config.kimi.model_name,
|
||||
messages=messages,
|
||||
max_tokens=kwargs.get("max_tokens", self.config.kimi.max_tokens),
|
||||
temperature=_reasoning_safe_temperature(self.config.kimi.model_name, kwargs.get("temperature", self.config.kimi.temperature)),
|
||||
top_p=kwargs.get("top_p", 0.95),
|
||||
frequency_penalty=kwargs.get("frequency_penalty", 0),
|
||||
presence_penalty=kwargs.get("presence_penalty", 0)
|
||||
)
|
||||
return response.choices[0].message.content
|
||||
except Exception as e:
|
||||
logger.error(f"Error generating response with Kimi K3: {e}")
|
||||
raise
|
||||
|
||||
async def agenerate(self, messages: List[Dict[str, str]], **kwargs) -> str:
|
||||
"""Async generate response using Kimi K3 model."""
|
||||
return await asyncio.to_thread(self.generate, messages, **kwargs)
|
||||
|
||||
|
||||
class Mem0Agent:
|
||||
"""Agent powered by Mem0 memory system and Kimi K3 model."""
|
||||
|
||||
def __init__(self, config: Optional[Config] = None):
|
||||
self.config = config or default_config
|
||||
self.config.validate()
|
||||
|
||||
# Initialize Kimi K3 client
|
||||
self.llm_client = KimiK3Client(self.config)
|
||||
|
||||
# Initialize Mem0 memory system
|
||||
self._init_memory()
|
||||
|
||||
# Agent state management
|
||||
self.active_contexts: Dict[str, AgentContext] = {}
|
||||
self.performance_metrics: Dict[str, List[float]] = defaultdict(list)
|
||||
|
||||
def _init_memory(self) -> None:
|
||||
"""Initialize Mem0 memory system."""
|
||||
# Mem0 runs its own LLM call for ADD-only fact extraction. Left unset,
|
||||
# mem0 defaults to
|
||||
# max_tokens=2000 / temperature=0.1, which is unsafe for reasoning
|
||||
# models (Kimi K3 wants temperature=1 and enough room for its thinking
|
||||
# tokens). Pin both explicitly so the pipeline is reasoning-safe.
|
||||
mem0_config = {
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"api_key": self.config.kimi.api_key,
|
||||
# mem0 >=1.0 names this field openai_base_url (not base_url);
|
||||
# it points the OpenAI-compatible client at Moonshot.
|
||||
"openai_base_url": self.config.kimi.api_base,
|
||||
"model": self.config.kimi.model_name,
|
||||
"temperature": _reasoning_safe_temperature(
|
||||
self.config.kimi.model_name, self.config.kimi.temperature
|
||||
),
|
||||
"max_tokens": max(self.config.kimi.max_tokens, 2048),
|
||||
}
|
||||
},
|
||||
"vector_store": self.config.mem0.vector_store_config,
|
||||
"embedder": {
|
||||
"provider": "openai",
|
||||
"config": {
|
||||
"model": self.config.mem0.embedding_model
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if self.config.mem0.backend == "local":
|
||||
self.memory = Memory.from_config(mem0_config)
|
||||
else:
|
||||
self.memory = MemoryClient(api_key=self.config.mem0.api_key)
|
||||
|
||||
logger.info(f"Initialized Mem0 memory system with {self.config.mem0.backend} backend")
|
||||
|
||||
def create_context(self, agent_id: str, user_id: str, session_id: str) -> AgentContext:
|
||||
"""Create a new agent context for a session."""
|
||||
context = AgentContext(
|
||||
agent_id=agent_id,
|
||||
user_id=user_id,
|
||||
session_id=session_id,
|
||||
metadata={
|
||||
"created_at": datetime.now().isoformat(),
|
||||
"model": self.config.kimi.model_name
|
||||
}
|
||||
)
|
||||
self.active_contexts[session_id] = context
|
||||
logger.info(f"Created context for agent {agent_id} in session {session_id}")
|
||||
return context
|
||||
|
||||
def get_context(self, session_id: str) -> Optional[AgentContext]:
|
||||
"""Get agent context for a session."""
|
||||
return self.active_contexts.get(session_id)
|
||||
|
||||
def _prepare_messages(self, context: AgentContext, user_input: str) -> List[Dict[str, str]]:
|
||||
"""Prepare messages for LLM including memory context."""
|
||||
messages = []
|
||||
|
||||
# System prompt
|
||||
system_prompt = f"""You are an intelligent agent participating in the LOCOMO benchmark.
|
||||
Your task is to maintain consistent and coherent conversations across multiple sessions.
|
||||
You have access to a memory system that helps you remember important information.
|
||||
|
||||
Agent ID: {context.agent_id}
|
||||
User ID: {context.user_id}
|
||||
Session ID: {context.session_id}
|
||||
Current Turn: {context.turn_count}
|
||||
|
||||
Guidelines:
|
||||
1. Maintain consistency with previous conversations
|
||||
2. Reference relevant past information when appropriate
|
||||
3. Build upon established context naturally
|
||||
4. Be concise but informative in your responses
|
||||
"""
|
||||
messages.append({"role": "system", "content": system_prompt})
|
||||
|
||||
# Retrieve relevant memories
|
||||
memories = _as_memory_list(self.memory.search(
|
||||
query=user_input,
|
||||
filters=_memory_filters(context.user_id, context.agent_id),
|
||||
top_k=5,
|
||||
))
|
||||
|
||||
if memories and len(memories) > 0:
|
||||
memory_context = "\n\nRelevant memories from past interactions:\n"
|
||||
for mem in memories:
|
||||
memory_context += f"- {mem.get('memory', mem.get('text', ''))}\n"
|
||||
messages.append({"role": "system", "content": memory_context})
|
||||
|
||||
# Add recent conversation history (last 10 turns)
|
||||
recent_history = context.conversation_history[-10:] if len(context.conversation_history) > 10 else context.conversation_history
|
||||
for turn in recent_history:
|
||||
messages.append({"role": turn["role"], "content": turn["content"]})
|
||||
|
||||
# Add current user input
|
||||
messages.append({"role": "user", "content": user_input})
|
||||
|
||||
return messages
|
||||
|
||||
def process_turn(self, session_id: str, user_input: str) -> Tuple[str, Dict[str, Any]]:
|
||||
"""Process a single turn in the conversation."""
|
||||
context = self.get_context(session_id)
|
||||
if not context:
|
||||
raise ValueError(f"No context found for session {session_id}")
|
||||
|
||||
# Record user input
|
||||
context.add_turn("user", user_input)
|
||||
|
||||
# Prepare messages with memory context
|
||||
messages = self._prepare_messages(context, user_input)
|
||||
|
||||
# Generate response using Kimi K3
|
||||
start_time = datetime.now()
|
||||
response = self.llm_client.generate(messages)
|
||||
generation_time = (datetime.now() - start_time).total_seconds()
|
||||
|
||||
# Record assistant response
|
||||
context.add_turn("assistant", response)
|
||||
|
||||
# Store interaction in memory. Mem0 v3 performs one ADD-only
|
||||
# extraction pass and returns the facts it appended.
|
||||
add_result = self.memory.add(
|
||||
messages=[
|
||||
{"role": "user", "content": user_input},
|
||||
{"role": "assistant", "content": response}
|
||||
],
|
||||
user_id=context.user_id,
|
||||
agent_id=context.agent_id,
|
||||
metadata={
|
||||
"session_id": session_id,
|
||||
"turn": context.turn_count - 1,
|
||||
"timestamp": datetime.now().isoformat()
|
||||
}
|
||||
)
|
||||
added_memories = _extract_added_memories(add_result)
|
||||
|
||||
# Calculate metrics
|
||||
metrics = {
|
||||
"generation_time": generation_time,
|
||||
"response_length": len(response),
|
||||
"turn_count": context.turn_count,
|
||||
"memory_count": len(self.get_all_memories(context.user_id, top_k=100)),
|
||||
"added_memories": added_memories,
|
||||
}
|
||||
|
||||
# Store performance metrics
|
||||
self.performance_metrics[session_id].append(generation_time)
|
||||
|
||||
logger.info(f"Processed turn {context.turn_count} for session {session_id} in {generation_time:.2f}s")
|
||||
|
||||
return response, metrics
|
||||
|
||||
async def process_turn_async(self, session_id: str, user_input: str) -> Tuple[str, Dict[str, Any]]:
|
||||
"""Async version of process_turn."""
|
||||
return await asyncio.to_thread(self.process_turn, session_id, user_input)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Direct memory operations (used by the CLI and the pipeline demo)
|
||||
# ------------------------------------------------------------------
|
||||
def add_memory(self, messages, user_id: str, agent_id: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None) -> List[Dict[str, str]]:
|
||||
"""Add a message/conversation to memory.
|
||||
|
||||
Returns the facts appended by mem0's ADD-only extraction. An empty
|
||||
list means that no new fact was extracted (including exact dedupes).
|
||||
``messages`` may be a plain string or an OpenAI-style message list.
|
||||
"""
|
||||
add_result = self.memory.add(
|
||||
messages=messages,
|
||||
user_id=user_id,
|
||||
agent_id=agent_id,
|
||||
metadata=metadata or {}
|
||||
)
|
||||
return _extract_added_memories(add_result)
|
||||
|
||||
def search_memory(self, query: str, user_id: str, agent_id: Optional[str] = None,
|
||||
top_k: int = 5) -> List[Dict[str, Any]]:
|
||||
"""Retrieve memories with mem0 v3's fused search signals."""
|
||||
return _as_memory_list(self.memory.search(
|
||||
query=query,
|
||||
filters=_memory_filters(user_id, agent_id),
|
||||
top_k=top_k,
|
||||
))
|
||||
|
||||
def get_all_memories(self, user_id: str, agent_id: Optional[str] = None,
|
||||
top_k: int = 100) -> List[Dict[str, Any]]:
|
||||
"""List up to ``top_k`` stored memories for a user."""
|
||||
kwargs = {"filters": _memory_filters(user_id, agent_id)}
|
||||
if isinstance(self.memory, MemoryClient):
|
||||
kwargs["page_size"] = top_k
|
||||
else:
|
||||
kwargs["top_k"] = top_k
|
||||
return _as_memory_list(self.memory.get_all(**kwargs))
|
||||
|
||||
def memory_history(self, memory_id: str) -> List[Dict[str, Any]]:
|
||||
"""Return the audit history of one memory."""
|
||||
return self.memory.history(memory_id)
|
||||
|
||||
def delete_memory(self, memory_id: str) -> str:
|
||||
"""Delete a single memory by id."""
|
||||
self.memory.delete(memory_id)
|
||||
return memory_id
|
||||
|
||||
def evaluate_consistency(self, session_id: str) -> float:
|
||||
"""Evaluate consistency of responses in a session."""
|
||||
context = self.get_context(session_id)
|
||||
if not context or len(context.conversation_history) < 2:
|
||||
return 1.0
|
||||
|
||||
# Simple consistency check based on response patterns
|
||||
responses = [turn["content"] for turn in context.conversation_history if turn["role"] == "assistant"]
|
||||
|
||||
if len(responses) < 2:
|
||||
return 1.0
|
||||
|
||||
# Calculate consistency score based on semantic similarity (simplified)
|
||||
# In a real implementation, you would use embeddings and cosine similarity
|
||||
consistency_scores = []
|
||||
for i in range(1, len(responses)):
|
||||
# Simplified: check for contradiction keywords
|
||||
prev_response = responses[i-1].lower()
|
||||
curr_response = responses[i].lower()
|
||||
|
||||
contradiction_words = ["however", "but actually", "correction", "i was wrong", "let me correct"]
|
||||
has_contradiction = any(word in curr_response for word in contradiction_words)
|
||||
|
||||
consistency_scores.append(0.5 if has_contradiction else 1.0)
|
||||
|
||||
return np.mean(consistency_scores) if consistency_scores else 1.0
|
||||
|
||||
def evaluate_coherence(self, session_id: str) -> float:
|
||||
"""Evaluate coherence of the conversation."""
|
||||
context = self.get_context(session_id)
|
||||
if not context or len(context.conversation_history) < 2:
|
||||
return 1.0
|
||||
|
||||
# Simple coherence check based on response relevance
|
||||
coherence_scores = []
|
||||
for i in range(0, len(context.conversation_history) - 1, 2):
|
||||
if i + 1 < len(context.conversation_history):
|
||||
user_turn = context.conversation_history[i]["content"]
|
||||
assistant_turn = context.conversation_history[i + 1]["content"]
|
||||
|
||||
# Check if response addresses the user input (simplified)
|
||||
user_keywords = set(user_turn.lower().split())
|
||||
assistant_keywords = set(assistant_turn.lower().split())
|
||||
|
||||
overlap = len(user_keywords.intersection(assistant_keywords))
|
||||
score = min(1.0, overlap / max(len(user_keywords), 1) * 2)
|
||||
coherence_scores.append(score)
|
||||
|
||||
return np.mean(coherence_scores) if coherence_scores else 1.0
|
||||
|
||||
def evaluate_memory_retention(self, user_id: str) -> float:
|
||||
"""Evaluate memory retention for a user."""
|
||||
memories = self.get_all_memories(user_id, top_k=100)
|
||||
|
||||
if not memories or len(memories) == 0:
|
||||
return 0.0
|
||||
|
||||
# Calculate retention score based on memory count and recency
|
||||
now = datetime.now()
|
||||
retention_scores = []
|
||||
|
||||
for memory in memories:
|
||||
created_at = memory.get("created_at", now.isoformat())
|
||||
if isinstance(created_at, str):
|
||||
created_at = datetime.fromisoformat(created_at.replace("Z", "+00:00"))
|
||||
|
||||
age_hours = (now - created_at).total_seconds() / 3600
|
||||
# Decay function: memories lose value over time
|
||||
retention_score = np.exp(-age_hours / 24) # Half-life of 24 hours
|
||||
retention_scores.append(retention_score)
|
||||
|
||||
return np.mean(retention_scores)
|
||||
|
||||
def get_performance_summary(self, session_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Get performance summary for a session or all sessions."""
|
||||
if session_id:
|
||||
context = self.get_context(session_id)
|
||||
if not context:
|
||||
return {}
|
||||
|
||||
metrics = self.performance_metrics.get(session_id, [])
|
||||
return {
|
||||
"session_id": session_id,
|
||||
"turn_count": context.turn_count,
|
||||
"avg_response_time": np.mean(metrics) if metrics else 0,
|
||||
"consistency_score": self.evaluate_consistency(session_id),
|
||||
"coherence_score": self.evaluate_coherence(session_id),
|
||||
"memory_retention": self.evaluate_memory_retention(context.user_id)
|
||||
}
|
||||
else:
|
||||
# Aggregate metrics for all sessions
|
||||
all_metrics = []
|
||||
for sid in self.active_contexts:
|
||||
all_metrics.append(self.get_performance_summary(sid))
|
||||
|
||||
if not all_metrics:
|
||||
return {}
|
||||
|
||||
return {
|
||||
"total_sessions": len(all_metrics),
|
||||
"avg_turn_count": np.mean([m["turn_count"] for m in all_metrics]),
|
||||
"avg_response_time": np.mean([m["avg_response_time"] for m in all_metrics]),
|
||||
"avg_consistency": np.mean([m["consistency_score"] for m in all_metrics]),
|
||||
"avg_coherence": np.mean([m["coherence_score"] for m in all_metrics]),
|
||||
"avg_memory_retention": np.mean([m["memory_retention"] for m in all_metrics])
|
||||
}
|
||||
|
||||
def display_metrics(self, session_id: Optional[str] = None) -> None:
|
||||
"""Display performance metrics in a formatted table."""
|
||||
summary = self.get_performance_summary(session_id)
|
||||
|
||||
if not summary:
|
||||
console.print("[yellow]No metrics available[/yellow]")
|
||||
return
|
||||
|
||||
table = Table(title="Performance Metrics")
|
||||
table.add_column("Metric", style="cyan")
|
||||
table.add_column("Value", style="green")
|
||||
|
||||
for key, value in summary.items():
|
||||
if isinstance(value, float):
|
||||
table.add_row(key.replace("_", " ").title(), f"{value:.4f}")
|
||||
else:
|
||||
table.add_row(key.replace("_", " ").title(), str(value))
|
||||
|
||||
console.print(table)
|
||||
|
||||
def reset(self) -> None:
|
||||
"""Reset the agent state."""
|
||||
self.active_contexts.clear()
|
||||
self.performance_metrics.clear()
|
||||
logger.info("Agent state reset")
|
||||
|
||||
def save_state(self, filepath: str) -> None:
|
||||
"""Save agent state to file."""
|
||||
state = {
|
||||
"contexts": {
|
||||
sid: {
|
||||
"agent_id": ctx.agent_id,
|
||||
"user_id": ctx.user_id,
|
||||
"session_id": ctx.session_id,
|
||||
"turn_count": ctx.turn_count,
|
||||
"conversation_history": ctx.conversation_history,
|
||||
"metadata": ctx.metadata
|
||||
}
|
||||
for sid, ctx in self.active_contexts.items()
|
||||
},
|
||||
"metrics": dict(self.performance_metrics),
|
||||
"timestamp": datetime.now().isoformat()
|
||||
}
|
||||
|
||||
with open(filepath, "w") as f:
|
||||
json.dump(state, f, indent=2)
|
||||
|
||||
logger.info(f"Agent state saved to {filepath}")
|
||||
|
||||
def load_state(self, filepath: str) -> None:
|
||||
"""Load agent state from file."""
|
||||
with open(filepath, "r") as f:
|
||||
state = json.load(f)
|
||||
|
||||
self.active_contexts.clear()
|
||||
for sid, ctx_data in state["contexts"].items():
|
||||
context = AgentContext(
|
||||
agent_id=ctx_data["agent_id"],
|
||||
user_id=ctx_data["user_id"],
|
||||
session_id=ctx_data["session_id"],
|
||||
turn_count=ctx_data["turn_count"],
|
||||
conversation_history=ctx_data["conversation_history"],
|
||||
metadata=ctx_data["metadata"]
|
||||
)
|
||||
self.active_contexts[sid] = context
|
||||
|
||||
self.performance_metrics = defaultdict(list, state["metrics"])
|
||||
|
||||
logger.info(f"Agent state loaded from {filepath}")
|
||||
@@ -0,0 +1,218 @@
|
||||
"""Configuration module for Mem0 agent with Kimi K3 integration."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Optional, Dict, Any
|
||||
from dataclasses import dataclass, field
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
|
||||
def _reasoning_safe_temperature(model, requested=1.0):
|
||||
"""Reasoning models (Kimi K3, GPT-5, ...) only accept temperature=1.
|
||||
Return 1 for those; otherwise the requested value so non-reasoning
|
||||
providers (Doubao, DeepSeek, older Moonshot) are unchanged."""
|
||||
m = str(model or "").lower().replace("/", "-")
|
||||
return 1 if ("kimi-k3" in m or "gpt-5" in m) else requested
|
||||
|
||||
|
||||
def _openrouter_model_id(model) -> str:
|
||||
"""Map a provider-native model name to an OpenRouter model id, used by the
|
||||
universal OpenRouter fallback. An explicit OPENROUTER_MODEL env var wins."""
|
||||
override = os.getenv("OPENROUTER_MODEL")
|
||||
if override:
|
||||
return override
|
||||
m = (model or "").strip()
|
||||
if not m:
|
||||
return "openai/gpt-5.6-luna"
|
||||
if "/" in m:
|
||||
return m # already an OpenRouter-style id (e.g. openai/gpt-5.6-luna)
|
||||
ml = m.lower()
|
||||
if ml.startswith(("gpt-", "o1", "o3", "o4", "chatgpt")):
|
||||
return "openai/" + m
|
||||
if ml.startswith("claude-"):
|
||||
return "anthropic/claude-opus-4.8"
|
||||
if ml.startswith("kimi"):
|
||||
# kimi-k3 is not on OpenRouter; moonshotai/kimi-k2.6 is the closest hosted id.
|
||||
return "moonshotai/kimi-k2.6"
|
||||
# Provider-native ids (kimi-*/doubao-*/qwen/deepseek-*) not hosted on
|
||||
# OpenRouter under the same name -> a widely-available OpenAI chat model.
|
||||
return "openai/gpt-5.6-luna"
|
||||
|
||||
|
||||
@dataclass
|
||||
class KimiConfig:
|
||||
"""Configuration for Kimi K3 model."""
|
||||
|
||||
api_key: str = field(default_factory=lambda: os.getenv("KIMI_API_KEY", ""))
|
||||
model_name: str = field(default_factory=lambda: os.getenv("MODEL_NAME", "kimi-k3"))
|
||||
max_tokens: int = field(default_factory=lambda: int(os.getenv("MAX_TOKENS", "128000")))
|
||||
temperature: float = field(default_factory=lambda: float(os.getenv("TEMPERATURE", "0.7")))
|
||||
api_base: str = field(default_factory=lambda: os.getenv("KIMI_API_BASE", "https://api.moonshot.cn/v1"))
|
||||
|
||||
def __post_init__(self):
|
||||
"""Universal OpenRouter fallback for the chat LLM: when KIMI_API_KEY is
|
||||
absent but OPENROUTER_API_KEY is present, route the chat model (used by
|
||||
KimiK3Client and threaded into mem0's own LLM config) through OpenRouter.
|
||||
NB: mem0's embedder still uses OpenAI embeddings (OpenRouter has no
|
||||
embeddings endpoint), so OPENAI_API_KEY remains needed for memory add."""
|
||||
if not self.api_key and os.getenv("OPENROUTER_API_KEY"):
|
||||
self.api_key = os.getenv("OPENROUTER_API_KEY")
|
||||
self.api_base = "https://openrouter.ai/api/v1"
|
||||
self.model_name = _openrouter_model_id(self.model_name)
|
||||
|
||||
def validate(self) -> bool:
|
||||
"""Validate Kimi configuration."""
|
||||
if not self.api_key:
|
||||
raise ValueError("KIMI_API_KEY is required (or set OPENROUTER_API_KEY for the fallback)")
|
||||
if self.max_tokens <= 0 or self.max_tokens > 128000:
|
||||
raise ValueError("MAX_TOKENS must be between 1 and 128000")
|
||||
if self.temperature < 0 or self.temperature > 2:
|
||||
raise ValueError("TEMPERATURE must be between 0 and 2")
|
||||
return True
|
||||
|
||||
|
||||
@dataclass
|
||||
class Mem0Config:
|
||||
"""Configuration for Mem0 memory system."""
|
||||
|
||||
api_key: Optional[str] = field(default_factory=lambda: os.getenv("MEM0_API_KEY"))
|
||||
backend: str = field(default_factory=lambda: os.getenv("MEMORY_BACKEND", "local"))
|
||||
collection_name: str = field(default_factory=lambda: os.getenv("MEMORY_COLLECTION", "locomo_benchmark"))
|
||||
embedding_model: str = field(default_factory=lambda: os.getenv("MEMORY_EMBEDDING_MODEL", "text-embedding-3-small"))
|
||||
vector_store_config: Dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
def __post_init__(self):
|
||||
"""Initialize vector store configuration based on backend."""
|
||||
if self.backend == "local":
|
||||
# NB: mem0 >=1.0 validates the chroma config against a fixed field
|
||||
# set (collection_name/path/host/port/api_key/tenant/client). The
|
||||
# embedding model belongs to the top-level "embedder" block (set in
|
||||
# agent.py), NOT here — passing embedding_function raises a
|
||||
# MemoryConfig validation error.
|
||||
self.vector_store_config = {
|
||||
"provider": "chroma",
|
||||
"config": {
|
||||
"collection_name": self.collection_name,
|
||||
"path": "./data/chroma_db",
|
||||
}
|
||||
}
|
||||
elif self.backend == "cloud":
|
||||
if not self.api_key:
|
||||
raise ValueError("MEM0_API_KEY is required for cloud backend")
|
||||
self.vector_store_config = {
|
||||
"provider": "mem0_cloud",
|
||||
"config": {
|
||||
"api_key": self.api_key,
|
||||
"collection_name": self.collection_name
|
||||
}
|
||||
}
|
||||
else:
|
||||
raise ValueError(f"Invalid backend: {self.backend}. Must be 'local' or 'cloud'")
|
||||
|
||||
def validate(self) -> bool:
|
||||
"""Validate Mem0 configuration."""
|
||||
if self.backend not in ["local", "cloud"]:
|
||||
raise ValueError("MEMORY_BACKEND must be 'local' or 'cloud'")
|
||||
if self.backend == "cloud" and not self.api_key:
|
||||
raise ValueError("MEM0_API_KEY is required for cloud backend")
|
||||
return True
|
||||
|
||||
|
||||
@dataclass
|
||||
class LOCOMOConfig:
|
||||
"""Configuration for LOCOMO benchmark."""
|
||||
|
||||
data_path: Path = field(default_factory=lambda: Path(os.getenv("BENCHMARK_DATA_PATH", "./data/locomo")))
|
||||
max_sessions: int = field(default_factory=lambda: int(os.getenv("MAX_SESSIONS", "100")))
|
||||
max_agents: int = field(default_factory=lambda: int(os.getenv("MAX_AGENTS", "10")))
|
||||
context_window_size: int = field(default_factory=lambda: int(os.getenv("CONTEXT_WINDOW_SIZE", "128000")))
|
||||
evaluation_metrics: list = field(default_factory=lambda: [
|
||||
"consistency_score",
|
||||
"coherence_score",
|
||||
"memory_retention",
|
||||
"context_utilization",
|
||||
"response_relevance"
|
||||
])
|
||||
|
||||
def __post_init__(self):
|
||||
"""Ensure data path exists."""
|
||||
self.data_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def validate(self) -> bool:
|
||||
"""Validate LOCOMO configuration."""
|
||||
if self.max_sessions <= 0:
|
||||
raise ValueError("MAX_SESSIONS must be positive")
|
||||
if self.max_agents <= 0:
|
||||
raise ValueError("MAX_AGENTS must be positive")
|
||||
if self.context_window_size <= 0:
|
||||
raise ValueError("CONTEXT_WINDOW_SIZE must be positive")
|
||||
return True
|
||||
|
||||
|
||||
@dataclass
|
||||
class LoggingConfig:
|
||||
"""Configuration for logging."""
|
||||
|
||||
level: str = field(default_factory=lambda: os.getenv("LOG_LEVEL", "INFO"))
|
||||
file_path: Optional[Path] = field(default_factory=lambda: Path(os.getenv("LOG_FILE", "./logs/mem0_agent.log")) if os.getenv("LOG_FILE") else None)
|
||||
|
||||
def __post_init__(self):
|
||||
"""Ensure log directory exists."""
|
||||
if self.file_path:
|
||||
self.file_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
"""Main configuration class."""
|
||||
|
||||
kimi: KimiConfig = field(default_factory=KimiConfig)
|
||||
mem0: Mem0Config = field(default_factory=Mem0Config)
|
||||
locomo: LOCOMOConfig = field(default_factory=LOCOMOConfig)
|
||||
logging: LoggingConfig = field(default_factory=LoggingConfig)
|
||||
|
||||
def validate(self) -> bool:
|
||||
"""Validate all configurations."""
|
||||
self.kimi.validate()
|
||||
self.mem0.validate()
|
||||
self.locomo.validate()
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "Config":
|
||||
"""Create configuration from environment variables."""
|
||||
return cls()
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert configuration to dictionary."""
|
||||
return {
|
||||
"kimi": {
|
||||
"model_name": self.kimi.model_name,
|
||||
"max_tokens": self.kimi.max_tokens,
|
||||
"temperature": _reasoning_safe_temperature(self.kimi.model_name, self.kimi.temperature),
|
||||
"api_base": self.kimi.api_base
|
||||
},
|
||||
"mem0": {
|
||||
"backend": self.mem0.backend,
|
||||
"collection_name": self.mem0.collection_name,
|
||||
"embedding_model": self.mem0.embedding_model
|
||||
},
|
||||
"locomo": {
|
||||
"data_path": str(self.locomo.data_path),
|
||||
"max_sessions": self.locomo.max_sessions,
|
||||
"max_agents": self.locomo.max_agents,
|
||||
"context_window_size": self.locomo.context_window_size,
|
||||
"evaluation_metrics": self.locomo.evaluation_metrics
|
||||
},
|
||||
"logging": {
|
||||
"level": self.logging.level,
|
||||
"file_path": str(self.logging.file_path) if self.logging.file_path else None
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
# Global configuration instance
|
||||
config = Config.from_env()
|
||||
@@ -0,0 +1,32 @@
|
||||
# API Keys
|
||||
KIMI_API_KEY=your_kimi_api_key_here
|
||||
MEM0_API_KEY=your_mem0_api_key_here # Optional: for cloud-based Mem0
|
||||
|
||||
# OpenRouter universal fallback (optional): if KIMI_API_KEY is missing but
|
||||
# OPENROUTER_API_KEY is set, the CHAT LLM (KimiK3Client and mem0's own LLM
|
||||
# config) is routed through OpenRouter. Model names are mapped automatically
|
||||
# (kimi-k3 -> moonshotai/kimi-k2.6; set OPENROUTER_MODEL to override).
|
||||
# NB: mem0's embedder still uses OpenAI embeddings (OpenRouter has no
|
||||
# embeddings endpoint), so OPENAI_API_KEY is still needed to store memories.
|
||||
OPENROUTER_API_KEY=your_openrouter_api_key_here
|
||||
# OPENROUTER_MODEL=openai/gpt-5.6-luna
|
||||
|
||||
# Model Configuration
|
||||
MODEL_NAME=kimi-k3
|
||||
MAX_TOKENS=128000
|
||||
TEMPERATURE=0.7
|
||||
|
||||
# Memory Configuration
|
||||
MEMORY_BACKEND=local # Options: local, cloud
|
||||
MEMORY_COLLECTION=locomo_benchmark
|
||||
MEMORY_EMBEDDING_MODEL=text-embedding-3-small
|
||||
|
||||
# LOCOMO Benchmark Settings
|
||||
BENCHMARK_DATA_PATH=./data/locomo
|
||||
MAX_SESSIONS=100
|
||||
MAX_AGENTS=10
|
||||
CONTEXT_WINDOW_SIZE=128000
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO
|
||||
LOG_FILE=./logs/mem0_agent.log
|
||||
@@ -0,0 +1,446 @@
|
||||
"""LOCOMO benchmark experiment runner for Mem0 agent."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import random
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Any, Tuple
|
||||
from datetime import datetime
|
||||
import argparse
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from rich.console import Console
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TimeElapsedColumn
|
||||
from rich.table import Table
|
||||
from rich.panel import Panel
|
||||
from rich.layout import Layout
|
||||
import matplotlib.pyplot as plt
|
||||
import seaborn as sns
|
||||
|
||||
from agent import Mem0Agent, AgentContext
|
||||
from config import Config
|
||||
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
class LOCOMOBenchmark:
|
||||
"""LOCOMO benchmark implementation for evaluating long-context multi-agent communication."""
|
||||
|
||||
def __init__(self, agent: Mem0Agent, config: Config):
|
||||
self.agent = agent
|
||||
self.config = config
|
||||
self.results = []
|
||||
self.scenario_data = []
|
||||
|
||||
def generate_scenario(self, scenario_id: int, num_agents: int = 3) -> Dict[str, Any]:
|
||||
"""Generate a LOCOMO benchmark scenario."""
|
||||
scenarios = [
|
||||
{
|
||||
"type": "collaborative_planning",
|
||||
"description": "Multiple agents collaborate to plan a complex project",
|
||||
"topics": ["project timeline", "resource allocation", "task dependencies", "risk assessment"],
|
||||
"context_requirements": ["maintain consistency across planning decisions", "remember previous constraints", "coordinate between agents"]
|
||||
},
|
||||
{
|
||||
"type": "information_sharing",
|
||||
"description": "Agents share and synthesize information across sessions",
|
||||
"topics": ["research findings", "data analysis", "hypothesis formation", "conclusion drawing"],
|
||||
"context_requirements": ["retain factual information", "build upon previous insights", "cross-reference between sources"]
|
||||
},
|
||||
{
|
||||
"type": "problem_solving",
|
||||
"description": "Agents work together to solve multi-step problems",
|
||||
"topics": ["problem decomposition", "solution strategies", "intermediate results", "final synthesis"],
|
||||
"context_requirements": ["remember partial solutions", "maintain logical consistency", "track progress across sessions"]
|
||||
},
|
||||
{
|
||||
"type": "negotiation",
|
||||
"description": "Agents engage in multi-round negotiations",
|
||||
"topics": ["initial positions", "concessions", "agreements", "conflict resolution"],
|
||||
"context_requirements": ["remember previous offers", "maintain negotiation stance", "track agreement points"]
|
||||
},
|
||||
{
|
||||
"type": "teaching_learning",
|
||||
"description": "Agents engage in educational dialogue",
|
||||
"topics": ["concept explanation", "question answering", "knowledge verification", "skill progression"],
|
||||
"context_requirements": ["track learning progress", "adapt to understanding level", "remember misconceptions"]
|
||||
}
|
||||
]
|
||||
|
||||
scenario = scenarios[scenario_id % len(scenarios)].copy()
|
||||
scenario["scenario_id"] = f"scenario_{scenario_id:03d}"
|
||||
scenario["num_agents"] = num_agents
|
||||
scenario["agents"] = [f"agent_{i:02d}" for i in range(num_agents)]
|
||||
scenario["num_sessions"] = random.randint(3, 8)
|
||||
scenario["turns_per_session"] = random.randint(5, 15)
|
||||
|
||||
return scenario
|
||||
|
||||
def generate_conversation_prompts(self, scenario: Dict[str, Any], session_num: int) -> List[str]:
|
||||
"""Generate conversation prompts for a scenario session."""
|
||||
prompts = []
|
||||
topic = random.choice(scenario["topics"])
|
||||
|
||||
base_prompts = {
|
||||
"collaborative_planning": [
|
||||
f"Let's discuss the {topic} for our project. What are your thoughts?",
|
||||
f"Based on our previous discussion, how should we adjust the {topic}?",
|
||||
f"Can you summarize what we've decided about {topic} so far?",
|
||||
f"What challenges do you foresee with the current {topic}?",
|
||||
f"How does the {topic} align with our overall objectives?"
|
||||
],
|
||||
"information_sharing": [
|
||||
f"What new information do you have about {topic}?",
|
||||
f"How does this relate to what we discussed about {topic} before?",
|
||||
f"Can you integrate the findings about {topic} with our previous data?",
|
||||
f"What patterns are emerging from our {topic} analysis?",
|
||||
f"What conclusions can we draw about {topic} at this point?"
|
||||
],
|
||||
"problem_solving": [
|
||||
f"What's our current approach to {topic}?",
|
||||
f"Have we made progress on {topic} since last time?",
|
||||
f"What obstacles are we facing with {topic}?",
|
||||
f"Can you propose an alternative solution for {topic}?",
|
||||
f"How can we validate our solution for {topic}?"
|
||||
],
|
||||
"negotiation": [
|
||||
f"What's your position on {topic}?",
|
||||
f"Can we find middle ground on {topic}?",
|
||||
f"What concessions are you willing to make regarding {topic}?",
|
||||
f"How does this affect our previous agreement on {topic}?",
|
||||
f"Let's finalize our agreement on {topic}."
|
||||
],
|
||||
"teaching_learning": [
|
||||
f"Can you explain {topic} in simple terms?",
|
||||
f"What questions do you have about {topic}?",
|
||||
f"How would you apply {topic} in practice?",
|
||||
f"What did we learn about {topic} last time?",
|
||||
f"Can you give an example of {topic}?"
|
||||
]
|
||||
}
|
||||
|
||||
scenario_prompts = base_prompts.get(scenario["type"], base_prompts["information_sharing"])
|
||||
|
||||
# Add session-specific context
|
||||
for i in range(scenario["turns_per_session"]):
|
||||
if session_num == 0:
|
||||
prompt = f"[Session {session_num + 1}, Turn {i + 1}] {random.choice(scenario_prompts)}"
|
||||
else:
|
||||
prompt = f"[Session {session_num + 1}, Turn {i + 1}] Continuing from our previous session, {random.choice(scenario_prompts)}"
|
||||
prompts.append(prompt)
|
||||
|
||||
return prompts
|
||||
|
||||
async def run_scenario_session(self, scenario: Dict[str, Any], session_num: int) -> Dict[str, Any]:
|
||||
"""Run a single session of a scenario."""
|
||||
session_id = f"{scenario['scenario_id']}_session_{session_num:02d}"
|
||||
user_id = f"user_{scenario['scenario_id']}"
|
||||
|
||||
# Create contexts for all agents
|
||||
agent_contexts = {}
|
||||
for agent_id in scenario["agents"]:
|
||||
context = self.agent.create_context(
|
||||
agent_id=agent_id,
|
||||
user_id=user_id,
|
||||
session_id=f"{session_id}_{agent_id}"
|
||||
)
|
||||
agent_contexts[agent_id] = context
|
||||
|
||||
# Generate conversation prompts
|
||||
prompts = self.generate_conversation_prompts(scenario, session_num)
|
||||
|
||||
session_results = {
|
||||
"session_id": session_id,
|
||||
"session_num": session_num,
|
||||
"turns": [],
|
||||
"metrics": {}
|
||||
}
|
||||
|
||||
# Run conversation turns
|
||||
for turn_idx, prompt in enumerate(prompts):
|
||||
# Randomly select which agent responds
|
||||
responding_agent = random.choice(scenario["agents"])
|
||||
agent_session_id = f"{session_id}_{responding_agent}"
|
||||
|
||||
# Process turn
|
||||
response, metrics = await self.agent.process_turn_async(agent_session_id, prompt)
|
||||
|
||||
session_results["turns"].append({
|
||||
"turn": turn_idx,
|
||||
"agent": responding_agent,
|
||||
"prompt": prompt,
|
||||
"response": response,
|
||||
"metrics": metrics
|
||||
})
|
||||
|
||||
# Small delay to simulate realistic conversation
|
||||
await asyncio.sleep(0.1)
|
||||
|
||||
# Calculate session-level metrics
|
||||
session_results["metrics"] = {
|
||||
"total_turns": len(prompts),
|
||||
"avg_response_time": np.mean([t["metrics"]["generation_time"] for t in session_results["turns"]]),
|
||||
"avg_response_length": np.mean([t["metrics"]["response_length"] for t in session_results["turns"]]),
|
||||
"consistency_scores": {
|
||||
agent_id: self.agent.evaluate_consistency(f"{session_id}_{agent_id}")
|
||||
for agent_id in scenario["agents"]
|
||||
},
|
||||
"coherence_scores": {
|
||||
agent_id: self.agent.evaluate_coherence(f"{session_id}_{agent_id}")
|
||||
for agent_id in scenario["agents"]
|
||||
}
|
||||
}
|
||||
|
||||
return session_results
|
||||
|
||||
async def run_scenario(self, scenario_id: int) -> Dict[str, Any]:
|
||||
"""Run a complete LOCOMO scenario."""
|
||||
scenario = self.generate_scenario(scenario_id)
|
||||
|
||||
console.print(f"\n[cyan]Running Scenario {scenario['scenario_id']}[/cyan]")
|
||||
console.print(f"Type: {scenario['type']}")
|
||||
console.print(f"Agents: {', '.join(scenario['agents'])}")
|
||||
console.print(f"Sessions: {scenario['num_sessions']}")
|
||||
|
||||
scenario_results = {
|
||||
"scenario": scenario,
|
||||
"sessions": [],
|
||||
"start_time": datetime.now().isoformat()
|
||||
}
|
||||
|
||||
# Run all sessions
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
|
||||
TimeElapsedColumn(),
|
||||
console=console
|
||||
) as progress:
|
||||
task = progress.add_task(
|
||||
f"Running {scenario['num_sessions']} sessions...",
|
||||
total=scenario["num_sessions"]
|
||||
)
|
||||
|
||||
for session_num in range(scenario["num_sessions"]):
|
||||
session_results = await self.run_scenario_session(scenario, session_num)
|
||||
scenario_results["sessions"].append(session_results)
|
||||
progress.update(task, advance=1)
|
||||
|
||||
# Delay between sessions
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
scenario_results["end_time"] = datetime.now().isoformat()
|
||||
|
||||
# Calculate scenario-level metrics
|
||||
scenario_results["overall_metrics"] = self.calculate_scenario_metrics(scenario_results)
|
||||
|
||||
return scenario_results
|
||||
|
||||
def calculate_scenario_metrics(self, scenario_results: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Calculate overall metrics for a scenario."""
|
||||
all_turns = []
|
||||
for session in scenario_results["sessions"]:
|
||||
all_turns.extend(session["turns"])
|
||||
|
||||
consistency_scores = []
|
||||
coherence_scores = []
|
||||
for session in scenario_results["sessions"]:
|
||||
consistency_scores.extend(list(session["metrics"]["consistency_scores"].values()))
|
||||
coherence_scores.extend(list(session["metrics"]["coherence_scores"].values()))
|
||||
|
||||
metrics = {
|
||||
"total_turns": len(all_turns),
|
||||
"total_sessions": len(scenario_results["sessions"]),
|
||||
"avg_response_time": np.mean([t["metrics"]["generation_time"] for t in all_turns]),
|
||||
"std_response_time": np.std([t["metrics"]["generation_time"] for t in all_turns]),
|
||||
"avg_response_length": np.mean([t["metrics"]["response_length"] for t in all_turns]),
|
||||
"avg_consistency": np.mean(consistency_scores) if consistency_scores else 0,
|
||||
"avg_coherence": np.mean(coherence_scores) if coherence_scores else 0,
|
||||
"memory_utilization": len(self.agent.get_all_memories(
|
||||
f"user_{scenario_results['scenario']['scenario_id']}", top_k=100
|
||||
))
|
||||
}
|
||||
|
||||
return metrics
|
||||
|
||||
async def run_benchmark(self, num_scenarios: int = 5) -> Dict[str, Any]:
|
||||
"""Run the complete LOCOMO benchmark."""
|
||||
console.print(Panel.fit(
|
||||
f"[bold cyan]LOCOMO Benchmark[/bold cyan]\n"
|
||||
f"Scenarios: {num_scenarios}\n"
|
||||
f"Model: {self.config.kimi.model_name}\n"
|
||||
f"Memory Backend: {self.config.mem0.backend}",
|
||||
title="Benchmark Configuration"
|
||||
))
|
||||
|
||||
benchmark_results = {
|
||||
"config": self.config.to_dict(),
|
||||
"start_time": datetime.now().isoformat(),
|
||||
"scenarios": []
|
||||
}
|
||||
|
||||
for i in range(num_scenarios):
|
||||
scenario_results = await self.run_scenario(i)
|
||||
benchmark_results["scenarios"].append(scenario_results)
|
||||
self.results.append(scenario_results)
|
||||
|
||||
# Display interim results
|
||||
self.display_scenario_results(scenario_results)
|
||||
|
||||
benchmark_results["end_time"] = datetime.now().isoformat()
|
||||
benchmark_results["overall_metrics"] = self.calculate_overall_metrics(benchmark_results)
|
||||
|
||||
return benchmark_results
|
||||
|
||||
def calculate_overall_metrics(self, benchmark_results: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Calculate overall benchmark metrics."""
|
||||
all_metrics = [s["overall_metrics"] for s in benchmark_results["scenarios"]]
|
||||
|
||||
return {
|
||||
"total_scenarios": len(benchmark_results["scenarios"]),
|
||||
"avg_response_time": np.mean([m["avg_response_time"] for m in all_metrics]),
|
||||
"std_response_time": np.std([m["avg_response_time"] for m in all_metrics]),
|
||||
"avg_consistency": np.mean([m["avg_consistency"] for m in all_metrics]),
|
||||
"std_consistency": np.std([m["avg_consistency"] for m in all_metrics]),
|
||||
"avg_coherence": np.mean([m["avg_coherence"] for m in all_metrics]),
|
||||
"std_coherence": np.std([m["avg_coherence"] for m in all_metrics]),
|
||||
"avg_memory_utilization": np.mean([m["memory_utilization"] for m in all_metrics]),
|
||||
"total_turns": sum([m["total_turns"] for m in all_metrics])
|
||||
}
|
||||
|
||||
def display_scenario_results(self, scenario_results: Dict[str, Any]) -> None:
|
||||
"""Display results for a single scenario."""
|
||||
metrics = scenario_results["overall_metrics"]
|
||||
|
||||
table = Table(title=f"Scenario {scenario_results['scenario']['scenario_id']} Results")
|
||||
table.add_column("Metric", style="cyan")
|
||||
table.add_column("Value", style="green")
|
||||
|
||||
table.add_row("Type", scenario_results["scenario"]["type"])
|
||||
table.add_row("Sessions", str(metrics["total_sessions"]))
|
||||
table.add_row("Total Turns", str(metrics["total_turns"]))
|
||||
table.add_row("Avg Response Time", f"{metrics['avg_response_time']:.3f}s")
|
||||
table.add_row("Consistency Score", f"{metrics['avg_consistency']:.3f}")
|
||||
table.add_row("Coherence Score", f"{metrics['avg_coherence']:.3f}")
|
||||
table.add_row("Memory Utilization", str(metrics["memory_utilization"]))
|
||||
|
||||
console.print(table)
|
||||
|
||||
def display_overall_results(self, benchmark_results: Dict[str, Any]) -> None:
|
||||
"""Display overall benchmark results."""
|
||||
metrics = benchmark_results["overall_metrics"]
|
||||
|
||||
console.print("\n")
|
||||
console.print(Panel.fit(
|
||||
f"[bold green]Benchmark Complete![/bold green]\n"
|
||||
f"Total Scenarios: {metrics['total_scenarios']}\n"
|
||||
f"Total Turns: {metrics['total_turns']}\n"
|
||||
f"Avg Response Time: {metrics['avg_response_time']:.3f}s ± {metrics['std_response_time']:.3f}s\n"
|
||||
f"Avg Consistency: {metrics['avg_consistency']:.3f} ± {metrics['std_consistency']:.3f}\n"
|
||||
f"Avg Coherence: {metrics['avg_coherence']:.3f} ± {metrics['std_coherence']:.3f}\n"
|
||||
f"Avg Memory Utilization: {metrics['avg_memory_utilization']:.1f}",
|
||||
title="Overall Results"
|
||||
))
|
||||
|
||||
def save_results(self, benchmark_results: Dict[str, Any], filepath: Path) -> None:
|
||||
"""Save benchmark results to file."""
|
||||
filepath.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(filepath, "w") as f:
|
||||
json.dump(benchmark_results, f, indent=2)
|
||||
console.print(f"[green]Results saved to {filepath}[/green]")
|
||||
|
||||
def generate_report(self, benchmark_results: Dict[str, Any], output_dir: Path) -> None:
|
||||
"""Generate a detailed report with visualizations."""
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Extract data for visualization
|
||||
scenarios_df = pd.DataFrame([
|
||||
{
|
||||
"scenario_id": s["scenario"]["scenario_id"],
|
||||
"type": s["scenario"]["type"],
|
||||
"consistency": s["overall_metrics"]["avg_consistency"],
|
||||
"coherence": s["overall_metrics"]["avg_coherence"],
|
||||
"response_time": s["overall_metrics"]["avg_response_time"],
|
||||
"memory_utilization": s["overall_metrics"]["memory_utilization"]
|
||||
}
|
||||
for s in benchmark_results["scenarios"]
|
||||
])
|
||||
|
||||
# Create visualizations
|
||||
fig, axes = plt.subplots(2, 2, figsize=(12, 10))
|
||||
|
||||
# Consistency scores by scenario type
|
||||
sns.boxplot(data=scenarios_df, x="type", y="consistency", ax=axes[0, 0])
|
||||
axes[0, 0].set_title("Consistency Scores by Scenario Type")
|
||||
axes[0, 0].set_xticklabels(axes[0, 0].get_xticklabels(), rotation=45)
|
||||
|
||||
# Coherence scores by scenario type
|
||||
sns.boxplot(data=scenarios_df, x="type", y="coherence", ax=axes[0, 1])
|
||||
axes[0, 1].set_title("Coherence Scores by Scenario Type")
|
||||
axes[0, 1].set_xticklabels(axes[0, 1].get_xticklabels(), rotation=45)
|
||||
|
||||
# Response time distribution
|
||||
axes[1, 0].hist(scenarios_df["response_time"], bins=20, edgecolor='black')
|
||||
axes[1, 0].set_title("Response Time Distribution")
|
||||
axes[1, 0].set_xlabel("Response Time (s)")
|
||||
axes[1, 0].set_ylabel("Frequency")
|
||||
|
||||
# Memory utilization vs performance
|
||||
axes[1, 1].scatter(scenarios_df["memory_utilization"],
|
||||
scenarios_df["consistency"],
|
||||
alpha=0.6, label="Consistency")
|
||||
axes[1, 1].scatter(scenarios_df["memory_utilization"],
|
||||
scenarios_df["coherence"],
|
||||
alpha=0.6, label="Coherence")
|
||||
axes[1, 1].set_title("Memory Utilization vs Performance")
|
||||
axes[1, 1].set_xlabel("Memory Utilization")
|
||||
axes[1, 1].set_ylabel("Score")
|
||||
axes[1, 1].legend()
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(output_dir / "benchmark_results.png", dpi=300)
|
||||
console.print(f"[green]Report generated in {output_dir}[/green]")
|
||||
|
||||
|
||||
async def main():
|
||||
"""Main function to run the LOCOMO benchmark."""
|
||||
parser = argparse.ArgumentParser(description="Run LOCOMO benchmark for Mem0 agent")
|
||||
parser.add_argument("--scenarios", type=int, default=5, help="Number of scenarios to run")
|
||||
parser.add_argument("--output", type=str, default="results", help="Output directory for results")
|
||||
parser.add_argument("--config", type=str, help="Path to configuration file")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Initialize configuration
|
||||
config = Config.from_env()
|
||||
|
||||
# Initialize agent
|
||||
console.print("[yellow]Initializing Mem0 agent...[/yellow]")
|
||||
agent = Mem0Agent(config)
|
||||
|
||||
# Initialize benchmark
|
||||
benchmark = LOCOMOBenchmark(agent, config)
|
||||
|
||||
# Run benchmark
|
||||
console.print(f"[yellow]Starting benchmark with {args.scenarios} scenarios...[/yellow]")
|
||||
benchmark_results = await benchmark.run_benchmark(num_scenarios=args.scenarios)
|
||||
|
||||
# Save results
|
||||
output_dir = Path(args.output)
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
results_file = output_dir / f"locomo_results_{timestamp}.json"
|
||||
benchmark.save_results(benchmark_results, results_file)
|
||||
|
||||
# Generate report
|
||||
report_dir = output_dir / f"report_{timestamp}"
|
||||
benchmark.generate_report(benchmark_results, report_dir)
|
||||
|
||||
# Display overall results
|
||||
benchmark.display_overall_results(benchmark_results)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,398 @@
|
||||
"""Main entry point for the Mem0 agent with Kimi K3."""
|
||||
|
||||
import asyncio
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
import sys
|
||||
|
||||
from rich.console import Console
|
||||
from rich.prompt import Prompt, Confirm
|
||||
from rich.panel import Panel
|
||||
from rich.markdown import Markdown
|
||||
|
||||
from agent import Mem0Agent
|
||||
from config import Config
|
||||
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
class InteractiveSession:
|
||||
"""Interactive session manager for Mem0 agent."""
|
||||
|
||||
def __init__(self, agent: Mem0Agent):
|
||||
self.agent = agent
|
||||
self.current_session = None
|
||||
self.current_user = None
|
||||
self.current_agent_id = None
|
||||
|
||||
def start_session(self) -> None:
|
||||
"""Start a new interactive session."""
|
||||
console.print(Panel.fit(
|
||||
"[bold cyan]Mem0 Agent Interactive Session[/bold cyan]\n"
|
||||
"Type 'help' for commands, 'exit' to quit",
|
||||
title="Welcome"
|
||||
))
|
||||
|
||||
# Get session details
|
||||
self.current_user = Prompt.ask("Enter user ID", default="user_001")
|
||||
self.current_agent_id = Prompt.ask("Enter agent ID", default="agent_001")
|
||||
session_id = Prompt.ask("Enter session ID", default="session_001")
|
||||
|
||||
# Create context
|
||||
context = self.agent.create_context(
|
||||
agent_id=self.current_agent_id,
|
||||
user_id=self.current_user,
|
||||
session_id=session_id
|
||||
)
|
||||
self.current_session = session_id
|
||||
|
||||
console.print(f"[green]Session started:[/green] {session_id}")
|
||||
console.print(f"[green]User:[/green] {self.current_user}")
|
||||
console.print(f"[green]Agent:[/green] {self.current_agent_id}")
|
||||
|
||||
def show_help(self) -> None:
|
||||
"""Display help information."""
|
||||
help_text = """
|
||||
# Available Commands
|
||||
|
||||
- **help** - Show this help message
|
||||
- **exit/quit** - Exit the session
|
||||
- **clear** - Clear the screen
|
||||
- **metrics** - Show performance metrics
|
||||
- **memories** - Show stored memories
|
||||
- **save** - Save conversation state
|
||||
- **load** - Load conversation state
|
||||
- **reset** - Reset the agent state
|
||||
- **new** - Start a new session
|
||||
"""
|
||||
console.print(Markdown(help_text))
|
||||
|
||||
def show_memories(self) -> None:
|
||||
"""Display stored memories."""
|
||||
if not self.current_user:
|
||||
console.print("[yellow]No active session[/yellow]")
|
||||
return
|
||||
|
||||
memories = self.agent.get_all_memories(user_id=self.current_user)
|
||||
|
||||
if not memories:
|
||||
console.print("[yellow]No memories found[/yellow]")
|
||||
return
|
||||
|
||||
console.print(f"\n[cyan]Memories for {self.current_user}:[/cyan]")
|
||||
for i, memory in enumerate(memories, 1):
|
||||
console.print(f"{i}. {memory.get('memory', memory.get('text', 'N/A'))}")
|
||||
|
||||
def save_state(self) -> None:
|
||||
"""Save the current state."""
|
||||
filepath = Prompt.ask("Enter filepath to save", default="state.json")
|
||||
self.agent.save_state(filepath)
|
||||
console.print(f"[green]State saved to {filepath}[/green]")
|
||||
|
||||
def load_state(self) -> None:
|
||||
"""Load a saved state."""
|
||||
filepath = Prompt.ask("Enter filepath to load", default="state.json")
|
||||
if Path(filepath).exists():
|
||||
self.agent.load_state(filepath)
|
||||
console.print(f"[green]State loaded from {filepath}[/green]")
|
||||
else:
|
||||
console.print(f"[red]File not found: {filepath}[/red]")
|
||||
|
||||
async def run(self) -> None:
|
||||
"""Run the interactive session."""
|
||||
self.start_session()
|
||||
|
||||
while True:
|
||||
try:
|
||||
# Get user input
|
||||
user_input = Prompt.ask("\n[bold]You[/bold]")
|
||||
|
||||
# Check for commands
|
||||
if user_input.lower() in ["exit", "quit"]:
|
||||
if Confirm.ask("Are you sure you want to exit?"):
|
||||
break
|
||||
elif user_input.lower() == "help":
|
||||
self.show_help()
|
||||
continue
|
||||
elif user_input.lower() == "clear":
|
||||
console.clear()
|
||||
continue
|
||||
elif user_input.lower() == "metrics":
|
||||
self.agent.display_metrics(self.current_session)
|
||||
continue
|
||||
elif user_input.lower() == "memories":
|
||||
self.show_memories()
|
||||
continue
|
||||
elif user_input.lower() == "save":
|
||||
self.save_state()
|
||||
continue
|
||||
elif user_input.lower() == "load":
|
||||
self.load_state()
|
||||
continue
|
||||
elif user_input.lower() == "reset":
|
||||
if Confirm.ask("Reset agent state?"):
|
||||
self.agent.reset()
|
||||
console.print("[green]Agent state reset[/green]")
|
||||
continue
|
||||
elif user_input.lower() == "new":
|
||||
self.start_session()
|
||||
continue
|
||||
|
||||
# Process the input through the agent
|
||||
console.print("[dim]Processing...[/dim]")
|
||||
response, metrics = await self.agent.process_turn_async(
|
||||
self.current_session,
|
||||
user_input
|
||||
)
|
||||
|
||||
# Display response
|
||||
console.print(f"\n[bold cyan]Agent[/bold cyan]: {response}")
|
||||
|
||||
# Display metrics (optional)
|
||||
if metrics.get("generation_time"):
|
||||
console.print(
|
||||
f"[dim]Generated in {metrics['generation_time']:.2f}s | "
|
||||
f"Turn {metrics['turn_count']} | "
|
||||
f"Memories: {metrics['memory_count']}[/dim]"
|
||||
)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
console.print("\n[yellow]Interrupted[/yellow]")
|
||||
if Confirm.ask("Exit session?"):
|
||||
break
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
|
||||
console.print("\n[cyan]Session ended. Goodbye![/cyan]")
|
||||
|
||||
|
||||
async def run_batch_mode(agent: Mem0Agent, input_file: Path, output_file: Path) -> None:
|
||||
"""Run the agent in batch mode."""
|
||||
console.print(f"[yellow]Processing batch file: {input_file}[/yellow]")
|
||||
|
||||
# Read input file
|
||||
with open(input_file, "r") as f:
|
||||
batch_data = f.read()
|
||||
|
||||
# Parse batch data (assuming JSON format)
|
||||
import json
|
||||
try:
|
||||
sessions = json.loads(batch_data)
|
||||
except json.JSONDecodeError:
|
||||
console.print("[red]Invalid JSON in input file[/red]")
|
||||
return
|
||||
|
||||
results = []
|
||||
|
||||
# Process each session
|
||||
for session_data in sessions:
|
||||
session_id = session_data.get("session_id", "batch_session")
|
||||
user_id = session_data.get("user_id", "batch_user")
|
||||
agent_id = session_data.get("agent_id", "batch_agent")
|
||||
turns = session_data.get("turns", [])
|
||||
|
||||
# Create context
|
||||
context = agent.create_context(
|
||||
agent_id=agent_id,
|
||||
user_id=user_id,
|
||||
session_id=session_id
|
||||
)
|
||||
|
||||
session_results = {
|
||||
"session_id": session_id,
|
||||
"user_id": user_id,
|
||||
"agent_id": agent_id,
|
||||
"turns": []
|
||||
}
|
||||
|
||||
# Process turns
|
||||
for turn in turns:
|
||||
response, metrics = await agent.process_turn_async(session_id, turn)
|
||||
session_results["turns"].append({
|
||||
"input": turn,
|
||||
"response": response,
|
||||
"metrics": metrics
|
||||
})
|
||||
|
||||
results.append(session_results)
|
||||
|
||||
# Save results
|
||||
with open(output_file, "w") as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
console.print(f"[green]Results saved to {output_file}[/green]")
|
||||
|
||||
|
||||
def _load_add_messages(text: str):
|
||||
"""Resolve the --text argument for a memory add operation.
|
||||
|
||||
If it points to an existing JSON file, load it (expects a message list
|
||||
or a string); otherwise treat the argument itself as a user utterance.
|
||||
"""
|
||||
if os.path.exists(text):
|
||||
with open(text, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return text
|
||||
|
||||
|
||||
async def run_memory_op(agent: Mem0Agent, args) -> None:
|
||||
"""Run a single direct memory operation (add/search/get-all/history/delete).
|
||||
|
||||
This exposes mem0 v3's ADD-only ingestion and retrieval API independently
|
||||
of the chat loop.
|
||||
"""
|
||||
op = args.op
|
||||
if not op:
|
||||
console.print("[red]memory 模式需要 --op 参数(add/search/get-all/history/delete)[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
result = None
|
||||
|
||||
if op == "add":
|
||||
if not args.text:
|
||||
console.print("[red]add 操作需要 --text 参数(一段对话文本,或 JSON 消息文件路径)[/red]")
|
||||
sys.exit(1)
|
||||
messages = _load_add_messages(args.text)
|
||||
added = await asyncio.to_thread(agent.add_memory, messages, args.user_id, args.agent_id)
|
||||
console.print("[green]写入完成,ADD-only 提取追加的事实:[/green]")
|
||||
if added:
|
||||
for memory in added:
|
||||
console.print(f" [ADD] {memory['memory']} [dim](id={memory['id']})[/dim]")
|
||||
else:
|
||||
console.print(" [dim](没有提取到需要追加的新事实)[/dim]")
|
||||
result = {"op": "add", "user_id": args.user_id, "added_memories": added}
|
||||
|
||||
elif op == "search":
|
||||
if not args.query:
|
||||
console.print("[red]search 操作需要 --query 参数[/red]")
|
||||
sys.exit(1)
|
||||
hits = await asyncio.to_thread(agent.search_memory, args.query, args.user_id, args.agent_id)
|
||||
console.print(f"[green]检索到 {len(hits)} 条相关记忆:[/green]")
|
||||
for mem in hits:
|
||||
console.print(f" - {mem.get('memory', mem.get('text', 'N/A'))} [dim](id={mem.get('id','')})[/dim]")
|
||||
result = {"op": "search", "query": args.query, "user_id": args.user_id, "memories": hits}
|
||||
|
||||
elif op == "get-all":
|
||||
memories = await asyncio.to_thread(agent.get_all_memories, args.user_id, args.agent_id)
|
||||
console.print(f"[green]用户 {args.user_id} 共有 {len(memories)} 条记忆:[/green]")
|
||||
for i, mem in enumerate(memories, 1):
|
||||
console.print(f" {i}. {mem.get('memory', mem.get('text', 'N/A'))} [dim](id={mem.get('id','')})[/dim]")
|
||||
result = {"op": "get-all", "user_id": args.user_id, "memories": memories}
|
||||
|
||||
elif op == "history":
|
||||
if not args.memory_id:
|
||||
console.print("[red]history 操作需要 --memory-id 参数[/red]")
|
||||
sys.exit(1)
|
||||
history = await asyncio.to_thread(agent.memory_history, args.memory_id)
|
||||
console.print(f"[green]记忆 {args.memory_id} 的修改历史:[/green]")
|
||||
for entry in history:
|
||||
console.print(f" - {entry}")
|
||||
result = {"op": "history", "memory_id": args.memory_id, "history": history}
|
||||
|
||||
elif op == "delete":
|
||||
if not args.memory_id:
|
||||
console.print("[red]delete 操作需要 --memory-id 参数[/red]")
|
||||
sys.exit(1)
|
||||
await asyncio.to_thread(agent.delete_memory, args.memory_id)
|
||||
console.print(f"[green]已删除记忆 {args.memory_id}[/green]")
|
||||
result = {"op": "delete", "memory_id": args.memory_id}
|
||||
|
||||
if args.output and result is not None:
|
||||
with open(args.output, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, ensure_ascii=False, indent=2, default=str)
|
||||
console.print(f"[green]结果已写入 {args.output}[/green]")
|
||||
|
||||
|
||||
CLI_EPILOG = """\
|
||||
示例:
|
||||
python main.py # 默认进入交互式对话(记忆随对话自动写入/检索)
|
||||
python main.py --mode demo --user-id u1 # 运行“北京→上海”的 ADD-only + 混合检索演示
|
||||
python main.py --mode memory --op add --text "我住在北京,是一名后端工程师" --user-id u1
|
||||
python main.py --mode memory --op search --query "这个用户住在哪里?" --user-id u1
|
||||
python main.py --mode memory --op get-all --user-id u1 --output mem.json
|
||||
python main.py --mode batch --input conversations.json --output results.json
|
||||
python main.py --mode benchmark --model kimi-k3
|
||||
|
||||
说明:memory / demo / interactive / batch / benchmark 均需要可用的 LLM API(KIMI_API_KEY)
|
||||
及向量存储;Mem0 的记忆提取与检索依赖在线模型调用。
|
||||
"""
|
||||
|
||||
|
||||
async def main():
|
||||
"""Main entry point."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Mem0 记忆智能体(Kimi K3)— 演示 Mem0 v3 的 ADD-only 提取与混合检索",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=CLI_EPILOG,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--mode",
|
||||
choices=["interactive", "batch", "benchmark", "memory", "demo"],
|
||||
default="interactive",
|
||||
help="运行模式:interactive 交互对话(默认)/ batch 批量对话 / benchmark 跑 LOCOMO 基准 / "
|
||||
"memory 直接调用记忆操作 / demo 记忆流水线演示",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--op",
|
||||
choices=["add", "search", "get-all", "history", "delete"],
|
||||
help="memory 模式下的记忆操作:add 写入 / search 检索 / get-all 列出全部 / "
|
||||
"history 查看某条记忆的修改历史 / delete 删除",
|
||||
)
|
||||
parser.add_argument("--text", type=str,
|
||||
help="add 操作的对话输入:一段文本,或指向 JSON 消息列表文件的路径")
|
||||
parser.add_argument("--query", type=str, help="search 操作的查询语句")
|
||||
parser.add_argument("--memory-id", type=str, help="history / delete 操作针对的记忆 ID")
|
||||
parser.add_argument("--user-id", type=str, default="user_001",
|
||||
help="记忆归属的用户 ID(默认 user_001)")
|
||||
parser.add_argument("--agent-id", type=str, default="agent_001",
|
||||
help="智能体 ID(默认 agent_001)")
|
||||
parser.add_argument("--model", type=str,
|
||||
help="覆盖 MODEL_NAME,指定对话模型(如 kimi-k3)")
|
||||
parser.add_argument("--input", type=str, help="batch 模式的输入 JSON 文件")
|
||||
parser.add_argument("--output", type=str,
|
||||
help="将结果写入的 JSON 文件(memory / batch 模式)")
|
||||
parser.add_argument("--config", type=str, help="配置文件路径(预留)")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Initialize configuration
|
||||
config = Config.from_env()
|
||||
if args.model:
|
||||
config.kimi.model_name = args.model
|
||||
|
||||
# Initialize agent
|
||||
console.print("[yellow]Initializing Mem0 agent...[/yellow]")
|
||||
try:
|
||||
agent = Mem0Agent(config)
|
||||
console.print("[green]Agent initialized successfully[/green]")
|
||||
except Exception as e:
|
||||
console.print(f"[red]Failed to initialize agent: {e}[/red]")
|
||||
sys.exit(1)
|
||||
|
||||
# Run based on mode
|
||||
if args.mode == "interactive":
|
||||
session = InteractiveSession(agent)
|
||||
await session.run()
|
||||
elif args.mode == "batch":
|
||||
if not args.input or not args.output:
|
||||
console.print("[red]Batch mode requires --input and --output arguments[/red]")
|
||||
sys.exit(1)
|
||||
await run_batch_mode(agent, Path(args.input), Path(args.output))
|
||||
elif args.mode == "memory":
|
||||
await run_memory_op(agent, args)
|
||||
elif args.mode == "demo":
|
||||
from quickstart import memory_pipeline_example
|
||||
await memory_pipeline_example(agent=agent, user_id=args.user_id)
|
||||
elif args.mode == "benchmark":
|
||||
# Import and run benchmark
|
||||
from experiment import LOCOMOBenchmark
|
||||
benchmark = LOCOMOBenchmark(agent, config)
|
||||
results = await benchmark.run_benchmark(num_scenarios=3)
|
||||
benchmark.display_overall_results(results)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,305 @@
|
||||
"""Quick start example for Mem0 agent with Kimi K3."""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from dotenv import load_dotenv
|
||||
from rich.console import Console
|
||||
|
||||
from agent import Mem0Agent
|
||||
from config import Config
|
||||
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def basic_example():
|
||||
"""Basic example of using Mem0 agent."""
|
||||
console.print("[bold cyan]Basic Mem0 Agent Example[/bold cyan]\n")
|
||||
|
||||
# Initialize configuration
|
||||
config = Config.from_env()
|
||||
|
||||
# Initialize agent
|
||||
console.print("[yellow]Initializing agent...[/yellow]")
|
||||
agent = Mem0Agent(config)
|
||||
|
||||
# Create a session context
|
||||
session_id = "quickstart_session"
|
||||
user_id = "quickstart_user"
|
||||
agent_id = "quickstart_agent"
|
||||
|
||||
context = agent.create_context(
|
||||
agent_id=agent_id,
|
||||
user_id=user_id,
|
||||
session_id=session_id
|
||||
)
|
||||
|
||||
console.print(f"[green]Session created: {session_id}[/green]\n")
|
||||
|
||||
# Example conversation
|
||||
conversations = [
|
||||
"Hello! I'm interested in learning about machine learning.",
|
||||
"I prefer Python for programming and have experience with scikit-learn.",
|
||||
"What would you recommend as the next step in my ML journey?",
|
||||
"Can you remind me what programming language I mentioned earlier?",
|
||||
"What libraries have I mentioned using?"
|
||||
]
|
||||
|
||||
for i, user_input in enumerate(conversations, 1):
|
||||
console.print(f"[bold]Turn {i} - User:[/bold] {user_input}")
|
||||
|
||||
# Process the turn
|
||||
response, metrics = await agent.process_turn_async(session_id, user_input)
|
||||
|
||||
console.print(f"[cyan]Agent:[/cyan] {response}")
|
||||
console.print(f"[dim]Response time: {metrics['generation_time']:.2f}s[/dim]\n")
|
||||
|
||||
# Small delay for readability
|
||||
await asyncio.sleep(0.5)
|
||||
|
||||
# Display final metrics
|
||||
console.print("\n[bold]Session Metrics:[/bold]")
|
||||
agent.display_metrics(session_id)
|
||||
|
||||
# Show stored memories
|
||||
console.print("\n[bold]Stored Memories:[/bold]")
|
||||
memories = agent.get_all_memories(user_id)
|
||||
for memory in memories:
|
||||
console.print(f"- {memory.get('memory', memory.get('text', 'N/A'))}")
|
||||
|
||||
|
||||
async def memory_pipeline_example(agent=None, user_id: str = "pipeline_user"):
|
||||
"""Demonstrate Mem0 v3's ADD-only extraction and hybrid retrieval.
|
||||
|
||||
The user first says they live in Beijing and later says they moved to
|
||||
Shanghai. Mem0 preserves both facts; retrieval is responsible for ranking
|
||||
the relevant, current one. The example also shows cross-session recall.
|
||||
|
||||
Requires a working LLM API (KIMI_API_KEY) and vector store — Mem0's fact
|
||||
extraction and semantic retrieval are online model calls.
|
||||
"""
|
||||
console.print("\n[bold cyan]Memory Pipeline Example (仅追加提取 + 混合检索)[/bold cyan]\n")
|
||||
|
||||
if agent is None:
|
||||
agent = Mem0Agent(Config.from_env())
|
||||
|
||||
def show_added(label, added):
|
||||
console.print(f"[bold]{label}[/bold]")
|
||||
if added:
|
||||
for memory in added:
|
||||
console.print(f" [magenta][ADD][/magenta] {memory['memory']} "
|
||||
f"[dim](id={memory['id']})[/dim]")
|
||||
else:
|
||||
console.print(" [dim](没有提取到需要追加的新事实)[/dim]")
|
||||
console.print()
|
||||
|
||||
# --- Session 1: establish facts about the user ---------------------------
|
||||
console.print("[yellow]Session 1 —— 首次对话,建立用户画像[/yellow]")
|
||||
events = await asyncio.to_thread(
|
||||
agent.add_memory,
|
||||
"我住在北京,在一家 AI 创业公司做后端工程师。",
|
||||
user_id,
|
||||
)
|
||||
show_added("写入「我住在北京 / 后端工程师」后追加的事实:", events)
|
||||
|
||||
events = await asyncio.to_thread(
|
||||
agent.add_memory,
|
||||
"我平时喜欢周末去爬山,也在学弹吉他。",
|
||||
user_id,
|
||||
)
|
||||
show_added("写入「爱好」后追加的事实:", events)
|
||||
|
||||
# --- Recall the stored memory (used later, across the session) -----------
|
||||
console.print("[yellow]检索 —— 从记忆中回忆用户信息(跨轮次复用)[/yellow]")
|
||||
hits = await asyncio.to_thread(
|
||||
agent.search_memory, "这个用户住在哪座城市?做什么工作?", user_id
|
||||
)
|
||||
console.print(f"[bold]检索到 {len(hits)} 条相关记忆:[/bold]")
|
||||
for mem in hits:
|
||||
console.print(f" - {mem.get('memory', mem.get('text', 'N/A'))}")
|
||||
console.print()
|
||||
|
||||
# --- Session 2 (later): the new fact is appended, not overwritten --------
|
||||
console.print("[yellow]Session 2(一段时间后)—— 用户搬家,出现冲突信息[/yellow]")
|
||||
events = await asyncio.to_thread(
|
||||
agent.add_memory,
|
||||
"更新一下,我上个月从北京搬到上海了。",
|
||||
user_id,
|
||||
)
|
||||
show_added("写入「搬到上海」后追加的事实:", events)
|
||||
|
||||
# --- Verify append-only history and current-state retrieval ---------------
|
||||
console.print("[yellow]核对 —— 旧事实保留,检索负责找出当前状态[/yellow]")
|
||||
memories = await asyncio.to_thread(agent.get_all_memories, user_id)
|
||||
console.print(f"[bold]用户 {user_id} 当前全部记忆({len(memories)} 条):[/bold]")
|
||||
for i, mem in enumerate(memories, 1):
|
||||
console.print(f" {i}. {mem.get('memory', mem.get('text', 'N/A'))}")
|
||||
console.print()
|
||||
current = await asyncio.to_thread(agent.search_memory, "用户现在住在哪里?", user_id)
|
||||
console.print("[bold]查询当前居住地的排序结果:[/bold]")
|
||||
for mem in current:
|
||||
console.print(f" - {mem.get('memory', mem.get('text', 'N/A'))}")
|
||||
console.print("[dim]提示:v3 可以保留北京与上海两条历史事实,并让时间感知检索优先返回当前事实。[/dim]")
|
||||
|
||||
|
||||
async def multi_session_example():
|
||||
"""Example showing memory persistence across sessions."""
|
||||
console.print("\n[bold cyan]Multi-Session Memory Example[/bold cyan]\n")
|
||||
|
||||
# Initialize agent
|
||||
config = Config.from_env()
|
||||
agent = Mem0Agent(config)
|
||||
|
||||
user_id = "persistent_user"
|
||||
|
||||
# First session
|
||||
console.print("[yellow]Starting Session 1...[/yellow]")
|
||||
session1_id = "session_001"
|
||||
context1 = agent.create_context(
|
||||
agent_id="agent_001",
|
||||
user_id=user_id,
|
||||
session_id=session1_id
|
||||
)
|
||||
|
||||
# First session conversation
|
||||
response1, _ = await agent.process_turn_async(
|
||||
session1_id,
|
||||
"Hi! I'm working on a project about renewable energy, specifically solar panels."
|
||||
)
|
||||
console.print(f"[cyan]Session 1 Response:[/cyan] {response1}\n")
|
||||
|
||||
response2, _ = await agent.process_turn_async(
|
||||
session1_id,
|
||||
"I need to analyze efficiency data from different manufacturers."
|
||||
)
|
||||
console.print(f"[cyan]Session 1 Response:[/cyan] {response2}\n")
|
||||
|
||||
# Second session (different session, same user)
|
||||
console.print("[yellow]Starting Session 2 (after some time)...[/yellow]")
|
||||
session2_id = "session_002"
|
||||
context2 = agent.create_context(
|
||||
agent_id="agent_001",
|
||||
user_id=user_id,
|
||||
session_id=session2_id
|
||||
)
|
||||
|
||||
# Second session should remember context from first session
|
||||
response3, _ = await agent.process_turn_async(
|
||||
session2_id,
|
||||
"What was I working on last time we talked?"
|
||||
)
|
||||
console.print(f"[cyan]Session 2 Response:[/cyan] {response3}\n")
|
||||
|
||||
response4, _ = await agent.process_turn_async(
|
||||
session2_id,
|
||||
"Can you help me continue with that project?"
|
||||
)
|
||||
console.print(f"[cyan]Session 2 Response:[/cyan] {response4}\n")
|
||||
|
||||
# Show all memories
|
||||
console.print("[bold]All Memories for User:[/bold]")
|
||||
memories = agent.get_all_memories(user_id)
|
||||
for memory in memories:
|
||||
console.print(f"- {memory.get('memory', memory.get('text', 'N/A'))}")
|
||||
|
||||
|
||||
async def multi_agent_example():
|
||||
"""Example with multiple agents collaborating."""
|
||||
console.print("\n[bold cyan]Multi-Agent Collaboration Example[/bold cyan]\n")
|
||||
|
||||
# Initialize agent
|
||||
config = Config.from_env()
|
||||
agent = Mem0Agent(config)
|
||||
|
||||
user_id = "collaboration_user"
|
||||
session_id = "collab_session"
|
||||
|
||||
# Create contexts for multiple agents
|
||||
agents = ["researcher", "analyst", "advisor"]
|
||||
contexts = {}
|
||||
|
||||
for agent_id in agents:
|
||||
contexts[agent_id] = agent.create_context(
|
||||
agent_id=agent_id,
|
||||
user_id=user_id,
|
||||
session_id=f"{session_id}_{agent_id}"
|
||||
)
|
||||
console.print(f"[green]Created context for {agent_id}[/green]")
|
||||
|
||||
# Collaborative conversation
|
||||
console.print("\n[yellow]Starting collaborative discussion...[/yellow]\n")
|
||||
|
||||
# Researcher starts
|
||||
response1, _ = await agent.process_turn_async(
|
||||
f"{session_id}_researcher",
|
||||
"I've found some interesting data on climate change impacts on agriculture."
|
||||
)
|
||||
console.print(f"[cyan]Researcher:[/cyan] {response1}\n")
|
||||
|
||||
# Analyst responds
|
||||
response2, _ = await agent.process_turn_async(
|
||||
f"{session_id}_analyst",
|
||||
"Based on what the researcher mentioned, what are the key metrics we should analyze?"
|
||||
)
|
||||
console.print(f"[cyan]Analyst:[/cyan] {response2}\n")
|
||||
|
||||
# Advisor provides guidance
|
||||
response3, _ = await agent.process_turn_async(
|
||||
f"{session_id}_advisor",
|
||||
"Considering both the research and analysis perspectives, what recommendations can we make?"
|
||||
)
|
||||
console.print(f"[cyan]Advisor:[/cyan] {response3}\n")
|
||||
|
||||
# Show metrics for all agents
|
||||
console.print("[bold]Performance Metrics:[/bold]")
|
||||
for agent_id in agents:
|
||||
console.print(f"\n[yellow]{agent_id.capitalize()}:[/yellow]")
|
||||
summary = agent.get_performance_summary(f"{session_id}_{agent_id}")
|
||||
for key, value in summary.items():
|
||||
if isinstance(value, float):
|
||||
console.print(f" {key}: {value:.3f}")
|
||||
else:
|
||||
console.print(f" {key}: {value}")
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run all examples."""
|
||||
console.print(Panel.fit(
|
||||
"[bold]Mem0 Agent Quickstart Examples[/bold]\n"
|
||||
"Demonstrating various capabilities of the Mem0 agent with Kimi K3",
|
||||
title="Welcome"
|
||||
))
|
||||
|
||||
# Check for API key
|
||||
if not os.getenv("KIMI_API_KEY"):
|
||||
console.print("[red]Error: KIMI_API_KEY not found in environment[/red]")
|
||||
console.print("Please set your Kimi API key in the .env file")
|
||||
return
|
||||
|
||||
try:
|
||||
# Run examples
|
||||
await memory_pipeline_example()
|
||||
await asyncio.sleep(1)
|
||||
|
||||
await basic_example()
|
||||
await asyncio.sleep(1)
|
||||
|
||||
await multi_session_example()
|
||||
await asyncio.sleep(1)
|
||||
|
||||
await multi_agent_example()
|
||||
|
||||
console.print("\n[green]All examples completed successfully![/green]")
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error running examples: {e}[/red]")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
from rich.panel import Panel
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,28 @@
|
||||
# Core dependencies
|
||||
mem0ai[nlp]>=2.0,<3
|
||||
openai>=1.54.0
|
||||
python-dotenv>=1.0.0
|
||||
|
||||
# Kimi K3 model integration
|
||||
httpx>=0.27.0
|
||||
pydantic>=2.9.0
|
||||
tenacity>=9.0.0
|
||||
|
||||
# LOCOMO benchmark requirements
|
||||
numpy>=1.26.0
|
||||
pandas>=2.2.0
|
||||
scikit-learn>=1.5.0
|
||||
tqdm>=4.66.0
|
||||
|
||||
# Memory storage
|
||||
chromadb>=0.5.0
|
||||
qdrant-client>=1.12.0
|
||||
|
||||
# Utilities
|
||||
rich>=13.7.0
|
||||
typer>=0.13.0
|
||||
click>=8.1.0
|
||||
|
||||
# Testing and evaluation
|
||||
pytest>=8.3.0
|
||||
pytest-asyncio>=0.24.0
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Contract tests for the Mem0 v3 companion helpers."""
|
||||
|
||||
from agent import MemoryClient, Mem0Agent, _extract_added_memories, _memory_filters
|
||||
|
||||
|
||||
class FakeMemory:
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
|
||||
def search(self, **kwargs):
|
||||
self.calls.append(("search", kwargs))
|
||||
return {"results": [{"id": "m1", "memory": "current fact"}]}
|
||||
|
||||
def get_all(self, **kwargs):
|
||||
self.calls.append(("get_all", kwargs))
|
||||
return {"results": [{"id": "m1", "memory": "stored fact"}]}
|
||||
|
||||
|
||||
class FakeMemoryClient(MemoryClient):
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
|
||||
def get_all(self, **kwargs):
|
||||
self.calls.append(("get_all", kwargs))
|
||||
return {"results": [{"id": "m1", "memory": "cloud fact"}]}
|
||||
|
||||
|
||||
def make_agent():
|
||||
agent = object.__new__(Mem0Agent)
|
||||
agent.memory = FakeMemory()
|
||||
return agent
|
||||
|
||||
|
||||
def test_memory_filters_omit_empty_agent_id():
|
||||
assert _memory_filters("u1") == {"user_id": "u1"}
|
||||
assert _memory_filters("u1", "a1") == {"user_id": "u1", "agent_id": "a1"}
|
||||
|
||||
|
||||
def test_search_uses_v3_filters_and_top_k():
|
||||
agent = make_agent()
|
||||
|
||||
assert agent.search_memory("where", "u1", "a1", top_k=7)[0]["id"] == "m1"
|
||||
assert agent.memory.calls == [(
|
||||
"search",
|
||||
{
|
||||
"query": "where",
|
||||
"filters": {"user_id": "u1", "agent_id": "a1"},
|
||||
"top_k": 7,
|
||||
},
|
||||
)]
|
||||
|
||||
|
||||
def test_get_all_uses_v3_filters_and_top_k():
|
||||
agent = make_agent()
|
||||
|
||||
assert agent.get_all_memories("u1", top_k=42)[0]["id"] == "m1"
|
||||
assert agent.memory.calls == [(
|
||||
"get_all",
|
||||
{"filters": {"user_id": "u1"}, "top_k": 42},
|
||||
)]
|
||||
|
||||
|
||||
def test_cloud_get_all_maps_top_k_to_page_size():
|
||||
agent = object.__new__(Mem0Agent)
|
||||
agent.memory = FakeMemoryClient()
|
||||
|
||||
assert agent.get_all_memories("u1", top_k=42)[0]["id"] == "m1"
|
||||
assert agent.memory.calls == [(
|
||||
"get_all",
|
||||
{"filters": {"user_id": "u1"}, "page_size": 42},
|
||||
)]
|
||||
|
||||
|
||||
def test_add_result_is_presented_as_added_memories_not_v2_decisions():
|
||||
result = {"results": [{"id": "m1", "memory": "new fact", "event": "ADD"}]}
|
||||
|
||||
assert _extract_added_memories(result) == [
|
||||
{"id": "m1", "memory": "new fact"},
|
||||
]
|
||||
@@ -0,0 +1,180 @@
|
||||
"""Simple test to verify the Mem0 agent setup."""
|
||||
|
||||
import os
|
||||
import asyncio
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
|
||||
def test_imports():
|
||||
"""Test that all modules can be imported."""
|
||||
print("Testing imports...")
|
||||
try:
|
||||
from agent import Mem0Agent, AgentContext, KimiK3Client
|
||||
print("✓ Agent module imported successfully")
|
||||
|
||||
from config import Config, KimiConfig, Mem0Config, LOCOMOConfig
|
||||
print("✓ Config module imported successfully")
|
||||
|
||||
from experiment import LOCOMOBenchmark
|
||||
print("✓ Experiment module imported successfully")
|
||||
|
||||
import mem0
|
||||
print("✓ Mem0 library available")
|
||||
|
||||
import openai
|
||||
print("✓ OpenAI library available")
|
||||
|
||||
return True
|
||||
except ImportError as e:
|
||||
print(f"✗ Import error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def test_configuration():
|
||||
"""Test configuration loading."""
|
||||
print("\nTesting configuration...")
|
||||
try:
|
||||
from config import Config
|
||||
|
||||
config = Config.from_env()
|
||||
|
||||
# Check if API key is set
|
||||
if not config.kimi.api_key:
|
||||
print("⚠ Warning: KIMI_API_KEY not set in environment")
|
||||
print(" Please create a .env file from env.example and add your API key")
|
||||
return False
|
||||
else:
|
||||
print(f"✓ Kimi API key configured (length: {len(config.kimi.api_key)})")
|
||||
|
||||
print(f"✓ Model: {config.kimi.model_name}")
|
||||
print(f"✓ Max tokens: {config.kimi.max_tokens}")
|
||||
print(f"✓ Memory backend: {config.mem0.backend}")
|
||||
print(f"✓ LOCOMO data path: {config.locomo.data_path}")
|
||||
|
||||
# Validate configuration
|
||||
config.validate()
|
||||
print("✓ Configuration validation passed")
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"✗ Configuration error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
async def test_agent_initialization():
|
||||
"""Test agent initialization."""
|
||||
print("\nTesting agent initialization...")
|
||||
try:
|
||||
from agent import Mem0Agent
|
||||
from config import Config
|
||||
|
||||
config = Config.from_env()
|
||||
|
||||
# Skip if no API key
|
||||
if not config.kimi.api_key:
|
||||
print("⚠ Skipping agent test (no API key)")
|
||||
return False
|
||||
|
||||
agent = Mem0Agent(config)
|
||||
print("✓ Agent initialized successfully")
|
||||
|
||||
# Create a test context
|
||||
context = agent.create_context(
|
||||
agent_id="test_agent",
|
||||
user_id="test_user",
|
||||
session_id="test_session"
|
||||
)
|
||||
print(f"✓ Context created: {context.session_id}")
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"✗ Agent initialization error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
async def test_memory_system():
|
||||
"""Test memory system initialization."""
|
||||
print("\nTesting memory system...")
|
||||
try:
|
||||
from mem0 import Memory
|
||||
from config import Config
|
||||
|
||||
config = Config.from_env()
|
||||
|
||||
if config.mem0.backend == "local":
|
||||
# Test local memory initialization
|
||||
mem0_config = {
|
||||
"vector_store": {
|
||||
"provider": "chroma",
|
||||
"config": {
|
||||
"collection_name": "test_collection",
|
||||
"path": "./data/test_chroma_db"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
memory = Memory.from_config(mem0_config)
|
||||
print("✓ Local memory system initialized")
|
||||
|
||||
# Clean up test database
|
||||
import shutil
|
||||
if os.path.exists("./data/test_chroma_db"):
|
||||
shutil.rmtree("./data/test_chroma_db")
|
||||
print("✓ Test database cleaned up")
|
||||
else:
|
||||
print("✓ Cloud memory backend configured")
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"✗ Memory system error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""Run all tests."""
|
||||
print("=" * 50)
|
||||
print("Mem0 Agent Setup Test")
|
||||
print("=" * 50)
|
||||
|
||||
all_passed = True
|
||||
|
||||
# Run tests
|
||||
if not test_imports():
|
||||
all_passed = False
|
||||
|
||||
if not test_configuration():
|
||||
all_passed = False
|
||||
|
||||
# Run async tests
|
||||
loop = asyncio.get_event_loop()
|
||||
|
||||
if not loop.run_until_complete(test_agent_initialization()):
|
||||
all_passed = False
|
||||
|
||||
if not loop.run_until_complete(test_memory_system()):
|
||||
all_passed = False
|
||||
|
||||
# Summary
|
||||
print("\n" + "=" * 50)
|
||||
if all_passed:
|
||||
print("✅ All tests passed! The agent is ready to use.")
|
||||
print("\nNext steps:")
|
||||
print("1. Run quickstart examples: python quickstart.py")
|
||||
print("2. Try interactive mode: python main.py")
|
||||
print("3. Run LOCOMO benchmark: python experiment.py")
|
||||
else:
|
||||
print("⚠️ Some tests failed. Please check the configuration.")
|
||||
print("\nTroubleshooting:")
|
||||
print("1. Create .env file from env.example")
|
||||
print("2. Add your KIMI_API_KEY to .env")
|
||||
print("3. Install all requirements: pip install -r requirements.txt")
|
||||
print("=" * 50)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user