ai-agent-book 精选快照(<2MB 代码与文档,来自 github.com/bojieli/ai-agent-book)
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
Build latest book artifacts / build (push) Canceled after 0s
dependency resolution / resolve (3.11) (push) Canceled after 0s
dependency resolution / resolve (3.13) (push) Canceled after 0s
deploy-pages / build (push) Canceled after 0s
deploy-pages / deploy (push) Canceled after 0s
i18n consistency check / check (push) Canceled after 0s
provider adoption tests / test (chapter2/context-compression) (push) Canceled after 0s
provider adoption tests / test (chapter2/prompt-injection) (push) Canceled after 0s
provider adoption tests / test (chapter2/system-hint) (push) Canceled after 0s
provider adoption tests / test (chapter3/log-sanitization) (push) Canceled after 0s
web-search-agent tests / test (push) Canceled after 0s
web-search-agent tests / agentbook (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,87 @@
|
||||
---
|
||||
title: "All Parameters"
|
||||
description: "Complete API reference for Browser Actor classes, methods, and parameters including BrowserSession, Page, Element, and Mouse"
|
||||
icon: "list"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
## Browser (BrowserSession)
|
||||
|
||||
Main browser session manager.
|
||||
|
||||
### Key Methods
|
||||
|
||||
```python
|
||||
from browser_use import Browser
|
||||
|
||||
browser = Browser()
|
||||
await browser.start()
|
||||
|
||||
# Page management
|
||||
page = await browser.new_page("https://example.com")
|
||||
pages = await browser.get_pages()
|
||||
current = await browser.get_current_page()
|
||||
await browser.close_page(page)
|
||||
|
||||
# To stop the browser session
|
||||
await browser.stop()
|
||||
```
|
||||
|
||||
### Constructor Parameters
|
||||
|
||||
See [Browser Parameters](../browser/all-parameters) for complete configuration options.
|
||||
|
||||
## Page
|
||||
|
||||
Browser tab/iframe for page-level operations.
|
||||
|
||||
### Navigation
|
||||
- `goto(url: str)` - Navigate to URL
|
||||
- `go_back()`, `go_forward()`, `reload()` - History navigation
|
||||
|
||||
### Element Finding
|
||||
- `get_elements_by_css_selector(selector: str) -> list[Element]` - CSS selector
|
||||
- `get_element(backend_node_id: int) -> Element` - By CDP node ID
|
||||
- `get_element_by_prompt(prompt: str, llm) -> Element | None` - AI-powered
|
||||
- `must_get_element_by_prompt(prompt: str, llm) -> Element` - AI (raises if not found)
|
||||
|
||||
### JavaScript & Controls
|
||||
- `evaluate(page_function: str, *args) -> str` - Execute JS (arrow function format)
|
||||
- `press(key: str)` - Send keyboard input ("Enter", "Control+A")
|
||||
- `set_viewport_size(width: int, height: int)` - Set viewport
|
||||
- `screenshot(format='jpeg', quality=None) -> str` - Take screenshot
|
||||
|
||||
### Information
|
||||
- `get_url() -> str`, `get_title() -> str` - Page info
|
||||
- `mouse -> Mouse` - Get mouse interface
|
||||
|
||||
### AI Features
|
||||
- `extract_content(prompt: str, structured_output: type[T], llm) -> T` - Extract data
|
||||
|
||||
## Element
|
||||
|
||||
Individual DOM element interactions.
|
||||
|
||||
### Interactions
|
||||
- `click(button='left', click_count=1, modifiers=None)` - Click element
|
||||
- `fill(text: str, clear_existing=True)` - Fill input
|
||||
- `hover()`, `focus()` - Mouse/focus actions
|
||||
- `check()` - Toggle checkbox/radio
|
||||
- `select_option(values: str | list[str])` - Select dropdown options
|
||||
- `drag_to(target: Element | Position)` - Drag and drop
|
||||
|
||||
### Properties
|
||||
- `get_attribute(name: str) -> str | None` - Get attribute
|
||||
- `get_bounding_box() -> BoundingBox | None` - Position/size
|
||||
- `get_basic_info() -> ElementInfo` - Complete element info
|
||||
- `screenshot(format='jpeg') -> str` - Element screenshot
|
||||
|
||||
## Mouse
|
||||
|
||||
Coordinate-based mouse operations.
|
||||
|
||||
### Operations
|
||||
- `click(x: int, y: int, button='left', click_count=1)` - Click at coordinates
|
||||
- `move(x: int, y: int, steps=1)` - Move mouse
|
||||
- `down(button='left')`, `up(button='left')` - Press/release buttons
|
||||
- `scroll(x=0, y=0, delta_x=None, delta_y=None)` - Scroll at coordinates
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
title: "Basics"
|
||||
description: "Low-level Playwright-like browser automation with direct and full CDP control and precise element interactions"
|
||||
icon: "code"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
## Core Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Browser] --> B[Page]
|
||||
B --> C[Element]
|
||||
B --> D[Mouse]
|
||||
B --> E[AI Features]
|
||||
C --> F[DOM Interactions]
|
||||
D --> G[Coordinate Operations]
|
||||
E --> H[LLM Integration]
|
||||
```
|
||||
|
||||
### Core Classes
|
||||
|
||||
- **Browser** (alias: **BrowserSession**): Main session manager
|
||||
- **Page**: Represents a browser tab/iframe
|
||||
- **Element**: Individual DOM element operations
|
||||
- **Mouse**: Coordinate-based mouse operations
|
||||
|
||||
## Basic Usage
|
||||
|
||||
```python
|
||||
from browser_use import Browser, Agent
|
||||
from browser_use.llm.openai import ChatOpenAI
|
||||
|
||||
async def main():
|
||||
llm = ChatOpenAI(api_key="your-api-key")
|
||||
browser = Browser()
|
||||
await browser.start()
|
||||
|
||||
# 1. Actor: Precise navigation and element interactions
|
||||
page = await browser.new_page("https://github.com/login")
|
||||
email_input = await page.must_get_element_by_prompt("username field", llm=llm)
|
||||
await email_input.fill("your-username")
|
||||
|
||||
# 2. Agent: AI-driven complex tasks
|
||||
agent = Agent(browser=browser, llm=llm)
|
||||
await agent.run("Complete login and navigate to my repositories")
|
||||
|
||||
await browser.stop()
|
||||
```
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **Not Playwright**: Actor is built on CDP, not Playwright. The API resembles Playwright as much as possible for easy migration, but is sorta subset.
|
||||
- **Immediate Returns**: `get_elements_by_css_selector()` doesn't wait for visibility
|
||||
- **Manual Timing**: You handle navigation timing and waiting
|
||||
- **JavaScript Format**: `evaluate()` requires arrow function format: `() => {}`
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
title: "Examples"
|
||||
description: "Comprehensive examples for Browser Actor automation tasks including forms, JavaScript, mouse operations, and AI features"
|
||||
icon: "code-simple"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
## Page Management
|
||||
|
||||
```python
|
||||
from browser_use import Browser
|
||||
|
||||
browser = Browser()
|
||||
await browser.start()
|
||||
|
||||
# Create pages
|
||||
page = await browser.new_page() # Blank tab
|
||||
page = await browser.new_page("https://example.com") # With URL
|
||||
|
||||
# Get all pages
|
||||
pages = await browser.get_pages()
|
||||
current = await browser.get_current_page()
|
||||
|
||||
# Close page
|
||||
await browser.close_page(page)
|
||||
await browser.stop()
|
||||
```
|
||||
|
||||
## Element Finding & Interactions
|
||||
|
||||
```python
|
||||
page = await browser.new_page('https://github.com')
|
||||
|
||||
# CSS selectors (immediate return)
|
||||
elements = await page.get_elements_by_css_selector("input[type='text']")
|
||||
buttons = await page.get_elements_by_css_selector("button.submit")
|
||||
|
||||
# Element actions
|
||||
await elements[0].click()
|
||||
await elements[0].fill("Hello World")
|
||||
await elements[0].hover()
|
||||
|
||||
# Page actions
|
||||
await page.press("Enter")
|
||||
screenshot = await page.screenshot()
|
||||
```
|
||||
|
||||
## LLM-Powered Features
|
||||
|
||||
```python
|
||||
from browser_use.llm.openai import ChatOpenAI
|
||||
from pydantic import BaseModel
|
||||
|
||||
llm = ChatOpenAI(api_key="your-api-key")
|
||||
|
||||
# Find elements using natural language
|
||||
button = await page.get_element_by_prompt("login button", llm=llm)
|
||||
await button.click()
|
||||
|
||||
# Extract structured data
|
||||
class ProductInfo(BaseModel):
|
||||
name: str
|
||||
price: float
|
||||
|
||||
product = await page.extract_content(
|
||||
"Extract product name and price",
|
||||
ProductInfo,
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
## JavaScript Execution
|
||||
|
||||
```python
|
||||
# Simple JavaScript evaluation
|
||||
title = await page.evaluate('() => document.title')
|
||||
|
||||
# JavaScript with arguments
|
||||
result = await page.evaluate('(x, y) => x + y', 10, 20)
|
||||
|
||||
# Complex operations
|
||||
stats = await page.evaluate('''() => ({
|
||||
url: location.href,
|
||||
links: document.querySelectorAll('a').length
|
||||
})''')
|
||||
```
|
||||
|
||||
## Mouse Operations
|
||||
|
||||
```python
|
||||
mouse = await page.mouse
|
||||
|
||||
# Click at coordinates
|
||||
await mouse.click(x=100, y=200)
|
||||
|
||||
# Drag and drop
|
||||
await mouse.down()
|
||||
await mouse.move(x=500, y=600)
|
||||
await mouse.up()
|
||||
|
||||
# Scroll
|
||||
await mouse.scroll(x=0, y=100, delta_y=-500)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Use `asyncio.sleep()` after actions that trigger navigation
|
||||
- Check URL/title changes to verify state transitions
|
||||
- Always check if elements exist before interaction
|
||||
- Implement retry logic for flaky elements
|
||||
- Call `browser.stop()` to clean up resources
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
title: "All Parameters"
|
||||
description: "Complete reference for all agent configuration options"
|
||||
icon: "sliders"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
## Available Parameters
|
||||
|
||||
### Core Settings
|
||||
- `tools`: Registry of [our tools](https://github.com/browser-use/browser-use/blob/main/browser_use/tools/service.py) the agent can call. [Example for custom tools](https://github.com/browser-use/browser-use/tree/main/examples/custom-functions)
|
||||
- `browser`: Browser object where you can specify the browser settings.
|
||||
- `output_model_schema`: Pydantic model class for structured output validation. [Example](https://github.com/browser-use/browser-use/blob/main/examples/features/custom_output.py)
|
||||
|
||||
### Vision & Processing
|
||||
- `use_vision` (default: `True`): Enable/disable vision capabilities for processing screenshots
|
||||
- `vision_detail_level` (default: `'auto'`): Screenshot detail level - `'low'`, `'high'`, or `'auto'`
|
||||
- `page_extraction_llm`: Separate LLM model for page content extraction. You can choose a small & fast model because it only needs to extract text from the page (default: same as `llm`)
|
||||
|
||||
### Actions & Behavior
|
||||
- `initial_actions`: List of actions to run before the main task without LLM. [Example](https://github.com/browser-use/browser-use/blob/main/examples/features/initial_actions.py)
|
||||
- `max_actions_per_step` (default: `10`): Maximum actions per step, e.g. for form filling the agent can output 10 fields at once. We execute the actions until the page changes.
|
||||
- `max_failures` (default: `3`): Maximum retries for steps with errors
|
||||
- `final_response_after_failure` (default: `True`): If True, attempt to force one final model call with intermediate output after max_failures is reached
|
||||
- `use_thinking` (default: `True`): Controls whether the agent uses its internal "thinking" field for explicit reasoning steps.
|
||||
- `flash_mode` (default: `False`): Fast mode that skips evaluation, next goal and thinking and only uses memory. If `flash_mode` is enabled, it overrides `use_thinking` and disables the thinking process entirely. [Example](https://github.com/browser-use/browser-use/blob/main/examples/getting_started/05_fast_agent.py)
|
||||
|
||||
### System Messages
|
||||
- `override_system_message`: Completely replace the default system prompt.
|
||||
- `extend_system_message`: Add additional instructions to the default system prompt. [Example](https://github.com/browser-use/browser-use/blob/main/examples/features/custom_system_prompt.py)
|
||||
|
||||
### File & Data Management
|
||||
- `save_conversation_path`: Path to save complete conversation history
|
||||
- `save_conversation_path_encoding` (default: `'utf-8'`): Encoding for saved conversations
|
||||
- `available_file_paths`: List of file paths the agent can access
|
||||
- `sensitive_data`: Dictionary of sensitive data to handle carefully. [Example](https://github.com/browser-use/browser-use/blob/main/examples/features/sensitive_data.py)
|
||||
|
||||
### Visual Output
|
||||
- `generate_gif` (default: `False`): Generate GIF of agent actions. Set to `True` or string path
|
||||
- `include_attributes`: List of HTML attributes to include in page analysis
|
||||
|
||||
### Performance & Limits
|
||||
- `max_history_items`: Maximum number of last steps to keep in the LLM memory. If `None`, we keep all steps.
|
||||
- `llm_timeout` (default: `90`): Timeout in seconds for LLM calls
|
||||
- `step_timeout` (default: `120`): Timeout in seconds for each step
|
||||
- `directly_open_url` (default: `True`): If we detect a url in the task, we directly open it.
|
||||
|
||||
### Advanced Options
|
||||
- `calculate_cost` (default: `False`): Calculate and track API costs
|
||||
- `display_files_in_done_text` (default: `True`): Show file information in completion messages
|
||||
|
||||
### Backwards Compatibility
|
||||
- `controller`: Alias for `tools` for backwards compatibility.
|
||||
- `browser_session`: Alias for `browser` for backwards compatibility.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
title: "Basics"
|
||||
description: ""
|
||||
icon: "play"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatOpenAI
|
||||
|
||||
agent = Agent(
|
||||
task="Search for latest news about AI",
|
||||
llm=ChatOpenAI(model="gpt-4.1-mini"),
|
||||
)
|
||||
|
||||
async def main():
|
||||
history = await agent.run(max_steps=100)
|
||||
```
|
||||
|
||||
- `task`: The task you want to automate.
|
||||
- `llm`: Your favorite LLM. See <a href="/customize/supported-models">Supported Models</a>.
|
||||
|
||||
|
||||
The agent is executed using the async `run()` method:
|
||||
|
||||
- `max_steps` (default: `100`): Maximum number of steps an agent can take.
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
title: "Output Format"
|
||||
description: ""
|
||||
icon: "arrow-right-to-bracket"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
## Agent History
|
||||
|
||||
The `run()` method returns an `AgentHistoryList` object with the complete execution history:
|
||||
|
||||
```python
|
||||
history = await agent.run()
|
||||
|
||||
# Access useful information
|
||||
history.urls() # List of visited URLs
|
||||
history.screenshot_paths() # List of screenshot paths
|
||||
history.screenshots() # List of screenshots as base64 strings
|
||||
history.action_names() # Names of executed actions
|
||||
history.extracted_content() # List of extracted content from all actions
|
||||
history.errors() # List of errors (with None for steps without errors)
|
||||
history.model_actions() # All actions with their parameters
|
||||
history.model_outputs() # All model outputs from history
|
||||
history.last_action() # Last action in history
|
||||
|
||||
# Analysis methods
|
||||
history.final_result() # Get the final extracted content (last step)
|
||||
history.is_done() # Check if agent completed successfully
|
||||
history.is_successful() # Check if agent completed successfully (returns None if not done)
|
||||
history.has_errors() # Check if any errors occurred
|
||||
history.model_thoughts() # Get the agent's reasoning process (AgentBrain objects)
|
||||
history.action_results() # Get all ActionResult objects from history
|
||||
history.action_history() # Get truncated action history with essential fields
|
||||
history.number_of_steps() # Get the number of steps in the history
|
||||
history.total_duration_seconds() # Get total duration of all steps in seconds
|
||||
|
||||
# Structured output (when using output_model_schema)
|
||||
history.structured_output # Property that returns parsed structured output
|
||||
```
|
||||
|
||||
See all helper methods in the [AgentHistoryList source code](https://github.com/browser-use/browser-use/blob/main/browser_use/agent/views.py#L301).
|
||||
|
||||
## Structured Output
|
||||
|
||||
For structured output, use the `output_model_schema` parameter with a Pydantic model. [Example](https://github.com/browser-use/browser-use/blob/main/examples/features/custom_output.py).
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
title: "Prompting Guide"
|
||||
description: "Tips and tricks "
|
||||
icon: "lightbulb"
|
||||
---
|
||||
|
||||
Prompting can trasticly improve performance and solve existing limitations of the library.
|
||||
|
||||
### 1. Be Specific vs Open-Ended
|
||||
|
||||
**✅ Specific (Recommended)**
|
||||
```python
|
||||
task = """
|
||||
1. Go to https://quotes.toscrape.com/
|
||||
2. Use extract_structured_data action with the query "first 3 quotes with their authors"
|
||||
3. Save results to quotes.csv using write_file action
|
||||
4. Do a google search for the first quote and find when it was written
|
||||
"""
|
||||
```
|
||||
|
||||
**❌ Open-Ended**
|
||||
```python
|
||||
task = "Go to web and make money"
|
||||
```
|
||||
|
||||
### 2. Name Actions Directly
|
||||
|
||||
When you know exactly what the agent should do, reference actions by name:
|
||||
|
||||
```python
|
||||
task = """
|
||||
1. Use search_google action to find "Python tutorials"
|
||||
2. Use click_element_by_index to open first result in a new tab
|
||||
3. Use scroll action to scroll down 2 pages
|
||||
4. Use extract_structured_data to extract the names of the first 5 items
|
||||
5. Wait for 2 seconds if the page is not loaded, refresh it and wait 10 sec
|
||||
6. Use send_keys action with "Tab Tab ArrowDown Enter"
|
||||
"""
|
||||
```
|
||||
|
||||
See [Available Tools](/customize/tools/available) for the complete list of actions.
|
||||
|
||||
|
||||
### 3. Handle interaction problems via keyboard navigation
|
||||
|
||||
Sometimes buttons can't be clicked (you found a bug in the library - open an issue).
|
||||
Good news - often you can work around it with keyboard navigation!
|
||||
|
||||
```python
|
||||
task = """
|
||||
If the submit button cannot be clicked:
|
||||
1. Use send_keys action with "Tab Tab Enter" to navigate and activate
|
||||
2. Or use send_keys with "ArrowDown ArrowDown Enter" for form submission
|
||||
"""
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
### 4. Custom Actions Integration
|
||||
|
||||
```python
|
||||
# When you have custom actions
|
||||
@controller.action("Get 2FA code from authenticator app")
|
||||
async def get_2fa_code():
|
||||
# Your implementation
|
||||
pass
|
||||
|
||||
task = """
|
||||
Login with 2FA:
|
||||
1. Enter username/password
|
||||
2. When prompted for 2FA, use get_2fa_code action
|
||||
3. NEVER try to extract 2FA codes from the page manually
|
||||
4. ALWAYS use the get_2fa_code action for authentication codes
|
||||
"""
|
||||
```
|
||||
|
||||
### 5. Error Recovery
|
||||
|
||||
```python
|
||||
task = """
|
||||
Robust data extraction:
|
||||
1. Go to openai.com to find their CEO
|
||||
2. If navigation fails due to anti-bot protection:
|
||||
- Use google search to find the CEO
|
||||
3. If page times out, use go_back and try alternative approach
|
||||
"""
|
||||
```
|
||||
|
||||
|
||||
|
||||
The key to effective prompting is being specific about actions.
|
||||
@@ -0,0 +1,254 @@
|
||||
---
|
||||
title: "Supported Models"
|
||||
description: "Choose your favorite LLM"
|
||||
icon: "robot"
|
||||
|
||||
---
|
||||
|
||||
### Recommendations
|
||||
|
||||
- Best accuracy: `O3`
|
||||
- Fastest: `llama4` on groq
|
||||
- Balanced: fast + cheap + clever: `gemini-2.5-flash` or `gpt-4.1-mini`
|
||||
|
||||
|
||||
### OpenAI [example](https://github.com/browser-use/browser-use/blob/main/examples/models/gpt-4.1.py)
|
||||
|
||||
`O3` model is recommended for best performance.
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatOpenAI
|
||||
|
||||
# Initialize the model
|
||||
llm = ChatOpenAI(
|
||||
model="o3",
|
||||
)
|
||||
|
||||
# Create agent with the model
|
||||
agent = Agent(
|
||||
task="...", # Your task here
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
Required environment variables:
|
||||
|
||||
```bash .env
|
||||
OPENAI_API_KEY=
|
||||
```
|
||||
|
||||
<Info>
|
||||
You can use any OpenAI compatible model by passing the model name to the
|
||||
`ChatOpenAI` class using a custom URL (or any other parameter that would go
|
||||
into the normal OpenAI API call).
|
||||
</Info>
|
||||
|
||||
### Anthropic [example](https://github.com/browser-use/browser-use/blob/main/examples/models/claude-4-sonnet.py)
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatAnthropic
|
||||
|
||||
# Initialize the model
|
||||
llm = ChatAnthropic(
|
||||
model="claude-sonnet-4-0",
|
||||
)
|
||||
|
||||
# Create agent with the model
|
||||
agent = Agent(
|
||||
task="...", # Your task here
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
And add the variable:
|
||||
|
||||
```bash .env
|
||||
ANTHROPIC_API_KEY=
|
||||
```
|
||||
|
||||
### Azure OpenAI [example](https://github.com/browser-use/browser-use/blob/main/examples/models/azure_openai.py)
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatAzureOpenAI
|
||||
from pydantic import SecretStr
|
||||
import os
|
||||
|
||||
# Initialize the model
|
||||
llm = ChatAzureOpenAI(
|
||||
model="o4-mini",
|
||||
)
|
||||
|
||||
# Create agent with the model
|
||||
agent = Agent(
|
||||
task="...", # Your task here
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
Required environment variables:
|
||||
|
||||
```bash .env
|
||||
AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com/
|
||||
AZURE_OPENAI_API_KEY=
|
||||
```
|
||||
|
||||
### Gemini [example](https://github.com/browser-use/browser-use/blob/main/examples/models/gemini.py)
|
||||
|
||||
> [!IMPORTANT] `GEMINI_API_KEY` was the old environment var name, it should be called `GOOGLE_API_KEY` as of 2025-05.
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatGoogle
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Read GOOGLE_API_KEY into env
|
||||
load_dotenv()
|
||||
|
||||
# Initialize the model
|
||||
llm = ChatGoogle(model='gemini-2.5-flash')
|
||||
|
||||
# Create agent with the model
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
Required environment variables:
|
||||
|
||||
```bash .env
|
||||
GOOGLE_API_KEY=
|
||||
```
|
||||
|
||||
### AWS Bedrock [example](https://github.com/browser-use/browser-use/blob/main/examples/models/aws.py)
|
||||
|
||||
AWS Bedrock provides access to multiple model providers through a single API. We support both a general AWS Bedrock client and provider-specific convenience classes.
|
||||
|
||||
#### General AWS Bedrock (supports all providers)
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatAWSBedrock
|
||||
|
||||
# Works with any Bedrock model (Anthropic, Meta, AI21, etc.)
|
||||
llm = ChatAWSBedrock(
|
||||
model="anthropic.claude-3-5-sonnet-20240620-v1:0", # or any Bedrock model
|
||||
aws_region="us-east-1",
|
||||
)
|
||||
|
||||
# Create agent with the model
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
#### Anthropic Claude via AWS Bedrock (convenience class)
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatAnthropicBedrock
|
||||
|
||||
# Anthropic-specific class with Claude defaults
|
||||
llm = ChatAnthropicBedrock(
|
||||
model="anthropic.claude-3-5-sonnet-20240620-v1:0",
|
||||
aws_region="us-east-1",
|
||||
)
|
||||
|
||||
# Create agent with the model
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
#### AWS Authentication
|
||||
|
||||
Required environment variables:
|
||||
|
||||
```bash .env
|
||||
AWS_ACCESS_KEY_ID=
|
||||
AWS_SECRET_ACCESS_KEY=
|
||||
AWS_DEFAULT_REGION=us-east-1
|
||||
```
|
||||
|
||||
You can also use AWS profiles or IAM roles instead of environment variables. The implementation supports:
|
||||
|
||||
- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`)
|
||||
- AWS profiles and credential files
|
||||
- IAM roles (when running on EC2)
|
||||
- Session tokens for temporary credentials
|
||||
- AWS SSO authentication (`aws_sso_auth=True`)
|
||||
|
||||
## Groq [example](https://github.com/browser-use/browser-use/blob/main/examples/models/llama4-groq.py)
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatGroq
|
||||
|
||||
llm = ChatGroq(model="meta-llama/llama-4-maverick-17b-128e-instruct")
|
||||
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=llm
|
||||
)
|
||||
```
|
||||
|
||||
Required environment variables:
|
||||
|
||||
```bash .env
|
||||
GROQ_API_KEY=
|
||||
```
|
||||
|
||||
## Ollama
|
||||
|
||||
1. Install Ollama: https://github.com/ollama/ollama
|
||||
2. Run `ollama serve` to start the server
|
||||
3. In a new terminal, install the model you want to use: `ollama pull llama3.1:8b` (this has 4.9GB)
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatOllama
|
||||
|
||||
llm = ChatOllama(model="llama3.1:8b")
|
||||
```
|
||||
|
||||
## Langchain
|
||||
|
||||
[Example](https://github.com/browser-use/browser-use/blob/main/examples/models/langchain) on how to use Langchain with Browser Use.
|
||||
|
||||
## Qwen [example](https://github.com/browser-use/browser-use/blob/main/examples/models/qwen.py)
|
||||
|
||||
Currently, only `qwen-vl-max` is recommended for Browser Use. Other Qwen models, including `qwen-max`, have issues with the action schema format.
|
||||
Smaller Qwen models may return incorrect action schema formats (e.g., `actions: [{"go_to_url": "google.com"}]` instead of `[{"go_to_url": {"url": "google.com"}}]`). If you want to use other models, add concrete examples of the correct action format to your prompt.
|
||||
|
||||
```python
|
||||
from browser_use import Agent, ChatOpenAI
|
||||
from dotenv import load_dotenv
|
||||
import os
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Get API key from https://modelstudio.console.alibabacloud.com/?tab=playground#/api-key
|
||||
api_key = os.getenv('ALIBABA_CLOUD')
|
||||
base_url = 'https://dashscope-intl.aliyuncs.com/compatible-mode/v1'
|
||||
|
||||
llm = ChatOpenAI(model='qwen-vl-max', api_key=api_key, base_url=base_url)
|
||||
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=llm,
|
||||
use_vision=True
|
||||
)
|
||||
```
|
||||
|
||||
Required environment variables:
|
||||
|
||||
```bash .env
|
||||
ALIBABA_CLOUD=
|
||||
```
|
||||
|
||||
|
||||
## Other models (DeepSeek, Novita, X...)
|
||||
|
||||
We support all other models that can be called via OpenAI compatible API. We are open to PRs for more providers.
|
||||
|
||||
**Examples available:**
|
||||
- [DeepSeek](https://github.com/browser-use/browser-use/blob/main/examples/models/deepseek-chat.py)
|
||||
- [Novita](https://github.com/browser-use/browser-use/blob/main/examples/models/novita.py)
|
||||
- [OpenRouter](https://github.com/browser-use/browser-use/blob/main/examples/models/openrouter.py)
|
||||
@@ -0,0 +1,117 @@
|
||||
---
|
||||
title: "All Parameters"
|
||||
description: "Complete reference for all browser configuration options"
|
||||
icon: "sliders"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
<Note>
|
||||
The `Browser` instance also provides all [Actor](/customize/actor/all-parameters) methods for direct browser control (page management, element interactions, etc.).
|
||||
</Note>
|
||||
|
||||
## Core Settings
|
||||
|
||||
- `cdp_url`: CDP URL for connecting to existing browser instance (e.g., `"http://localhost:9222"`)
|
||||
|
||||
## Display & Appearance
|
||||
|
||||
- `headless` (default: `None`): Run browser without UI. Auto-detects based on display availability (`True`/`False`/`None`)
|
||||
- `window_size`: Browser window size for headful mode. Use dict `{'width': 1920, 'height': 1080}` or `ViewportSize` object
|
||||
- `window_position` (default: `{'width': 0, 'height': 0}`): Window position from top-left corner in pixels
|
||||
- `viewport`: Content area size, same format as `window_size`. Use `{'width': 1280, 'height': 720}` or `ViewportSize` object
|
||||
- `no_viewport` (default: `None`): Disable viewport emulation, content fits to window size
|
||||
- `device_scale_factor`: Device scale factor (DPI). Set to `2.0` or `3.0` for high-resolution screenshots
|
||||
|
||||
## Browser Behavior
|
||||
|
||||
- `keep_alive` (default: `None`): Keep browser running after agent completes
|
||||
- `allowed_domains`: Restrict navigation to specific domains. Domain pattern formats:
|
||||
- `'example.com'` - Matches only `https://example.com/*`
|
||||
- `'*.example.com'` - Matches `https://example.com/*` and any subdomain `https://*.example.com/*`
|
||||
- `'http*://example.com'` - Matches both `http://` and `https://` protocols
|
||||
- `'chrome-extension://*'` - Matches any Chrome extension URL
|
||||
- **Security**: Wildcards in TLD (e.g., `example.*`) are **not allowed** for security
|
||||
- Use list like `['*.google.com', 'https://example.com', 'chrome-extension://*']`
|
||||
- `enable_default_extensions` (default: `True`): Load automation extensions (uBlock Origin, cookie handlers, ClearURLs)
|
||||
- `cross_origin_iframes` (default: `False`): Enable cross-origin iframe support (may cause complexity)
|
||||
- `is_local` (default: `True`): Whether this is a local browser instance. Set to `False` for remote browsers. If we have a `executable_path` set, it will be automatically set to `True`. This can effect your download behavior.
|
||||
|
||||
## User Data & Profiles
|
||||
|
||||
- `user_data_dir` (default: auto-generated temp): Directory for browser profile data. Use `None` for incognito mode
|
||||
- `profile_directory` (default: `'Default'`): Chrome profile subdirectory name (`'Profile 1'`, `'Work Profile'`, etc.)
|
||||
- `storage_state`: Browser storage state (cookies, localStorage). Can be file path string or dict object
|
||||
|
||||
## Network & Security
|
||||
|
||||
- `proxy`: Proxy configuration using `ProxySettings(server='http://host:4242', bypass='localhost,127.0.0.1', username='user', password='pass')`
|
||||
- `permissions` (default: `['clipboardReadWrite', 'notifications']`): Browser permissions to grant. Use list like `['camera', 'microphone', 'geolocation']`
|
||||
|
||||
- `headers`: Additional HTTP headers for connect requests (remote browsers only)
|
||||
|
||||
## Browser Launch
|
||||
|
||||
- `executable_path`: Path to browser executable for custom installations. Platform examples:
|
||||
- macOS: `'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'`
|
||||
- Windows: `'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'`
|
||||
- Linux: `'/usr/bin/google-chrome'`
|
||||
- `channel`: Browser channel (`'chromium'`, `'chrome'`, `'chrome-beta'`, `'msedge'`, etc.)
|
||||
- `args`: Additional command-line arguments for the browser. Use list format: `['--disable-gpu', '--custom-flag=value', '--another-flag']`
|
||||
- `env`: Environment variables for browser process. Use dict like `{'DISPLAY': ':0', 'LANG': 'en_US.UTF-8', 'CUSTOM_VAR': 'test'}`
|
||||
- `chromium_sandbox` (default: `True` except in Docker): Enable Chromium sandboxing for security
|
||||
- `devtools` (default: `False`): Open DevTools panel automatically (requires `headless=False`)
|
||||
- `ignore_default_args`: List of default args to disable, or `True` to disable all. Use list like `['--enable-automation', '--disable-extensions']`
|
||||
|
||||
## Timing & Performance
|
||||
|
||||
- `minimum_wait_page_load_time` (default: `0.25`): Minimum time to wait before capturing page state in seconds
|
||||
- `wait_for_network_idle_page_load_time` (default: `0.5`): Time to wait for network activity to cease in seconds
|
||||
- `wait_between_actions` (default: `0.5`): Time to wait between agent actions in seconds
|
||||
|
||||
## AI Integration
|
||||
|
||||
- `highlight_elements` (default: `True`): Highlight interactive elements for AI vision
|
||||
- `paint_order_filtering` (default: `True`): Enable paint order filtering to optimize DOM tree by removing elements hidden behind others. Slightly experimental
|
||||
|
||||
## Downloads & Files
|
||||
|
||||
- `accept_downloads` (default: `True`): Automatically accept all downloads
|
||||
- `downloads_path`: Directory for downloaded files. Use string like `'./downloads'` or `Path` object
|
||||
- `auto_download_pdfs` (default: `True`): Automatically download PDFs instead of viewing in browser
|
||||
|
||||
## Device Emulation
|
||||
|
||||
- `user_agent`: Custom user agent string. Example: `'Mozilla/5.0 (iPhone; CPU iPhone OS 14_0 like Mac OS X)'`
|
||||
- `screen`: Screen size information, same format as `window_size`
|
||||
|
||||
## Recording & Debugging
|
||||
|
||||
- `record_video_dir`: Directory to save video recordings as `.mp4` files
|
||||
- `record_video_size` (default: `ViewportSize`): The frame size (width, height) of the video recording.
|
||||
- `record_video_framerate` (default: `30`): The framerate to use for the video recording.
|
||||
- `record_har_path`: Path to save network trace files as `.har` format
|
||||
- `traces_dir`: Directory to save complete trace files for debugging
|
||||
- `record_har_content` (default: `'embed'`): HAR content mode (`'omit'`, `'embed'`, `'attach'`)
|
||||
- `record_har_mode` (default: `'full'`): HAR recording mode (`'full'`, `'minimal'`)
|
||||
|
||||
## Advanced Options
|
||||
|
||||
- `disable_security` (default: `False`): ⚠️ **NOT RECOMMENDED** - Disables all browser security features
|
||||
- `deterministic_rendering` (default: `False`): ⚠️ **NOT RECOMMENDED** - Forces consistent rendering but reduces performance
|
||||
|
||||
---
|
||||
|
||||
## Outdated BrowserProfile
|
||||
|
||||
For backward compatibility, you can pass all the parameters from above to the `BrowserProfile` and then to the `Browser`.
|
||||
|
||||
```python
|
||||
from browser_use import BrowserProfile
|
||||
profile = BrowserProfile(headless=False)
|
||||
browser = Browser(browser_profile=profile)
|
||||
```
|
||||
|
||||
## Browser vs BrowserSession
|
||||
|
||||
`Browser` is an alias for `BrowserSession` - they are exactly the same class:
|
||||
Use `Browser` for cleaner, more intuitive code.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
title: "Basics"
|
||||
description: ""
|
||||
icon: "play"
|
||||
---
|
||||
|
||||
|
||||
---
|
||||
|
||||
```python
|
||||
from browser_use import Agent, Browser, ChatOpenAI
|
||||
|
||||
browser = Browser(
|
||||
headless=False, # Show browser window
|
||||
window_size={'width': 1000, 'height': 700}, # Set window size
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
task='Search for Browser Use',
|
||||
browser=browser,
|
||||
llm=ChatOpenAI(model='gpt-4.1-mini'),
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
await agent.run()
|
||||
```
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
title: "Real Browser"
|
||||
description: ""
|
||||
icon: "arrow-right-to-bracket"
|
||||
---
|
||||
|
||||
Connect your existing Chrome browser to preserve authentication.
|
||||
|
||||
## Basic Example
|
||||
|
||||
```python
|
||||
from browser_use import Agent, Browser, ChatOpenAI
|
||||
|
||||
# Connect to your existing Chrome browser
|
||||
browser = Browser(
|
||||
executable_path='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
||||
user_data_dir='~/Library/Application Support/Google/Chrome',
|
||||
profile_directory='Default',
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
task='Visit https://duckduckgo.com and search for "browser-use founders"',
|
||||
browser=browser,
|
||||
llm=ChatOpenAI(model='gpt-4.1-mini'),
|
||||
)
|
||||
async def main():
|
||||
await agent.run()
|
||||
```
|
||||
|
||||
> **Note:** You need to fully close chrome before running this example. Also, Google blocks this approach currently so we use DuckDuckGo instead.
|
||||
|
||||
|
||||
|
||||
|
||||
## How it Works
|
||||
|
||||
1. **`executable_path`** - Path to your Chrome installation
|
||||
2. **`user_data_dir`** - Your Chrome profile folder (keeps cookies, extensions, bookmarks)
|
||||
3. **`profile_directory`** - Specific profile name (Default, Profile 1, etc.)
|
||||
|
||||
|
||||
## Platform Paths
|
||||
|
||||
```python
|
||||
# macOS
|
||||
executable_path='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
|
||||
user_data_dir='~/Library/Application Support/Google/Chrome'
|
||||
|
||||
# Windows
|
||||
executable_path='C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
|
||||
user_data_dir='%LOCALAPPDATA%\\Google\\Chrome\\User Data'
|
||||
|
||||
# Linux
|
||||
executable_path='/usr/bin/google-chrome'
|
||||
user_data_dir='~/.config/google-chrome'
|
||||
```
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: "Remote Browser"
|
||||
description: ""
|
||||
icon: "cloud"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
|
||||
### Browser-Use Cloud Browser or CDP URL
|
||||
|
||||
The easiest way to use a cloud browser is with the built-in Browser-Use cloud service:
|
||||
|
||||
```python
|
||||
from browser_use import Agent, Browser, ChatOpenAI
|
||||
|
||||
# Use Browser-Use cloud browser service
|
||||
browser = Browser(
|
||||
use_cloud=True, # Automatically provisions a cloud browser
|
||||
# cdp_url="http://remote-server:9222" # CDP URL from your favorite browser provider like AnchorBrowser, HyperBrowser, BrowserBase, Steel.dev, etc.
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=ChatOpenAI(model='gpt-4.1-mini'),
|
||||
browser=browser,
|
||||
)
|
||||
```
|
||||
|
||||
**Prerequisites:**
|
||||
1. Get an API key from [cloud.browser-use.com](https://cloud.browser-use.com)
|
||||
2. Set BROWSER_USE_API_KEY environment variable
|
||||
|
||||
**Benefits:**
|
||||
- ✅ No local browser setup required
|
||||
- ✅ Scalable and fast cloud infrastructure
|
||||
- ✅ Automatic provisioning and teardown
|
||||
- ✅ Built-in authentication handling
|
||||
- ✅ Optimized for browser automation
|
||||
|
||||
### Third-Party Cloud Browsers
|
||||
Get a CDP URL from your favorite browser provider like AnchorBrowser, HyperBrowser, BrowserBase, Steel.dev, etc.
|
||||
|
||||
|
||||
|
||||
|
||||
### Proxy Connection
|
||||
|
||||
```python
|
||||
|
||||
from browser_use import Agent, Browser, ChatOpenAI
|
||||
from browser_use.browser import ProxySettings
|
||||
|
||||
browser = Browser(
|
||||
headless=False,
|
||||
proxy=ProxySettings(
|
||||
server="http://proxy-server:4242",
|
||||
username="proxy-user",
|
||||
password="proxy-pass"
|
||||
)
|
||||
cdp_url="http://remote-server:9222"
|
||||
)
|
||||
|
||||
|
||||
agent = Agent(
|
||||
task="Your task here",
|
||||
llm=ChatOpenAI(model='gpt-4.1-mini'),
|
||||
browser=browser,
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,119 @@
|
||||
---
|
||||
title: "Lifecycle Hooks"
|
||||
description: "Customize agent behavior with lifecycle hooks"
|
||||
icon: "Wrench"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
Browser-Use provides lifecycle hooks that allow you to execute custom code at specific points during the agent's execution.
|
||||
Hook functions can be used to read and modify agent state while running, implement custom logic, change configuration, integrate the Agent with external applications.
|
||||
|
||||
## Available Hooks
|
||||
|
||||
Currently, Browser-Use provides the following hooks:
|
||||
|
||||
| Hook | Description | When it's called |
|
||||
| --------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------- |
|
||||
| `on_step_start` | Executed at the beginning of each agent step | Before the agent processes the current state and decides on the next action |
|
||||
| `on_step_end` | Executed at the end of each agent step | After the agent has executed all the actions for the current step, before it starts the next step |
|
||||
|
||||
```python
|
||||
await agent.run(on_step_start=..., on_step_end=...)
|
||||
```
|
||||
|
||||
Each hook should be an `async` callable function that accepts the `agent` instance as its only parameter.
|
||||
|
||||
### Basic Example
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
from browser_use import Agent, ChatOpenAI
|
||||
from browser_use.browser.events import ScreenshotEvent
|
||||
|
||||
|
||||
async def my_step_hook(agent: Agent):
|
||||
# inside a hook you can access all the state and methods under the Agent object:
|
||||
# agent.settings, agent.state, agent.task
|
||||
# agent.tools, agent.llm, agent.browser_session
|
||||
# agent.pause(), agent.resume(), agent.add_new_task(...), etc.
|
||||
|
||||
# You also have direct access to the browser state
|
||||
state = await agent.browser_session.get_browser_state_summary()
|
||||
|
||||
current_url = state.url
|
||||
visit_log = agent.history.urls()
|
||||
previous_url = visit_log[-2] if len(visit_log) >= 2 else None
|
||||
print(f'Agent was last on URL: {previous_url} and is now on {current_url}')
|
||||
cdp_session = await agent.browser_session.get_or_create_cdp_session()
|
||||
|
||||
# Example: Get page HTML content
|
||||
doc = await cdp_session.cdp_client.send.DOM.getDocument(session_id=cdp_session.session_id)
|
||||
html_result = await cdp_session.cdp_client.send.DOM.getOuterHTML(
|
||||
params={'nodeId': doc['root']['nodeId']}, session_id=cdp_session.session_id
|
||||
)
|
||||
page_html = html_result['outerHTML']
|
||||
|
||||
# Example: Take a screenshot using the event system
|
||||
screenshot_event = agent.browser_session.event_bus.dispatch(ScreenshotEvent(full_page=False))
|
||||
await screenshot_event
|
||||
result = await screenshot_event.event_result(raise_if_any=True, raise_if_none=True)
|
||||
|
||||
# Example: pause agent execution and resume it based on some custom code
|
||||
if '/finished' in current_url:
|
||||
agent.pause()
|
||||
Path('result.txt').write_text(page_html)
|
||||
input('Saved "finished" page content to result.txt, press [Enter] to resume...')
|
||||
agent.resume()
|
||||
|
||||
|
||||
async def main():
|
||||
agent = Agent(
|
||||
task='Search for the latest news about AI',
|
||||
llm=ChatOpenAI(model='gpt-5-mini'),
|
||||
)
|
||||
|
||||
await agent.run(
|
||||
on_step_start=my_step_hook,
|
||||
# on_step_end=...
|
||||
max_steps=10,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Data Available in Hooks
|
||||
|
||||
When working with agent hooks, you have access to the entire `Agent` instance. Here are some useful data points you can access:
|
||||
|
||||
- `agent.task` lets you see what the main task is, `agent.add_new_task(...)` lets you queue up a new one
|
||||
- `agent.tools` give access to the `Tools()` object and `Registry()` containing the available actions
|
||||
- `agent.tools.registry.execute_action('click_element_by_index', {'index': 123}, browser_session=agent.browser_session)`
|
||||
- `agent.context` lets you access any user-provided context object passed in to `Agent(context=...)`
|
||||
- `agent.sensitive_data` contains the sensitive data dict, which can be updated in-place to add/remove/modify items
|
||||
- `agent.settings` contains all the configuration options passed to the `Agent(...)` at init time
|
||||
- `agent.llm` gives direct access to the main LLM object (e.g. `ChatOpenAI`)
|
||||
- `agent.state` gives access to lots of internal state, including agent thoughts, outputs, actions, etc.
|
||||
- `agent.history` gives access to historical data from the agent's execution:
|
||||
- `agent.history.model_thoughts()`: Reasoning from Browser Use's model.
|
||||
- `agent.history.model_outputs()`: Raw outputs from the Browser Use's model.
|
||||
- `agent.history.model_actions()`: Actions taken by the agent
|
||||
- `agent.history.extracted_content()`: Content extracted from web pages
|
||||
- `agent.history.urls()`: URLs visited by the agent
|
||||
- `agent.browser_session` gives direct access to the `BrowserSession` and CDP interface
|
||||
- `agent.browser_session.agent_focus`: Get the current CDP session the agent is focused on
|
||||
- `agent.browser_session.get_or_create_cdp_session()`: Get the current CDP session for browser interaction
|
||||
- `agent.browser_session.get_tabs()`: Get all tabs currently open
|
||||
- `agent.browser_session.get_current_page_url()`: Get the URL of the current active tab
|
||||
- `agent.browser_session.get_current_page_title()`: Get the title of the current active tab
|
||||
|
||||
## Tips for Using Hooks
|
||||
|
||||
- **Avoid blocking operations**: Since hooks run in the same execution thread as the agent, keep them efficient and avoid blocking operations.
|
||||
- **Use custom tools instead**: hooks are fairly advanced, most things can be implemented with [custom tools](/customize/tools/basics) instead
|
||||
- **Increase step_timeout**: If your hook is doing something that takes a long time, you can increase the `step_timeout` parameter in the `Agent(...)` constructor.
|
||||
|
||||
---
|
||||
@@ -0,0 +1,191 @@
|
||||
---
|
||||
title: "MCP Server"
|
||||
description: "Expose browser-use capabilities via Model Context Protocol for AI assistants like Claude Desktop"
|
||||
icon: "server"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The MCP (Model Context Protocol) Server allows you to expose browser-use's browser automation capabilities to AI assistants like Claude Desktop, Cline, and other MCP-compatible clients. This enables AI assistants to perform web automation tasks directly through browser-use.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Start MCP Server
|
||||
|
||||
```bash
|
||||
uvx browser-use --mcp
|
||||
```
|
||||
|
||||
The server will start in stdio mode, ready to accept MCP connections.
|
||||
|
||||
## Claude Desktop Integration
|
||||
|
||||
The most common use case is integrating with Claude Desktop. Add this configuration to your Claude Desktop config file:
|
||||
|
||||
### macOS
|
||||
Edit `~/Library/Application Support/Claude/claude_desktop_config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"browser-use": {
|
||||
"command": "uvx",
|
||||
"args": ["browser-use", "--mcp"],
|
||||
"env": {
|
||||
"OPENAI_API_KEY": "your-openai-api-key-here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Windows
|
||||
Edit `%APPDATA%\Claude\claude_desktop_config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"browser-use": {
|
||||
"command": "uvx",
|
||||
"args": ["browser-use", "--mcp"],
|
||||
"env": {
|
||||
"OPENAI_API_KEY": "your-openai-api-key-here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
You can configure browser-use through environment variables:
|
||||
|
||||
- `OPENAI_API_KEY` - Your OpenAI API key (required)
|
||||
- `ANTHROPIC_API_KEY` - Your Anthropic API key (alternative to OpenAI)
|
||||
- `BROWSER_USE_HEADLESS` - Set to `false` to show browser window
|
||||
- `BROWSER_USE_DISABLE_SECURITY` - Set to `true` to disable browser security features
|
||||
|
||||
## Available Tools
|
||||
|
||||
The MCP server exposes these browser automation tools:
|
||||
|
||||
### Autonomous Agent Tools
|
||||
- **`retry_with_browser_use_agent`** - Run a complete browser automation task with an AI agent (use as last resort when direct control fails)
|
||||
|
||||
### Direct Browser Control
|
||||
- **`browser_navigate`** - Navigate to a URL
|
||||
- **`browser_click`** - Click on an element by index
|
||||
- **`browser_type`** - Type text into an element
|
||||
- **`browser_get_state`** - Get current page state and interactive elements
|
||||
- **`browser_scroll`** - Scroll the page
|
||||
- **`browser_go_back`** - Go back in browser history
|
||||
|
||||
### Tab Management
|
||||
- **`browser_list_tabs`** - List all open browser tabs
|
||||
- **`browser_switch_tab`** - Switch to a specific tab
|
||||
- **`browser_close_tab`** - Close a tab
|
||||
|
||||
### Content Extraction
|
||||
- **`browser_extract_content`** - Extract structured content from the current page
|
||||
|
||||
### Session Management
|
||||
- **`browser_list_sessions`** - List all active browser sessions with details
|
||||
- **`browser_close_session`** - Close a specific browser session by ID
|
||||
- **`browser_close_all`** - Close all active browser sessions
|
||||
|
||||
## Example Usage
|
||||
|
||||
Once configured with Claude Desktop, you can ask Claude to perform browser automation tasks:
|
||||
|
||||
```
|
||||
"Please navigate to example.com and take a screenshot"
|
||||
|
||||
"Search for 'browser automation' on Google and summarize the first 3 results"
|
||||
|
||||
"Go to GitHub, find the browser-use repository, and tell me about the latest release"
|
||||
```
|
||||
|
||||
Claude will use the MCP server to execute these tasks through browser-use.
|
||||
|
||||
## Programmatic Usage
|
||||
|
||||
You can also connect to the MCP server programmatically:
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
|
||||
async def use_browser_mcp():
|
||||
# Connect to browser-use MCP server
|
||||
server_params = StdioServerParameters(
|
||||
command="uvx",
|
||||
args=["browser-use", "--mcp"]
|
||||
)
|
||||
|
||||
async with stdio_client(server_params) as (read, write):
|
||||
async with ClientSession(read, write) as session:
|
||||
await session.initialize()
|
||||
|
||||
# Navigate to a website
|
||||
result = await session.call_tool(
|
||||
"browser_navigate",
|
||||
arguments={"url": "https://example.com"}
|
||||
)
|
||||
print(result.content[0].text)
|
||||
|
||||
# Get page state
|
||||
result = await session.call_tool(
|
||||
"browser_get_state",
|
||||
arguments={"include_screenshot": True}
|
||||
)
|
||||
print("Page state retrieved!")
|
||||
|
||||
asyncio.run(use_browser_mcp())
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**"MCP SDK is required" Error**
|
||||
```bash
|
||||
uv pip install 'browser-use'
|
||||
```
|
||||
|
||||
**Browser doesn't start**
|
||||
- Check that you have Chrome/Chromium installed
|
||||
- Try setting `BROWSER_USE_HEADLESS=false` to see browser window
|
||||
- Ensure no other browser instances are using the same profile
|
||||
|
||||
**API Key Issues**
|
||||
- Verify your `OPENAI_API_KEY` is set correctly
|
||||
- Check API key permissions and billing status
|
||||
- Try using `ANTHROPIC_API_KEY` as an alternative
|
||||
|
||||
**Connection Issues in Claude Desktop**
|
||||
- Restart Claude Desktop after config changes
|
||||
- Check the config file syntax is valid JSON
|
||||
- Verify the file path is correct for your OS
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Enable debug logging by setting:
|
||||
```bash
|
||||
export BROWSER_USE_LOG_LEVEL=DEBUG
|
||||
uvx browser-use --mcp
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- The MCP server has access to your browser and file system
|
||||
- Only connect trusted MCP clients
|
||||
- Be cautious with sensitive websites and data
|
||||
- Consider running in a sandboxed environment for untrusted automation
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Explore the [examples directory](https://github.com/browser-use/browser-use/tree/main/examples/mcp) for more usage patterns
|
||||
- Check out [MCP documentation](https://modelcontextprotocol.io/) to learn more about the protocol
|
||||
- Join our [Discord](https://link.browser-use.com/discord) for support and discussions
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: "Add Tools"
|
||||
description: ""
|
||||
icon: "plus"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
|
||||
Examples:
|
||||
- deterministic clicks
|
||||
- file handling
|
||||
- calling APIs
|
||||
- human-in-the-loop
|
||||
- browser interactions
|
||||
- calling LLMs
|
||||
- get 2fa codes
|
||||
- send emails
|
||||
- Playwright integration (see [GitHub example](https://github.com/browser-use/browser-use/blob/main/examples/browser/playwright_integration.py))
|
||||
- ...
|
||||
|
||||
Simply add `@tools.action(...)` to your function.
|
||||
|
||||
```python
|
||||
from browser_use import Tools, Agent
|
||||
|
||||
tools = Tools()
|
||||
|
||||
@tools.action(description='Ask human for help with a question')
|
||||
def ask_human(question: str) -> ActionResult:
|
||||
answer = input(f'{question} > ')
|
||||
return f'The human responded with: {answer}'
|
||||
```
|
||||
|
||||
```python
|
||||
agent = Agent(task='...', llm=llm, tools=tools)
|
||||
```
|
||||
|
||||
- **`description`** *(required)* - What the tool does, the LLM uses this to decide when to call it.
|
||||
- **`allowed_domains`** - List of domains where tool can run (e.g. `['*.example.com']`), defaults to all domains
|
||||
|
||||
The Agent fills your function parameters based on their names, type hints, & defaults.
|
||||
|
||||
|
||||
## Available Objects
|
||||
|
||||
Your function has access to these objects:
|
||||
|
||||
- **`browser_session: BrowserSession`** - Current browser session for CDP access
|
||||
- **`cdp_client`** - Direct Chrome DevTools Protocol client
|
||||
- **`page_extraction_llm: BaseChatModel`** - The LLM you pass into agent. This can be used to do a custom llm call here.
|
||||
- **`file_system: FileSystem`** - File system access
|
||||
- **`available_file_paths: list[str]`** - Available files for upload/processing
|
||||
- **`has_sensitive_data: bool`** - Whether action contains sensitive data
|
||||
|
||||
## Pydantic Input
|
||||
|
||||
You can use Pydantic for the tool parameters:
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel
|
||||
|
||||
class Cars(BaseModel):
|
||||
name: str = Field(description='The name of the car, e.g. "Toyota Camry"')
|
||||
price: int = Field(description='The price of the car as int in USD, e.g. 25000')
|
||||
|
||||
@tools.action(description='Save cars to file')
|
||||
def save_cars(cars: list[Cars]) -> str:
|
||||
with open('cars.json', 'w') as f:
|
||||
json.dump(cars, f)
|
||||
return f'Saved {len(cars)} cars to file'
|
||||
|
||||
task = "find cars and save them to file"
|
||||
```
|
||||
## Domain Restrictions
|
||||
|
||||
Limit tools to specific domains:
|
||||
|
||||
```python
|
||||
@tools.action(
|
||||
description='Fill out banking forms',
|
||||
allowed_domains=['https://mybank.com']
|
||||
)
|
||||
def fill_bank_form(account_number: str) -> str:
|
||||
# Only works on mybank.com
|
||||
return f'Filled form for account {account_number}'
|
||||
```
|
||||
|
||||
## Advanced Example
|
||||
|
||||
For a comprehensive example of custom tools with Playwright integration, see:
|
||||
**[Playwright Integration Example](https://github.com/browser-use/browser-use/blob/main/examples/browser/playwright_integration.py)**
|
||||
|
||||
This shows how to create custom actions that use Playwright's precise browser automation alongside Browser-Use.
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: "Available Tools"
|
||||
description: "Here is the [source code](https://github.com/browser-use/browser-use/blob/main/browser_use/tools/service.py) for the default tools:"
|
||||
icon: "list"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
|
||||
|
||||
|
||||
### Navigation & Browser Control
|
||||
- **`search_google`** - Search queries in Google
|
||||
- **`go_to_url`** - Navigate to URLs
|
||||
- **`go_back`** - Go back in browser history
|
||||
- **`wait`** - Wait for specified seconds
|
||||
|
||||
### Page Interaction
|
||||
- **`click_element_by_index`** - Click elements by their index
|
||||
- **`input_text`** - Input text into form fields
|
||||
- **`upload_file_to_element`** - Upload files to file inputs
|
||||
- **`scroll`** - Scroll the page up/down
|
||||
- **`scroll_to_text`** - Scroll to specific text on page
|
||||
- **`send_keys`** - Send special keys (Enter, Escape, etc.)
|
||||
|
||||
### Tab Management
|
||||
- **`switch_tab`** - Switch between browser tabs
|
||||
- **`close_tab`** - Close browser tabs
|
||||
|
||||
### Content Extraction
|
||||
- **`extract_structured_data`** - Extract data from webpages using LLM
|
||||
|
||||
### Form Controls
|
||||
- **`get_dropdown_options`** - Get dropdown option values
|
||||
- **`select_dropdown_option`** - Select dropdown options
|
||||
|
||||
### File Operations
|
||||
- **`write_file`** - Write content to files
|
||||
- **`read_file`** - Read file contents
|
||||
- **`replace_file_str`** - Replace text in files
|
||||
|
||||
### Task Completion
|
||||
- **`done`** - Complete the task (always available)
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
title: "Basics"
|
||||
description: "Tools are the functions that the agent has to interact with the world."
|
||||
icon: "play"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
|
||||
## Quick Example
|
||||
|
||||
|
||||
```python
|
||||
from browser_use import Tools, ActionResult, Browser
|
||||
|
||||
tools = Tools()
|
||||
|
||||
@tools.action('Ask human for help with a question')
|
||||
def ask_human(question: str, browser: Browser) -> ActionResult:
|
||||
answer = input(f'{question} > ')
|
||||
return f'The human responded with: {answer}'
|
||||
|
||||
agent = Agent(
|
||||
task='Ask human for help',
|
||||
llm=llm,
|
||||
tools=tools,
|
||||
)
|
||||
```
|
||||
|
||||
<Note>
|
||||
Use `browser` parameter in tools for deterministic [Actor](/customize/actor/basics) actions.
|
||||
</Note>
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: "Remove Tools"
|
||||
description: "You can exclude default tools:"
|
||||
icon: "minus"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
|
||||
```python
|
||||
from browser_use import Tools
|
||||
|
||||
tools = Tools(exclude_actions=['search_google', 'wait'])
|
||||
agent = Agent(task='...', llm=llm, tools=tools)
|
||||
```
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "Tool Response"
|
||||
description: ""
|
||||
icon: "arrow-turn-down-left"
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
Tools return results using `ActionResult` or simple strings.
|
||||
|
||||
## Return Types
|
||||
|
||||
```python
|
||||
@tools.action('My tool')
|
||||
def my_tool() -> str:
|
||||
return "Task completed successfully"
|
||||
|
||||
@tools.action('Advanced tool')
|
||||
def advanced_tool() -> ActionResult:
|
||||
return ActionResult(
|
||||
extracted_content="Main result",
|
||||
long_term_memory="Remember this info",
|
||||
error="Something went wrong",
|
||||
is_done=True,
|
||||
success=True,
|
||||
attachments=["file.pdf"],
|
||||
)
|
||||
```
|
||||
|
||||
## ActionResult Properties
|
||||
|
||||
- `extracted_content` (default: `None`) - Main result passed to LLM, this is equivalent to returning a string.
|
||||
- `include_extracted_content_only_once` (default: `False`) - Set to `True` for large content to include it only once in the LLM input.
|
||||
- `long_term_memory` (default: `None`) - This is always included in the LLM input for all future steps.
|
||||
- `error` (default: `None`) - Error message, we catch exceptions and set this automatically. This is always included in the LLM input.
|
||||
- `is_done` (default: `False`) - Tool completes entire task
|
||||
- `success` (default: `None`) - Task success (only valid with `is_done=True`)
|
||||
- `attachments` (default: `None`) - Files to show user
|
||||
- `metadata` (default: `None`) - Debug/observability data
|
||||
|
||||
## Why `extracted_content` and `long_term_memory`?
|
||||
With this you control the context for the LLM.
|
||||
|
||||
### 1. Include short content always in context
|
||||
```python
|
||||
def simple_tool() -> str:
|
||||
return "Hello, world!" # Keep in context for all future steps
|
||||
```
|
||||
|
||||
### 2. Show long content once, remember subset in context
|
||||
```python
|
||||
return ActionResult(
|
||||
extracted_content="[500 lines of product data...]", # Shows to LLM once
|
||||
include_extracted_content_only_once=True, # Never show full output again
|
||||
long_term_memory="Found 50 products" # Only this in future steps
|
||||
)
|
||||
```
|
||||
We save the full `extracted_content` to files which the LLM can read in future steps.
|
||||
|
||||
### 3. Dont show long content, remember subset in context
|
||||
```python
|
||||
return ActionResult(
|
||||
extracted_content="[500 lines of product data...]", # The LLM never sees this because `long_term_memory` overrides it and `include_extracted_content_only_once` is not used
|
||||
long_term_memory="Saved user's favorite products", # This is shown to the LLM in future steps
|
||||
)
|
||||
```
|
||||
|
||||
## Terminating the Agent
|
||||
|
||||
Set `is_done=True` to stop the agent completely. Use when your tool finishes the entire task:
|
||||
|
||||
```python
|
||||
@tools.action(description='Complete the task')
|
||||
def finish_task() -> ActionResult:
|
||||
return ActionResult(
|
||||
extracted_content="Task completed!",
|
||||
is_done=True, # Stops the agent
|
||||
success=True # Task succeeded
|
||||
)
|
||||
```
|
||||
Reference in New Issue
Block a user