insready/agent-browser repository overview

⁠agent-browser

A Docker image that gives an AI agent a real, headed Chromium to drive, and gives you a live view of it in your browser.

• The agent controls the browser with Microsoft's official @playwright/cli⁠ (playwright-cli) through docker exec.
• You watch the same screen at http://localhost:6080⁠ over noVNC.
• An LLM skill (skills/agent-browser/SKILL.md⁠) teaches the agent how to use it.
• Works with Claude, OpenAI, Grok, DeepSeek, and local Ollama models. See Connect it to your LLM⁠.

There is no custom control server and no MCP server. Only the noVNC port is published.

⁠Why

• CLI over MCP for token efficiency. A short playwright-cli snapshot or click e15 costs far fewer tokens than loading a large MCP tool schema and verbose page trees into context on every turn.
• Headed, not headless. Chromium runs on a virtual X display (Xvfb), so pages behave like a real desktop browser and you can see what the agent does.
• Live view. noVNC streams that display to any web browser. You can watch, and step in to help (for example, a login or captcha).
• Persistent sessions. A named Docker volume keeps the playwright-cli session cache across container restarts.
• Pinned and reproducible. Base image mcr.microsoft.com/playwright:v1.63.0-noble, @playwright/[email protected].

⁠Quick start

⁠Docker Hub image

docker run -d --name agent-browser \
  -p 127.0.0.1:6080:6080 \
  --shm-size=1g \
  -v "$PWD/artifacts:/artifacts" \
  -v agent-browser-sessions:/home/pwuser/.cache/ms-playwright \
  insready/agent-browser:latest

⁠Docker Compose (from source)

git clone https://github.com/skyred/agent-browser.git
cd agent-browser
docker compose up -d --build     # or: make up

Then open the live view:

http://localhost:6080/vnc.html?autoconnect=true&resize=scale⁠

Stop with docker compose down (or make down).

⁠Drive the browser

Run playwright-cli inside the container. With Compose, use docker compose exec agent-browser …; with docker run, use docker exec agent-browser ….

docker compose exec agent-browser playwright-cli open https://example.com --headed
docker compose exec agent-browser playwright-cli snapshot          # a11y tree with element refs (e3, e15, …)
docker compose exec agent-browser playwright-cli click e15
docker compose exec agent-browser playwright-cli fill e7 "hello" --submit
docker compose exec agent-browser playwright-cli screenshot --filename=/artifacts/page.png
docker compose exec agent-browser playwright-cli close

Shortcut: make cli CMD='open https://example.com --headed'. Inside the image, pw is an alias for playwright-cli.

⁠Cheat sheet

playwright-cli open [url] --headed           # start headed browser (visible on noVNC)
playwright-cli goto <url>
playwright-cli snapshot                      # a11y tree + element refs
playwright-cli click <ref>
playwright-cli type <text>
playwright-cli fill <ref> <text> [--submit]
playwright-cli press <key>
playwright-cli screenshot [--filename=/artifacts/x.png]
playwright-cli eval <js>
playwright-cli -s=name open <url> --headed   # named session
playwright-cli list
playwright-cli close | close-all

Full reference: playwright-cli --help and microsoft/playwright-cli⁠.

⁠Examples

• examples/navigate-and-screenshot.sh⁠: start the stack, open example.com, snapshot, save ./artifacts/example.png.
• examples/snapshot-demo.sh⁠: open a page and print its snapshot.

⁠Install the agent skill

skills/agent-browser/SKILL.md⁠ tells an LLM agent when to use this stack, how to start it, the open → snapshot → click/fill → screenshot loop, sessions, and troubleshooting. Copy or symlink the folder into your agent's skills directory, for example:

# Claude Code / compatible agents
mkdir -p ~/.claude/skills
ln -s "$PWD/skills/agent-browser" ~/.claude/skills/agent-browser

The skill uses docker compose commands, so the agent should run them from this project directory.

⁠Connect it to your LLM

There is no server to register. The model drives the browser by running shell commands such as docker compose exec agent-browser playwright-cli snapshot. Every integration is the same two pieces:

1. A shell tool the model can call, restricted to playwright-cli inside the container.
2. The instructions in skills/agent-browser/SKILL.md⁠, loaded as a skill or as the system prompt.

Start the stack first (docker compose up -d) and open the live view⁠ to watch.

⁠Coding agents (skill-based)

Claude Code. Skills are folders with a SKILL.md in ~/.claude/skills/<name>/ (all projects) or .claude/skills/<name>/ (this repo). Symlinks are supported.

mkdir -p ~/.claude/skills
ln -s "$PWD/skills/agent-browser" ~/.claude/skills/agent-browser

Claude loads the skill when a task needs a browser, or invoke it with /agent-browser. To skip the permission prompt for browser commands only, add an allow rule such as Bash(docker compose exec agent-browser playwright-cli *) in /permissions. Don't allow Bash(*).

OpenAI Codex CLI. Codex reads skills from .agents/skills/ (repo) or ~/.agents/skills/ (user), and follows symlinks:

mkdir -p ~/.agents/skills
ln -s "$PWD/skills/agent-browser" ~/.agents/skills/agent-browser

Mention it with $ or /skills, or let Codex pick it from the description. Keep Codex's sandbox/approval settings on so it asks before running commands outside the allowlist.

Other agents (Cursor, Aider, …). Any agent that can run shell commands and read instructions works. Point it at the skill file, for example from your project's AGENTS.md ("For browser tasks, follow skills/agent-browser/SKILL.md"), or aider --read skills/agent-browser/SKILL.md. Restrict its terminal to playwright-cli commands where the agent supports it.

⁠API models (generic agent loop)

examples/agent_loop.py⁠ is a ~150-line agent loop for any OpenAI-compatible Chat Completions API. It uses the skill as the system prompt and exposes one tool, run_shell, that only runs playwright-cli in the container.

pip install openai
docker compose up -d
export BASE_URL=... API_KEY=... MODEL=...     # see the provider table below
python examples/agent_loop.py "Open example.com and tell me the main heading"

The tool definition the model sees:

{"type": "function", "function": {
    "name": "run_shell",
    "description": "Run ONE playwright-cli command inside the agent-browser container ...",
    "parameters": {"type": "object",
                   "properties": {"command": {"type": "string"}},
                   "required": ["command"]}}}

Optional env vars: CONTAINER (default agent-browser), SKILL_FILE, MAX_STEPS (default 30).

Model names change often; check each provider's model list.

OpenAI

export BASE_URL=https://api.openai.com/v1 API_KEY=$OPENAI_API_KEY MODEL=gpt-6.1-sol

Claude API / Claude Desktop. The loop works through Anthropic's OpenAI SDK compatibility⁠ layer (Anthropic recommends the native API for production; port the same single tool to the Messages API if you need its full features):

export BASE_URL=https://api.anthropic.com/v1/ API_KEY=$ANTHROPIC_API_KEY MODEL=claude-opus-5-5

Claude Desktop has no built-in shell, so use Claude Code (above) for this stack.

Grok (xAI)

export BASE_URL=https://api.x.ai/v1 API_KEY=$XAI_API_KEY MODEL=grok-4.7

DeepSeek

export BASE_URL=https://api.deepseek.com API_KEY=$DEEPSEEK_API_KEY MODEL=deepseek-flash

Ollama (local, no API key)

ollama pull qwen3
export BASE_URL=http://localhost:11434/v1 API_KEY=ollama MODEL=qwen3

Use a model that supports tool calling (see Ollama tool calling⁠); small models may loop or misuse refs, so keep MAX_STEPS low. Ollama doesn't need to reach the container: the script runs on the host, talks to Ollama on localhost:11434, and drives the browser with docker exec. If you run the agent loop inside another container, use http://host.docker.internal:11434/v1 (add extra_hosts: ["host.docker.internal:host-gateway"] on Linux) and mount the Docker socket only if you accept that risk.

⁠How it works

Your browser ──► :6080 (noVNC)
                    │
                    ▼
   ┌─ container ───────────────────────────────┐
   │  noVNC / websockify → x11vnc (localhost)   │
   │        → Xvfb :99 ← openbox                │
   │        → headed Chromium (Playwright)      │
   │                                            │
   │  Agent: docker exec → playwright-cli       │
   └────────────────────────────────────────────┘

The entrypoint (scripts/entrypoint.sh) starts Xvfb, openbox, x11vnc (bound to 127.0.0.1:5900 inside the container), and noVNC, then idles so you can docker exec into it. The default CLI config (config/cli.config.json) forces headed Chromium with Docker-friendly flags (--no-sandbox, --disable-dev-shm-usage, --disable-gpu), a 1280×800 viewport, and output to /artifacts.

⁠Configuration

⁠Ports

⁠Environment variables

⁠Volumes

Set shm_size: 1gb (or --shm-size=1g) so Chromium doesn't crash on heavy pages.

⁠Makefile

⁠Security

• This is a full browser with outbound network access. Treat what the agent browses as untrusted.
• Anyone who can reach port 6080 can watch and control the browser. VNC has no password by default. docker-compose.yml publishes 6080:6080, which binds to all host interfaces; change it to 127.0.0.1:6080:6080 on shared machines, and set NOVNC_PASSWORD or put it behind an authenticated proxy before exposing it further. noVNC is plain HTTP.
• The persistent session volume can hold cookies and logins. Delete it (docker volume rm agent-browser-sessions) when you're done with sensitive accounts.
• No secrets are baked into the image or compose file.

⁠Publishing

.github/workflows/docker-publish.yml builds multi-arch images (linux/amd64, linux/arm64) and pushes them to insready/agent-browser⁠:

To release: git tag v2.0.0 && git push origin v2.0.0.

⁠Related projects

• vercel-labs/agent-browser⁠: a different tool with the same name. It's a Rust browser-automation CLI for agents, with no container or live view.
• auto-browser⁠ and mcp-playwright-novnc⁠: similar headed-browser-with-noVNC setups, controlled over MCP instead of a CLI.
• microsoft/playwright-cli⁠: the CLI this image wraps.

⁠License

MIT. See LICENSE⁠.