Claude Computer Use API Guide: New Toolset, Models, Docker Demo, Python Loop and Costs

Claude’s computer use tool lets the model see a screen through screenshots and control it with mouse clicks and key presses. You run the actions on your own machine or container, and Claude decides what to do next.

The tool got a big update with the new computer_toolset_20260801 version. This guide covers the current version, supported models, the Docker demo, a working Python loop, safety and real token costs.

How Computer Use Works

Computer use is a client-side tool. Claude never touches your machine directly. It sends back actions, and your code carries them out.

The loop goes like this:

1. You send a task: for example, “Save a picture of a cat to my desktop”, plus the computer toolset.
2. Claude asks for a screenshot: it returns a tool_use block named screenshot.
3. Your code runs it: you capture the screen and send the image back as a tool_result.
4. Claude picks the next step: a click, some typing or a key press. Your code runs those too.
5. Repeat until done: when Claude stops asking for tools, the task is complete.

Note: Because your code runs every action, screenshots and actions stay in your environment. Anthropic lists computer use as eligible for zero data retention and HIPAA.

Tool Versions and Supported Models

There are three versions in Anthropic’s docs right now. The newest is the one to build on.

Platform notes

• Claude API: full support for the current toolset.
• Amazon Bedrock: the docs say Opus 5.5, Sonnet 5.5 and Haiku 5.5 also accept computer_20251124 with its beta header there.
• Claude Managed Agents: the new toolset is not supported yet.
• SDKs: the Python and TypeScript SDKs mark the toolset as beta, even though the API needs no header.

Model availability differs a little per cloud, so check the availability list in the docs for your platform before you ship.

What Actions Claude Can Take

In the new toolset, each action is its own member tool. A tool_use block has name set to the member and toolset_name set to "computer". There is no action field anymore.

Claude can also return several actions in one turn. You must run them in order and stop at the first failure.

Try the Reference Docker Demo

Anthropic’s computer-use-demo in the claude-quickstarts repo is the fastest way to see this working. It packs a Linux desktop (Xvfb, Mutter, Tint2), apps, an agent loop and a web UI into one container.

1. Install Docker. Any recent Docker Desktop or Docker Engine works.
2. Get an API key. Create one in the Claude Console and export it as ANTHROPIC_API_KEY in your shell.
3. Run the container. Use the command below. It maps four ports and saves settings in ~/.anthropic.
4. Open the UI. Go to http://localhost:8080 for the chat plus desktop view.
5. Give it a task. Start small, like “open the browser and search for today’s weather”.

docker run \
    -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
    -v $HOME/.anthropic:/home/computeruse/.anthropic \
    -p 5900:5900 \
    -p 8501:8501 \
    -p 6080:6080 \
    -p 8080:8080 \
    -it ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest

What each port does

• 8080: combined chat and desktop view.
• 8501: the Streamlit chat only.
• 6080: the desktop only, at /vnc.html.
• 5900: raw VNC for a VNC client.

Tip: Set WIDTH and HEIGHT to change the screen size. The README suggests staying at XGA (1024×768) or close to it, since big screens hurt accuracy and speed.

The README also covers Bedrock (mount ~/.aws and set API_PROVIDER=bedrock) and Vertex (set API_PROVIDER=vertex with your project and region). It warns that the demo serves one session at a time and is not built for production.

Build Your Own Python Agent Loop

Here is a small, complete loop based on the patterns in Anthropic’s docs. It uses pyautogui to control the screen, so run it only inside a VM or container you do not mind Claude using.

Step 1: Install

pip install anthropic pyautogui pillow
export ANTHROPIC_API_KEY="your-key-here"

Step 2: The action handler

import base64, io, math, time
import anthropic
import pyautogui

client = anthropic.Anthropic()
MODEL = "claude-opus-5-5"

# Screenshots must fit the API limit (2,576 px long edge, ~3.75 MP)
SCREEN_W, SCREEN_H = pyautogui.size()
SCALE = min(1.0, 2576 / max(SCREEN_W, SCREEN_H),
            math.sqrt(3_750_000 / (SCREEN_W * SCREEN_H)))

def to_screen(coord):
    x, y = coord
    return int(x / SCALE), int(y / SCALE)

def screenshot():
    img = pyautogui.screenshot()
    if SCALE < 1:
        img = img.resize((int(img.width * SCALE), int(img.height * SCALE)))
    buf = io.BytesIO()
    img.save(buf, format="PNG")
    data = base64.b64encode(buf.getvalue()).decode()
    return [{"type": "image",
             "source": {"type": "base64", "media_type": "image/png", "data": data}}]

def handle_computer_action(name, inp):
    if name == "screenshot":
        return screenshot()
    if name in ("left_click", "right_click", "double_click"):
        button = "right" if name == "right_click" else "left"
        clicks = 2 if name == "double_click" else 1
        if "coordinate" in inp:
            pyautogui.click(*to_screen(inp["coordinate"]), clicks=clicks, button=button)
        else:
            pyautogui.click(clicks=clicks, button=button)
        return "done"
    if name == "mouse_move":
        pyautogui.moveTo(*to_screen(inp["coordinate"]))
        return "done"
    if name == "type":
        pyautogui.write(inp["text"], interval=0.02)
        return "done"
    if name == "key":
        keys = inp["text"].lower().split("+")
        for _ in range(inp.get("repeat", 1)):
            pyautogui.hotkey(*keys)
        return "done"
    if name == "scroll":
        if "coordinate" in inp:
            pyautogui.moveTo(*to_screen(inp["coordinate"]))
        amount = inp["scroll_amount"]
        pyautogui.scroll(amount if inp["scroll_direction"] == "up" else -amount)
        return "done"
    if name == "wait":
        time.sleep(inp.get("duration", 1))
        return "done"
    raise ValueError(f"Unsupported member: {name}")

Step 3: The loop

TOOLS = [{"type": "computer_toolset_20260801",
          "configs": {"zoom": {"enabled": False}}}]
HALT = "Not executed: an earlier computer action in this turn failed."

def process_tool_calls(response):
    results, failed = [], False
    for block in response.content:
        if block.type != "tool_use" or block.toolset_name != "computer":
            continue
        result = {"type": "tool_result", "tool_use_id": block.id,
                  "toolset_name": "computer"}
        if failed:
            result["content"], result["is_error"] = HALT, True
        else:
            try:
                result["content"] = handle_computer_action(block.name, block.input)
            except Exception as err:
                result["content"], result["is_error"] = f"Error: {err}", True
                failed = True
        results.append(result)
    return results

def run(task, max_iterations=15):
    messages = [{"role": "user", "content": task}]
    for _ in range(max_iterations):
        response = client.messages.create(
            model=MODEL, max_tokens=4096, messages=messages, tools=TOOLS)
        messages.append({"role": "assistant", "content": response.content})
        results = process_tool_calls(response)
        if not results:
            return response
        messages.append({"role": "user", "content": results})

run("Open the text editor and type 'Hello from Claude'.")

Warning: This handler covers the common members only. Claude may also call members like triple_click or left_click_drag. Either add handlers or turn those members off in configs, or the run will fail with an error result.

Key names can also differ. Claude may send names like Return or Page_Down, which you may need to map to your library’s names. For a deeper look at the SDK itself, see our Claude API Python tutorial.

Screenshot Size and Coordinate Scaling

This is the part most first builds get wrong. The new toolset does not downscale images. An oversized screenshot is rejected with a validation error.

• Current models: up to 2,576 px on the long edge and about 3.75 megapixels (4,784 visual tokens).
• Older models: up to 1,568 px on the long edge and about 1.15 megapixels.
• Coordinates: Claude replies in the scaled image’s pixels, so divide by your scale factor before clicking.

Smaller screens also mean cheaper screenshots and better click accuracy. That is why the demo suggests a 1024×768 desktop.

Pricing and Token Overhead

Computer use is billed like normal tool use. Screenshots count as image input, and tool results count as input tokens.

Model prices

Per Anthropic’s pricing page, per million tokens:

• Opus 5.5: $4 input, $20 output.
• Sonnet 5.5: $2 input, $10 output.
• Haiku 5.5: $0.10 input, $0.50 output for prompts up to 100K tokens.

As plain arithmetic, a request holding the toolset plus 10 screenshots in history is roughly 20,000 input tokens. On Sonnet 5.5 that is about $0.04, and it grows every step as history piles up.

Tip: Add cache_control to the toolset entry and use prompt caching for the stable part of the conversation. Our prompt caching guide and Claude API pricing guide show the savings.

Safety Checklist

An agent that clicks around a real computer can do real damage. Anthropic’s own list of precautions is short and worth following exactly:

• Isolate it: use a dedicated VM or container with minimal privileges.
• No secrets: do not give it access to login credentials or sensitive data.
• Limit the internet: allowlist only the domains the task needs.
• Human in the loop: confirm money moves, cookie banners and terms acceptance by hand.
• Prompt injection: web pages and images can carry hidden instructions. Anthropic runs classifiers on screenshots, but you should still isolate the agent.
• Tell your users: explain the risks and get consent before turning it on in a product.

Known limits

• Speed: it can be slower than a human doing the same task.
• Clicks: Claude can misjudge coordinates; ask it to take a screenshot after each step to check.
• Scrolling: if scroll does nothing, Page Down often works better.
• Complex apps: spreadsheets and niche apps may need several tries.

If you just want Claude to use your own desktop apps without coding, our Claude desktop control guide covers the consumer feature.

Migrating from computer_20251124

If you already have an older integration, these are the main changes from Anthropic’s nine-step migration list:

1. Drop the beta header and call the standard Messages client.
2. Change the tool entry to {"type": "computer_toolset_20260801"} and remove name, display_width_px, display_height_px, display_number and enable_zoom.
3. Decide on zoom: it is on by default now.
4. Handle every block in a turn and dispatch on name plus toolset_name.
5. Stop at the first failure and answer the rest with the halt text.
6. Echo toolset_name on every tool_result.
7. Support repeat on the key member.
8. Resize screenshots yourself.
9. Move defer_loading into configs.

Troubleshooting: Common Problems and Fixes

Validation error on the screenshot

The image is too big. Resize it so the long edge is at most 2,576 px and the total stays near 3.75 MP, or 1,568 px and 1.15 MP on older models.

Clicks land in the wrong place

You resized the screenshot but did not scale the coordinates back. Divide Claude’s x and y by your scale factor before clicking.

Error about unknown parameters in the tool entry

You kept old fields like display_width_px or name. The new toolset rejects them, so remove them.

Claude repeats the same action

Your tool result may be empty or missing the screenshot. Return an image after actions that change the screen, or let Claude call screenshot again.

Old code with input[“action”] breaks

The new toolset has no action field. Read block.name instead, and check block.toolset_name == "computer".

Frequently Asked Questions

What is the latest Claude computer use tool version?

It is computer_toolset_20260801. It has 17 member tools and needs no beta header on the API.

Which Claude models support computer use?

The current toolset works with Fable 5 and 5.1, Mythos 5 and 5.1, Opus 5, Opus 5.5, Opus 4.8, Sonnet 5, Sonnet 5.5 and Haiku 5.5. Older models such as Sonnet 4.6 and Haiku 4.5 use earlier versions with a beta header.

How much does computer use cost?

There is no separate fee. You pay normal token prices, plus about 4,500 tokens for the toolset and about 1,000 to 1,800 tokens per screenshot.

Does Claude run actions on my computer by itself?

No. Claude only returns the actions it wants. Your code carries them out, so you control the environment and can block anything.

Can I use computer use with Amazon Bedrock or Google Cloud?

Yes, computer use is listed for Bedrock, Google Cloud, Microsoft Foundry and Claude Platform on AWS. Supported models and versions vary a little per platform.

Is the Docker demo safe for production?

No. Anthropic describes it as a demo with weakly separated parts that handles one session at a time. Use it to learn, then build your own hardened setup.

Can I use computer use in Claude Managed Agents?

Not yet. The docs say the new toolset is not currently available in Claude Managed Agents. Use the Messages API with your own loop, or see our Claude Agent SDK guide.

Sources

• Claude docs: Computer use tool
• anthropics/claude-quickstarts on GitHub
• computer-use-demo README
• Claude API pricing
• Simon Willison: Initial explorations of Anthropic’s computer use