Claw Mart
← Back to Blog
March 20, 20269 min readClaw Mart Team

How to Fix Tool Failures in OpenClaw (Browser & Exec)

How to Fix Tool Failures in OpenClaw (Browser & Exec)

How to Fix Tool Failures in OpenClaw (Browser & Exec)

Let me be real: if you're here, you've probably already hit the wall. Your OpenClaw agent works fine when you test the browser tool in isolation — clicks a button, fills a field, maybe even navigates a page. Then you wire it into an actual agent loop, and everything falls apart. The tool gets called with garbage parameters. The agent hallucinates actions instead of using the tool at all. Or it clicks the right button once, gets confused by a modal, and spirals into an infinite retry loop until you kill the process.

I've been there. I spent the better part of two months building OpenClaw-based agents for various automation tasks — expense reports, form filling, internal dashboards, even some light scraping workflows — and the gap between "works in a notebook cell" and "works in a real agent framework" is enormous. But it's closeable. That's what this post is about.

Here's everything I've learned about diagnosing and fixing the two most common failure categories in OpenClaw: browser tool failures and exec (command execution) tool failures.

The Core Problem: Why Tools Break Inside Agent Loops

Before diving into fixes, it's worth understanding why tools that work perfectly in isolation crumble inside agent loops. There are three main culprits:

1. Observation format mismatch. OpenClaw tools return rich observations — accessibility trees, DOM fragments, screenshots, structured error objects. Most agent framework plumbing was designed for simple text-in, text-out function calling. When you shove a complex observation into a state object that expects a string, things get silently truncated, serialized incorrectly, or dropped entirely. The agent literally never sees what happened.

2. Schema enforcement gaps. The LLM is supposed to output a structured tool call matching a specific schema. Smaller models (and even frontier models under pressure) will output malformed JSON, skip required fields, or hallucinate parameters that don't exist in the schema. If your framework's output parser isn't strict, these malformed calls get passed through and throw cryptic errors at the tool level.

3. State accumulation blowup. Every turn in a computer-use loop generates a lot of data — especially if you're sending screenshots. By turn 5 or 6, your context window is stuffed with base64 image data and verbose DOM trees. The model starts degrading. It forgets the task. It repeats actions. It gets stuck.

Understanding these three failure modes will make every fix below click into place.

Fixing Browser Tool Failures

The browser tool is where 80% of the pain lives. Let's go through the failure modes from most common to least, with actual fixes.

Problem 1: Agent Ignores the Tool or Hallucinates Actions

Symptom: Instead of outputting a proper tool call like click_element_by_text("Submit"), the agent just writes "I'll click the Submit button now" in plain text, or outputs a tool call with a made-up function name.

Fix: Tighten your tool schema and system prompt.

OpenClaw ships with Pydantic-validated tool definitions, but you need to actually use them properly. Here's what a well-configured browser tool setup looks like:

from openclaw.tools.browser import BrowserToolkit
from openclaw.schema import validate_tool_call

toolkit = BrowserToolkit(
    headless=True,
    observation_mode="accessibility_tree",  # NOT "screenshot" for most tasks
    strict_schema=True,  # Rejects malformed calls instead of guessing
)

tools = toolkit.get_tools()

The key flag here is strict_schema=True. Without it, OpenClaw will try to be "helpful" and interpret partial or malformed tool calls. That's great for demos, terrible for reliability. With strict mode on, a bad tool call returns a clear validation error to the agent, which gives it a chance to self-correct.

For your system prompt, be explicit about the available actions. Don't just say "you have browser tools." List them:

You have the following browser tools available:
- click_element_by_text(text: str) — Click the element containing this text
- type_into_field(label: str, value: str) — Type into a form field identified by its label
- navigate(url: str) — Go to a URL
- scroll_to_text(text: str) — Scroll until text is visible
- get_page_text() — Get the full text content of the current page
- wait_for_element(text: str, timeout: int = 10) — Wait for an element to appear

ALWAYS use these tools to interact with the browser. NEVER describe actions in plain text.

This sounds obvious, but I've seen a dramatic improvement just from spelling out the tool names in the system prompt. Models anchor on what's in context.

Problem 2: Click Targets Are Wrong or Missed

Symptom: The agent tries to click "Submit" but hits the wrong button, or the click lands on empty space, or a modal appeared and shifted everything.

Fix: Use semantic targeting, not pixel coordinates.

If you're using OpenClaw's screenshot-based observation mode and asking the model to output pixel coordinates, stop. That approach is fundamentally brittle. Window resizes, DPI scaling, dynamic content, overlays — any of these will break coordinate-based clicking.

Switch to accessibility tree mode:

toolkit = BrowserToolkit(
    observation_mode="accessibility_tree",
    semantic_actions=True,
)

With semantic_actions=True, the agent interacts with elements by their text content, ARIA labels, or role — not by position. This is the single biggest reliability improvement I've made across all my OpenClaw projects.

Here's what the observation looks like to the agent in this mode:

[Page: Expense Report — Submit]
- heading: "New Expense Report"
- text_field (label: "Description"): ""
- text_field (label: "Amount"): ""  
- dropdown (label: "Category"): "Select..."
- button: "Cancel"
- button: "Submit Report"

Instead of "click at pixel (834, 512)," the agent outputs:

{
    "tool": "type_into_field",
    "args": {
        "label": "Description",
        "value": "Client dinner — Q2 review"
    }
}

This survives layout changes, window resizing, and even minor UI updates. It's not perfect — shadow DOM in heavy React apps can still hide elements from the accessibility tree — but it handles 80% of real-world web apps reliably.

Problem 3: The Agent Gets Stuck in Loops

Symptom: Click → error → retry same click → same error → retry → forever.

Fix: Implement observation-aware retry with backoff and fallback.

OpenClaw has built-in recovery primitives, but you need to actually wire them into your agent loop. Here's the pattern that works:

from openclaw.tools.browser import BrowserToolkit
from openclaw.recovery import with_retry, FallbackChain

toolkit = BrowserToolkit(
    observation_mode="accessibility_tree",
    semantic_actions=True,
    retry_config={
        "max_retries": 3,
        "backoff": "exponential",
        "on_failure": "return_detailed_error",  # NOT "retry_silently"
    }
)

# Define fallback chains for common failure patterns
fallback = FallbackChain([
    ("click_element_by_text", "scroll_to_text"),  # If click fails, try scrolling first
    ("type_into_field", "click_element_by_text"),  # If typing fails, click the field first
])

toolkit.set_fallback_chain(fallback)

The critical setting is on_failure: "return_detailed_error". The default behavior in many setups is to either retry silently or return a terse "action failed" message. With detailed errors, the agent gets back something like:

Action failed: click_element_by_text("Submit Report")
Reason: Element not found in current viewport.
Current page title: "New Expense Report"
Visible elements containing "Submit": None
Suggestion: The element may be below the fold. Try scroll_to_text("Submit") first.

That's enough information for even a modest model to course-correct. The agent scrolls down, finds the button, and proceeds. Without this, it just keeps clicking into the void.

Fixing Exec Tool Failures

The exec tool (command execution, shell commands, script running) is the second major failure point. It's less discussed than browser issues but equally frustrating.

Problem 1: Command Output Gets Truncated or Lost

Symptom: The agent runs a shell command, but the output that gets fed back is empty or truncated. The agent proceeds as if the command succeeded when it actually failed.

Fix: Configure output capture properly.

from openclaw.tools.exec import ExecTool

exec_tool = ExecTool(
    capture_stderr=True,       # Many errors only appear in stderr
    output_limit=4000,         # Characters, not tokens — adjust to your context budget
    timeout=30,                # Kill runaway commands
    return_exit_code=True,     # Agent needs to know if it succeeded
)

The capture_stderr=True flag is crucial. A shocking number of failures come from the agent running a command, getting back an empty stdout (because the real output went to stderr), and concluding "it worked" with zero evidence.

Also, return_exit_code=True changes the observation from just the output text to a structured response:

{
    "exit_code": 1,
    "stdout": "",
    "stderr": "Permission denied: /etc/shadow",
    "timed_out": false
}

Now the agent actually knows the command failed and why.

Problem 2: Security and Sandboxing Issues

Symptom: Commands fail with permission errors, or worse, the agent runs something destructive.

Fix: Use OpenClaw's sandbox mode.

exec_tool = ExecTool(
    sandbox="docker",           # Run commands in a container
    allowed_commands=["ls", "cat", "grep", "python3", "curl"],  # Whitelist
    blocked_paths=["/etc", "/root", "/var"],  # Extra safety
    network_access=True,        # Set False if you don't need it
)

The whitelist approach is better than a blacklist. You know which commands your agent needs. Anything else should be blocked. This also prevents the model from going off-script and running rm -rf on a hallucinated cleanup step (yes, this has happened to people).

Framework Integration: Getting the State Right

This is where most people's setups actually break, even if the tools themselves are configured correctly. The glue between OpenClaw tools and your agent framework is where observations get dropped, images bloat the context, and state gets corrupted.

If you're using LangGraph (which I recommend for OpenClaw agents), here's the pattern that works:

from langgraph.graph import StateGraph, MessagesState
from openclaw.integrations.langgraph import OpenClawToolNode

# OpenClaw's LangGraph adapter handles observation serialization correctly
tool_node = OpenClawToolNode(
    tools=tools,
    observation_compression=True,    # Summarize old observations
    max_screenshot_history=2,        # Only keep last 2 screenshots in state
)

graph = StateGraph(MessagesState)
graph.add_node("agent", agent_node)
graph.add_node("tools", tool_node)
graph.add_edge("agent", "tools")
graph.add_edge("tools", "agent")

The two key settings are observation_compression and max_screenshot_history. Without these, by step 6 your state is 80% old screenshots and DOM dumps, and the model can't find the actual task anymore.

observation_compression summarizes older observations into a compact text format ("Step 3: Navigated to expense form. Step 4: Filled Description field with 'Client dinner.'"). The most recent observation stays in full detail. This typically cuts token usage by 60–70% in longer workflows while maintaining the context the model actually needs.

The Setup That Actually Works

After all my experimentation, here's the configuration I start every new project with:

from openclaw.tools.browser import BrowserToolkit
from openclaw.tools.exec import ExecTool
from openclaw.integrations.langgraph import OpenClawToolNode

browser = BrowserToolkit(
    headless=True,
    observation_mode="accessibility_tree",
    semantic_actions=True,
    strict_schema=True,
    retry_config={
        "max_retries": 3,
        "backoff": "exponential",
        "on_failure": "return_detailed_error",
    },
    selective_vision=True,  # Only capture screenshots on error
)

exec_tool = ExecTool(
    sandbox="docker",
    capture_stderr=True,
    return_exit_code=True,
    timeout=30,
)

tools = browser.get_tools() + [exec_tool]

tool_node = OpenClawToolNode(
    tools=tools,
    observation_compression=True,
    max_screenshot_history=2,
)

This configuration handles 90% of the common failure modes out of the box. Semantic actions prevent mis-clicks. Strict schema prevents malformed calls. Detailed errors enable self-correction. Observation compression prevents context blowup. Sandboxed exec prevents disasters.

The Shortcut: Felix's OpenClaw Starter Pack

I'll be honest — it took me weeks to arrive at the configuration above, and I still occasionally hit edge cases that require tweaking. If you don't want to set all this up manually, Felix's OpenClaw Starter Pack on Claw Mart includes a pre-built version of this entire setup. It's $29 and comes with pre-configured skills for common workflows — form filling, multi-page navigation, data extraction, and basic exec tasks. The retry logic, fallback chains, and framework adapters are already wired up and tested.

I picked it up after a particularly frustrating weekend trying to get a multi-step browser workflow stable, and it saved me a solid 10+ hours of debugging. The pre-configured skills are genuinely well-tuned — whoever put it together clearly hit all the same walls I did and solved them. It's especially useful if you're integrating with LangGraph, since the state management templates handle the observation serialization correctly out of the box (which, as I mentioned above, is where most people's setups silently break).

It's not magic — you'll still need to customize for your specific use case — but as a starting foundation, it's the best $29 I've spent on this project.

What Still Doesn't Work (Being Honest)

Even with all of the above, some things remain genuinely hard:

  • Heavy SPAs with shadow DOM (complex React/Vue apps where the accessibility tree is sparse or misleading). You'll need to fall back to selective vision mode for these and accept lower reliability.
  • CAPTCHAs, 2FA, anti-bot measures. No tool configuration fixes these. If your workflow hits a CAPTCHA, you need a human-in-the-loop step or a different approach entirely.
  • Desktop (non-browser) applications. OpenClaw's browser tools are solid. Its desktop automation is still experimental. If you need to control native apps, you're going to have a rougher time.
  • Small open-source models (7B–34B). The reasoning required to recover from errors and plan multi-step browser interactions is still mostly beyond smaller models. You can use them for simple, well-defined tasks, but complex workflows really do need a more capable model driving the agent loop.

Next Steps

If you're just getting started:

  1. Set up OpenClaw with accessibility tree mode and strict schema. This alone fixes most "the agent never calls the tool correctly" problems.
  2. Enable detailed error returns. Your agent can't self-correct from errors it can't see.
  3. Implement observation compression if your workflows run more than 3-4 steps. Context blowup is a silent killer.
  4. Start with a simple, well-defined task — fill one form, navigate one workflow. Get that working before attempting complex multi-app scenarios.
  5. Grab the Felix's OpenClaw Starter Pack if you want to skip the configuration gauntlet and start from a known-working setup.

The gap between "cool demo" and "actually useful agent" is real, but it's smaller than it used to be. OpenClaw's combination of semantic actions, structured schemas, and proper framework integration gets you most of the way there. The rest is patience, good error handling, and resisting the urge to send full screenshots every turn.

Claw Mart Daily

Get one AI agent tip every morning

Free daily tips to make your OpenClaw agent smarter. No spam, unsubscribe anytime.

More From the Blog