microsoft/autogen · error · ValueError

Unknown tool '{name}'. Please choose from: {tool_names}

Error message

Unknown tool '{name}'. Please choose from:

{tool_names}

What it means

In MultimodalWebSurfer's action dispatch (visit_page, click, input_text, scroll_down/up, scroll_element_up/down, hover, sleep, ...), any tool name not matching a known branch falls through to ValueError listing the valid tool_names. In practice this error is produced by the LLM's own tool call, not the application: the model hallucinated a tool name or used one from a different agent/prompt version.

Source

Thrown at python/packages/autogen-ext/src/autogen_ext/agents/web_surfer/_multimodal_web_surfer.py:768

            # Summarize the DOM. No need to take further action. Browser state does not change.
            action_description = "I summarized the current web page"
            return await self._summarize_page(cancellation_token=cancellation_token)

        elif name == "hover":
            target_id = str(args.get("target_id"))
            target_name = self._target_name(target_id, rects)
            if target_name:
                action_description = f"I hovered over '{target_name}'."
            else:
                action_description = "I hovered over the control."
            await self._playwright_controller.hover_id(self._page, target_id)

        elif name == "sleep":
            action_description = "I am waiting a short period of time before taking further action."
            await self._playwright_controller.sleep(self._page, 3)

        else:
            raise ValueError(f"Unknown tool '{name}'. Please choose from:\n\n{tool_names}")

        await self._page.wait_for_load_state()
        await self._playwright_controller.sleep(self._page, 3)

        # Handle downloads
        if self._last_download is not None and self.downloads_folder is not None:
            fname = os.path.join(self.downloads_folder, self._last_download.suggested_filename)
            await self._last_download.save_as(fname)  # type: ignore
            page_body = f"<html><head><title>Download Successful</title></head><body style=\"margin: 20px;\"><h1>Successfully downloaded '{self._last_download.suggested_filename}' to local path:<br><br>{fname}</h1></body></html>"
            await self._page.goto(
                "data:text/html;base64," + base64.b64encode(page_body.encode("utf-8")).decode("utf-8")
            )
            await self._page.wait_for_load_state()

        # Handle metadata
        page_metadata = json.dumps(await self._playwright_controller.get_page_metadata(self._page), indent=4)
        metadata_hash = hashlib.md5(page_metadata.encode("utf-8")).hexdigest()
        if metadata_hash != self._prior_metadata_hash:

View on GitHub (pinned to 027ecf0a37)

Solutions

  1. Update/repair the system prompt or tool schema so only supported tool names are advertised
  2. Catch ValueError from the agent's message handling and feed the error text back to the model so it retries with a valid name
  3. Use a stronger function-calling model for the web surfer
  4. Ensure the tool call is routed to the agent whose schema produced it

Example fix

// before
result = await surfer.on_messages([TextMessage(content='go', source='user')], cts)

// after
try:
    result = await surfer.on_messages(msgs, cts)
except ValueError as e:
    # echo valid tool names back so the model self-corrects
    msgs.append(TextMessage(content=f'Invalid action: {e}. Retry with a listed tool.', source='user'))
Defensive patterns

Strategy: retry

Validate before calling

KNOWN_TOOLS = {'visit_page','click','input_text','scroll_down','scroll_up','scroll_element_up','scroll_element_down','hover','sleep','read_page','answer_question'}
def valid_tool_name(name: str) -> bool:
    return name in KNOWN_TOOLS

Try / catch

try:
    result = await surfer.on_messages(messages, token)
except ValueError as e:
    if 'Unknown tool' in str(e):
        messages = messages + [TextMessage(content=f'That tool does not exist. {e}. Pick again.', source='user')]
        result = await surfer.on_messages(messages, token)  # model self-corrects
    else:
        raise

Prevention

When it happens

Trigger: The model emits a function call named e.g. 'navigate', 'type', or 'screenshot' that is not in the web surfer's tool list, or an application forwards a tool call intended for another agent into this agent's handler.

Common situations: Prompt changes where the advertised tool list drifted from the dispatcher, small models with weak tool-name adherence, team setups where tool calls get routed to the wrong agent's handle_request/on_messages.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/ad07146bb88fdd8d. Report an issue: GitHub.