microsoft/autogen · error · ValueError

Unexpected tool call type from LlamaCpp model.

Error message

Unexpected tool call type from LlamaCpp model.

What it means

When the response contains tool_calls, each element of that list must be a dict with 'id' and 'function'->{'arguments','name'} keys. If llama-cpp-python returns tool calls as objects (or any non-dict), the per-call isinstance check raises this ValueError. Like error 804 it indicates the installed llama-cpp-python does not match the response shape this client parses.

Source

Thrown at python/packages/autogen-ext/src/autogen_ext/models/llama_cpp/_llama_cpp_completion_client.py:353

        self._total_usage["prompt_tokens"] += response["usage"]["prompt_tokens"]
        self._total_usage["completion_tokens"] += response["usage"]["completion_tokens"]

        # Parse the response
        response_tool_calls: ChatCompletionTool | None = None
        response_text: str | None = None
        if "choices" in response and len(response["choices"]) > 0:
            if "message" in response["choices"][0]:
                response_text = response["choices"][0]["message"]["content"]
            if "tool_calls" in response["choices"][0]:
                response_tool_calls = response["choices"][0]["tool_calls"]  # type: ignore

        content: List[FunctionCall] | str = ""
        thought: str | None = None
        if response_tool_calls:
            content = []
            for tool_call in response_tool_calls:
                if not isinstance(tool_call, dict):
                    raise ValueError("Unexpected tool call type from LlamaCpp model.")
                content.append(
                    FunctionCall(
                        id=tool_call["id"],
                        arguments=tool_call["function"]["arguments"],
                        name=normalize_name(tool_call["function"]["name"]),
                    )
                )
            if response_text and len(response_text) > 0:
                thought = response_text
        else:
            if response_text:
                content = response_text

        # Detect tool usage in the response
        if not response_tool_calls and not response_text:
            logger.debug("DEBUG: No response text found. Returning empty response.")
            return CreateResult(
                content="", usage=RequestUsage(prompt_tokens=0, completion_tokens=0), finish_reason="stop", cached=False

View on GitHub (pinned to 027ecf0a37)

Solutions

  1. Align versions: reinstall a llama-cpp-python release that autogen-ext was tested with (check the package's dependency pins)
  2. If the model should not emit tool calls, drop tools from the create() call or use tool_choice='none'
  3. In tests, return tool calls as dicts: [{'id': '1', 'function': {'name': 'f', 'arguments': '{}'}}]

Example fix

# test fake fix
# before
fake.create_chat_completion.return_value = {"choices": [{"message": {"tool_calls": ["call(foo)"]}}]}
# after
fake.create_chat_completion.return_value = {"choices": [{"message": {"tool_calls": [{"id": "1", "function": {"name": "foo", "arguments": "{}"}}]}}]}
Defensive patterns

Strategy: fallback

Validate before calling

if response_tool_calls and not all(isinstance(tc, dict) for tc in response_tool_calls):
    # only possible in a patched client; re-normalize if you control the wrapper
    response_tool_calls = [tc if isinstance(tc, dict) else tc.model_dump() for tc in response_tool_calls]

Type guard

def is_dict_tool_call(tc: object) -> TypeGuard[dict]:
    return isinstance(tc, dict) and "id" in tc and isinstance(tc.get("function"), dict)

Try / catch

try:
    result = await client.create(messages, tools=tools)
except ValueError as e:
    if "Unexpected tool call type" in str(e):
        # version mismatch: retry once without tools so text output still completes
        result = await client.create(messages, tool_choice="none")
    else:
        raise

Prevention

When it happens

Trigger: A model emits tool_calls and the installed llama-cpp-python serializes them as objects/namedtuples instead of dicts; a mocked Llama returning string-encoded tool calls; partial upgrade where the wheel's chat-completion schema changed.

Common situations: Upgrading llama-cpp-python independently of autogen-ext (or vice versa); running tool-calling tests against a hand-rolled fake that returns JSON strings; GPU/CPU wheels from different builds mixed in one environment.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/1ac5e7e61b9ea7ee. Report an issue: GitHub.