microsoft/autogen · error · ValueError

tool_choice specified but model does not support function ca

Error message

tool_choice specified but model does not support function calling

What it means

create_stream() validates tool_choice before anything else (streaming itself is not implemented). If tool_choice is anything other than 'auto' or 'none' (i.e. 'required', a Tool, or a required-mode value) and the model_info says function_calling is False, this ValueError is raised. The check exists so misconfigured streaming calls fail fast instead of silently misbehaving.

Source

Thrown at python/packages/autogen-ext/src/autogen_ext/models/llama_cpp/_llama_cpp_completion_client.py:421

        )
        return create_result

    async def create_stream(
        self,
        messages: Sequence[LLMMessage],
        *,
        tools: Sequence[Tool | ToolSchema] = [],
        tool_choice: Tool | Literal["auto", "required", "none"] = "auto",
        # None means do not override the default
        # A value means to override the client default - often specified in the constructor
        json_output: Optional[bool | type[BaseModel]] = None,
        extra_create_args: Mapping[str, Any] = {},
        cancellation_token: Optional[CancellationToken] = None,
    ) -> AsyncGenerator[Union[str, CreateResult], None]:
        # Validate tool_choice parameter even though streaming is not implemented
        if tool_choice != "auto" and tool_choice != "none":
            if not self.model_info["function_calling"]:
                raise ValueError("tool_choice specified but model does not support function calling")
            if len(tools) == 0:
                raise ValueError("tool_choice specified but no tools provided")
            logger.warning("tool_choice parameter specified but may not be supported by llama-cpp-python")

        raise NotImplementedError("Stream not yet implemented for LlamaCppChatCompletionClient")
        yield ""

    # Implement abstract methods
    def actual_usage(self) -> RequestUsage:
        return RequestUsage(
            prompt_tokens=self._total_usage.get("prompt_tokens", 0),
            completion_tokens=self._total_usage.get("completion_tokens", 0),
        )

    @property
    def capabilities(self) -> ModelInfo:
        return self.model_info

View on GitHub (pinned to 027ecf0a37)

Solutions

Set tool_choice='auto' or 'none' when the model does not support function calling
If the model genuinely supports tools, pass model_info with 'function_calling': True to the constructor (DEFAULT_MODEL_INFO has it False)
Note streaming is unimplemented anyway — use create() instead of create_stream()

Example fix

# before
client = LlamaCppChatCompletionClient(model_path="m.gguf")
async for chunk in client.create_stream(msgs, tool_choice="required"): ...

# after
client = LlamaCppChatCompletionClient(model_path="m.gguf", model_info={"function_calling": True, ...})
result = await client.create(msgs, tool_choice="required")

Defensive patterns

Strategy: validation

Validate before calling

choice = "required" if (tools and client.capabilities["function_calling"]) else "auto"
# note: create_stream raises NotImplementedError regardless; use create()
result = await client.create(messages, tools=tools, tool_choice=choice)

Type guard

def supports_forced_tools(client: LlamaCppChatCompletionClient) -> bool:
    return bool(client.capabilities.get("function_calling"))

Try / catch

try:
    ...  # call site with tool_choice
except ValueError as e:
    if "does not support function calling" in str(e):
        result = await client.create(messages, tool_choice="auto")
    else:
        raise

Prevention

Set function_calling accurately in model_info at construction
Branch tool_choice on client.capabilities['function_calling'] in shared call sites
Prefer create() over create_stream() for this client

When it happens

Trigger: Calling create_stream(messages, tools=..., tool_choice='required') on a client whose model_info (default DEFAULT_MODEL_INFO or user-supplied) has function_calling: False; passing tool_choice=some_tool with a non-tool-calling model.

Common situations: Using a shared call site that switches between create() and create_stream(); copy-pasting tool-calling parameters from a GPT-4 config into a small local llama.cpp model; forgetting to set function_calling: True in a custom model_info dict.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/913664b09ec1349f. Report an issue: GitHub.