zylon-ai/private-gpt · error · ValueError

Configured model does not support function calling

Error message

Configured model does not support function calling

What it means

ValueError raised at the very start of AsyncChatEngine._initialize_run: the LLM resolved for request.system.model via the LLM component is not an instance of FunctionCallingLLM. The engine is built around tool calling, so a completion-only or non-function-calling wrapper LLM is rejected before any context stack or sampling params are processed.

Source

Thrown at private_gpt/components/engines/chat/async_chat_engine.py:1610

            handler.emit(result_start)
            handler.emit(RawContentBlockStopEvent.from_start(result_start))

        return _ToolExecutionResult(status=_ToolExecutionStatus.EXECUTED)

    # ------------------------------------------------------------------
    # Initialization and interceptor phases
    # ------------------------------------------------------------------

    def _initialize_run(
        self,
        request: ChatRequest,
        context_stack: ContextStack | None = None,
        hooks: ExecutionHooks | None = None,
        original_input: ChatInputState | None = None,
    ) -> _Run:
        llm = self._llm_component.get_llm(request.system.model)
        if not isinstance(llm, FunctionCallingLLM):
            raise ValueError("Configured model does not support function calling")

        if not isinstance(request, ResolvedChatRequest) and context_stack is None:
            raise ValueError("Configured context stack is required")

        llm_kwargs = ChatLLMParameters.model_validate(request.sampling_params)
        if request.thinking.enabled and request.thinking.type:
            llm_kwargs = llm_kwargs.model_copy(
                update={
                    "reasoning_effort": ReasoningEffort.from_str(request.thinking.type)
                }
            )
        if request.response_format and request.response_format.output_cls:
            structured = StructuredOutputsParams.from_optional(
                output_cls=request.response_format.output_cls,
            )
            if structured is not None:
                llm_kwargs = llm_kwargs.model_copy(
                    update={"structured_outputs": structured}

View on GitHub (pinned to 4a030776a3)

Solutions

1. Configure/use a model that supports function calling (one whose LLM instance is a FunctionCallingLLM)
2. If wrapping an LLM, subclass FunctionCallingLLM (llama_index.core.llms) and implement the required methods
3. Route non-tool models through a chat path that does not require function calling, if available
4. For tests, use a mock LLM that extends FunctionCallingLLM

Example fix

# before
class MyLLM(CustomLLM): ...  # rejected: not a FunctionCallingLLM

# after
from llama_index.core.llms import FunctionCallingLLM
class MyLLM(FunctionCallingLLM):
    ...

llm = llm_component.get_llm(request.system.model)
if not isinstance(llm, FunctionCallingLLM):
    raise HTTPException(400, f'model {request.system.model!r} does not support function calling')

from llama_index.core.llms import FunctionCallingLLM

def supports_function_calling(llm) -> TypeGuard[FunctionCallingLLM]:
    return isinstance(llm, FunctionCallingLLM)

try:
    run = engine._initialize_run(request, context_stack)
except ValueError as e:
    if 'function calling' in str(e):
        return error_response(400, 'selected model lacks tool support')
    raise

When it happens

Trigger: Sending a chat request whose system.model resolves to a non-function-calling LLM (e.g. a basic completion model, a mock, or a custom LLM class not subclassing FunctionCallingLLM) into the async chat engine path.

Common situations: Switching the configured model to one without tool support to cut costs; a custom LLM wrapper that forgets to inherit from FunctionCallingLLM; test mocks using a generic LLM class; provider integration that returns the base LLM class for certain model ids.

Related errors

• INVALID_REQUEST_ERROR
• Audio blocks found but no audio-capable LLM provided.
• Expected tool calls in response but found none. Message: {re
• Invalid reasoning_effort budget: {budget}. Must be a number
• LLM mode '{mode}' is not supported. Available: {available}

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/1f3aca25b1753474. Report an issue: GitHub.