{"record":{"id":"1f3aca25b1753474","repo":"zylon-ai/private-gpt","slug":"configured-model-does-not-support-function-calling","errorCode":null,"errorMessage":"Configured model does not support function calling","messagePattern":"Configured model does not support function calling","errorType":"exception","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"private_gpt/components/engines/chat/async_chat_engine.py","lineNumber":1610,"sourceCode":"            handler.emit(result_start)\n            handler.emit(RawContentBlockStopEvent.from_start(result_start))\n\n        return _ToolExecutionResult(status=_ToolExecutionStatus.EXECUTED)\n\n    # ------------------------------------------------------------------\n    # Initialization and interceptor phases\n    # ------------------------------------------------------------------\n\n    def _initialize_run(\n        self,\n        request: ChatRequest,\n        context_stack: ContextStack | None = None,\n        hooks: ExecutionHooks | None = None,\n        original_input: ChatInputState | None = None,\n    ) -> _Run:\n        llm = self._llm_component.get_llm(request.system.model)\n        if not isinstance(llm, FunctionCallingLLM):\n            raise ValueError(\"Configured model does not support function calling\")\n\n        if not isinstance(request, ResolvedChatRequest) and context_stack is None:\n            raise ValueError(\"Configured context stack is required\")\n\n        llm_kwargs = ChatLLMParameters.model_validate(request.sampling_params)\n        if request.thinking.enabled and request.thinking.type:\n            llm_kwargs = llm_kwargs.model_copy(\n                update={\n                    \"reasoning_effort\": ReasoningEffort.from_str(request.thinking.type)\n                }\n            )\n        if request.response_format and request.response_format.output_cls:\n            structured = StructuredOutputsParams.from_optional(\n                output_cls=request.response_format.output_cls,\n            )\n            if structured is not None:\n                llm_kwargs = llm_kwargs.model_copy(\n                    update={\"structured_outputs\": structured}","sourceCodeStart":1592,"sourceCodeEnd":1628,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/engines/chat/async_chat_engine.py#L1592-L1628","documentation":"ValueError raised at the very start of AsyncChatEngine._initialize_run: the LLM resolved for request.system.model via the LLM component is not an instance of FunctionCallingLLM. The engine is built around tool calling, so a completion-only or non-function-calling wrapper LLM is rejected before any context stack or sampling params are processed.","triggerScenarios":"Sending a chat request whose system.model resolves to a non-function-calling LLM (e.g. a basic completion model, a mock, or a custom LLM class not subclassing FunctionCallingLLM) into the async chat engine path.","commonSituations":"Switching the configured model to one without tool support to cut costs; a custom LLM wrapper that forgets to inherit from FunctionCallingLLM; test mocks using a generic LLM class; provider integration that returns the base LLM class for certain model ids.","solutions":["Configure/use a model that supports function calling (one whose LLM instance is a FunctionCallingLLM)","If wrapping an LLM, subclass FunctionCallingLLM (llama_index.core.llms) and implement the required methods","Route non-tool models through a chat path that does not require function calling, if available","For tests, use a mock LLM that extends FunctionCallingLLM"],"exampleFix":"# before\nclass MyLLM(CustomLLM): ...  # rejected: not a FunctionCallingLLM\n\n# after\nfrom llama_index.core.llms import FunctionCallingLLM\nclass MyLLM(FunctionCallingLLM):\n    ...","handlingStrategy":"type-guard","validationCode":"llm = llm_component.get_llm(request.system.model)\nif not isinstance(llm, FunctionCallingLLM):\n    raise HTTPException(400, f'model {request.system.model!r} does not support function calling')","typeGuard":"from llama_index.core.llms import FunctionCallingLLM\n\ndef supports_function_calling(llm) -> TypeGuard[FunctionCallingLLM]:\n    return isinstance(llm, FunctionCallingLLM)","tryCatchPattern":"try:\n    run = engine._initialize_run(request, context_stack)\nexcept ValueError as e:\n    if 'function calling' in str(e):\n        return error_response(400, 'selected model lacks tool support')\n    raise","preventionTips":["Whitelist function-calling-capable models in the model picker exposed to users","Make custom LLM wrappers subclass FunctionCallingLLM","Validate the resolved LLM type in a pre-flight check before starting the run"],"tags":["chat-engine","llm","function-calling","configuration"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}