run-llama/llama_index · error · NotImplementedError

get_tool_calls_from_response is not supported by default.

Error message

get_tool_calls_from_response is not supported by default.

What it means

NotImplementedError from the base FunctionCallingLLM.get_tool_calls_from_response(): the base class cannot parse tool calls out of a raw ChatResponse because the format is provider-specific. Concrete function-calling LLMs (OpenAI, Anthropic, etc.) override it; hitting this means the LLM class you are using inherited the stub.

Source

Thrown at llama-index-core/llama_index/core/llms/function_calling.py:198

    def _validate_chat_with_tools_response(
        self,
        response: ChatResponse,
        tools: Sequence["BaseTool"],
        allow_parallel_tool_calls: bool = False,
        **kwargs: Any,
    ) -> ChatResponse:
        """Validate the response from chat_with_tools."""
        return response

    def get_tool_calls_from_response(
        self,
        response: ChatResponse,
        error_on_no_tool_call: bool = True,
        **kwargs: Any,
    ) -> List[ToolSelection]:
        """Predict and call the tool."""
        raise NotImplementedError(
            "get_tool_calls_from_response is not supported by default."
        )

    def predict_and_call(
        self,
        tools: Sequence["BaseTool"],
        user_msg: Optional[Union[str, ChatMessage]] = None,
        chat_history: Optional[List[ChatMessage]] = None,
        verbose: bool = False,
        allow_parallel_tool_calls: bool = False,
        error_on_no_tool_call: bool = True,
        error_on_tool_error: bool = False,
        **kwargs: Any,
    ) -> "AgentChatResponse":
        """Predict and call the tool."""
        from llama_index.core.chat_engine.types import AgentChatResponse
        from llama_index.core.tools.calling import (
            call_tool_with_selection,

View on GitHub (pinned to afd0fef371)

Solutions

1. Use a fully implemented function-calling LLM (OpenAILLM, Anthropic, etc.) for tool-calling flows.
2. If subclassing FunctionCallingLLM, implement get_tool_calls_from_response() to extract ToolSelection objects from response.additional_kwargs / tool_calls.
3. Do not advertise function-calling support for wrappers that cannot parse tool calls.

Example fix

# before
class MyLLM(FunctionCallingLLM):  # missing get_tool_calls_from_response
    ...
resp = my_llm.chat_with_tools(tools, user_msg='hi')  # raises NotImplementedError

# after
class MyLLM(FunctionCallingLLM):
    def get_tool_calls_from_response(self, response, error_on_no_tool_call=True, **kwargs):
        tool_calls = response.message.additional_kwargs.get('tool_calls', [])
        selections = [ToolSelection.from_openai_tool_call(tc) for tc in tool_calls]
        if not selections and error_on_no_tool_call:
            raise ValueError('No tool call found')
        return selections

import inspect
from llama_index.core.llms.function_calling import FunctionCallingLLM

def supports_tool_parsing(llm) -> bool:
    return (
        isinstance(llm, FunctionCallingLLM)
        and FunctionCallingLLM.get_tool_calls_from_response
        is not type(llm).get_tool_calls_from_response
    )

import inspect
from llama_index.core.llms.function_calling import FunctionCallingLLM

def supports_tool_parsing(llm: FunctionCallingLLM) -> bool:
    return type(llm).get_tool_calls_from_response is not FunctionCallingLLM.get_tool_calls_from_response

try:
    calls = llm.get_tool_calls_from_response(resp)
except NotImplementedError:
    raise NotImplementedError(
        f'{type(llm).__name__} does not implement tool-call parsing; use an OpenAI/Anthropic-compatible LLM'
    )

When it happens

Trigger: Calling llm.get_tool_calls_from_response(response) or llm.chat_with_tools(...) / agent code paths that invoke it on an LLM whose class inherits from FunctionCallingLLM but does not implement the parser — e.g. a custom LLM wrapper marked as function-calling, or a base-class instantiation used directly.

Common situations: Custom LLM subclasses that set is_function_calling_model=True (or subclass FunctionCallingLLM) for structured-output support but never implement tool-call parsing; calling the abstract base during testing; providers whose integration only partially implements the interface.

Related errors

• Invalid
• This object node mapping does not support persist method.
• Could not extract final answer from input text: {input_text}
• Could not parse output: {output}
• 'handoff' is a reserved tool name. Please use a different na

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/7d95f1a4c1a18fc1. Report an issue: GitHub.