JuliusBrussee/caveman · error · TypeError

Expected an existing native LlamaIndex LLM

Error message

Expected an existing native LlamaIndex LLM

What it means

Raised in CavemanLLM.__init__: the object passed as the model to wrap is not an existing native LlamaIndex LLM instance. CavemanLLM is a pure delegate that forwards public provider methods to a real native LLM, so it must be constructed with one.

Solutions

1. Instantiate the provider LLM first, e.g. OpenAI(model="gpt-4o"), and pass that object
2. Verify the object is an instance of llama_index.core.llms.LLM before wrapping
3. Ensure a single compatible llama-index-core installation is in the environment

Example fix

// before
llm = caveman_llm("gpt-4o", runtime=rt, scope=scope)
// after
from llama_index.llms.openai import OpenAI
llm = caveman_llm(OpenAI(model="gpt-4o"), runtime=rt, scope=scope)

from llama_index.core.llms import LLM
if not isinstance(wrapped, LLM):
    raise TypeError("pass an instantiated llama_index LLM, not a model name")

from llama_index.core.llms import LLM
def is_llama_llm(obj) -> bool:
    return isinstance(obj, LLM)

try:
    wrapped_llm = caveman_llm(provider_llm, runtime=rt, scope=scope)
except TypeError as e:
    log.error("wrap requires an LLM instance: %s", e)
    raise

When it happens

Trigger: Calling the wrapper with a model identifier string instead of an instantiated LLM, with None, or with an object from a different/incompatible LlamaIndex installation.

Common situations: Migrating from other frameworks where you pass a model name, forgetting to instantiate the provider class, or having multiple llama-index packages where LLM classes diverge.

Related errors

• Expected a native Strands Model
• Expected a native ToolSelection
• Expected an AutoGen ChatCompletionClient
• Expected an AutoGen Workbench or list of workbenches
• Expected an existing native Pydantic AI Model

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/b6c2a3bcf38ac27e. Report an issue: GitHub.

Appendix: source

Thrown at packages/middleware/python/caveman_middleware/llama_index.py:352

            return await self.iterator.aclose()


class CavemanLLM(FunctionCallingLLM):
    """Delegate to an existing native LLM. Model-only calls are recovery-free.

    Public provider methods retain native serialization, retries, callbacks,
    response objects, and tool parsing. This class never runs a tool loop.
    """
    wrapped: LLM = Field(exclude=True)
    runtime: Any = Field(exclude=True)
    scope: Any = Field(exclude=True)
    protocol: str | None = Field(exclude=True)
    passthrough_reason: str = Field(exclude=True)
    registration: Any = Field(default=None, exclude=True)

    def __init__(self, wrapped: LLM, *, runtime, scope, registration=None):
        if not isinstance(wrapped, LLM):
            raise TypeError("Expected an existing native LlamaIndex LLM")
        # Copy public prompt settings so inherited predict/structured helpers
        # build exactly the same native input as the caller's LLM.
        settings = {name: getattr(wrapped, name) for name in LLM.model_fields}
        supported = _check_version(runtime)
        protocol = _protocol(wrapped, runtime) if supported else None
        known_provider = (type(wrapped).__module__, type(wrapped).__name__) in {
            ("llama_index.llms.openai.base", "OpenAI"), ("llama_index.llms.anthropic.base", "Anthropic")}
        super().__init__(wrapped=wrapped, runtime=runtime, scope=scope, protocol=protocol,
                         passthrough_reason="unsupported_version" if not supported or (known_provider and protocol is None) else "unsupported_provider",
                         registration=registration, **settings)

    @property
    def metadata(self):
        return self.wrapped.metadata

    def _passive(self, reason):
        if owner.get() is not None:
            return None

View on GitHub (pinned to 3ee70a1026)