run-llama/llama_index · error · ValueError

llm must start with str 'local' or of type LLM or BaseLangua

Error message

llm must start with str 'local' or of type LLM or BaseLanguageModel

What it means

resolve_llm accepts either an LLM instance, a LangChain BaseLanguageModel, or a string — but strings are only valid in the form 'local[:<model_path>]' which triggers LlamaCPP loading. Any other string (a bare model name like 'gpt-4', a path, an arbitrary label) fails this check with a ValueError.

Source

Thrown at llama-index-core/llama_index/core/llms/utils.py:78

                "`llama-index-llms-openai` package not found, "
                "please run `pip install llama-index-llms-openai`"
            )
        except ValueError as e:
            raise ValueError(
                "\n******\n"
                "Could not load OpenAI model. "
                "If you intended to use OpenAI, please check your OPENAI_API_KEY.\n"
                "Original error:\n"
                f"{e!s}"
                "\n******"
            )

    if isinstance(llm, str):
        splits = llm.split(":", 1)
        is_local = splits[0]
        model_path = splits[1] if len(splits) > 1 else None
        if is_local != "local":
            raise ValueError(
                "llm must start with str 'local' or of type LLM or BaseLanguageModel"
            )
        try:
            from llama_index.llms.llama_cpp.llama_utils import (
                completion_to_prompt,
                messages_to_prompt,
            )  # pants: no-infer-dep

            from llama_index.llms.llama_cpp import LlamaCPP  # pants: no-infer-dep

            llm = LlamaCPP(
                model_path=model_path,
                messages_to_prompt=messages_to_prompt,
                completion_to_prompt=completion_to_prompt,
                model_kwargs={"n_gpu_layers": 1},
            )
        except ImportError:
            raise ImportError(

View on GitHub (pinned to afd0fef371)

Solutions

To use OpenAI: from llama_index.llms.openai import OpenAI; Settings.llm = OpenAI(model='gpt-4o').
To use a local GGUF model: pass 'local:/path/to/model.gguf' (and install llama-index-llms-llama-cpp).
Pass an actual LLM/LangChainLLM instance instead of a string.

Example fix

# before
Settings.llm = "gpt-4o"  # ValueError
# after
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o")
# local model path
Settings.llm = "local:/models/llama-2-7b.gguf"

Defensive patterns

Strategy: type-guard

Validate before calling

from llama_index.core.llms import LLM
from llama_index.core.llms.utils import BaseLanguageModel

def is_acceptable_llm_value(llm) -> bool:
    if isinstance(llm, LLM):
        return True
    if BaseLanguageModel is not None and isinstance(llm, BaseLanguageModel):
        return True
    return isinstance(llm, str) and llm.split(":", 1)[0] == "local"

Type guard

def is_local_llm_spec(value) -> bool:
    return isinstance(value, str) and value.split(":", 1)[0] == "local"

Try / catch

try:
    resolved = resolve_llm(llm_value)
except ValueError as e:
    if "must start with str 'local'" in str(e):
        raise ValueError("Pass an LLM instance or 'local:<model_path>', not a model name") from e
    raise

Prevention

Never pass bare model-name strings as the llm value.
Construct provider LLM objects (OpenAI(...), etc.) for hosted models.
Use the 'local:<path>' form only for llama.cpp GGUF models.

When it happens

Trigger: Passing Settings.llm = 'gpt-4o' or llm='some-model-name' to a constructor that resolves LLMs; passing a filesystem path string that doesn't start with 'local:'.

Common situations: Assuming the string is a model identifier (a common expectation from other SDKs); migrating configs where llm was previously an object and got serialized to a string.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/c4ee420428f90f2b. Report an issue: GitHub.