run-llama/llama_index · error · ValueError

llm must start with str 'local' or of type LLM or…

Error message

llm must start with str 'local' or of type LLM or BaseLanguageModel

What it means

resolve_llm accepts either an LLM instance, a LangChain BaseLanguageModel, or a string — but strings are only valid in the form 'local[:<model_path>]' which triggers LlamaCPP loading. Any other string (a bare model name like 'gpt-4', a path, an arbitrary label) fails this check with a ValueError.

Solutions

To use OpenAI: from llama_index.llms.openai import OpenAI; Settings.llm = OpenAI(model='gpt-4o').
To use a local GGUF model: pass 'local:/path/to/model.gguf' (and install llama-index-llms-llama-cpp).
Pass an actual LLM/LangChainLLM instance instead of a string.

Example fix

# before
Settings.llm = "gpt-4o"  # ValueError
# after
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o")
# local model path
Settings.llm = "local:/models/llama-2-7b.gguf"

Defensive patterns

Strategy: type-guard

Validate before calling

from llama_index.core.llms import LLM
from llama_index.core.llms.utils import BaseLanguageModel

def is_acceptable_llm_value(llm) -> bool:
    if isinstance(llm, LLM):
        return True
    if BaseLanguageModel is not None and isinstance(llm, BaseLanguageModel):
        return True
    return isinstance(llm, str) and llm.split(":", 1)[0] == "local"

Type guard

def is_local_llm_spec(value) -> bool:
    return isinstance(value, str) and value.split(":", 1)[0] == "local"

Try / catch

try:
    resolved = resolve_llm(llm_value)
except ValueError as e:
    if "must start with str 'local'" in str(e):
        raise ValueError("Pass an LLM instance or 'local:<model_path>', not a model name") from e
    raise

Prevention

Never pass bare model-name strings as the llm value.
Construct provider LLM objects (OpenAI(...), etc.) for hosted models.
Use the 'local:<path>' form only for llama.cpp GGUF models.

When it happens

Trigger: Passing Settings.llm = 'gpt-4o' or llm='some-model-name' to a constructor that resolves LLMs; passing a filesystem path string that doesn't start with 'local:'.

Common situations: Assuming the string is a model identifier (a common expectation from other SDKs); migrating configs where llm was previously an object and got serialized to a string.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/c4ee420428f90f2b. Report an issue: GitHub.

Appendix: source

Thrown at llama-index-core/llama_index/core/llms/utils.py:78

                "`llama-index-llms-openai` package not found, "
                "please run `pip install llama-index-llms-openai`"
            )
        except ValueError as e:
            raise ValueError(
                "\n******\n"
                "Could not load OpenAI model. "
                "If you intended to use OpenAI, please check your OPENAI_API_KEY.\n"
                "Original error:\n"
                f"{e!s}"
                "\n******"
            )

    if isinstance(llm, str):
        splits = llm.split(":", 1)
        is_local = splits[0]
        model_path = splits[1] if len(splits) > 1 else None
        if is_local != "local":
            raise ValueError(
                "llm must start with str 'local' or of type LLM or BaseLanguageModel"
            )
        try:
            from llama_index.llms.llama_cpp.llama_utils import (
                completion_to_prompt,
                messages_to_prompt,
            )  # pants: no-infer-dep

            from llama_index.llms.llama_cpp import LlamaCPP  # pants: no-infer-dep

            llm = LlamaCPP(
                model_path=model_path,
                messages_to_prompt=messages_to_prompt,
                completion_to_prompt=completion_to_prompt,
                model_kwargs={"n_gpu_layers": 1},
            )
        except ImportError:
            raise ImportError(

View on GitHub (pinned to afd0fef371)