langchain-ai/langchain · error · ValueError

run_name must be a list of the same length as prompts

Error message

run_name must be a list of the same length as prompts

What it means

Raised by BaseLLM.generate when run_name is supplied with per-prompt callbacks but is not a list whose length equals the number of prompts. Each prompt needs its own run name entry when per-prompt callback configs are used.

Source

Thrown at libs/core/langchain_core/language_models/llms.py:954

            # We've received a list of callbacks args to apply to each input
            if len(callbacks) != len(prompts):
                msg = "callbacks must be the same length as prompts"
                raise ValueError(msg)
            if tags is not None and not (
                isinstance(tags, list) and len(tags) == len(prompts)
            ):
                msg = "tags must be a list of the same length as prompts"
                raise ValueError(msg)
            if metadata is not None and not (
                isinstance(metadata, list) and len(metadata) == len(prompts)
            ):
                msg = "metadata must be a list of the same length as prompts"
                raise ValueError(msg)
            if run_name is not None and not (
                isinstance(run_name, list) and len(run_name) == len(prompts)
            ):
                msg = "run_name must be a list of the same length as prompts"
                raise ValueError(msg)
            tags_list = cast("list[list[str] | None]", tags or ([None] * len(prompts)))
            metadata_list = cast(
                "list[builtins.dict[str, Any] | None]",
                metadata or ([{}] * len(prompts)),
            )
            run_name_list = run_name or cast(
                "list[str | None]", ([None] * len(prompts))
            )
            params = self._dict_for_compat()
            params["stop"] = stop
            callback_managers = [
                CallbackManager.configure(
                    callback,
                    self.callbacks,
                    self.verbose,
                    tag,
                    self.tags,
                    meta,

View on GitHub (pinned to e32fa9a52e)

Solutions

1. Provide one run name per prompt: run_name=['run-1', 'run-2', ...] matching len(prompts)
2. Or pass run_name=None and let LangChain auto-generate run names
3. Or use shared callbacks instead of per-prompt callbacks if a single run_name is desired

Example fix

# before
llm.generate(prompts, callbacks=per_prompt_cbs, run_name='batch-run')
# after
llm.generate(prompts, callbacks=per_prompt_cbs, run_name=[f'batch-run-{i}' for i in range(len(prompts))])

n = len(prompts)
if run_name is not None:
    assert isinstance(run_name, list) and len(run_name) == n, f'run_name must be a list of {n} strings'

def valid_batch_run_name(rn: object, n: int) -> bool:
    return rn is None or (isinstance(rn, list) and len(rn) == n and all(isinstance(x, str) for x in rn))

try:
    llm.generate(prompts, callbacks=cbs, run_name=run_name)
except ValueError as e:
    if 'run_name must be a list' in str(e) and isinstance(run_name, str):
        llm.generate(prompts, callbacks=cbs, run_name=[run_name] * len(prompts))
    else:
        raise

When it happens

Trigger: llm.generate(prompts, callbacks=[[cb], [cb]], run_name='my-run') with more than one prompt; or run_name=['a', 'b'] for three prompts. Triggered only when callbacks is a per-prompt list.

Common situations: Naming LangSmith runs for batched requests and passing a single string run_name out of habit from invoke(); partial lists after editing the prompt batch.

Related errors

• tags must be a list of the same length as prompts
• metadata must be a list of the same length as prompts
• Number of manually provided run_id's does not match batch le
• Argument 'prompts' is expected to be of type list[str], rece
• invalid IP address

AI-assisted analysis of langchain-ai/langchain@e32fa9a52e (2026-08-14). Data as JSON: /api/errors/4fe56aab9b4a821b. Report an issue: GitHub.