invoke-ai/InvokeAI · error · ValueError

Model '{model_key}' is not a LLaVA OneVision model (got {mod

Error message

Model '{model_key}' is not a LLaVA OneVision model (got {model_config.type})

What it means

ValueError raised by _run_image_to_prompt when the model for model_key exists but its type is not ModelType.LlavaOnevision — the image-to-prompt (captioning) path only accepts LLaVA OneVision models. Surfaced as HTTP 422 by the endpoint's ValueError handler.

Source

Thrown at invokeai/app/api/routers/utilities.py:252

class ImageToPromptResponse(BaseModel):
    prompt: str
    error: str | None = None


def _run_image_to_prompt(
    image_name: str,
    model_key: str,
    instruction: str,
    task_id: str | None,
    user_id: str,
) -> str:
    """Run LLaVA OneVision inference synchronously (called from thread)."""
    model_manager = ApiDependencies.invoker.services.model_manager
    events = ApiDependencies.invoker.services.events
    model_config = model_manager.store.get_model(model_key)

    if model_config.type != ModelType.LlavaOnevision:
        raise ValueError(f"Model '{model_key}' is not a LLaVA OneVision model (got {model_config.type})")

    if task_id is not None:
        events.emit_llm_task_progress(task_id=task_id, user_id=user_id, phase="loading_model", message="Loading model")

    with _model_load_lock:
        loaded_model = model_manager.load.load_model(model_config, user_id=user_id)

    # Load the image from InvokeAI's image store
    image = ApiDependencies.invoker.services.images.get_pil_image(image_name)
    image = image.convert("RGB")

    with torch.no_grad(), loaded_model.model_on_device() as (_, model):
        if not isinstance(model, LlavaOnevisionForConditionalGeneration):
            raise TypeError(f"Expected LlavaOnevisionForConditionalGeneration, got {type(model).__name__}")

        model_abs_path = _resolve_model_path(model_config.path)
        processor = AutoProcessor.from_pretrained(model_abs_path, local_files_only=True)
        if not isinstance(processor, LlavaOnevisionProcessor):

View on GitHub (pinned to 0b6a024f2f)

Solutions

Select a model_key whose type is LlavaOnevision (check the models listing)
Install a LLaVA OneVision model if none is available
Fix the configured default model_key for the image-to-prompt feature
Upgrade InvokeAI if your LLaVA variant is not classified as LlavaOnevision

Example fix

// before
image_to_prompt(model_key="text_llm:my-llm-model")
// after
image_to_prompt(model_key="llava_onevision:llava-onevision-qwen2-7b")

Defensive patterns

Strategy: type-guard

Validate before calling

const models = await api.listModels();
const cfg = models.find(m => m.key === modelKey);
if (!cfg || cfg.type !== 'llava_onevision') {
  throw new Error(`image-to-prompt requires a LlavaOnevision model (got ${cfg ? cfg.type : 'unknown'})`);
}

Type guard

function isLlavaOnevisionModel(config) {
  return config != null && config.type === 'llava_onevision' && typeof config.key === 'string';
}

Try / catch

try {
  await api.imageToPrompt({ modelKey, image });
} catch (e) {
  if (e.status === 422 && /not a LLaVA OneVision model/.test(e.detail)) {
    console.error('Pick a LlavaOnevision model for captioning');
  } else throw e;
}

Prevention

Filter captioning UI to models with type llava_onevision
Validate configured default captioning model type at startup
Install a LLaVA OneVision model before enabling the image-to-prompt feature

When it happens

Trigger: POST image-to-prompt with a model_key resolving to a Main, TextLLM, or other non-LlavaOnevision model config.

Common situations: Pointing the captioning feature at a text LLM thinking any LLM works; UI not filtering to LlavaOnevision models; installed model family differs (e.g. Llava 1.5 older variant) so the type string doesn't match.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/6b991192ce658af0. Report an issue: GitHub.