invoke-ai/InvokeAI · error · TypeError

Expected LlavaOnevisionForConditionalGeneration, got {type(m

Error message

Expected LlavaOnevisionForConditionalGeneration, got {type(model).__name__}

What it means

_run_image_to_prompt loads a LLaVA OneVision model via the model manager and, once it is resident on the device, asserts the loaded object is a transformers LlavaOnevisionForConditionalGeneration. If InvokeAI's loader returned some other module object (different model type, stub, or class from an incompatible transformers version), the isinstance check fails and this TypeError is raised. It guards the downstream LlavaOnevisionPipeline, which can only accept that exact class.

Source

Thrown at invokeai/app/api/routers/utilities.py:266

    events = ApiDependencies.invoker.services.events
    model_config = model_manager.store.get_model(model_key)

    if model_config.type != ModelType.LlavaOnevision:
        raise ValueError(f"Model '{model_key}' is not a LLaVA OneVision model (got {model_config.type})")

    if task_id is not None:
        events.emit_llm_task_progress(task_id=task_id, user_id=user_id, phase="loading_model", message="Loading model")

    with _model_load_lock:
        loaded_model = model_manager.load.load_model(model_config, user_id=user_id)

    # Load the image from InvokeAI's image store
    image = ApiDependencies.invoker.services.images.get_pil_image(image_name)
    image = image.convert("RGB")

    with torch.no_grad(), loaded_model.model_on_device() as (_, model):
        if not isinstance(model, LlavaOnevisionForConditionalGeneration):
            raise TypeError(f"Expected LlavaOnevisionForConditionalGeneration, got {type(model).__name__}")

        model_abs_path = _resolve_model_path(model_config.path)
        processor = AutoProcessor.from_pretrained(model_abs_path, local_files_only=True)
        if not isinstance(processor, LlavaOnevisionProcessor):
            raise TypeError(f"Expected LlavaOnevisionProcessor, got {type(processor).__name__}")

        pipeline = LlavaOnevisionPipeline(model, processor)
        model_device = next(model.parameters()).device

        progress_callback = _make_progress_callback(events, task_id, user_id)

        output = pipeline.run(
            prompt=instruction,
            images=[image],
            device=model_device,
            dtype=TorchDevice.choose_torch_dtype(),
            progress_callback=progress_callback,
        )

View on GitHub (pinned to 0b6a024f2f)

Solutions

Upgrade/align the installed transformers version so LlavaOnevisionForConditionalGeneration imports from the same package InvokeAI's loader instantiates
Verify the model record (model_key) actually points to a LLaVA OneVision checkpoint and its config is not corrupted
Check for duplicate transformers installs or a vendored copy shadowing the real package (pip show transformers, sys.path order)
Inspect the loader path (model_manager.load.load_model / model_on_device) to confirm what object it returns

Example fix

# before
with torch.no_grad(), loaded_model.model_on_device() as (_, model):
    pipeline = LlavaOnevisionPipeline(model, processor)
# after
with torch.no_grad(), loaded_model.model_on_device() as (_, model):
    if not isinstance(model, LlavaOnevisionForConditionalGeneration):
        raise TypeError(f"Expected LlavaOnevisionForConditionalGeneration, got {type(model).__name__}")
    pipeline = LlavaOnevisionPipeline(model, processor)

Defensive patterns

Strategy: type-guard

Validate before calling

import transformers
from transformers import LlavaOnevisionForConditionalGeneration
assert hasattr(transformers, "LlavaOnevisionForConditionalGeneration"), "upgrade transformers >= 4.45"

Type guard

def is_llava_onevision_model(model) -> bool:
    from transformers import LlavaOnevisionForConditionalGeneration
    return isinstance(model, LlavaOnevisionForConditionalGeneration)

Try / catch

try:
    prompt = requests.post(f"{base}/utilities/image_to_prompt", json={...}).raise_for_status().json()["prompt"]
except requests.HTTPError as e:
    if e.response.status_code == 422 and "Expected LlavaOnevisionForConditionalGeneration" in e.response.text:
        fix_model_install()

Prevention

Pin the transformers version InvokeAI was tested against
Validate the checkpoint is a genuine llava-onevision download (config.json architectures field)
Watch for duplicate transformers installs in the environment

When it happens

Trigger: POST /utilities/image_to_prompt with a model_key whose loaded model is not a LlavaOnevisionForConditionalGeneration — e.g. the model manager load path returned a raw module wrapper, the installed transformers version lacks/moved LlavaOnevisionForConditionalGeneration, or the model_config was mutated so a non-LLaVA checkpoint is loaded under a LlavaOnevision config.

Common situations: transformers version drift (LlavaOnevision classes only exist in transformers >= 4.45), custom model-manager loaders returning BaseModelClass instances instead of the raw HF module, checkpoints mislabeled as LlavaOnevision in the model record, or monkeypatched test doubles.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/7aec4eceb404bbb5. Report an issue: GitHub.