huggingface/transformers · error · RuntimeError

decompose_multimodal failed for {type(model).name}. Inpu

Error message

decompose_multimodal failed for {type(model).__name__}. Inputs passed: {list(inputs.keys())}.

What it means

After multimodal submodules are found, decompose_multimodal runs a full model(**inputs) pass with forward-capture hooks attached to each submodule. If that forward pass raises for any reason (bad inputs, device/dtype mismatch, hook incompatibility), the utility wraps it in this RuntimeError naming the model class and input keys. The root cause is preserved as __cause__.

Source

Thrown at src/transformers/exporters/utils.py:883

    Raises:
        `ValueError`: if no known multi-modal submodules are found on the model.
    """
    submodules = _find_multimodal_submodules(model)
    if not submodules:
        raise ValueError(
            f"decompose_multimodal found no multi-modal submodules on {type(model).__name__}. "
            f"Expected an image/audio encoder + language model, found neither."
        )

    try:
        with contextlib.ExitStack() as stack, torch.no_grad():
            submodule_inputs = {
                name: stack.enter_context(_capture_forward(module)) for name, module in submodules.items()
            }
            model(**copy.deepcopy(inputs))
    except Exception as e:
        raise RuntimeError(
            f"decompose_multimodal failed for {type(model).__name__}. Inputs passed: {list(inputs.keys())}."
        ) from e

    return {
        name: (module, submodule_inputs[name][-1])
        for name, module in submodules.items()
        if submodule_inputs[name]  # skip submodules not called (e.g. lm_head on base models)
    }


def decompose_for_generation(
    model: PreTrainedModel, inputs: dict[str, Any], generation_config: Any = None, multi_token_decode: bool = False
) -> dict[str, tuple[torch.nn.Module, dict]]:
    """Decompose a generative model into independently exportable `(model, forward_inputs)` pairs.

    Runs `decompose_prefill_decode` to capture prefill and decode forward kwargs from a real
    `model.generate(**inputs, max_new_tokens=2)`. If the prefill is multi-modal (per `is_multimodal`),
    further splits it into one entry per submodule (vision/audio encoder, projector, language model,

View on GitHub (pinned to a597f97485)

Solutions

Call model(**inputs) directly (outside export) and inspect the chained exception to find the real failure
Supply complete inputs for every modality via the model's processor (images/audio + text)
Ensure tensors have the expected dtype/device and required keys are present
Re-run the decomposition once the plain forward works

Example fix

// before
decompose_multimodal(vlm, {"input_ids": ids})

// after
batch = processor(images=imgs, text=prompts, return_tensors="pt")
decompose_multimodal(vlm, batch)

Defensive patterns

Strategy: try-catch

Validate before calling

def forward_ok(model, inputs) -> bool:
    try:
        with torch.no_grad():
            model(**inputs)
        return True
    except Exception:
        return False

Try / catch

try:
    decompose_multimodal(model, inputs)
except RuntimeError as e:
    raise RuntimeError(f"multimodal decomposition failed: {e.__cause__!r}") from e.__cause__

Prevention

Run one plain forward with the exact export inputs first
Provide inputs for every modality via the processor
Keep dtype/device uniform between inputs and model

When it happens

Trigger: Calling the multimodal export decomposition with inputs the full model rejects: missing modality inputs (no pixel_values/audio), wrong shapes, numpy instead of tensors, or CPU/GPU device mismatches.

Common situations: Export scripts feeding only the text branch to a VLM; raw unprocessed media passed instead of processor outputs; models with forward-time validation that fires on dummy inputs.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/ea81f5891fe3b52f. Report an issue: GitHub.