huggingface/transformers · error · ValueError

decompose_multimodal found no multi-modal submodules on {typ

Error message

decompose_multimodal found no multi-modal submodules on {type(model).__name__}. Expected an image/audio encoder + language model, found neither.

What it means

decompose_multimodal splits a multimodal model into separate exportable submodules (vision/audio encoder, projector, language model, lm_head) discovered via _find_multimodal_submodules. If that discovery finds none of the expected submodule names/attributes, the model is not recognizable as multimodal and the decomposition raises ValueError.

Source

Thrown at src/transformers/exporters/utils.py:871

    Detects all known multi-modal submodules by attribute name (vision tower, projector,
    language model, lm_head, …) and captures their forward kwargs during one
    `model(**inputs)` call.

    Each submodule is returned as a separate `name: (module, inputs)` entry for
    independent export. The token-merge step (e.g. `masked_scatter` for multi-modal models)
    is intentionally left outside the exported graphs — it is the caller's responsibility
    to assemble `inputs_embeds` from the encoder outputs before running the decoder.

    Returns:
        `dict[str, tuple[torch.nn.Module, dict]]`: One `name: (module, inputs)`
        entry per detected submodule (image/audio encoder, projector, language model, lm_head).

    Raises:
        `ValueError`: if no known multi-modal submodules are found on the model.
    """
    submodules = _find_multimodal_submodules(model)
    if not submodules:
        raise ValueError(
            f"decompose_multimodal found no multi-modal submodules on {type(model).__name__}. "
            f"Expected an image/audio encoder + language model, found neither."
        )

    try:
        with contextlib.ExitStack() as stack, torch.no_grad():
            submodule_inputs = {
                name: stack.enter_context(_capture_forward(module)) for name, module in submodules.items()
            }
            model(**copy.deepcopy(inputs))
    except Exception as e:
        raise RuntimeError(
            f"decompose_multimodal failed for {type(model).__name__}. Inputs passed: {list(inputs.keys())}."
        ) from e

    return {
        name: (module, submodule_inputs[name][-1])
        for name, module in submodules.items()

View on GitHub (pinned to a597f97485)

Solutions

1. Verify you actually passed a multimodal model (e.g. AutoModel for a VLM/ASR config, not the text backbone)
2. If the model is custom, rename/attach the expected submodules (encoder/projector/language model) so discovery finds them
3. For unimodal export, use the regular export path instead of the multimodal decomposition
4. Check _find_multimodal_submodules in the installed version for the recognized names

Example fix

// before
decompose_multimodal(text_backbone, inputs)

// after
vlm = AutoModel.from_pretrained("some/vlm-checkpoint")
decompose_multimodal(vlm, processor_inputs)

from transformers.exporters.utils import _find_multimodal_submodules

def has_multimodal_parts(model) -> bool:
    return bool(_find_multimodal_submodules(model))

def is_multimodal_exportable(model) -> bool:
    subs = _find_multimodal_submodules(model)
    return any(k in subs for k in ("vision_encoder", "audio_encoder", "language_model"))

try:
    decompose_multimodal(model, inputs)
except ValueError as e:
    if "no multi-modal submodules" in str(e):
        logger.info("falling back to unimodal export")
        export_model(model, inputs)
    else:
        raise

When it happens

Trigger: Calling decompose_multimodal (directly or via an export API) on a unimodal model (text-only LM, plain ViT) or a multimodal model whose submodule names are not among those _find_multimodal_submodules recognizes.

Common situations: Passing the wrong model class to a multimodal export flow; custom multimodal architectures with non-standard attribute names; new models not yet allow-listed by the detection helper.

Related errors

• decompose_multimodal failed for {type(model).__name__}. Inpu
• Found 'model.config.return_loss=True'. Loss computation is n
• Found 'return_loss=True' in inputs. Loss computation is not
• decompose_prefill_decode failed for {type(model).__name__}.
• decompose_prefill_decode expected at least {num_new_tokens}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/dee30ad0efafd5a2. Report an issue: GitHub.