{"record":{"id":"dee30ad0efafd5a2","repo":"huggingface/transformers","slug":"decompose-multimodal-found-no-multi-modal-submodul","errorCode":null,"errorMessage":"decompose_multimodal found no multi-modal submodules on {type(model).__name__}. Expected an image/audio encoder + language model, found neither.","messagePattern":"decompose_multimodal found no multi-modal submodules on (.+?)\\. Expected an image/audio encoder \\+ language model, found neither\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/exporters/utils.py","lineNumber":871,"sourceCode":"    Detects all known multi-modal submodules by attribute name (vision tower, projector,\n    language model, lm_head, …) and captures their forward kwargs during one\n    `model(**inputs)` call.\n\n    Each submodule is returned as a separate `name: (module, inputs)` entry for\n    independent export. The token-merge step (e.g. `masked_scatter` for multi-modal models)\n    is intentionally left outside the exported graphs — it is the caller's responsibility\n    to assemble `inputs_embeds` from the encoder outputs before running the decoder.\n\n    Returns:\n        `dict[str, tuple[torch.nn.Module, dict]]`: One `name: (module, inputs)`\n        entry per detected submodule (image/audio encoder, projector, language model, lm_head).\n\n    Raises:\n        `ValueError`: if no known multi-modal submodules are found on the model.\n    \"\"\"\n    submodules = _find_multimodal_submodules(model)\n    if not submodules:\n        raise ValueError(\n            f\"decompose_multimodal found no multi-modal submodules on {type(model).__name__}. \"\n            f\"Expected an image/audio encoder + language model, found neither.\"\n        )\n\n    try:\n        with contextlib.ExitStack() as stack, torch.no_grad():\n            submodule_inputs = {\n                name: stack.enter_context(_capture_forward(module)) for name, module in submodules.items()\n            }\n            model(**copy.deepcopy(inputs))\n    except Exception as e:\n        raise RuntimeError(\n            f\"decompose_multimodal failed for {type(model).__name__}. Inputs passed: {list(inputs.keys())}.\"\n        ) from e\n\n    return {\n        name: (module, submodule_inputs[name][-1])\n        for name, module in submodules.items()","sourceCodeStart":853,"sourceCodeEnd":889,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/exporters/utils.py#L853-L889","documentation":"decompose_multimodal splits a multimodal model into separate exportable submodules (vision/audio encoder, projector, language model, lm_head) discovered via _find_multimodal_submodules. If that discovery finds none of the expected submodule names/attributes, the model is not recognizable as multimodal and the decomposition raises ValueError.","triggerScenarios":"Calling decompose_multimodal (directly or via an export API) on a unimodal model (text-only LM, plain ViT) or a multimodal model whose submodule names are not among those _find_multimodal_submodules recognizes.","commonSituations":"Passing the wrong model class to a multimodal export flow; custom multimodal architectures with non-standard attribute names; new models not yet allow-listed by the detection helper.","solutions":["Verify you actually passed a multimodal model (e.g. AutoModel for a VLM/ASR config, not the text backbone)","If the model is custom, rename/attach the expected submodules (encoder/projector/language model) so discovery finds them","For unimodal export, use the regular export path instead of the multimodal decomposition","Check _find_multimodal_submodules in the installed version for the recognized names"],"exampleFix":"// before\ndecompose_multimodal(text_backbone, inputs)\n\n// after\nvlm = AutoModel.from_pretrained(\"some/vlm-checkpoint\")\ndecompose_multimodal(vlm, processor_inputs)","handlingStrategy":"type-guard","validationCode":"from transformers.exporters.utils import _find_multimodal_submodules\n\ndef has_multimodal_parts(model) -> bool:\n    return bool(_find_multimodal_submodules(model))","typeGuard":"def is_multimodal_exportable(model) -> bool:\n    subs = _find_multimodal_submodules(model)\n    return any(k in subs for k in (\"vision_encoder\", \"audio_encoder\", \"language_model\"))","tryCatchPattern":"try:\n    decompose_multimodal(model, inputs)\nexcept ValueError as e:\n    if \"no multi-modal submodules\" in str(e):\n        logger.info(\"falling back to unimodal export\")\n        export_model(model, inputs)\n    else:\n        raise","preventionTips":["Load models through AutoModel with the full multimodal checkpoint","Verify submodule discovery before wiring the export pipeline","Keep custom architecture names aligned with the recognized submodule set"],"tags":["export","multimodal","model-structure","transformers"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}