{"record":{"id":"4e3a728282f26b03","repo":"huggingface/transformers","slug":"unknown-modality-for-model-classname","errorCode":null,"errorMessage":"Unknown modality for: {model_classname}","messagePattern":"Unknown modality for: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cli/serving/model_manager.py","lineNumber":444,"sourceCode":"        \"\"\"\n        if processor is not None and isinstance(processor, PreTrainedTokenizerBase):\n            return Modality.LLM\n\n        from transformers.models.auto.modeling_auto import (\n            MODEL_FOR_CAUSAL_LM_MAPPING_NAMES,\n            MODEL_FOR_IMAGE_TEXT_TO_TEXT_MAPPING_NAMES,\n            MODEL_FOR_MULTIMODAL_LM_MAPPING_NAMES,\n        )\n\n        model_classname = model.__class__.__name__\n        if model_classname in MODEL_FOR_MULTIMODAL_LM_MAPPING_NAMES.values():\n            return Modality.MULTIMODAL\n        elif model_classname in MODEL_FOR_IMAGE_TEXT_TO_TEXT_MAPPING_NAMES.values():\n            return Modality.VLM\n        elif model_classname in MODEL_FOR_CAUSAL_LM_MAPPING_NAMES.values():\n            return Modality.LLM\n        else:\n            raise ValueError(f\"Unknown modality for: {model_classname}\")\n\n    @staticmethod\n    @lru_cache\n    def get_gen_models(cache_dir: str | None = None) -> list[dict]:\n        \"\"\"List generative models (LLMs and VLMs) available in the HuggingFace cache.\n\n        Args:\n            cache_dir (`str`, *optional*): Path to the HuggingFace cache directory.\n                Defaults to the standard cache location.\n\n        Returns:\n            `list[dict]`: OpenAI-compatible model list entries with ``id``, ``object``, etc.\n        \"\"\"\n        from transformers.models.auto.modeling_auto import (\n            MODEL_FOR_CAUSAL_LM_MAPPING_NAMES,\n            MODEL_FOR_IMAGE_TEXT_TO_TEXT_MAPPING_NAMES,\n            MODEL_FOR_MULTIMODAL_LM_MAPPING_NAMES,\n        )","sourceCodeStart":426,"sourceCodeEnd":462,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cli/serving/model_manager.py#L426-L462","documentation":"ModelManager.get_model_modality classifies a loaded model by matching its class name against the auto-mappings MODEL_FOR_MULTIMODAL_LM / MODEL_FOR_IMAGE_TEXT_TO_TEXT / MODEL_FOR_CAUSAL_LM. If the model's class is registered in none of them (custom architecture, non-generative model, or a class not in the mappings' values), it raises ValueError 'Unknown modality'.","triggerScenarios":"Loading a custom PreTrainedModel subclass (e.g. your own architecture) and hitting any endpoint that needs modality routing; a generative model class missing from the three mappings (e.g. seq2seq or audio models); a model registered only under MODEL_FOR_SEQ2SEQ or MODEL_FOR_SPEECH mappings.","commonSituations":"Serving a custom fine-tuned architecture; serving encoder-decoder (seq2seq) models the serve CLI does not route; brand-new model classes before their mapping lands in your transformers version.","solutions":["Serve a model class that is registered in the causal-LM / image-text-to-text / multimodal auto mappings","For custom classes, register them in the mapping or wrap/serv them with your own FastAPI app","Upgrade transformers so newer model classes appear in the mappings","Check membership first: any(model.__class__.__name__ in m.values() for m in (MODEL_FOR_CAUSAL_LM_MAPPING_NAMES, ...))"],"exampleFix":"# before: custom class\nserve --model_id ./my-custom-arch  # ValueError: Unknown modality\n\n# after: serve a mapped architecture\nserve --model_id meta-llama/Llama-3.1-8B-Instruct","handlingStrategy":"type-guard","validationCode":"from transformers.models.auto.modeling_auto import (\n    MODEL_FOR_CAUSAL_LM_MAPPING_NAMES,\n    MODEL_FOR_IMAGE_TEXT_TO_TEXT_MAPPING_NAMES,\n    MODEL_FOR_MULTIMODAL_LM_MAPPING_NAMES,\n)\n\nclassname = type(model).__name__\nknown = any(\n    classname in m.values()\n    for m in (MODEL_FOR_CAUSAL_LM_MAPPING_NAMES, MODEL_FOR_IMAGE_TEXT_TO_TEXT_MAPPING_NAMES, MODEL_FOR_MULTIMODAL_LM_MAPPING_NAMES)\n)\nif not known:\n    print(\"Model class not routable by serve; use a mapped architecture\")","typeGuard":"def is_servable_class(model) -> bool:\n    name = model.__class__.__name__\n    return any(\n        name in m.values()\n        for m in (\n            MODEL_FOR_CAUSAL_LM_MAPPING_NAMES,\n            MODEL_FOR_IMAGE_TEXT_TO_TEXT_MAPPING_NAMES,\n            MODEL_FOR_MULTIMODAL_LM_MAPPING_NAMES,\n        )\n    )","tryCatchPattern":"try:\n    modality = manager.get_model_modality(model, processor=processor)\nexcept ValueError as e:\n    if \"Unknown modality\" in str(e):\n        raise SystemExit(\"Serve a causal-LM/VLM/multimodal model or register your class\") from e\n    raise","preventionTips":["Serve standard causal-LM or VLM checkpoints via the CLI","Check AutoModelForCausalLM.from_pretrained works on the checkpoint before serving","Register custom classes in the auto mappings or write a custom server app"],"tags":["serving","model-registry","modality","valueerror","custom-model"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}