{"record":{"id":"0896554bf03f648e","repo":"sgl-project/sglang","slug":"unknown-multimodal-item-type-type-item","errorCode":null,"errorMessage":"Unknown multimodal item type: {type(item)}","messagePattern":"Unknown multimodal item type: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/processors/base_processor.py","lineNumber":1685,"sourceCode":"            input_ids = tokenizer(\n                base_output.input_text,\n                return_tensors=\"pt\",\n                add_special_tokens=True,\n            ).input_ids.flatten()\n            return [], input_ids, {}\n\n        dict_items, raw_images, raw_audios, raw_videos = [], [], [], []\n        for modality, item in all_loaded_data:\n            if isinstance(item, dict):\n                dict_items.append((modality, item))\n            elif modality == Modality.IMAGE:\n                raw_images.append(item)\n            elif modality == Modality.AUDIO:\n                raw_audios.append(item)\n            elif modality == Modality.VIDEO:\n                raw_videos.append(item)\n            else:\n                raise ValueError(f\"Unknown multimodal item type: {type(item)}\")\n        # Process items and get input_ids\n        all_collected_items: list[MultimodalDataItem] = []\n        input_ids = None\n        # Handle raw items (need processing)\n        if raw_images or raw_audios or raw_videos:\n            if processor_override is not None:\n                kwargs[\"processor\"] = processor\n            collected_items, input_ids, ret = self._process_and_collect_mm_items(\n                input_text=base_output.input_text,\n                images=raw_images,\n                audios=raw_audios,\n                videos=raw_videos,\n                **kwargs,\n            )\n            all_collected_items = collected_items\n\n            # When SGLANG_MM_AVOID_RETOKENIZE is on, keep the user's exact tokens to avoid retokenize drift.\n            # Drift happens when Retokenization is not identity: Decode(X) => String => Re-tokenize => Y, X != Y.","sourceCodeStart":1667,"sourceCodeEnd":1703,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/processors/base_processor.py#L1667-L1703","documentation":"process_and_combine_mm_data sorts incoming items by modality; an item whose modality is not IMAGE, AUDIO, or VIDEO falls through to this ValueError. It guards against new/unknown modality enum values reaching the combine stage.","triggerScenarios":"Passing an item whose Modality enum value is something other than IMAGE/AUDIO/VIDEO (e.g. a future or custom modality) into process_and_combine_mm_data.","commonSituations":"Extending Modality with a new kind without updating this router; mixed-content dicts misclassified upstream; version skew between components enumerating modalities differently.","solutions":["Filter items to Modality.IMAGE/AUDIO/VIDEO before calling, or route other modalities through their dedicated path","If adding a new modality, extend this router (raw_* handling) in the same change","Check for version mismatch between sglang components after upgrades"],"exampleFix":"// before\nitems = all_mm_items  # contains Modality.EMBEDDING\nawait processor.process_and_combine_mm_data(items, ...)\n// after\nitems = [i for i in all_mm_items if i.modality in (Modality.IMAGE, Modality.AUDIO, Modality.VIDEO)]\nawait processor.process_and_combine_mm_data(items, ...)","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"from sglang.srt.multimodal import Modality\nSUPPORTED = {Modality.IMAGE, Modality.AUDIO, Modality.VIDEO}\ndef routable(item):\n    return getattr(item, 'modality', None) in SUPPORTED","tryCatchPattern":null,"preventionTips":["Filter items to known modalities before the combine stage","When adding a new Modality enum value, update the router in the same PR"],"tags":["multimodal","routing","input-validation"],"backgroundTag":"unsupported-modality","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}