{"record":{"id":"e6ce68d40714487a","repo":"sgl-project/sglang","slug":"empty-multimodal-encoder-output","errorCode":null,"errorMessage":"Empty multimodal encoder output.","messagePattern":"Empty multimodal encoder output\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/transformers.py","lineNumber":1425,"sourceCode":"            if value.is_floating_point() and dtype is not None:\n                return value.to(dtype=dtype, device=device)\n            return value\n        if isinstance(value, dict):\n            return {k: self._cast_mm_value(v, dtype, device) for k, v in value.items()}\n        if isinstance(value, list):\n            return [self._cast_mm_value(v, dtype, device) for v in value]\n        if isinstance(value, tuple):\n            return tuple(self._cast_mm_value(v, dtype, device) for v in value)\n        return value\n\n    def _to_tensor_output(self, output) -> torch.Tensor:\n        if hasattr(output, \"pooler_output\") and output.pooler_output is not None:\n            output = output.pooler_output\n        if isinstance(output, tuple):\n            output = output[0]\n        if isinstance(output, (list, tuple)):\n            if len(output) == 0:\n                raise ValueError(\"Empty multimodal encoder output.\")\n            if all(torch.is_tensor(x) for x in output):\n                output = torch.cat(\n                    [x.reshape(-1, x.shape[-1]) if x.ndim > 2 else x for x in output],\n                    dim=0,\n                )\n            else:\n                output = output[0]\n        elif hasattr(output, \"last_hidden_state\"):\n            output = output.last_hidden_state\n        elif isinstance(output, dict):\n            if output.get(\"pooler_output\", None) is not None:\n                output = output[\"pooler_output\"]\n            else:\n                output = next(v for v in output.values() if torch.is_tensor(v))\n            if isinstance(output, (list, tuple)):\n                if len(output) == 0:\n                    raise ValueError(\"Empty multimodal encoder output.\")\n                if all(torch.is_tensor(x) for x in output):","sourceCodeStart":1407,"sourceCodeEnd":1443,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/transformers.py#L1407-L1443","documentation":"After unwrapping the encoder output (pooler_output, tuple[0]), a list/tuple output of length 0 means no embeddings were produced, which downstream concatenation cannot handle.","triggerScenarios":"An encoder returning [] (e.g. zero images passed through, filtered batch, or a model returning empty last_hidden_state list) during _encode_modality_items.","commonSituations":"Empty image batch after filtering; processor returning empty pixel_values but not None; edge-case prompts with only special tokens.","solutions":["Ensure pixel inputs are None (skip encoding) rather than empty when the batch has no items","Guard callers to not invoke the encoder with zero items","Upgrade if a fix handles empty batches gracefully"],"exampleFix":"// before\nif isinstance(output, (list, tuple)) and len(output) == 0:\n    raise ValueError(\"Empty multimodal encoder output.\")\n// after\nif isinstance(output, (list, tuple)) and len(output) == 0:\n    return torch.empty(0, hidden_size, device=..., dtype=...)","handlingStrategy":"validation","validationCode":"if items is None or len(items) == 0:\n    embeds = None  # skip encoder call\nelse:\n    embeds = model.get_multimodal_embeddings(...)","typeGuard":null,"tryCatchPattern":"try:\n    out = model._to_tensor_output(enc_out)\nexcept ValueError as e:\n    if 'Empty multimodal encoder output' in str(e):\n        out = torch.empty(0, hidden_size)\n    else: raise","preventionTips":["Never call encoders with zero items","Filter empty batches before the mm path"],"tags":["multimodal","empty-batch","encoder-output"],"backgroundTag":"empty-encoder-output","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}