{"record":{"id":"1c74d9d6c0943168","repo":"sgl-project/sglang","slug":"cuda-vmm-feature-transport-requires-each-feature-f","errorCode":null,"errorMessage":"CUDA VMM feature transport requires each feature field to contain a single tensor","messagePattern":"CUDA VMM feature transport requires each feature field to contain a single tensor","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_transport_utils.py","lineNumber":1010,"sourceCode":"            if len(pack_candidates) >= 2:\n                packed = self.pool.wrap_tensors(\n                    [tensor for _, tensor in pack_candidates]\n                )\n                if packed is not None:\n                    for (item, tensor), proxy in zip(\n                        pack_candidates, packed, strict=True\n                    ):\n                        item.feature = proxy\n                        updates.append((item, \"feature\", tensor, proxy))\n\n            for item in mm_items:\n                fields = (\n                    (\"feature\", item.feature),\n                    (\"precomputed_embeddings\", item.precomputed_embeddings),\n                )\n                for field, tensor in fields:\n                    if _contains_tensor_container(tensor):\n                        raise TypeError(\n                            \"CUDA VMM feature transport requires each feature \"\n                            \"field to contain a single tensor\"\n                        )\n                    if not isinstance(tensor, torch.Tensor):\n                        continue\n                    wrapped = self.pool.wrap_tensor(tensor)\n                    setattr(item, field, wrapped)\n                    updates.append((item, field, tensor, wrapped))\n        except BaseException as error:\n            rollback_errors = []\n            for item, field, tensor, wrapped in reversed(updates):\n                try:\n                    if isinstance(wrapped, CudaVmmTensorTransportProxy):\n                        self.pool.cancel_proxy(wrapped)\n                except BaseException as rollback_error:\n                    rollback_errors.append(rollback_error)\n                finally:\n                    setattr(item, field, tensor)","sourceCodeStart":992,"sourceCodeEnd":1028,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_transport_utils.py#L992-L1028","documentation":"wrap_items validates that each feature field ('feature', 'precomputed_embeddings') is a single torch.Tensor or non-tensor value — never a list/dict/nested structure containing tensors. Packed VMM transport maps each field to one contiguous pool slice, so heterogeneous tensor containers are unsupported and fail closed.","triggerScenarios":"A model returning feature=[t1, t2] or a dict of tensors in MultiModalItem fields; precomputed_embeddings as a nested list; new model integration with multi-tensor features fed into cuda_vmm transport.","commonSituations":"Integrating a new multimodal model whose processor emits per-frame/per-crop tensor lists; switching transport from default (which tolerates containers) to cuda_vmm.","solutions":["Flatten/collapse multi-tensor fields into a single tensor (cat/stack) in the model's mm processor before dispatch","Disable cuda_vmm transport for models with container-shaped features","Add a pre-dispatch validation step that rejects container fields early with a clear message"],"exampleFix":"# before\nitem.feature = [feat_frame0, feat_frame1]\n\n# after\nitem.feature = torch.cat([feat_frame0, feat_frame1], dim=0)","handlingStrategy":"type-guard","validationCode":"def is_flat_tensor(v) -> bool:\n    return v is None or isinstance(v, torch.Tensor)","typeGuard":"def is_wrap_safe(item) -> bool:\n    return all(\n        _contains_tensor_container(v) is False\n        for v in (item.feature, item.precomputed_embeddings)\n    )","tryCatchPattern":"try:\n    transport.wrap_items(items)\nexcept TypeError as e:\n    if \"single tensor\" in str(e):\n        items = flatten_feature_tensors(items)\n        transport.wrap_items(items)","preventionTips":["Keep feature fields a single tensor (cat/stack multi-part features)","Validate item shape before dispatch when using cuda_vmm transport"],"tags":["cuda","vmm","multimodal","validation","type-error"],"backgroundTag":"unsupported-data-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}