{"record":{"id":"16afd88a90874f4c","repo":"vllm-project/vllm","slug":"unsupported-dynamic-dimensions-dims-for-argument","errorCode":null,"errorMessage":"Unsupported dynamic dimensions {dims} for argument {k} with type {type(arg)}.","messagePattern":"Unsupported dynamic dimensions (.+?) for argument (.+?) with type (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/decorators.py","lineNumber":482,"sourceCode":"\n            if arg is not None:\n                dims = list(dim_to_shape_id.keys())\n\n                if isinstance(arg, torch.Tensor):\n                    dim_shape_pairs = [\n                        (arg.ndim + d if d < 0 else d, dim_to_shape_id.get(d))\n                        for d in dims\n                    ]\n                    mark_dynamic(arg, dim_shape_pairs)\n                elif isinstance(arg, IntermediateTensors):\n                    for tensor in arg.tensors.values():\n                        dim_shape_pairs = [\n                            (tensor.ndim + d if d < 0 else d, dim_to_shape_id.get(d))\n                            for d in dims\n                        ]\n                        mark_dynamic(tensor, dim_shape_pairs)\n                else:\n                    raise ValueError(\n                        f\"Unsupported dynamic dimensions {dims} \"\n                        f\"for argument {k} with type {type(arg)}.\"\n                    )\n\n        if mark_unbacked_dims:\n            for k, dims_val in mark_unbacked_dims.items():\n                arg = bound_args.arguments.get(k)\n                if arg is not None:\n                    dims = [dims_val] if isinstance(dims_val, int) else list(dims_val)\n                    if isinstance(arg, torch.Tensor):\n                        dims = [arg.ndim + d if d < 0 else d for d in dims]\n                        if is_torch_equal_or_newer(\"2.10.0\"):\n                            for dim in dims:\n                                torch._dynamo.decorators.mark_unbacked(\n                                    arg, dim, hint_override=arg.size()[dim]\n                                )\n                        else:\n                            torch._dynamo.decorators.mark_unbacked(arg, dims)","sourceCodeStart":464,"sourceCodeEnd":500,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/decorators.py#L464-L500","documentation":"When marking dynamic dimensions per forward argument, vLLM only knows how to handle torch.Tensor arguments and IntermediateTensors containers (marking dim 0 / given dims, normalizing negative dims). If dynamic_arg_dims (or mark_unbacked_dims) names an argument that is neither, it raises ValueError because there is no defined semantics for marking dims on that type.","triggerScenarios":"Passing dynamic_arg_dims={'attn_mask': 0} (or mark_unbacked_dims) where the forward parameter attn_mask is e.g. a list, tuple, dataclass, or plain object rather than torch.Tensor or IntermediateTensors; also when the argument is None at call time through the tensor branches is fine, but non-tensor types hit the else branch.","commonSituations":"Model forwards that take lists of tensors or custom input dataclasses; copying dynamic_arg_dims maps from multimodal models with different argument types.","solutions":["Remove the entry for the non-tensor argument from dynamic_arg_dims / mark_unbacked_dims.","Change the forward parameter type to torch.Tensor or IntermediateTensors if it genuinely holds tensors.","For containers, wrap them in IntermediateTensors or mark dims on the individual tensor parameters instead."],"exampleFix":"# before\n@support_torch_compile(dynamic_arg_dims={\"kv_cache\": 0})\nclass L(nn.Module):\n    def forward(self, hidden_states, kv_cache: list): ...\n# after\n@support_torch_compile(dynamic_arg_dims={\"hidden_states\": 0})\nclass L(nn.Module):\n    def forward(self, hidden_states, kv_cache: list): ...","handlingStrategy":"validation","validationCode":"import inspect, torch\nfrom vllm.distributed import IntermediateTensors\n\ndef valid_dynamic_arg_dims(cls, dims: dict) -> bool:\n    hints = {k: v.annotation for k, v in inspect.signature(cls.forward).parameters.items()}\n    return all(hints.get(k) in (torch.Tensor, torch.Tensor | None, IntermediateTensors, IntermediateTensors | None) for k in dims)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only list tensor/IntermediateTensors parameters in dynamic_arg_dims","Add a config test that runs the decorator on a dummy input","Avoid list/tuple tensor containers in compiled forward signatures"],"tags":["torch-compile","dynamic-shapes","type-validation","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}