{"record":{"id":"97fcea18962be3be","repo":"vllm-project/vllm","slug":"no-dynamic-dimensions-found-in-the-forward-method","errorCode":null,"errorMessage":"No dynamic dimensions found in the forward method of {cls}. Please provide dynamic_arg_dims explicitly.","messagePattern":"No dynamic dimensions found in the forward method of (.+?)\\. Please provide dynamic_arg_dims explicitly\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/decorators.py","lineNumber":228,"sourceCode":"            for k, v in sig.parameters.items():\n                if v.annotation in [\n                    torch.Tensor,\n                    torch.Tensor | None,\n                    torch.FloatTensor,\n                    torch.FloatTensor | None,\n                    IntermediateTensors,\n                    IntermediateTensors | None,\n                ]:\n                    inferred_dynamic_arg_dims[k] = 0\n\n            logger.debug(\n                (\"Inferred dynamic dimensions for forward method of %s: %s\"),\n                cls,\n                list(inferred_dynamic_arg_dims.keys()),\n            )\n\n        if len(inferred_dynamic_arg_dims) == 0:\n            raise ValueError(\n                \"No dynamic dimensions found in the forward method of \"\n                f\"{cls}. Please provide dynamic_arg_dims explicitly.\"\n            )\n\n        for k in inferred_dynamic_arg_dims:\n            if k not in sig.parameters:\n                raise ValueError(\n                    f\"Argument {k} not found in the forward method of {cls}\"\n                )\n\n        return _support_torch_compile(\n            cls,\n            inferred_dynamic_arg_dims,\n            mark_unbacked_dims,\n            enable_if,\n            is_encoder,\n        )\n","sourceCodeStart":210,"sourceCodeEnd":246,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/decorators.py#L210-L246","documentation":"When dynamic_arg_dims is not supplied, the decorator infers dynamic dimensions by scanning forward's parameters whose type annotation is a tensor-like type (torch.Tensor, optional tensors, IntermediateTensors) and marking dim 0 dynamic. If no parameter carries such an annotation, inference produces an empty dict and vLLM raises ValueError asking you to pass dynamic_arg_dims explicitly.","triggerScenarios":"Using @support_torch_compile on a class whose forward parameters are unannotated (no type hints), or annotated only with non-tensor types, without providing the dynamic_arg_dims argument to the decorator.","commonSituations":"Custom models written without type annotations; forward signatures using generic Any or dict annotations; encoder/multimodal wrappers whose tensor inputs are buried in kwargs.","solutions":["Pass dynamic_arg_dims explicitly, e.g. @support_torch_compile(dynamic_arg_dims={\"hidden_states\": 0, \"positions\": 0}).","Or annotate the tensor parameters of forward with torch.Tensor / torch.Tensor | None / IntermediateTensors so inference finds them.","Verify each key you list matches a real parameter name in forward."],"exampleFix":"# before\n@support_torch_compile\nclass MyLayer(nn.Module):\n    def forward(self, hidden_states, positions): ...\n# after\n@support_torch_compile(dynamic_arg_dims={\"hidden_states\": 0})\nclass MyLayer(nn.Module):\n    def forward(self, hidden_states, positions): ...","handlingStrategy":"validation","validationCode":"import inspect, torch\nfrom vllm.distributed import IntermediateTensors\n\nTENSOR_TYPES = {torch.Tensor, torch.Tensor | None, torch.FloatTensor, torch.FloatTensor | None, IntermediateTensors, IntermediateTensors | None}\n\ndef needs_explicit_dynamic_args(cls) -> bool:\n    sig = inspect.signature(cls.forward)\n    return not any(p.annotation in TENSOR_TYPES for p in sig.parameters.values())\n# if needs_explicit_dynamic_args(MyLayer): pass dynamic_arg_dims=...","typeGuard":"def forward_has_tensor_params(cls) -> bool:\n    return any(p.annotation in TENSOR_TYPES for p in inspect.signature(cls.forward).parameters.values())","tryCatchPattern":null,"preventionTips":["Always pass dynamic_arg_dims explicitly in custom models","Annotate forward parameters with torch.Tensor","Add a unit test that applies the decorator to catch config errors at import time"],"tags":["torch-compile","decorators","dynamic-shapes","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}