{"record":{"id":"5c8a5288826f7e19","repo":"vllm-project/vllm","slug":"attribute-key-not-exists-in-the-runnable-of-cuda","errorCode":null,"errorMessage":"Attribute {key} not exists in the runnable of cudagraph wrapper: {self._runnable_str}","messagePattern":"Attribute (.+?) not exists in the runnable of cudagraph wrapper: (.+?)","errorType":"exception","errorClass":"AttributeError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/cuda_graph.py","lineNumber":216,"sourceCode":"        # streams, it might not be safe to share a global pool.\n        # only investigate this when we use multiple streams\n        self.graph_pool = current_platform.get_global_graph_pool()\n\n        if cudagraph_options is None:\n            cudagraph_options = CUDAGraphOptions()\n        self.cudagraph_options = cudagraph_options\n        # the entries for different batch descriptors that we need to capture\n        # cudagraphs for.\n        self.concrete_cudagraph_entries: dict[BatchDescriptor, CUDAGraphEntry] = {}\n\n        CUDAGraphWrapper._all_instances.add(self)\n\n    def __getattr__(self, key: str) -> Any:\n        # allow accessing the attributes of the runnable.\n        if hasattr(self.runnable, key):\n            return getattr(self.runnable, key)\n        if self.is_debugging_mode:\n            raise AttributeError(\n                f\"Attribute {key} not exists in the runnable of \"\n                f\"cudagraph wrapper: {self._runnable_str}\"\n            )\n        raise AttributeError\n\n    def unwrap(self) -> Callable[..., Any]:\n        # in case we need to access the original runnable.\n        return self.runnable\n\n    @property\n    def cudagraph_wrapper(self) -> \"CUDAGraphWrapper\":\n        return self\n\n    def clear_graphs(self) -> None:\n        self.concrete_cudagraph_entries.clear()\n\n    def __call__(self, *args: Any, **kwargs: Any) -> Any | None:\n        if not is_forward_context_available():","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/cuda_graph.py#L198-L234","documentation":"CUDAGraphWrapper delegates unknown attribute access to the wrapped runnable via __getattr__. When the attribute does not exist on the runnable either, the fallback raises a plain AttributeError; in debugging mode it raises a descriptive AttributeError naming the missing key and the runnable, so developers can see what the wrapper failed to proxy.","triggerScenarios":"Accessing any attribute on a model wrapped by vllm.compilation.cuda_graph.CUDAGraphWrapper that neither the wrapper nor the underlying runnable defines (e.g. model.some_layer when some_layer only exists on a different class), while self.is_debugging_mode is True. Also triggered by hasattr-style probing APIs (e.g. copy.deepcopy, pickle, pytest) that poke arbitrary dunder attributes.","commonSituations":"Custom model code that reaches into decoder layers (self.model.layers[i]) after vLLM wrapped the runnable for cudagraph capture; serialization or introspection tooling probing __deepcopy__/__getstate__; typos in attribute names in downstream code.","solutions":["Call .unwrap() on the wrapper to get the original runnable and access the attribute there: model.unwrap().some_layer.","Fix the attribute name / verify the attribute really exists on the wrapped module's class.","Move the access before cudagraph capture so it runs against the unwrapped object, or store a reference to the runnable prior to wrapping."],"exampleFix":"# before\nlayer = cudagraph_wrapped_model.model.layers[0]  # AttributeError: Attribute model not exists...\n# after\nlayer = cudagraph_wrapped_model.unwrap().model.layers[0]","handlingStrategy":"type-guard","validationCode":"from vllm.compilation.cuda_graph import CUDAGraphWrapper\nrunnable = m.unwrap() if isinstance(m, CUDAGraphWrapper) else m\nassert hasattr(runnable, 'some_layer'), 'attribute missing on runnable'","typeGuard":"from vllm.compilation.cuda_graph import CUDAGraphWrapper\n\ndef has_attr_on_runnable(obj: Any, name: str) -> bool:\n    target = obj.unwrap() if isinstance(obj, CUDAGraphWrapper) else obj\n    return hasattr(target, name)","tryCatchPattern":"try:\n    layer = model.some_layer\nexcept AttributeError:\n    layer = model.unwrap().some_layer  # or fix the name","preventionTips":["Keep a reference to the runnable before vLLM wraps it","Prefer .unwrap() when introspecting wrapped models","Write tests that access model internals via unwrap()"],"tags":["cudagraph","attribute-access","wrapper","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}