vllm-project/vllm · error · AttributeError

Attribute not exists in the runnable of cudagraph wrapper

Error message

Attribute {key} not exists in the runnable of cudagraph wrapper: {self._runnable_str}

What it means

CUDAGraphWrapper delegates unknown attribute access to the wrapped runnable via __getattr__. When the attribute does not exist on the runnable either, the fallback raises a plain AttributeError; in debugging mode it raises a descriptive AttributeError naming the missing key and the runnable, so developers can see what the wrapper failed to proxy.

Solutions

Call .unwrap() on the wrapper to get the original runnable and access the attribute there: model.unwrap().some_layer.
Fix the attribute name / verify the attribute really exists on the wrapped module's class.
Move the access before cudagraph capture so it runs against the unwrapped object, or store a reference to the runnable prior to wrapping.

Example fix

# before
layer = cudagraph_wrapped_model.model.layers[0]  # AttributeError: Attribute model not exists...
# after
layer = cudagraph_wrapped_model.unwrap().model.layers[0]

Defensive patterns

Strategy: type-guard

Validate before calling

from vllm.compilation.cuda_graph import CUDAGraphWrapper
runnable = m.unwrap() if isinstance(m, CUDAGraphWrapper) else m
assert hasattr(runnable, 'some_layer'), 'attribute missing on runnable'

Type guard

from vllm.compilation.cuda_graph import CUDAGraphWrapper

def has_attr_on_runnable(obj: Any, name: str) -> bool:
    target = obj.unwrap() if isinstance(obj, CUDAGraphWrapper) else obj
    return hasattr(target, name)

Try / catch

try:
    layer = model.some_layer
except AttributeError:
    layer = model.unwrap().some_layer  # or fix the name

Prevention

Keep a reference to the runnable before vLLM wraps it
Prefer .unwrap() when introspecting wrapped models
Write tests that access model internals via unwrap()

When it happens

Trigger: Accessing any attribute on a model wrapped by vllm.compilation.cuda_graph.CUDAGraphWrapper that neither the wrapper nor the underlying runnable defines (e.g. model.some_layer when some_layer only exists on a different class), while self.is_debugging_mode is True. Also triggered by hasattr-style probing APIs (e.g. copy.deepcopy, pickle, pytest) that poke arbitrary dunder attributes.

Common situations: Custom model code that reaches into decoder layers (self.model.layers[i]) after vLLM wrapped the runnable for cudagraph capture; serialization or introspection tooling probing __deepcopy__/__getstate__; typos in attribute names in downstream code.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5c8a5288826f7e19. Report an issue: GitHub.

Appendix: source

Thrown at vllm/compilation/cuda_graph.py:216

        # streams, it might not be safe to share a global pool.
        # only investigate this when we use multiple streams
        self.graph_pool = current_platform.get_global_graph_pool()

        if cudagraph_options is None:
            cudagraph_options = CUDAGraphOptions()
        self.cudagraph_options = cudagraph_options
        # the entries for different batch descriptors that we need to capture
        # cudagraphs for.
        self.concrete_cudagraph_entries: dict[BatchDescriptor, CUDAGraphEntry] = {}

        CUDAGraphWrapper._all_instances.add(self)

    def __getattr__(self, key: str) -> Any:
        # allow accessing the attributes of the runnable.
        if hasattr(self.runnable, key):
            return getattr(self.runnable, key)
        if self.is_debugging_mode:
            raise AttributeError(
                f"Attribute {key} not exists in the runnable of "
                f"cudagraph wrapper: {self._runnable_str}"
            )
        raise AttributeError

    def unwrap(self) -> Callable[..., Any]:
        # in case we need to access the original runnable.
        return self.runnable

    @property
    def cudagraph_wrapper(self) -> "CUDAGraphWrapper":
        return self

    def clear_graphs(self) -> None:
        self.concrete_cudagraph_entries.clear()

    def __call__(self, *args: Any, **kwargs: Any) -> Any | None:
        if not is_forward_context_available():

View on GitHub (pinned to c794754062)