vllm-project/vllm · error · AttributeError

Attribute {key} not exists in the runnable of cudagraph wrap

Error message

Attribute {key} not exists in the runnable of cudagraph wrapper: {self._runnable_str}

What it means

CUDAGraphWrapper delegates unknown attribute access to the wrapped runnable via __getattr__. When the attribute does not exist on the runnable either, the fallback raises a plain AttributeError; in debugging mode it raises a descriptive AttributeError naming the missing key and the runnable, so developers can see what the wrapper failed to proxy.

Source

Thrown at vllm/compilation/cuda_graph.py:216

        # streams, it might not be safe to share a global pool.
        # only investigate this when we use multiple streams
        self.graph_pool = current_platform.get_global_graph_pool()

        if cudagraph_options is None:
            cudagraph_options = CUDAGraphOptions()
        self.cudagraph_options = cudagraph_options
        # the entries for different batch descriptors that we need to capture
        # cudagraphs for.
        self.concrete_cudagraph_entries: dict[BatchDescriptor, CUDAGraphEntry] = {}

        CUDAGraphWrapper._all_instances.add(self)

    def __getattr__(self, key: str) -> Any:
        # allow accessing the attributes of the runnable.
        if hasattr(self.runnable, key):
            return getattr(self.runnable, key)
        if self.is_debugging_mode:
            raise AttributeError(
                f"Attribute {key} not exists in the runnable of "
                f"cudagraph wrapper: {self._runnable_str}"
            )
        raise AttributeError

    def unwrap(self) -> Callable[..., Any]:
        # in case we need to access the original runnable.
        return self.runnable

    @property
    def cudagraph_wrapper(self) -> "CUDAGraphWrapper":
        return self

    def clear_graphs(self) -> None:
        self.concrete_cudagraph_entries.clear()

    def __call__(self, *args: Any, **kwargs: Any) -> Any | None:
        if not is_forward_context_available():

View on GitHub (pinned to c794754062)

Solutions

1. Call .unwrap() on the wrapper to get the original runnable and access the attribute there: model.unwrap().some_layer.
2. Fix the attribute name / verify the attribute really exists on the wrapped module's class.
3. Move the access before cudagraph capture so it runs against the unwrapped object, or store a reference to the runnable prior to wrapping.

Example fix

# before
layer = cudagraph_wrapped_model.model.layers[0]  # AttributeError: Attribute model not exists...
# after
layer = cudagraph_wrapped_model.unwrap().model.layers[0]

from vllm.compilation.cuda_graph import CUDAGraphWrapper
runnable = m.unwrap() if isinstance(m, CUDAGraphWrapper) else m
assert hasattr(runnable, 'some_layer'), 'attribute missing on runnable'

from vllm.compilation.cuda_graph import CUDAGraphWrapper

def has_attr_on_runnable(obj: Any, name: str) -> bool:
    target = obj.unwrap() if isinstance(obj, CUDAGraphWrapper) else obj
    return hasattr(target, name)

try:
    layer = model.some_layer
except AttributeError:
    layer = model.unwrap().some_layer  # or fix the name

When it happens

Trigger: Accessing any attribute on a model wrapped by vllm.compilation.cuda_graph.CUDAGraphWrapper that neither the wrapper nor the underlying runnable defines (e.g. model.some_layer when some_layer only exists on a different class), while self.is_debugging_mode is True. Also triggered by hasattr-style probing APIs (e.g. copy.deepcopy, pickle, pytest) that poke arbitrary dunder attributes.

Common situations: Custom model code that reaches into decoder layers (self.model.layers[i]) after vLLM wrapped the runnable for cudagraph capture; serialization or introspection tooling probing __deepcopy__/__getstate__; typos in attribute names in downstream code.

Related errors

• CUDA graph capturing detected at an inappropriate time. This
• cudagraph_capture_sizes not supported in compile_sizes.This
• Assigning / modifying buffers of nn.Module during forward pa
• HTTP request failed: {0}
• JSON error: {0}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5c8a5288826f7e19. Report an issue: GitHub.