vllm-project/vllm · error · RuntimeError

aot_compile is not supported by the current configuration. P

Error message

aot_compile is not supported by the current configuration. Please make sure torch.compile is enabled with the latest version of PyTorch (current using torch: {torch.__version__})

What it means

VllmBackend's compiled artifact exposes aot_compile() (used by vLLM's AOT compilation tooling) only when the backend was created with torch.compile enabled under a recent PyTorch (enable_aot_compile support). The wrapper method delegates to self._compiled_callable.aot_compile; if that attribute is absent — wrong compilation mode or old torch — it raises RuntimeError with the current torch version.

Source

Thrown at vllm/compilation/wrapper.py:164

        with aot_context:
            self._compiled_callable = torch.compile(
                compiled_ptr,
                fullgraph=True,
                dynamic=False,
                backend=backend,
                options=options,
            )

        if envs.VLLM_USE_BYTECODE_HOOK and mode != CompilationMode.STOCK_TORCH_COMPILE:
            self._bytecode_hook_handle = (
                torch._dynamo.convert_frame.register_bytecode_hook(self.bytecode_hook)
            )
            self._compiled_bytecode: CodeType | None = None

    def aot_compile(self, *args: Any, **kwargs: Any) -> Any:
        if not hasattr(self._compiled_callable, "aot_compile"):
            raise RuntimeError(
                "aot_compile is not supported by the current configuration. "
                "Please make sure torch.compile is enabled with the latest "
                f"version of PyTorch (current using torch: {torch.__version__})"
            )
        return self._compiled_callable.aot_compile((args, kwargs))

    def __call__(self, *args: Any, **kwargs: Any) -> Any:
        if envs.VLLM_USE_BYTECODE_HOOK:
            if (
                self.vllm_config.compilation_config.mode
                == CompilationMode.STOCK_TORCH_COMPILE
            ):
                return self._compiled_callable(*args, **kwargs)

            if not self._compiled_bytecode:
                # Make sure a compilation is triggered by clearing dynamo
                # cache.
                torch._dynamo.eval_frame.remove_from_cache(self.original_code_object())

View on GitHub (pinned to c794754062)

Solutions

1. Upgrade PyTorch to the latest version supported by your vLLM build.
2. Set compilation_config.mode to CompilationMode.VLLM_COMPILE (the vLLM backend exposes aot_compile) before wrapping.
3. If AOT is not needed, skip calling aot_compile and run normal serving.

Example fix

# before
wrapper = TorchCompileWrapper(module, ...)  # mode=STOCK_TORCH_COMPILE
artifacts = wrapper.aot_compile(*args)  # RuntimeError
# after
vllm_config.compilation_config.mode = CompilationMode.VLLM_COMPILE
wrapper = TorchCompileWrapper(module, ...)
artifacts = wrapper.aot_compile(*args)

from vllm.utils import is_torch_equal_or_newer

def aot_supported(wrapper) -> bool:
    return hasattr(getattr(wrapper, '_compiled_callable', None), 'aot_compile') and is_torch_equal_or_newer('2.8')

if not hasattr(wrapper._compiled_callable, 'aot_compile'):
    raise SystemExit('AOT compile requires VLLM_COMPILE mode and latest torch')
artifacts = wrapper.aot_compile(*args)

When it happens

Trigger: Calling TorchCompileWrapper.aot_compile(...) when the underlying torch.compile'd callable (built with mode!=VLLM_COMPILE, or on a PyTorch lacking torch._dynamo.config.enable_aot_compile) has no aot_compile attribute. Typical when running the AOT compile script against a config compiled in stock mode or an old torch wheel.

Common situations: Using vLLM's AOT pre-compilation tooling on nightly/old torch mixes; setting compilation mode to STOCK_TORCH_COMPILE or NONE and still invoking aot_compile; running aot compile on CPU-only old builds.

Related errors

• shape_id='{shape_id}' requires PyTorch >= 2.11.0
• vLLM failed to compile the model. The most likely reason for
• decorated class should have a forward method.
• No dynamic dimensions found in the forward method of {cls}.
• Argument {k} not found in the forward method of {cls}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/f5f1052cc303d1d0. Report an issue: GitHub.