vllm-project/vllm · error · RuntimeError

Compilation mode cannot be NO_COMPILATION

Error message

Compilation mode cannot be NO_COMPILATION

What it means

TorchCompileWrapper.__init__ builds a torch.compile wrapper around the module and needs a concrete compilation backend; the backend is chosen from vllm_config.compilation_config.mode. If mode is None (CompilationMode.NO_COMPILATION / eager), constructing the wrapper is contradictory and vLLM raises RuntimeError immediately.

Source

Thrown at vllm/compilation/wrapper.py:88

        return callable_fn(*args, **kwargs)

    def __init__(
        self,
        compile_prefix: str = "",
        is_encoder: bool = False,
    ) -> None:
        self.compiled = False
        self._compile_prefix = compile_prefix
        self._is_encoder = is_encoder

        vllm_config = get_current_vllm_config()
        self.vllm_config = vllm_config
        mode = vllm_config.compilation_config.mode
        self.layerwise_nvtx_tracing_enabled = (
            vllm_config.observability_config.enable_layerwise_nvtx_tracing
        )
        if mode is None:
            raise RuntimeError("Compilation mode cannot be NO_COMPILATION")

        backend = vllm_config.compilation_config.init_backend(
            vllm_config, prefix=compile_prefix, is_encoder=is_encoder
        )
        options = {}

        if isinstance(backend, str) and backend == "inductor":
            options = vllm_config.compilation_config.inductor_compile_config

        self.first_compile = True
        self.evaluate_guards = (
            vllm_config.compilation_config.dynamic_shapes_config.evaluate_guards
        )

        ds_type = vllm_config.compilation_config.dynamic_shapes_config.type

        if mode != CompilationMode.STOCK_TORCH_COMPILE:
            # Drop all the guards.

View on GitHub (pinned to c794754062)

Solutions

1. Only construct the wrapper when compilation_config.mode is not None; guard the call site.
2. Set an explicit mode if you want compilation: CompilationConfig(mode='VLLM_COMPILE') (or run with -O3).
3. If you intended eager, skip the wrapper entirely instead of instantiating it.

Example fix

# before
wrapper = TorchCompileWrapper(my_module, compilation_prefix='model')
# with compilation_config.mode is None
# after
from vllm.config.compilation import CompilationMode
if vllm_config.compilation_config.mode is not None:
    wrapper = TorchCompileWrapper(my_module, compilation_prefix='model')
else:
    wrapper = my_module

from vllm.config.compilation import CompilationMode

def can_wrap(vllm_config) -> bool:
    return vllm_config.compilation_config.mode is not CompilationMode.NONE and vllm_config.compilation_config.mode is not None

When it happens

Trigger: Manually instantiating TorchCompileWrapper (or a subclass like VllmCompiledNode / CompilationWrapper) while the current vllm_config has compilation mode None — e.g. LLM(model=..., compilation_config=CompilationConfig(mode=None)) or enforce_eager=True paths, then calling the wrapper.

Common situations: Custom integration code that always wraps models in TorchCompileWrapper regardless of config; disabling compilation globally but a code path still constructs the wrapper; tests constructing wrappers with a default VllmConfig.

Related errors

• invalid --allowed-methods value {method:?}: {e}
• invalid --allowed-headers value {header:?}: {e}
• --ssl-certfile is required to enable TLS; --ssl-keyfile/--ss
• --ssl-cert-reqs must be 0 (none), 1 (optional), or 2 (requir
• --ssl-ca-certs is required when --ssl-cert-reqs is {} (clien

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/b0874463aef19f72. Report an issue: GitHub.