vllm-project/vllm · error · RuntimeError

Source code has changed since the last compilation. Recompil

Error message

Source code has changed since the last compilation. Recompiling the model.

What it means

During cache verification, vLLM compares the checksum of the source code recorded at trace time (inlined_sources content embedded in the compilation artifact) against the checksum of the same files on disk. A mismatch means the .py files that produced the compiled artifact were edited afterwards, so the cached compiled graph no longer corresponds to the running code, and vLLM refuses to use it (RuntimeError telling you to recompile).

Source

Thrown at vllm/compilation/decorators.py:279

    sha256_hash.update(str(fn.__code__.co_firstlineno).encode())
    return sha256_hash.hexdigest()


def _verify_source_unchanged(
    source_info: "SourceInfo", vllm_config: VllmConfig
) -> None:
    from .caching import _compute_code_hash, _compute_code_hash_with_content

    file_contents = {}
    for source in source_info.inlined_sources:
        module = sys.modules[source.module]
        file = inspect.getfile(module)
        vllm_config.compilation_config.traced_files.add(file)
        file_contents[file] = source.content
    expected_checksum = _compute_code_hash_with_content(file_contents)
    actual_checksum = _compute_code_hash(set(file_contents.keys()))
    if expected_checksum != actual_checksum:
        raise RuntimeError(
            "Source code has changed since the last compilation. Recompiling the model."
        )


def _try_load_aot_compiled_fn(
    model: Any,
    aot_compilation_path: str,
) -> Any | None:
    """Try to load an AOT-compiled function from disk.

    Returns the loaded callable on success, or None on failure.
    Re-raises on failure when ``VLLM_FORCE_AOT_LOAD`` is set.
    """
    try:
        with monitor_torch_compile(model.vllm_config, is_encoder=model._is_encoder):
            with (
                set_current_vllm_config(model.vllm_config),
                open(aot_compilation_path, "rb") as f,

View on GitHub (pinned to c794754062)

Solutions

1. Clear the compilation cache so vLLM recompiles against current sources: rm -rf ~/.cache/vllm/torch_compile_cache.
2. Keep source files byte-identical between compile and load runs (don't edit model code, switch branches, or reinstall a different version in between).
3. In CI, key the compiled-artifact cache on the source checksum, not just the model name.

Example fix

# before
git checkout my-model-experiment  # edit model source after compiling
python run_vllm.py  # RuntimeError: Source code has changed since the last compilation
# after
rm -rf ~/.cache/vllm/torch_compile_cache
git checkout my-model-experiment
python run_vllm.py

import hashlib, pathlib

def fingerprint(paths: list[str]) -> str:
    h = hashlib.sha256()
    for p in sorted(paths):
        h.update(pathlib.Path(p).read_bytes())
    return h.hexdigest()
# record fingerprint(model_files) at compile time; compare before loading cache

try:
    run_with_compilation_cache()
except RuntimeError as e:
    if 'Source code has changed' in str(e):
        clear_vllm_compile_cache(); run_with_compilation_cache()
    else:
        raise

When it happens

Trigger: Loading a cached/AOT compilation artifact (torch_compile_cache or AOT compilation path) after editing any traced source file — model implementation files registered in compilation_config.traced_files — between the compile run and the load run.

Common situations: Iterating on model code while reusing a persistent compilation cache; a git branch switch or pip reinstall changing file contents/mtimes between compile and serve; CI caching compiled artifacts across code changes.

Related errors

• vLLM failed to compile the model. The most likely reason for
• Input {arg} to maybe_inplace node {node} is used again after
• cudagraph_capture_sizes not supported in compile_sizes.This
• HTTP request failed: {0}
• JSON error: {0}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/adc34a3f67a0796a. Report an issue: GitHub.