vllm-project/vllm · error · TypeError

Expected {ty} but got {type(value)} for {value}

Error message

Expected {ty} but got {type(value)} for {value}

What it means

The Inductor compilation-cache backend persists compiled-graph keys to vllm_compile_cache.py and reloads them with ast.literal_eval, then re-validates the structure with check_type(). If the cached literal contains a value whose Python type no longer matches the expected schema (int graph index, str compiler name, Range/tuple range), TypeError is raised with expected vs. actual type.

Source

Thrown at vllm/compilation/backends.py:195

        base cache dir of /path/to/hash_str/rank_i_j/ ,
        to store some common compilation artifacts.
        """

        self.disable_cache = disable_cache
        self.cache_dir = cache_dir
        self.cache_file_path = os.path.join(cache_dir, "vllm_compile_cache.py")

        if not disable_cache and os.path.exists(self.cache_file_path):
            # load the cache from the file
            with open(self.cache_file_path) as f:
                # we use ast.literal_eval to parse the data
                # because it is a safe way to parse Python literals.
                # do not use eval(), it is unsafe.
                cache = ast.literal_eval(f.read())

            def check_type(value: Any, ty: type) -> None:
                if not isinstance(value, ty):
                    raise TypeError(f"Expected {ty} but got {type(value)} for {value}")

            def parse_key(key: Any) -> tuple[Range, int, str]:
                range_tuple, graph_index, compiler_name = key
                check_type(graph_index, int)
                check_type(compiler_name, str)
                if isinstance(range_tuple, tuple):
                    start, end = range_tuple
                    check_type(start, int)
                    check_type(end, int)
                    range_tuple = Range(start=start, end=end)
                check_type(range_tuple, Range)
                return range_tuple, graph_index, compiler_name

            self.cache = {parse_key(key): value for key, value in cache.items()}

        self.compiler.initialize_cache(
            cache_dir=cache_dir, disable_cache=disable_cache, prefix=prefix
        )

View on GitHub (pinned to c794754062)

Solutions

1. Delete the compilation cache directory (default under ~/.cache/vllm, the cache_dir holding vllm_compile_cache.py) so it is rebuilt from scratch
2. Pin matching vLLM/torch versions on machines that share a cache directory, or give each version its own cache_dir
3. Set disable_cache / VLLM_DISABLE_COMPILE_CACHE=1 if cache staleness keeps recurring

Example fix

# before: reuse stale cache from older vllm -> TypeError on load
# after
$ rm -rf ~/.cache/vllm/compilation  # or the configured cache_dir
$ # restart vllm, cache regenerates

cache = os.path.join(cache_dir, "vllm_compile_cache.py")
if os.path.exists(cache) and cache_written_by_different_version(cache):
    shutil.rmtree(cache_dir)  # regenerate

try:
    backend.load_cached(...)
except (TypeError, SyntaxError, ValueError):
    shutil.rmtree(cache_dir, ignore_errors=True)
    backend.load_cached(...)  # rebuild from scratch

When it happens

Trigger: Loading a stale vllm_compile_cache.py written by an older/newer vLLM version whose key schema differed, or a hand-edited/truncated cache file; e.g. compiler_name stored as int, or range_tuple as a list instead of tuple/Range.

Common situations: Upgrading vLLM (or torch) while reusing ~/.cache/vllm/compilation cache across versions; a crashed process leaving a partially-written cache file; version-skew between nodes sharing a cache volume.

Related errors

• The compiled artifact is not serializable. This usually mean
• vLLM failed to compile the model. The most likely reason for
• PostGradPassManager can not be kept in CompilationConfig.
• call_module is not allowed for codegen target {target}.
• Unsupported node from codegen: {node.format_node()}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a5b9d03ba46463f9. Report an issue: GitHub.