vllm-project/vllm · error · RuntimeError

The compiled artifact is not serializable. This usually mean

Error message

The compiled artifact is not serializable. This usually means that the model code has something that is not serializable by torch.compile in it. You can fix this by either figuring out what is not serializable and rewriting it, filing a bug report, or suppressing this error by disabling vLLM's compilation cache via VLLM_DISABLE_COMPILE_CACHE=1 (this will greatly increase vLLM server warm start times).

What it means

The standalone Inductor adapter saves compiled artifacts for fast restarts; saveability requires exactly one AOT autograd artifact (checked by the is_saveable_2_10 shim on torch 2.10, using compiled_artifact._artifacts). If the model graph produced something torch.compile cannot serialize (multiple AOT artifacts / non-serializable payloads), saving is refused with RuntimeError rather than writing a corrupt cache.

Source

Thrown at vllm/compilation/compiler_interface.py:401

            # just return the compiled graph and a key
            # since we can serialize the bytes using to_bytes
            # and reload it using the key when reading
            return compiled_graph, None

        # Save the compiled artifact to disk in the specified path
        assert key is not None
        path = os.path.join(self.cache_dir, key)

        def is_saveable_2_10(compiled_artifact):
            # can just use compiled_artifact.is_saveable in 2.11
            if compiled_artifact._artifacts is None:
                return False
            _, cache_info = compiled_artifact._artifacts
            return len(cache_info.aot_autograd_artifacts) == 1

        if is_compile_cache_enabled(compiler_config):
            if not is_saveable_2_10(compiled_graph):
                raise RuntimeError(
                    "The compiled artifact is not serializable. This usually means "
                    "that the model code has something that is not serializable "
                    "by torch.compile in it. You can fix this by either "
                    "figuring out what is not serializable and rewriting it, "
                    "filing a bug report, "
                    "or suppressing this error by "
                    "disabling vLLM's compilation cache via "
                    "VLLM_DISABLE_COMPILE_CACHE=1 "
                    "(this will greatly increase vLLM server warm start times)."
                )
            compiled_graph.save(path=path, format=self.save_format)
            compilation_counter.num_compiled_artifacts_saved += 1
        return compiled_graph, (key, path)

    def load(
        self,
        handle: Any,
        graph: fx.GraphModule,

View on GitHub (pinned to c794754062)

Solutions

1. Set VLLM_DISABLE_COMPILE_CACHE=1 as an immediate workaround (server still compiles, it just will not persist artifacts; slower warm start)
2. Upgrade (or pin) vLLM and torch to a matched pair where artifact serialization for your model is supported
3. Reduce graph breaks / non-serializable constructs in the model code (custom autograd functions, closures over unpicklable objects) so a single serializable artifact is produced

Example fix

# before
$ vllm serve model --enforce-eager  # or with compile cache on -> RuntimeError on save
# after
$ VLLM_DISABLE_COMPILE_CACHE=1 vllm serve model

import os
if os.environ.get("VLLM_DISABLE_COMPILE_CACHE") != "1":
    # cheap preflight: run one compile+save in CI to catch non-serializable models early
    pass

try:
    adapter.compile_and_save(graph, inputs, cfg, key)
except RuntimeError as e:
    if "not serializable" in str(e):
        os.environ["VLLM_DISABLE_COMPILE_CACHE"] = "1"  # proceed without cache
        adapter.compile_and_save(graph, inputs, cfg, key)
    else:
        raise

When it happens

Trigger: Running with the compile cache enabled (VLLM_DISABLE_COMPILE_CACHE unset) on torch 2.10, where the compiled graph's cache_info.aot_autograd_artifacts is empty or has more than one entry, or _artifacts is None; the save path triggers after successful compilation.

Common situations: Models whose graph splits into multiple autograd artifacts (custom modules, mutation, graph breaks); torch/vLLM version mismatch in the serialization format; the error message's own escape hatch (VLLM_DISABLE_COMPILE_CACHE=1) trades warm-start speed for correctness.

Related errors

• Expected {ty} but got {type(value)} for {value}
• vLLM failed to compile the model. The most likely reason for
• utility call `{method}` returned an invalid result (call_id=
• PostGradPassManager can not be kept in CompilationConfig.
• call_module is not allowed for codegen target {target}.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a40a22109f0af804. Report an issue: GitHub.