{"record":{"id":"a40a22109f0af804","repo":"vllm-project/vllm","slug":"the-compiled-artifact-is-not-serializable-this-us","errorCode":null,"errorMessage":"The compiled artifact is not serializable. This usually means that the model code has something that is not serializable by torch.compile in it. You can fix this by either figuring out what is not serializable and rewriting it, filing a bug report, or suppressing this error by disabling vLLM's compilation cache via VLLM_DISABLE_COMPILE_CACHE=1 (this will greatly increase vLLM server warm start times).","messagePattern":"The compiled artifact is not serializable\\. This usually means that the model code has something that is not serializable by torch\\.compile in it\\. You can fix this by either figuring out what is not serializable and rewriting it, filing a bug report, or suppressing this error by disabling vLLM's compilation cache via VLLM_DISABLE_COMPILE_CACHE=1 \\(this will greatly increase vLLM server warm start times\\)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/compiler_interface.py","lineNumber":401,"sourceCode":"            # just return the compiled graph and a key\n            # since we can serialize the bytes using to_bytes\n            # and reload it using the key when reading\n            return compiled_graph, None\n\n        # Save the compiled artifact to disk in the specified path\n        assert key is not None\n        path = os.path.join(self.cache_dir, key)\n\n        def is_saveable_2_10(compiled_artifact):\n            # can just use compiled_artifact.is_saveable in 2.11\n            if compiled_artifact._artifacts is None:\n                return False\n            _, cache_info = compiled_artifact._artifacts\n            return len(cache_info.aot_autograd_artifacts) == 1\n\n        if is_compile_cache_enabled(compiler_config):\n            if not is_saveable_2_10(compiled_graph):\n                raise RuntimeError(\n                    \"The compiled artifact is not serializable. This usually means \"\n                    \"that the model code has something that is not serializable \"\n                    \"by torch.compile in it. You can fix this by either \"\n                    \"figuring out what is not serializable and rewriting it, \"\n                    \"filing a bug report, \"\n                    \"or suppressing this error by \"\n                    \"disabling vLLM's compilation cache via \"\n                    \"VLLM_DISABLE_COMPILE_CACHE=1 \"\n                    \"(this will greatly increase vLLM server warm start times).\"\n                )\n            compiled_graph.save(path=path, format=self.save_format)\n            compilation_counter.num_compiled_artifacts_saved += 1\n        return compiled_graph, (key, path)\n\n    def load(\n        self,\n        handle: Any,\n        graph: fx.GraphModule,","sourceCodeStart":383,"sourceCodeEnd":419,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/compiler_interface.py#L383-L419","documentation":"The standalone Inductor adapter saves compiled artifacts for fast restarts; saveability requires exactly one AOT autograd artifact (checked by the is_saveable_2_10 shim on torch 2.10, using compiled_artifact._artifacts). If the model graph produced something torch.compile cannot serialize (multiple AOT artifacts / non-serializable payloads), saving is refused with RuntimeError rather than writing a corrupt cache.","triggerScenarios":"Running with the compile cache enabled (VLLM_DISABLE_COMPILE_CACHE unset) on torch 2.10, where the compiled graph's cache_info.aot_autograd_artifacts is empty or has more than one entry, or _artifacts is None; the save path triggers after successful compilation.","commonSituations":"Models whose graph splits into multiple autograd artifacts (custom modules, mutation, graph breaks); torch/vLLM version mismatch in the serialization format; the error message's own escape hatch (VLLM_DISABLE_COMPILE_CACHE=1) trades warm-start speed for correctness.","solutions":["Set VLLM_DISABLE_COMPILE_CACHE=1 as an immediate workaround (server still compiles, it just will not persist artifacts; slower warm start)","Upgrade (or pin) vLLM and torch to a matched pair where artifact serialization for your model is supported","Reduce graph breaks / non-serializable constructs in the model code (custom autograd functions, closures over unpicklable objects) so a single serializable artifact is produced"],"exampleFix":"# before\n$ vllm serve model --enforce-eager  # or with compile cache on -> RuntimeError on save\n# after\n$ VLLM_DISABLE_COMPILE_CACHE=1 vllm serve model","handlingStrategy":"fallback","validationCode":"import os\nif os.environ.get(\"VLLM_DISABLE_COMPILE_CACHE\") != \"1\":\n    # cheap preflight: run one compile+save in CI to catch non-serializable models early\n    pass","typeGuard":null,"tryCatchPattern":"try:\n    adapter.compile_and_save(graph, inputs, cfg, key)\nexcept RuntimeError as e:\n    if \"not serializable\" in str(e):\n        os.environ[\"VLLM_DISABLE_COMPILE_CACHE\"] = \"1\"  # proceed without cache\n        adapter.compile_and_save(graph, inputs, cfg, key)\n    else:\n        raise","preventionTips":["Exercise compile+save in CI for every served model so serialization failures surface before deploy","Keep vLLM and torch versions matched","Know the VLLM_DISABLE_COMPILE_CACHE=1 escape hatch and its warm-start cost"],"tags":["compilation","cache","serialization","torch-compile","version-skew"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}