{"record":{"id":"0e477cce035a77ed","repo":"vllm-project/vllm","slug":"caching-is-not-supported","errorCode":null,"errorMessage":"caching is not supported","messagePattern":"caching is not supported","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/compiler_interface.py","lineNumber":114,"sourceCode":"        `cache_dir/key`.\n        \"\"\"\n        return None, None\n\n    def load(\n        self,\n        handle: Any,\n        graph: fx.GraphModule,\n        example_inputs: list[Any],\n        graph_index: int,\n        compile_range: Range,\n    ) -> Callable[..., Any]:\n        \"\"\"\n        Load the compiled function from the handle.\n        Raises an error if the handle is invalid.\n\n        The handle is the second return value of the `compile` function.\n        \"\"\"\n        raise NotImplementedError(\"caching is not supported\")\n\n\nclass AlwaysHitShapeEnv:\n    \"\"\"\n    Why do we need this class:\n\n    For normal `torch.compile` usage, every compilation will have\n    one Dynamo bytecode compilation and one Inductor compilation.\n    The Inductor compilation happens under the context of the\n    Dynamo bytecode compilation, and that context is used to\n    determine the dynamic shape information, etc.\n\n    For our use case, we only run Dynamo bytecode compilation once,\n    and run Inductor compilation multiple times with different shapes\n    plus a general shape. The compilation for specific shapes happens\n    outside of the context of the Dynamo bytecode compilation. At that\n    time, we don't have shape environment to provide to Inductor, and\n    it will fail the Inductor code cache lookup.","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/compiler_interface.py#L96-L132","documentation":"CompilerInterface is the base class for vLLM's compilation backends; load() implements cache retrieval and its default body raises NotImplementedError('caching is not supported'). A custom/other compiler backend that returns a handle from compile() without implementing load() will blow up as soon as the compilation cache tries to reuse the artifact.","triggerScenarios":"Using a third-party or custom CompilerInterface subclass that does not override load() (returns the base implementation) while the compile cache attempts load_from_cache with a previously stored handle; also directly calling interface.load(...) on the base class in tests.","commonSituations":"Plugging a custom inductor-like compiler into vLLM's compilation backend registry; a backend that returns a path string from compile() but forgot the matching load(); version skew where the cache records handles a backend cannot reload.","solutions":["Implement load(handle, graph, example_inputs, graph_index, compile_range) in your CompilerInterface subclass, or return None as the handle from compile() so caching is skipped","If you do not need caching, disable the compile cache for that backend (VLLM_DISABLE_COMPILE_CACHE=1 or disable_cache in compilation config)","Clear the on-disk compile cache so stale handles from other backends are not loaded"],"exampleFix":"# before\nclass MyCompiler(CompilerInterface):\n    def compile(self, *a, **k):\n        return fn, \"my_handle\"   # handle returned, load() not implemented\n# after\nclass MyCompiler(CompilerInterface):\n    def compile(self, *a, **k):\n        return fn, \"my_handle\"\n    def load(self, handle, graph, example_inputs, graph_index, compile_range):\n        return deserialize_artifact(handle)","handlingStrategy":"validation","validationCode":"import inspect\nassert type(compiler).load is not CompilerInterface.load, \"backend must implement load() to use the cache\"","typeGuard":"def backend_supports_cache(compiler) -> bool:\n    from vllm.compilation.compiler_interface import CompilerInterface\n    return type(compiler).load is not CompilerInterface.load","tryCatchPattern":"try:\n    fn = compiler.load(handle, graph, inputs, idx, rng)\nexcept NotImplementedError:\n    fn = compiler.compile(graph, inputs, cfg, rng, None)[0]  # recompile without cache","preventionTips":["Return a None handle from compile() if your backend cannot reload artifacts","Implement load() whenever you return a non-None handle"],"tags":["compilation","cache","custom-backend","api-contract"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}