{"record":{"id":"566c5bdce5bc7be3","repo":"sgl-project/sglang","slug":"hicache-does-not-support-inkling-mtp-draft-state-y","errorCode":null,"errorMessage":"HiCache does not support Inkling MTP draft state yet.","messagePattern":"HiCache does not support Inkling MTP draft state yet\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/base_spec_worker.py","lineNumber":262,"sourceCode":"    def _build_hicache_draft_plan(self) -> HiCacheDraftPlan:\n        target_model_runner = self.target_worker.model_runner\n        target_model_runner.mtp_draft_device_pools = ()\n        spec_algorithm = target_model_runner.spec_algorithm\n        if not (\n            get_memory().enable_hierarchical_cache\n            or get_disagg().disaggregation_decode_retraction_backup == \"host_pool\"\n        ):\n            return HiCacheDraftPlan()\n\n        draft_runners = self._draft_model_runners()\n        if not draft_runners:\n            return HiCacheDraftPlan()\n        draft_pools = tuple(runner.token_to_kv_pool for runner in draft_runners)\n        if (\n            \"InklingForConditionalGenerationMTP\"\n            in draft_runners[0].model_config.hf_config.architectures\n        ):\n            raise NotImplementedError(\n                \"HiCache does not support Inkling MTP draft state yet.\"\n            )\n\n        if _can_pack_hicache_mtp(spec_algorithm, draft_runners):\n            target_model_runner.mtp_draft_device_pools = draft_pools\n            return HiCacheDraftPlan(\n                mode=HiCacheDraftMode.PACKED,\n                device_pools=draft_pools,\n            )\n\n        return HiCacheDraftPlan(\n            mode=HiCacheDraftMode.SIDECAR,\n            # Preserve the legacy non-packed HiCache behavior: multi-layer\n            # EAGLE registers only the first draft runner as the sidecar.\n            device_pools=draft_pools[:1],\n        )\n\n    def init_hicache_draft_plan(self) -> None:","sourceCodeStart":244,"sourceCodeEnd":280,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/base_spec_worker.py#L244-L280","documentation":"Raised when building a HiCache draft plan and the draft model's architecture list contains 'InklingForConditionalGenerationMTP'. HiCache (hierarchical KV cache offload) cannot yet manage or pack the draft KV state produced by Inkling's MTP draft model, so the combination is explicitly unsupported and initialization aborts.","triggerScenarios":"Enabling speculative decoding with an Inkling MTP draft model together with --enable-hicache (HiCache/hierarchical cache); init_hicache_draft_plan inspects draft_runners[0].model_config.hf_config.architectures and raises when the Inkling MTP architecture string is present.","commonSituations":"Server startup with a newer Inkling checkpoint that ships an MTP head while HiCache offload is enabled; copying a launch command from another model family without checking feature compatibility.","solutions":["Disable HiCache (drop --enable-hicache / set enable_hicache=False) and keep speculative decoding with the Inkling MTP draft model","Or disable speculative decoding (remove MTP/draft config) and keep HiCache on the target model","Track the SGLang release notes and retry once Inkling MTP draft-state support lands in _build_hicache_draft_plan"],"exampleFix":"# before\npython -m sglang.launch_server --model inkling-mtp --speculative-algorithm EAGLE --enable-hicache\n# after\npython -m sglang.launch_server --model inkling-mtp --speculative-algorithm EAGLE","handlingStrategy":"validation","validationCode":"archs = draft_model_config.hf_config.architectures\nif \"InklingForConditionalGenerationMTP\" in archs and server_args.enable_hicache:\n    raise SystemExit(\"Inkling MTP + HiCache unsupported; disable one of them\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate feature flags on model architecture compatibility at startup","Keep a per-model matrix of supported speculative/Hicache combinations in CI"],"tags":["sglang","speculative-decoding","hicache","mtp","not-implemented"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}