sgl-project/sglang · error · NotImplementedError

HiCache does not support Inkling MTP draft state yet.

Error message

HiCache does not support Inkling MTP draft state yet.

What it means

Raised when building a HiCache draft plan and the draft model's architecture list contains 'InklingForConditionalGenerationMTP'. HiCache (hierarchical KV cache offload) cannot yet manage or pack the draft KV state produced by Inkling's MTP draft model, so the combination is explicitly unsupported and initialization aborts.

Source

Thrown at python/sglang/srt/speculative/base_spec_worker.py:262

    def _build_hicache_draft_plan(self) -> HiCacheDraftPlan:
        target_model_runner = self.target_worker.model_runner
        target_model_runner.mtp_draft_device_pools = ()
        spec_algorithm = target_model_runner.spec_algorithm
        if not (
            get_memory().enable_hierarchical_cache
            or get_disagg().disaggregation_decode_retraction_backup == "host_pool"
        ):
            return HiCacheDraftPlan()

        draft_runners = self._draft_model_runners()
        if not draft_runners:
            return HiCacheDraftPlan()
        draft_pools = tuple(runner.token_to_kv_pool for runner in draft_runners)
        if (
            "InklingForConditionalGenerationMTP"
            in draft_runners[0].model_config.hf_config.architectures
        ):
            raise NotImplementedError(
                "HiCache does not support Inkling MTP draft state yet."
            )

        if _can_pack_hicache_mtp(spec_algorithm, draft_runners):
            target_model_runner.mtp_draft_device_pools = draft_pools
            return HiCacheDraftPlan(
                mode=HiCacheDraftMode.PACKED,
                device_pools=draft_pools,
            )

        return HiCacheDraftPlan(
            mode=HiCacheDraftMode.SIDECAR,
            # Preserve the legacy non-packed HiCache behavior: multi-layer
            # EAGLE registers only the first draft runner as the sidecar.
            device_pools=draft_pools[:1],
        )

    def init_hicache_draft_plan(self) -> None:

View on GitHub (pinned to 0132848349)

Solutions

1. Disable HiCache (drop --enable-hicache / set enable_hicache=False) and keep speculative decoding with the Inkling MTP draft model
2. Or disable speculative decoding (remove MTP/draft config) and keep HiCache on the target model
3. Track the SGLang release notes and retry once Inkling MTP draft-state support lands in _build_hicache_draft_plan

Example fix

# before
python -m sglang.launch_server --model inkling-mtp --speculative-algorithm EAGLE --enable-hicache
# after
python -m sglang.launch_server --model inkling-mtp --speculative-algorithm EAGLE

archs = draft_model_config.hf_config.architectures
if "InklingForConditionalGenerationMTP" in archs and server_args.enable_hicache:
    raise SystemExit("Inkling MTP + HiCache unsupported; disable one of them")

When it happens

Trigger: Enabling speculative decoding with an Inkling MTP draft model together with --enable-hicache (HiCache/hierarchical cache); init_hicache_draft_plan inspects draft_runners[0].model_config.hf_config.architectures and raises when the Inkling MTP architecture string is present.

Common situations: Server startup with a newer Inkling checkpoint that ships an MTP head while HiCache offload is enabled; copying a launch command from another model family without checking feature compatibility.

Related errors

• CuteDSLKDAKernel does not support target_verify
• NvidiaKDAKernel does not support target_verify
• PtxKDAKernel does not support target_verify
• {self.__class__.__name__} does not support target_verify
• trtllm_mla does not forward the cyclic DCP metadata to its d

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/566c5bdce5bc7be3. Report an issue: GitHub.