{"record":{"id":"5026c062eb5a3b3d","repo":"sgl-project/sglang","slug":"online-c128-does-not-support-mtp","errorCode":null,"errorMessage":"online c128 does not support MTP","messagePattern":"online c128 does not support MTP","errorType":"validation","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/deepseek_v4_memory_pool.py","lineNumber":43,"sourceCode":"from sglang.srt.utils import ceil_div, is_hip\n\nlogger = logging.getLogger(__name__)\n\n_is_hip = is_hip()\n\nONLINE_C128 = not _is_hip and envs.SGLANG_OPT_USE_ONLINE_COMPRESS.get()\n\n\ndef get_compress_state_ring_size(\n    compress_ratio: int, is_speculative: bool = False\n) -> int:\n    assert compress_ratio in [4, 128], f\"Unsupported {compress_ratio = }\"\n    # Online c128 keeps a single (max, sum, kv) state per index instead of a\n    # 128-slot ring buffer of raw tokens, so ring_size collapses to 1. Online\n    # is incompatible with speculative decode for now.\n    if compress_ratio == 128 and ONLINE_C128:\n        if is_speculative and not envs.SGLANG_EXPERIMENTAL_ONLINE_C128_MTP.get():\n            raise AssertionError(\"online c128 does not support MTP\")\n        return 1\n    if is_speculative:\n        return 16 if compress_ratio == 4 else 256\n    else:\n        return 8 if compress_ratio == 4 else 128\n\n\ndef get_compress_state_write_pad(compress_ratio: int, ring_size: int) -> int:\n    \"\"\"Largest draft-token count this ring can serve; mirrors `mtp_pad` in `c_plan.cuh`\n    (the bound is derived there). Zero for a non-speculative ring, which is exactly one\n    window wide.\"\"\"\n    window_size = compress_ratio * (2 if compress_ratio == 4 else 1)\n    return ring_size - window_size + 2 if ring_size > window_size else 0\n\n\nclass DeepSeekV4SingleKVPool(KVCache):\n    def __init__(\n        self,","sourceCodeStart":25,"sourceCodeEnd":61,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/deepseek_v4_memory_pool.py#L25-L61","documentation":"Online c128 compression (compress_ratio=128 with ONLINE_C128 enabled) keeps a single (max, sum, kv) state per index instead of a speculative ring buffer, so it cannot accommodate speculative decoding (MTP). When is_speculative is set and the SGLANG_EXPERIMENTAL_ONLINE_C128_MTP escape hatch is unset, this AssertionError fires from get_compress_state_ring_size.","triggerScenarios":"Launching a DeepSeek-V4 c128 model with ONLINE_C128 enabled together with speculative decoding (--speculative-algorithm / MTP) and without SGLANG_EXPERIMENTAL_ONLINE_C128_MTP=...; called during memory pool init (get_ring_size/__init__).","commonSituations":"Enabling MTP speculative decoding on a DSv4 online-c128 deployment for throughput; upgrading SGLang where online c128 became the default; test code that exercises ring capacity with speculative flags set.","solutions":["Disable speculative decoding (remove --speculative-algorithm / MTP flags) for online c128 models","Use compress_ratio=4 instead of 128 if MTP is required","If you accept the experimental risk, set SGLANG_EXPERIMENTAL_ONLINE_C128_MTP env var to enable the unsupported combination"],"exampleFix":"# before\n--model deepseek-ai/DeepSeek-V4 --speculative-algorithm EAGLE ...\n# after\n--model deepseek-ai/DeepSeek-V4   # no speculative flags with online c128","handlingStrategy":"validation","validationCode":"from sglang.srt.mem_cache.deepseek_v4_memory_pool import ONLINE_C128\nif ONLINE_C128 and server_args.speculative_algorithm and compress_ratio == 128:\n    assert envs.SGLANG_EXPERIMENTAL_ONLINE_C128_MTP.get(), \"online c128 + MTP unsupported\"","typeGuard":"def online_c128_mtp_ok(is_speculative: bool) -> bool:\n    return (not is_speculative) or bool(envs.SGLANG_EXPERIMENTAL_ONLINE_C128_MTP.get())","tryCatchPattern":null,"preventionTips":["Do not combine --speculative-algorithm with c128 online compression","Document the MTP limitation next to DSv4 deployment configs","Add launch-script checks for incompatible flag pairs"],"tags":["deepseek-v4","c128","speculative-decoding","mtp","incompatible-features"],"backgroundTag":"incompatible-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}