{"record":{"id":"1c1e6f4c9a927948","repo":"sgl-project/sglang","slug":"replayssm-cache-length-must-be-at-least-1","errorCode":null,"errorMessage":"ReplaySSM cache length must be at least 1.","messagePattern":"ReplaySSM cache length must be at least 1\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/helion/kda_replayssm.py","lineNumber":736,"sourceCode":"        initial_state,\n        out,\n        ssm_state_indices,\n    )\n\n    if write_pos.ndim != 1 or write_pos.dtype is not torch.int32:\n        raise ValueError(\"`write_pos` must be a 1D int32 tensor.\")\n    if write_pos.shape != (batch,):\n        raise ValueError(f\"`write_pos` must have shape {(batch,)}.\")\n    if force_flush is not None and (\n        force_flush.ndim != 1\n        or force_flush.dtype is not torch.int32\n        or force_flush.shape != (batch,)\n    ):\n        raise ValueError(\"`force_flush` must be a length-B int32 tensor or None.\")\n\n    cache_length = d_cache.size(2)\n    if cache_length < 1:\n        raise ValueError(\"ReplaySSM cache length must be at least 1.\")\n    if d_cache.shape[1:] != (num_v_heads, cache_length, value_dim):\n        raise ValueError(\"`d_cache` must have shape [slots, HV, L, V].\")\n    if k_cache.shape[1:] != (num_q_heads, cache_length, key_dim):\n        raise ValueError(\"`k_cache` must have shape [slots, H, L, K].\")\n    if g_cache.shape[1:] != (num_v_heads, cache_length, key_dim):\n        raise ValueError(\"`g_cache` must have shape [slots, HV, L, K].\")\n    if g_cache.dtype is not torch.float32:\n        raise ValueError(\"`g_cache` must have dtype torch.float32.\")\n\n    device = mixed_qkv.device\n    if any(\n        tensor.device != device for tensor in (d_cache, k_cache, g_cache, write_pos)\n    ):\n        raise ValueError(\"ReplaySSM inputs must be on the same device.\")\n    if force_flush is not None and force_flush.device != device:\n        raise ValueError(\"`force_flush` must be on the same device as the inputs.\")\n\n    cache_block = helion.next_power_of_2(max(16, cache_length))","sourceCodeStart":718,"sourceCodeEnd":754,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/helion/kda_replayssm.py#L718-L754","documentation":"The ReplaySSM caches (d_cache, k_cache, g_cache) are ring buffers along a length axis; the kernel reads d_cache.size(2) as the cache length and requires it >= 1. A zero-length cache (empty third dim) has no slot to write flushed rows into, so the guard raises.","triggerScenarios":"Allocating d_cache as torch.empty([slots, HV, 0, V]) — e.g. a pool sized from a config field that evaluated to 0 (cache_len=0 or replay window misconfigured).","commonSituations":"Configuring replay/window length to 0 when the feature is meant to be disabled instead of passing None; allocating caches from a shape tuple with a typo; a model config default of 0 for a new field feeding cache allocation.","solutions":["Allocate the cache with a positive length, at least 1: d_cache = torch.empty(slots, HV, L, V) with L >= 1","If replay is disabled, use the non-replayssm decode entry point instead of a zero-length cache","Validate cache config at startup: assert cache_len >= 1 when replayssm is enabled"],"exampleFix":"// before\nd_cache = torch.empty(slots, HV, 0, V, device=dev, dtype=dt)  # cache_len=0\n// after\nd_cache = torch.empty(slots, HV, max(1, cache_len), V, device=dev, dtype=dt)","handlingStrategy":"validation","validationCode":"assert d_cache.size(2) >= 1, f\"cache length {d_cache.size(2)} must be >= 1\"\n# or at allocation:\nL = max(1, cfg.replay_cache_len)\nd_cache = torch.empty(slots, HV, L, V, device=dev, dtype=dt)","typeGuard":"def valid_d_cache(t: torch.Tensor) -> bool:\n    return t.size(2) >= 1","tryCatchPattern":null,"preventionTips":["Validate cache config (>0) at server startup","Use the non-replay path when replay is disabled instead of zero-length caches"],"tags":["kda","replayssm","cache-allocation","zero-size"],"backgroundTag":"invalid-cache-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}