{"record":{"id":"1206c12cfa18b923","repo":"sgl-project/sglang","slug":"force-flush-must-be-a-length-b-int32-tensor-or-n","errorCode":null,"errorMessage":"`force_flush` must be a length-B int32 tensor or None.","messagePattern":"`force_flush` must be a length-B int32 tensor or None\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/helion/kda_replayssm.py","lineNumber":732,"sourceCode":"        flat_a,\n        b,\n        A_log,\n        flat_dt_bias,\n        initial_state,\n        out,\n        ssm_state_indices,\n    )\n\n    if write_pos.ndim != 1 or write_pos.dtype is not torch.int32:\n        raise ValueError(\"`write_pos` must be a 1D int32 tensor.\")\n    if write_pos.shape != (batch,):\n        raise ValueError(f\"`write_pos` must have shape {(batch,)}.\")\n    if force_flush is not None and (\n        force_flush.ndim != 1\n        or force_flush.dtype is not torch.int32\n        or force_flush.shape != (batch,)\n    ):\n        raise ValueError(\"`force_flush` must be a length-B int32 tensor or None.\")\n\n    cache_length = d_cache.size(2)\n    if cache_length < 1:\n        raise ValueError(\"ReplaySSM cache length must be at least 1.\")\n    if d_cache.shape[1:] != (num_v_heads, cache_length, value_dim):\n        raise ValueError(\"`d_cache` must have shape [slots, HV, L, V].\")\n    if k_cache.shape[1:] != (num_q_heads, cache_length, key_dim):\n        raise ValueError(\"`k_cache` must have shape [slots, H, L, K].\")\n    if g_cache.shape[1:] != (num_v_heads, cache_length, key_dim):\n        raise ValueError(\"`g_cache` must have shape [slots, HV, L, K].\")\n    if g_cache.dtype is not torch.float32:\n        raise ValueError(\"`g_cache` must have dtype torch.float32.\")\n\n    device = mixed_qkv.device\n    if any(\n        tensor.device != device for tensor in (d_cache, k_cache, g_cache, write_pos)\n    ):\n        raise ValueError(\"ReplaySSM inputs must be on the same device.\")","sourceCodeStart":714,"sourceCodeEnd":750,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/helion/kda_replayssm.py#L714-L750","documentation":"force_flush optionally forces a per-row flush of the ReplaySSM cache. When provided it must be a 1D int32 tensor of length batch (or None). The guard rejects wrong dtype, wrong rank, or wrong length so the kernel's per-row branch on force_flush[row] is valid.","triggerScenarios":"Passing force_flush as a Python bool, a bool tensor, an int64 tensor, or a [B,1] tensor to helion_fused_recurrent_kda_replayssm_decode.","commonSituations":"Calling with force_flush=True for an unconditional flush instead of a per-row tensor; broadcasting a single flag to [B] via expand (non-contiguous is fine here but dtype stays wrong); porting from a bool-mask API.","solutions":["Pass None when no forced flush is needed","Build a per-row flag: torch.zeros/ones(batch, dtype=torch.int32, device=...) (or torch.where(cond, 1, 0).int())","Never pass Python bools; the API has no scalar overload"],"exampleFix":"// before\nout = helion_fused_recurrent_kda_replayssm_decode(..., force_flush=True)\n// after\nforce = torch.ones(mixed_qkv.size(0), dtype=torch.int32, device=mixed_qkv.device)\nout = helion_fused_recurrent_kda_replayssm_decode(..., force_flush=force)","handlingStrategy":"type-guard","validationCode":"def prep_force_flush(batch, device, flag=False):\n    if flag is None or flag is False:\n        return None\n    return torch.full((batch,), int(flag), dtype=torch.int32, device=device)\n\nforce_flush = prep_force_flush(mixed_qkv.size(0), mixed_qkv.device, want_flush)","typeGuard":"def valid_force_flush(t, batch) -> bool:\n    return t is None or (isinstance(t, torch.Tensor) and t.ndim == 1 and t.dtype is torch.int32 and t.shape == (batch,))","tryCatchPattern":null,"preventionTips":["Never pass Python bools to force_flush","Build per-row int32 flags from request metadata each step"],"tags":["kda","replayssm","dtype","int32","optional-argument"],"backgroundTag":"wrong-tensor-dtype","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}