{"record":{"id":"c5c4e51fcdf52647","repo":"sgl-project/sglang","slug":"unexpected-initial-state-source-shape-initial-st","errorCode":null,"errorMessage":"Unexpected initial_state_source shape: {initial_state_source.shape}","messagePattern":"Unexpected initial_state_source shape: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/cutedsl_gdn.py","lineNumber":1409,"sourceCode":"    B_q, T_q, H, K = q.shape\n    HV = v.shape[2]\n    V = v.shape[3]\n    N = initial_state_indices.shape[0]\n\n    is_varlen_decode = B_q == 1 and T_q == N and N > 1\n    if scale is None:\n        scale = K**-0.5\n\n    use_small_batch = N < SMALL_BATCH_THRESHOLD\n\n    if initial_state_source.dim() == 1:\n        pool_size = initial_state_source.numel() // (HV * K * V)\n        h0_source = initial_state_source.view(pool_size, HV, K, V)\n    elif initial_state_source.dim() == 4:\n        pool_size = initial_state_source.shape[0]\n        h0_source = initial_state_source\n    else:\n        raise ValueError(\n            f\"Unexpected initial_state_source shape: {initial_state_source.shape}\"\n        )\n\n    if is_varlen_decode:\n        if a.dim() == 3:\n            a = a.squeeze(0)\n        if b.dim() == 3:\n            b = b.squeeze(0)\n        o = q.new_empty(1, N, HV, V, dtype=torch.bfloat16)\n    else:\n        if a.dim() == 2:\n            a = a.unsqueeze(1)\n        if b.dim() == 2:\n            b = b.unsqueeze(1)\n        o = q.new_empty(N, 1, HV, V, dtype=torch.bfloat16)\n\n    q, k, v = [t.contiguous() for t in (q, k, v)]\n","sourceCodeStart":1391,"sourceCodeEnd":1427,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/cutedsl_gdn.py#L1391-L1427","documentation":"The fused sigmoid-gating GDN delta-rule update accepts an initial recurrent-state tensor that is either a flat pool (dim 2, interpreted as (pool_size, HV, K, V) via numel arithmetic) or an explicit 4D (pool_size, HV, K, V) batch. Any other dimensionality raises ValueError('Unexpected initial_state_source shape: ...').","triggerScenarios":"Calling cutedsl_fused_sigmoid_gating_delta_rule_update with an initial_state_source of dim 3 or dim >= 5 — e.g. passing (1, HV, K, V) with an unwanted leading batch dim of size 1 that is actually meant to be 5D, or a per-request (HV,K,V) tensor without the pool dimension.","commonSituations":"Wiring a custom GDN/Gated-DeltaNet model's recurrent state cache directly into the fused kernel; migration from another attention backend whose state layout is (batch, heads, d_k, d_v) but squeezed incorrectly; passing cu_seqlen varlen states without the pool dimension.","solutions":["Reshape to 4D (pool_size, HV, K, V) before the call","Or pass the flat pool tensor and let the kernel view it: tensor.view(numel // (HV*K*V), HV, K, V)","Print initial_state_source.shape at the call site and compare with the HV, K, V arguments to find the mismatched dim"],"exampleFix":"# before\nstate = state.squeeze(0)          # dim 3 -> ValueError\nupdate = cutedsl_fused_sigmoid_gating_delta_rule_update(..., initial_state_source=state)\n# after\nstate = state.reshape(pool_size, HV, K, V)  # dim 4\nupdate = cutedsl_fused_sigmoid_gating_delta_rule_update(..., initial_state_source=state)","handlingStrategy":"validation","validationCode":"HV_K_V = HV * K * V\nassert initial_state_source.dim() in (2, 4), initial_state_source.shape\nassert initial_state_source.numel() % HV_K_V == 0, 'state numel must be a multiple of HV*K*V'","typeGuard":"def is_valid_initial_state(t) -> bool:\n    return t.dim() == 4 or (t.dim() == 2 and t.numel() % (HV * K * V) == 0)","tryCatchPattern":"try:\n    out = cutedsl_fused_sigmoid_gating_delta_rule_update(..., initial_state_source=state)\nexcept ValueError as e:\n    if 'initial_state_source' in str(e):\n        state = state.reshape(-1, HV, K, V)\n        out = cutedsl_fused_sigmoid_gating_delta_rule_update(..., initial_state_source=state)\n    else:\n        raise","preventionTips":["Canonicalize the recurrent state to (pool_size, HV, K, V) immediately after cache allocation","Log state.shape next to HV/K/V at model init to catch layout drift early"],"tags":["gdn","linear-attention","shape-validation","cutedsl"],"backgroundTag":"tensor-rank-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}