{"record":{"id":"fbc7f55327653685","repo":"sgl-project/sglang","slug":"invalid-stacked-eps-shape-for-fused-kv-materializa","errorCode":null,"errorMessage":"Invalid stacked eps shape for fused KV materialization: got {tuple(eps.shape)}, expected {(n_layers,)}.","messagePattern":"Invalid stacked eps shape for fused KV materialization: got (.+?), expected (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":169,"sourceCode":"\n    kv_size = num_kv_heads * head_dim\n    if kv_dim != kv_size * 2:\n        raise ValueError(\n            \"Invalid fused KV projection shape: \"\n            f\"got {tuple(kv.shape)}, expected trailing dim {kv_size * 2}.\"\n        )\n    if rotary_dim <= 0 or rotary_dim > head_dim or rotary_dim % 2 != 0:\n        raise ValueError(\n            \"Invalid fused KV rotary/head dim pair: \"\n            f\"rotary_dim={rotary_dim}, head_dim={head_dim}.\"\n        )\n    if k_norm_weight.shape != (n_layers, head_dim):\n        raise ValueError(\n            \"Invalid stacked k_norm_weight shape for fused KV materialization: \"\n            f\"got {tuple(k_norm_weight.shape)}, expected {(n_layers, head_dim)}.\"\n        )\n    if eps.shape != (n_layers,):\n        raise ValueError(\n            \"Invalid stacked eps shape for fused KV materialization: \"\n            f\"got {tuple(eps.shape)}, expected {(n_layers,)}.\"\n        )\n\n    half_rotary_dim = rotary_dim // 2\n    BLOCK_HD = triton.next_power_of_2(head_dim)\n\n    if positions.device != kv.device:\n        positions = positions.to(device=kv.device, dtype=torch.int64)\n    elif positions.dtype != torch.int64:\n        positions = positions.to(torch.int64)\n\n    expected_shape = (n_layers, total_ctx, num_kv_heads, head_dim)\n    if k_out is None:\n        k_out = torch.empty(expected_shape, dtype=kv.dtype, device=kv.device)\n    else:\n        if k_out.shape != expected_shape:\n            raise ValueError(","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L151-L187","documentation":"The per-layer RMSNorm epsilon values must be supplied as a 1D tensor of length n_layers, one eps per layer. Any other shape is rejected.","triggerScenarios":"Passing a scalar eps tensor, a Python-float, or an eps tensor whose length doesn't match the layer count in the stacked kv tensor.","commonSituations":"Building the fused inputs manually from model config where rms_norm_eps is a scalar and was not expanded per layer.","solutions":["Build eps as torch.full((n_layers,), config.rms_norm_eps, device=...).","Ensure n_layers matches kv.shape[1].","Use the materializer class __init__ which collects eps per layer automatically."],"exampleFix":"// before\neps = torch.tensor(1e-5)\n// after\neps = torch.full((n_layers,), 1e-5, device=kv.device)","handlingStrategy":"validation","validationCode":"assert eps.shape == (n_layers,) and eps.numel() == kv.shape[1]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Build eps with torch.full((n_layers,), cfg.rms_norm_eps)."],"tags":["shape-validation","rmsnorm","speculative-decoding"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}