{"record":{"id":"0ff6f38be2131b96","repo":"sgl-project/sglang","slug":"shape-mismatch-a-k-k-b-k-b-shape-1-2","errorCode":null,"errorMessage":"Shape mismatch: A K={k}, B K={b.shape[1] * 2}","messagePattern":"Shape mismatch: A K=(.+?), B K=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py","lineNumber":2878,"sourceCode":"    if ab_dtype != \"float4_e2m1fn\" or c_dtype != \"float4_e2m1fn\":\n        raise ValueError(\n            \"nvfp4_gemm_swiglu_nvfp4_quant currently supports NVFP4 input \"\n            \"and output only\"\n        )\n    if a.device.type != \"cuda\" or b.device.type != \"cuda\":\n        raise ValueError(\"nvfp4_gemm_swiglu_nvfp4_quant requires CUDA tensors\")\n\n    major, minor = get_compute_capability(a.device)\n    if major != 10:\n        raise ValueError(\n            f\"nvfp4_gemm_swiglu_nvfp4_quant requires SM100, got SM{major}{minor}\"\n        )\n\n    m = a.shape[0]\n    k = a.shape[1] * 2\n    n = b.shape[0]\n    if b.shape[1] * 2 != k:\n        raise ValueError(f\"Shape mismatch: A K={k}, B K={b.shape[1] * 2}\")\n    if n % 2 != 0:\n        raise ValueError(f\"Interleaved FC1 N must be even, got {n}\")\n\n    l = 1\n    n_out = n // 2\n    if n_out % sf_vec_size != 0:\n        raise ValueError(\n            f\"Output N={n_out} must be divisible by sf_vec_size={sf_vec_size}\"\n        )\n    scale_n_out = n_out // sf_vec_size\n    padded_m = _round_up(m, 128)\n    padded_scale_n = _round_up(scale_n_out, 4)\n\n    ab_dtype_cutlass = get_cutlass_dtype(ab_dtype)\n    sf_dtype_cutlass = get_cutlass_dtype(sf_dtype)\n    c_dtype_cutlass = get_cutlass_dtype(c_dtype)\n\n    if m <= 128:","sourceCodeStart":2860,"sourceCodeEnd":2896,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py#L2860-L2896","documentation":"The fused kernel computes K from the activation as a.shape[1]*2 (FP4 elements are packed two per byte) and from the weight as b.shape[1]*2. If these disagree, the GEMM contraction dimensions are inconsistent and the kernel cannot proceed.","triggerScenarios":"Calling nvfp4_gemm_swiglu_nvfp4_quant where a.shape[1]*2 != b.shape[1]*2 — e.g. passing a packed-K activation with an unpacked weight or weights from a different hidden_size.","commonSituations":"Loading a checkpoint whose hidden_size doesn't match the config, passing scale tensors/weights prepared for a different layer, or accidentally transposing b so its packed-K axis lands in shape[0].","solutions":["Verify a.shape[1] == b.shape[1] (packed K) before the call","Check that b is in the expected K-major layout ([K_packed, N] per the kernel's expectation) and re-repack weights if needed","Confirm the model config hidden_size matches the checkpoint"],"exampleFix":"// before\nassert a.shape[1]*2 == b.shape[1]*2\n// after\nif a.shape[1]*2 != b.shape[1]*2:\n    b = repack_b_for_nvfp4(b_raw)  # fix layout\nout = nvfp4_gemm_swiglu_nvfp4_quant(a, b, ...)","handlingStrategy":"validation","validationCode":"assert a.shape[1] == b.shape[1], (a.shape, b.shape)","typeGuard":"def k_dims_match(a, b): return a.shape[1]*2 == b.shape[1]*2","tryCatchPattern":null,"preventionTips":["Validate checkpoint hidden_size against config before loading"],"tags":["nvfp4","shape-mismatch","gemm","quantization"],"backgroundTag":"matmul-dimension-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}