{"record":{"id":"79b0b86522140b4d","repo":"sgl-project/sglang","slug":"num-token-non-padded-must-be-a-single-element-tens","errorCode":null,"errorMessage":"num_token_non_padded must be a single-element tensor, got shape {tuple(num_token_non_padded.shape)}","messagePattern":"num_token_non_padded must be a single-element tensor, got shape (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/moe/fill_padded_rows.py","lineNumber":59,"sourceCode":"    fill_value,\n) -> None:\n    \"\"\"Set ``x[row, :] = fill_value`` for every padded row (row index\n    ``>= num_token_non_padded``) using a single Triton launch.\n\n    Replaces the eager ``arange + (>=) + boolean index_put_`` sequence, which\n    issues several launch-latency-bound kernels per call. The grid is static\n    (one program per row) and the pad count is read from device memory inside\n    the kernel, so this is safe to capture inside a CUDA/HIP graph.\n    \"\"\"\n    # Metadata-only checks (no device sync): the kernel reads a single scalar\n    # routing count from device memory, so it must be a 1-element integer tensor\n    # on the same device as ``x``. Use explicit raises (not asserts) so the\n    # checks survive ``python -O`` and invalid inputs fail loudly instead of\n    # turning into opaque Triton/memory errors.\n    if not isinstance(num_token_non_padded, torch.Tensor):\n        raise TypeError(\"num_token_non_padded must be a torch.Tensor\")\n    if num_token_non_padded.numel() != 1:\n        raise ValueError(\n            \"num_token_non_padded must be a single-element tensor, got shape \"\n            f\"{tuple(num_token_non_padded.shape)}\"\n        )\n    if num_token_non_padded.dtype.is_floating_point:\n        raise TypeError(\n            \"num_token_non_padded must be an integer tensor, got \"\n            f\"{num_token_non_padded.dtype}\"\n        )\n    if num_token_non_padded.device != x.device:\n        raise ValueError(\"num_token_non_padded and x must be on the same device\")\n    n_rows, n_cols = x.shape\n    _fill_padded_rows_kernel[(n_rows,)](\n        x,\n        num_token_non_padded,\n        n_cols,\n        fill_value,\n        x.stride(0),\n        BLOCK_COLS=triton.next_power_of_2(n_cols),","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/moe/fill_padded_rows.py#L41-L77","documentation":"ValueError raised when num_token_non_padded is a torch.Tensor but does not contain exactly one element. The Triton kernel dereferences a single scalar from device memory, so multi-element tensors (shapes like (n,) or ()) with more than one value are rejected.","triggerScenarios":"Passing a full per-token padding-count vector or an accidentally broadcast tensor (e.g. shape (1,1) works via numel==1? no — (1,1).numel()==1 passes; shapes like (2,) or (n,) fail) to _fill_padded_rows via _mask_topk_ids_padded_region / _zero_topk_weights_padded_region.","commonSituations":"Plumbing a whole counts tensor through a helper meant for one scalar; slicing mistakes that keep a dimension; test fixtures constructing tensors with torch.zeros(n) instead of torch.tensor(n).","solutions":["Reduce to a single element: counts[0], counts.squeeze(), or torch.tensor(total, ...)","Verify upstream producer — usually num_token_non_padded comes from the scheduler as a 0-dim or (1,) tensor; fix the source that reshaped it"],"exampleFix":"// before\nnum = torch.tensor([n_real, n_pad], device=x.device)\n// after\nnum = torch.tensor(n_real, dtype=torch.int32, device=x.device)","handlingStrategy":"validation","validationCode":"assert num_token_non_padded.numel() == 1, f\"expected scalar, got {num_token_non_padded.shape}\"","typeGuard":"def is_scalar_count(t: torch.Tensor) -> bool:\n    return isinstance(t, torch.Tensor) and t.numel() == 1","tryCatchPattern":null,"preventionTips":["Use 0-dim tensors (torch.tensor(n)) as the canonical scalar representation","Never pass per-token count vectors into scalar-count APIs"],"tags":["moe","triton","tensor-shape","validation"],"backgroundTag":"invalid-tensor-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}