{"record":{"id":"635dc157b3659b27","repo":"sgl-project/sglang","slug":"num-token-non-padded-must-be-a-torch-tensor","errorCode":null,"errorMessage":"num_token_non_padded must be a torch.Tensor","messagePattern":"num_token_non_padded must be a torch\\.Tensor","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/moe/fill_padded_rows.py","lineNumber":57,"sourceCode":"    x: torch.Tensor,\n    num_token_non_padded: torch.Tensor,\n    fill_value,\n) -> None:\n    \"\"\"Set ``x[row, :] = fill_value`` for every padded row (row index\n    ``>= num_token_non_padded``) using a single Triton launch.\n\n    Replaces the eager ``arange + (>=) + boolean index_put_`` sequence, which\n    issues several launch-latency-bound kernels per call. The grid is static\n    (one program per row) and the pad count is read from device memory inside\n    the kernel, so this is safe to capture inside a CUDA/HIP graph.\n    \"\"\"\n    # Metadata-only checks (no device sync): the kernel reads a single scalar\n    # routing count from device memory, so it must be a 1-element integer tensor\n    # on the same device as ``x``. Use explicit raises (not asserts) so the\n    # checks survive ``python -O`` and invalid inputs fail loudly instead of\n    # turning into opaque Triton/memory errors.\n    if not isinstance(num_token_non_padded, torch.Tensor):\n        raise TypeError(\"num_token_non_padded must be a torch.Tensor\")\n    if num_token_non_padded.numel() != 1:\n        raise ValueError(\n            \"num_token_non_padded must be a single-element tensor, got shape \"\n            f\"{tuple(num_token_non_padded.shape)}\"\n        )\n    if num_token_non_padded.dtype.is_floating_point:\n        raise TypeError(\n            \"num_token_non_padded must be an integer tensor, got \"\n            f\"{num_token_non_padded.dtype}\"\n        )\n    if num_token_non_padded.device != x.device:\n        raise ValueError(\"num_token_non_padded and x must be on the same device\")\n    n_rows, n_cols = x.shape\n    _fill_padded_rows_kernel[(n_rows,)](\n        x,\n        num_token_non_padded,\n        n_cols,\n        fill_value,","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/moe/fill_padded_rows.py#L39-L75","documentation":"A TypeError raised by the input validation inside _fill_padded_rows. The kernel reads a routing count scalar directly from device memory, so num_token_non_padded must be a 1-element torch.Tensor; passing a Python int or anything else fails this check before launching the Triton kernel.","triggerScenarios":"Calling _mask_topk_ids_padded_region or _zero_topk_weights_padded_region (or _fill_padded_rows directly) with a Python int/float/numpy scalar for num_token_non_padded instead of a device-resident tensor.","commonSituations":"Refactoring a MoE padding path where the token count used to be a host-side int; passing cpu_tensor.item() results; porting test code that lazily passes a plain number.","solutions":["Wrap the count in a tensor: torch.tensor(n, dtype=torch.int32, device=x.device)","Check the caller supplying the value — the scheduler usually already carries it as a device tensor; use that instead of a host int","If a host int is all you have, accept the (small) sync and move it to device with .to(x.device)"],"exampleFix":"// before\n_mask_topk_ids_padded_region(topk_ids, n, fill_value=-1)\n// after\nn_t = torch.tensor(n, dtype=torch.int32, device=topk_ids.device)\n_mask_topk_ids_padded_region(topk_ids, n_t, fill_value=-1)","handlingStrategy":"type-guard","validationCode":"def as_pad_count(v, device):\n    if not isinstance(v, torch.Tensor):\n        v = torch.tensor(v, dtype=torch.int32, device=device)\n    return v","typeGuard":"def is_valid_pad_count(v) -> bool:\n    return isinstance(v, torch.Tensor) and v.numel() == 1 and not v.dtype.is_floating_point","tryCatchPattern":null,"preventionTips":["Always construct num_token_non_padded with device=x.device and dtype=torch.int32","Centralize scalar-tensor creation in one helper so all call sites agree"],"tags":["moe","triton","type-validation","tensor-shape"],"backgroundTag":"invalid-tensor-argument-type","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}