{"record":{"id":"c1b0354faa454b17","repo":"sgl-project/sglang","slug":"num-token-non-padded-must-be-an-integer-tensor-go","errorCode":null,"errorMessage":"num_token_non_padded must be an integer tensor, got {num_token_non_padded.dtype}","messagePattern":"num_token_non_padded must be an integer tensor, got (.+?)","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/moe/fill_padded_rows.py","lineNumber":64,"sourceCode":"    Replaces the eager ``arange + (>=) + boolean index_put_`` sequence, which\n    issues several launch-latency-bound kernels per call. The grid is static\n    (one program per row) and the pad count is read from device memory inside\n    the kernel, so this is safe to capture inside a CUDA/HIP graph.\n    \"\"\"\n    # Metadata-only checks (no device sync): the kernel reads a single scalar\n    # routing count from device memory, so it must be a 1-element integer tensor\n    # on the same device as ``x``. Use explicit raises (not asserts) so the\n    # checks survive ``python -O`` and invalid inputs fail loudly instead of\n    # turning into opaque Triton/memory errors.\n    if not isinstance(num_token_non_padded, torch.Tensor):\n        raise TypeError(\"num_token_non_padded must be a torch.Tensor\")\n    if num_token_non_padded.numel() != 1:\n        raise ValueError(\n            \"num_token_non_padded must be a single-element tensor, got shape \"\n            f\"{tuple(num_token_non_padded.shape)}\"\n        )\n    if num_token_non_padded.dtype.is_floating_point:\n        raise TypeError(\n            \"num_token_non_padded must be an integer tensor, got \"\n            f\"{num_token_non_padded.dtype}\"\n        )\n    if num_token_non_padded.device != x.device:\n        raise ValueError(\"num_token_non_padded and x must be on the same device\")\n    n_rows, n_cols = x.shape\n    _fill_padded_rows_kernel[(n_rows,)](\n        x,\n        num_token_non_padded,\n        n_cols,\n        fill_value,\n        x.stride(0),\n        BLOCK_COLS=triton.next_power_of_2(n_cols),\n    )\n","sourceCodeStart":46,"sourceCodeEnd":79,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/moe/fill_padded_rows.py#L46-L79","documentation":"TypeError raised when num_token_non_padded is a floating-point tensor. The routing count is used as an integer row bound inside the Triton kernel, so a float dtype would silently truncate or misinterpret; the check enforces an integer dtype up front.","triggerScenarios":"Calling _fill_padded_rows (via _mask_topk_ids_padded_region / _zero_topk_weights_padded_region) with num_token_non_padded of dtype float16/bfloat16/float32/float64, e.g. torch.tensor(64.0, device=...).","commonSituations":"Reusing a tensor produced by a float computation or an averaged count; default dtype set to float via torch.set_default_dtype; converting from a model intermediate without casting.","solutions":["Create the tensor with an integer dtype: torch.tensor(n, dtype=torch.int32, device=x.device)","Cast an existing tensor: num.to(torch.int32) (only after confirming the value is integral)","Audit the producer of the scalar — it should never be a float in the first place"],"exampleFix":"// before\nnum = torch.tensor(64.0, device=x.device)\n// after\nnum = torch.tensor(64, dtype=torch.int32, device=x.device)","handlingStrategy":"validation","validationCode":"if num_token_non_padded.dtype.is_floating_point:\n    num_token_non_padded = num_token_non_padded.to(torch.int32)","typeGuard":"def is_int_scalar(t: torch.Tensor) -> bool:\n    return isinstance(t, torch.Tensor) and t.numel() == 1 and not t.dtype.is_floating_point","tryCatchPattern":null,"preventionTips":["Create count tensors with an explicit dtype=torch.int32","Treat floating-point counts as a bug in the producer, not something to cast around"],"tags":["moe","dtype","triton","validation"],"backgroundTag":"invalid-tensor-dtype","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}