{"record":{"id":"6e8983290751ab1c","repo":"sgl-project/sglang","slug":"input-probs-contains-nan","errorCode":null,"errorMessage":"Input probs contains NaN.","messagePattern":"Input probs contains NaN\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/musa.py","lineNumber":248,"sourceCode":"        indices,\n        maybe_top_p_arr,\n        top_p_val,\n        deterministic,\n        generator,\n    )\n    return samples\n\n\ndef top_p_sampling_from_probs(\n    probs: torch.Tensor,\n    top_p: Union[torch.Tensor, float],\n    indices: Optional[torch.Tensor] = None,\n    deterministic: bool = True,\n    generator: Optional[torch.Generator] = None,\n    check_nan: bool = False,\n) -> torch.Tensor:\n    if check_nan and torch.any(torch.isnan(probs)):\n        raise ValueError(\"Input probs contains NaN.\")\n    return _top_p_sampling_from_probs_internal(\n        probs, indices, *_to_tensor_scalar_tuple(top_p), deterministic, generator\n    )\n\n\ndef _top_k_top_p_sampling_from_probs_internal(\n    probs: torch.Tensor,\n    indices: Optional[torch.Tensor],\n    maybe_top_k_arr: Optional[torch.Tensor],\n    top_k_val: int,\n    maybe_top_p_arr: Optional[torch.Tensor],\n    top_p_val: float,\n    deterministic: bool,\n    generator: Optional[torch.Generator],\n) -> torch.Tensor:\n    device = probs.device\n    probs = probs.float()\n    maybe_top_k_arr = maybe_top_k_arr.int() if maybe_top_k_arr is not None else None","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/musa.py#L230-L266","documentation":"The MUSA top-p sampling kernel wrapper optionally checks the probability tensor for NaN values before sampling. When check_nan=True and any element of probs is NaN, it raises immediately, because sampling from NaN probabilities yields garbage tokens.","triggerScenarios":"Calling top_p_sampling_from_probs(probs, top_p, check_nan=True) where torch.any(torch.isnan(probs)) is true; also hit indirectly via top_k_top_p_sampling_from_probs with top_k_first order.","commonSituations":"Upstream numerics producing NaN logits (inf - inf, overflow in bf16 softmax, masked-out rows of all -inf), which the check converts into a loud failure instead of silent garbage.","solutions":["Inspect logits before softmax for inf/NaN and fix masking so each row has at least one finite logit","Keep check_nan=False only after verifying upstream numerics are sound; otherwise leave it on in tests/debug","Clamp or normalize logits (e.g. subtract max, use softmax with dtype float32) before converting to probs"],"exampleFix":"# before\nsampled = top_p_sampling_from_probs(probs, top_p, check_nan=True)\n# after\nassert not torch.isnan(logits).any()\nprobs = torch.softmax(logits.float(), dim=-1)\nsampled = top_p_sampling_from_probs(probs, top_p, check_nan=True)","handlingStrategy":"validation","validationCode":"if check_nan and torch.isnan(probs).any(): raise RuntimeError('NaN probs from upstream logits')","typeGuard":"def probs_finite(p): return bool(torch.isfinite(p).all())","tryCatchPattern":"except ValueError as e: if 'NaN' in str(e): log offending rows and fix logits","preventionTips":["Validate logits are finite before softmax","Use float32 softmax"],"tags":["musa","sampling","top-p","nan"],"backgroundTag":"nan-in-probabilities","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}