{"record":{"id":"683f52e309087a3a","repo":"vllm-project/vllm","slug":"numerics-check-failed-for-case-case-n-e","errorCode":null,"errorMessage":"Numerics check failed for case {case}:\\n{e}","messagePattern":"Numerics check failed for case (.+?):\\\\n(.+?)","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"scripts/benchmark_helion_kernels.py","lineNumber":361,"sourceCode":"\n        configured_atol = getattr(settings, \"autotune_baseline_atol\", None)\n        configured_rtol = getattr(settings, \"autotune_baseline_rtol\", None)\n        atol = 1e-2 if configured_atol is None else configured_atol\n        rtol = 1e-2 if configured_rtol is None else configured_rtol\n        _assert_close(\n            kernel_output,\n            baseline_output,\n            atol=atol,\n            rtol=rtol,\n        )\n        _assert_close(\n            kernel_inputs,\n            baseline_inputs,\n            atol=atol,\n            rtol=rtol,\n        )\n    except AssertionError as e:\n        raise AssertionError(f\"Numerics check failed for case {case}:\\n{e}\") from e\n\n\n@dataclass\nclass CorrectnessResult:\n    \"\"\"Outcome of the numerics check for a single shape case.\"\"\"\n\n    case: str\n    passed: bool\n    error: str | None = None\n\n\ndef check_kernel_correctness(\n    kernel: Any,\n    baseline_fn: Callable,\n    inputs_dict: dict[Any, tuple[Any, ...]] | None = None,\n) -> list[CorrectnessResult]:\n    \"\"\"Run the per-shape numerics check for a kernel, continuing past failures.\n","sourceCodeStart":343,"sourceCodeEnd":379,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/scripts/benchmark_helion_kernels.py#L343-L379","documentation":"Wrapper assertion in scripts/benchmark_helion_kernels.py (line 361): re-raises any AssertionError from the numerics comparison (`_assert_close` on outputs and on mutated inputs) with the failing case label prepended, so you know which shape/config case diverged. The inner message carries the real cause: either a structure mismatch (error 114) or values outside atol/rtol tolerance; FP8 leaves get a fixed atol=1, rtol=0.","triggerScenarios":"Running `check_kernel_correctness` across shape cases: the candidate kernel's outputs (or its in-place modifications to inputs) differ from baseline by more than atol/rtol for the named case.","commonSituations":"Numerically sloppy kernel rewrites (different reduction order, missing masking, fused ops losing precision); too-tight atol/rtol defaults for bf16/fp16 accumulations; nondeterministic atomics producing run-to-run drift.","solutions":["Read the inner error: 'Output structure mismatch' means fix shapes/dtypes first; otherwise values diverge beyond tolerance","Compare the kernel's math against the baseline op-by-op (masking, accumulation dtype, reduction order) for the failing case","If the divergence is within the noise floor of the dtype, widen atol/rtol explicitly for that case rather than globally"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"for leaf in tree_flatten(kernel_output)[0]:\n    if isinstance(leaf, torch.Tensor):\n        assert leaf.isfinite().all(), \"kernel produced NaN/inf before tolerance check\"","typeGuard":null,"tryCatchPattern":"try:\n    check_kernel_correctness(case, ...)\nexcept AssertionError as e:\n    log_case_failure(case, str(e))  # keep the case label; inspect inner tolerance/structure message\n    raise","preventionTips":["Validate finiteness of kernel outputs before comparing to baseline","Set per-dtype tolerances explicitly (bf16/fp16 need looser atol) and fix the math before loosening further"],"tags":["python","benchmark","helion","numerics","testing"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}