{"record":{"id":"6a06a448f45097b2","repo":"affaan-m/ECC","slug":"scores-must-be-finite-and-between-0-and-1","errorCode":null,"errorMessage":"scores must be finite and between 0 and 1","messagePattern":"scores must be finite and between 0 and 1","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"ecc2/src/harness_eval.rs","lineNumber":539,"sourceCode":"        if samples.is_empty() {\n            bail!(\"samples cannot be empty\");\n        }\n        if samples\n            .iter()\n            .map(|sample| sample.seed)\n            .collect::<BTreeSet<_>>()\n            .len()\n            != samples.len()\n        {\n            bail!(\"sample seeds must be unique\");\n        }\n        if samples.iter().any(|s| {\n            !s.candidate_score.is_finite()\n                || !s.baseline_score.is_finite()\n                || !(0.0..=1.0).contains(&s.candidate_score)\n                || !(0.0..=1.0).contains(&s.baseline_score)\n        }) {\n            bail!(\"scores must be finite and between 0 and 1\");\n        }\n        let count = samples.len();\n        let candidate_mean = samples.iter().map(|s| s.candidate_score).sum::<f64>() / count as f64;\n        let baseline_mean = samples.iter().map(|s| s.baseline_score).sum::<f64>() / count as f64;\n        let mean_delta = candidate_mean - baseline_mean;\n        let win_rate = samples\n            .iter()\n            .filter(|s| s.candidate_score > s.baseline_score)\n            .count() as f64\n            / count as f64;\n        let mut failures = Vec::new();\n        if count < self.min_samples {\n            failures.push(format!(\n                \"minimum sample count is {}, got {count}\",\n                self.min_samples\n            ));\n        }\n        if mean_delta < self.min_mean_delta {","sourceCodeStart":521,"sourceCodeEnd":557,"githubUrl":"https://github.com/affaan-m/ECC/blob/8321021c54d670126ce3b2969d5deb880b4b0c2a/ecc2/src/harness_eval.rs#L521-L557","documentation":"`compare` validates each sample's `candidate_score` and `baseline_score`: both must be finite (not NaN or infinity) and within the inclusive range 0.0..=1.0. Non-finite or out-of-range values would corrupt mean, delta, and win-rate calculations. Any single bad sample causes the whole comparison to be rejected.","triggerScenarios":"Calling `compare` with a sample whose score is NaN, infinite, negative, or greater than 1.0 — typically from an unnormalized metric or a failed/errored evaluation recorded as NaN.","commonSituations":"Metrics that are not normalized to [0,1] passed in directly, division-by-zero in score computation producing NaN, deserialized JSON containing null/defaulted scores, or a task that timed out and produced an infinite penalty score.","solutions":["Normalize scores to [0,1] before constructing PairedSample (clamp and map the metric).","Sanitize or drop non-finite scores (NaN/inf) before collecting samples.","Check the upstream scoring function for NaN sources (empty outputs, division by zero)."],"exampleFix":"// before\nlet score = raw_value;\n// after\nlet score = if raw_value.is_finite() { raw_value.clamp(0.0, 1.0) } else { 0.0 };","handlingStrategy":"validation","validationCode":"anyhow::ensure!(\n    samples.iter().all(|s| s.candidate_score.is_finite()\n        && s.baseline_score.is_finite()\n        && (0.0..=1.0).contains(&s.candidate_score)\n        && (0.0..=1.0).contains(&s.baseline_score)),\n    \"sample scores must be finite and in [0,1]\"\n);","typeGuard":null,"tryCatchPattern":"match evaluator.compare(&samples) {\n    Ok(c) => c,\n    Err(e) if e.to_string().contains(\"finite and between 0 and 1\") => {\n        eprintln!(\"Score out of range; check scoring/normalization step\");\n        return Err(e);\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Clamp and normalize metrics into [0,1] at the scoring boundary","Replace non-finite scores (NaN/inf) with a sentinel before sampling","Add a sanity check on scores right after each task evaluation"],"tags":["validation","numeric-range","nan","evaluation-harness"],"backgroundTag":"value-out-of-range","analyzedSha":"8321021c54d670126ce3b2969d5deb880b4b0c2a","analyzedAt":"2026-09-16T10:08:13.343Z","contentChangedAt":"2026-09-16T10:08:13.343Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}