{"record":{"id":"43f08ae26af880d8","repo":"Hmbown/CodeWhale","slug":"metric-s-regressed-past-the-1-threshold","errorCode":null,"errorMessage":"{} metric(s) regressed past the {:.1}% threshold","messagePattern":"(.+?) metric\\(s\\) regressed past the (.+?)% threshold","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/tui/src/lib.rs","lineNumber":2865,"sourceCode":"            \"per_turn\": card.per_turn,\n            \"metrics\": card.metrics,\n            \"regressions\": regressions,\n        });\n        println!(\"{}\", serde_json::to_string_pretty(&out)?);\n    } else {\n        print!(\"{}\", card.to_summary());\n        for r in &regressions {\n            println!(\n                \"REGRESSION {}: baseline {:.4} -> current {:.4} (+{:.1}%)\",\n                r.metric, r.baseline, r.current, r.pct_increase\n            );\n        }\n    }\n\n    if regressions.is_empty() {\n        Ok(())\n    } else {\n        bail!(\n            \"{} metric(s) regressed past the {:.1}% threshold\",\n            regressions.len(),\n            args.threshold\n        )\n    }\n}\n\nasync fn run_fleet_command(workspace: &Path, config: &Config, args: FleetArgs) -> Result<()> {\n    use crate::fleet::alerts::{\n        FleetAlertAdapterConfig, FleetAlertConfig, FleetAlertDispatcher, FleetAlertEvent,\n        FleetEnvSecretResolver,\n    };\n    use crate::fleet::control as fleet_control;\n    use crate::fleet::executor::FleetExecutor;\n    use crate::fleet::manager::{FleetManager, FleetStatusSnapshot, FleetWorkerInspection};\n    use codewhale_lane::{ControlOperation, ControlSurface};\n    use codewhale_protocol::fleet::{FleetAlertEventClass, FleetArtifactKind, FleetRunId};\n","sourceCodeStart":2847,"sourceCodeEnd":2883,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/0c42157ee52f9d55af2b506d71b46249910f77d3/crates/tui/src/lib.rs#L2847-L2883","documentation":"run_scorecard scores recorded turns from --input JSON into token/cache/cost metrics and, when --baseline is supplied, flags every metric that increased more than --threshold percent (default 5.0). A non-empty regression list fails the command so it can act as a release gate; the printed REGRESSION lines name each metric with its baseline -> current movement and percent increase.","triggerScenarios":"`codewhale scorecard --input turns.json --baseline baseline.json` where any scored metric grew past the threshold: costlier routing per turn, lost prompt-cache hits, larger contexts, or a baseline recorded against a cheaper model mix. Also a deliberately strict threshold applied to known-noisy metrics.","commonSituations":"Release/CI gates tripping after a model or routing change; prompt changes that bust the cache; baselines from an older pricing tier; legitimate traffic growth mistaken for regression.","solutions":["Read each REGRESSION line and identify which metric (cost, tokens, cache) drove it","Check for real regressions first: routing changes, cache-busting prompt edits, context growth per turn","If the growth is accepted, regenerate the baseline from the current run so the gate tracks the new normal","Only if a metric is known-noisy, widen --threshold deliberately and record why"],"exampleFix":"# before\ncodewhale scorecard --input turns.json --baseline baseline.json\n# 2 metric(s) regressed past the 5.0% threshold\n\n# after (accepted growth: re-baseline the new normal)\ncodewhale scorecard --input turns.json --json > baseline.json\ngit add baseline.json && git commit -m 'scorecard: re-baseline after routing change'","handlingStrategy":"try-catch","validationCode":"# Dry-check before wiring the gate: run without --baseline to inspect metrics first\ncodewhale scorecard --input turns.json --json | jq '.metrics'","typeGuard":null,"tryCatchPattern":"if ! codewhale scorecard --input turns.json --baseline baseline.json; then\n  # output lists each REGRESSION metric; triage real regressions vs accepted growth\n  exit 1\nfi","preventionTips":["Commit the baseline file and regenerate it deliberately after accepted changes","Watch scorecard in baseline-less mode while tuning prompts/routing","Record why whenever --threshold is widened"],"tags":["scorecard","regression","release-gate","metrics"],"backgroundTag":"performance-regression-gate","analyzedSha":"0c42157ee52f9d55af2b506d71b46249910f77d3","analyzedAt":"2026-08-20T21:50:45.477Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}