{"record":{"id":"a68b2f3eaa2303ef","repo":"affaan-m/ECC","slug":"model-failed-promotion-gates-failures","errorCode":null,"errorMessage":"Model failed promotion gates: {failures}","messagePattern":"Model failed promotion gates: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"skills/mle-workflow/SKILL.md","lineNumber":287,"sourceCode":"    \"calibration_error\": (\"max\", 0.04),\n    \"p95_latency_ms\": (\"max\", 80),\n}\n\n\ndef assert_promotion_ready(metrics: dict[str, float]) -> None:\n    missing = sorted(name for name in PROMOTION_GATES if name not in metrics)\n    if missing:\n        raise ValueError(f\"Model promotion metrics missing required gates: {missing}\")\n\n    failures = {\n        name: value\n        for name, (direction, threshold) in PROMOTION_GATES.items()\n        for value in [metrics[name]]\n        if (direction == \"min\" and value < threshold)\n        or (direction == \"max\" and value > threshold)\n    }\n    if failures:\n        raise ValueError(f\"Model failed promotion gates: {failures}\")\n```\n\nUse offline metrics as gates, not guarantees. When the model changes product behavior, plan shadow evaluation, canary rollout, or A/B testing before full rollout.\n\n### 5. Package for Serving\n\nAn ML artifact is production-ready only when the serving contract is testable:\n\n- Model artifact includes version, training data reference, config, and preprocessing\n- Input schema rejects invalid, stale, or out-of-range features\n- Output schema includes model version and confidence or explanation fields when useful\n- Serving path has timeout, batching, resource limits, and fallback behavior\n- CPU/GPU requirements are explicit and tested\n- Prediction logs avoid PII and include enough identifiers for debugging and label joins\n- Integration tests cover missing features, stale features, bad types, empty batches, and fallback path\n\nNever let training-only feature code diverge from serving feature code without a test that proves equivalence.\n","sourceCodeStart":269,"sourceCodeEnd":305,"githubUrl":"https://github.com/affaan-m/ECC/blob/01e15490f04e29cfefe3896951f43db46994d8ee/skills/mle-workflow/SKILL.md#L269-L305","documentation":"After confirming all gate keys exist (597), `assert_promotion_ready` evaluates each gate using its direction (`min`/`max`) and threshold. Any metric on the wrong side of its threshold is collected into `failures`; if non-empty, the function raises `ValueError(\"Model failed promotion gates: {failures}\")`. Promotion is all-or-nothing.","triggerScenarios":"A model passes some gates but fails at least one: AUC below 0.82, calibration_error above 0.04, or p95_latency_ms above 80. Any one failure blocks promotion.","commonSituations":"A new model has better AUC but worse latency (cross-gate trade-off). Thresholds were calibrated on a different dataset. Eval ran on a slower machine inflating latency. Calibration regressed after a preprocessing change.","solutions":["Inspect the `failures` dict in the error message — it names the failing metric and its value.","For latency failures, re-run on the production-representative hardware profile.","For AUC/calibration, revisit features/preprocessing or retrain; do not relax the gate without sign-off.","If the trade-off is acceptable, formally revise `PROMOTION_GATES` (with rationale) so the change is auditable."],"exampleFix":"# before\nfailures = {\n    name: value\n    for name, (direction, threshold) in PROMOTION_GATES.items()\n    for value in [metrics[name]]\n    if (direction == \"min\" and value < threshold)\n    or (direction == \"max\" and value > threshold)\n}\nif failures:\n    raise ValueError(f\"Model failed promotion gates: {failures}\")\n\n# after — include direction+threshold so the failure is actionable\nfailures = {}\nfor name, (direction, threshold) in PROMOTION_GATES.items():\n    value = metrics[name]\n    ok = value >= threshold if direction == \"min\" else value <= threshold\n    if not ok:\n        failures[name] = {\"value\": value, \"direction\": direction, \"threshold\": threshold}\nif failures:\n    raise ValueError(f\"Model failed promotion gates: {failures}\")","handlingStrategy":"try-catch","validationCode":"null","typeGuard":"null","tryCatchPattern":"try:\n    assert_promotion_ready(metrics)\nexcept ValueError as e:\n    if 'failed promotion gates' in str(e):\n        block_deployment(e)  # do not ship; investigate failures dict\n    raise","preventionTips":["Treat all gates as hard blockers; never relax a threshold without sign-off.","Run latency evals on production-representative hardware.","Version `PROMOTION_GATES` so threshold changes are auditable."],"tags":["mlops","model-evaluation","deployment-gates","validation"],"backgroundTag":null,"analyzedSha":"01e15490f04e29cfefe3896951f43db46994d8ee","analyzedAt":"2026-08-13T00:31:08.655Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}