{"record":{"id":"8dd191137b1c77b7","repo":"affaan-m/ECC","slug":"model-promotion-metrics-missing-required-gates-m","errorCode":null,"errorMessage":"Model promotion metrics missing required gates: {missing}","messagePattern":"Model promotion metrics missing required gates: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"skills/mle-workflow/SKILL.md","lineNumber":277,"sourceCode":"- Primary metric aligned to product behavior\n- Guardrail metrics for latency, calibration, fairness slices, cost, and error concentration\n- Slice metrics for important cohorts, geographies, devices, languages, or data sources\n- Confidence intervals or repeated-run variance when metrics are noisy\n- Failure examples reviewed by a human for high-impact models\n- Explicit \"do not ship\" thresholds\n\n```python\nPROMOTION_GATES = {\n    \"auc\": (\"min\", 0.82),\n    \"calibration_error\": (\"max\", 0.04),\n    \"p95_latency_ms\": (\"max\", 80),\n}\n\n\ndef assert_promotion_ready(metrics: dict[str, float]) -> None:\n    missing = sorted(name for name in PROMOTION_GATES if name not in metrics)\n    if missing:\n        raise ValueError(f\"Model promotion metrics missing required gates: {missing}\")\n\n    failures = {\n        name: value\n        for name, (direction, threshold) in PROMOTION_GATES.items()\n        for value in [metrics[name]]\n        if (direction == \"min\" and value < threshold)\n        or (direction == \"max\" and value > threshold)\n    }\n    if failures:\n        raise ValueError(f\"Model failed promotion gates: {failures}\")\n```\n\nUse offline metrics as gates, not guarantees. When the model changes product behavior, plan shadow evaluation, canary rollout, or A/B testing before full rollout.\n\n### 5. Package for Serving\n\nAn ML artifact is production-ready only when the serving contract is testable:\n","sourceCodeStart":259,"sourceCodeEnd":295,"githubUrl":"https://github.com/affaan-m/ECC/blob/01e15490f04e29cfefe3896951f43db46994d8ee/skills/mle-workflow/SKILL.md#L259-L295","documentation":"`assert_promotion_ready(metrics)` enforces that every name in `PROMOTION_GATES` (auc, calibration_error, p95_latency_ms) is present in the supplied `metrics` dict. If any are missing, it raises `ValueError(\"Model promotion metrics missing required gates: {missing}\")` before evaluating thresholds. The missing-key check runs first to distinguish 'not measured' from 'measured and failed.'","triggerScenarios":"A candidate model's eval report omits one of the gated metrics (e.g. the latency harness was skipped, or `calibration_error` not computed for a regressor). Promotion is attempted with a partial metrics dict.","commonSituations":"Eval pipeline changed and dropped a metric; threshold renamed but metrics dict still uses the old key; classifier vs regressor eval report reused; CI gates run on a stub metrics dict.","solutions":["Re-run the full eval suite so the metrics dict contains every gate key.","Align key names between `PROMOTION_GATES` and the eval report (case-sensitive).","If a gate is not applicable to this model class, formally exclude it (e.g. a per-model gate config) rather than omit the metric.","Add a unit test that promotion fails on a partial dict and succeeds on a complete passing dict."],"exampleFix":"# before\nmissing = sorted(name for name in PROMOTION_GATES if name not in metrics)\nif missing:\n    raise ValueError(f\"Model promotion metrics missing required gates: {missing}\")\n\n# after — explicit gate applicability per model kind\nAPPLICABLE = {\"classifier\": [\"auc\",\"calibration_error\",\"p95_latency_ms\"], \"regressor\": [\"p95_latency_ms\"]}\ngates = {k: PROMOTION_GATES[k] for k in APPLICABLE[model_kind]}\nmissing = sorted(name for name in gates if name not in metrics)\nif missing:\n    raise ValueError(f\"Missing required gates for {model_kind}: {missing}\")","handlingStrategy":"validation","validationCode":"def has_all_gates(metrics: dict) -> bool:\n    return all(name in metrics for name in PROMOTION_GATES)","typeGuard":"def is_complete_metrics(metrics: dict) -> bool:\n    return isinstance(metrics, dict) and all(\n        isinstance(metrics.get(k), (int, float)) for k in PROMOTION_GATES\n    )","tryCatchPattern":"try:\n    assert_promotion_ready(metrics)\nexcept ValueError as e:\n    if 'missing required gates' in str(e):\n        rerun_full_eval_suite()\n    raise","preventionTips":["Run the full eval suite (all gated metrics) on every candidate, not a subset.","Keep `PROMOTION_GATES` key names in sync with the eval report (case-sensitive).","Per-model-kind applicability config prevents partial dicts for regressor vs classifier."],"tags":["mlops","model-evaluation","validation","deployment-gates"],"backgroundTag":null,"analyzedSha":"01e15490f04e29cfefe3896951f43db46994d8ee","analyzedAt":"2026-08-13T00:31:08.655Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}