affaan-m/ECC · error · ValueError

Model promotion metrics missing required gates

Error message

Model promotion metrics missing required gates: {missing}

What it means

Illustrative gate check from the mle-workflow skill: assert_promotion_ready first verifies every metric named in PROMOTION_GATES is present in the metrics dict; any missing key (auc, calibration_error, p95_latency_ms, ...) aborts promotion before threshold comparison. An incomplete evaluation report is the input at fault.

Solutions

1. Fail CI when the metrics artifact is missing any declared gate
2. Compute all gate metrics in the same evaluation run that produces them
3. List required gates in the promotion config schema so omissions surface early

When it happens

Trigger: Thrown at skills/mle-workflow/SKILL.md:277 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of affaan-m/ECC@d8409a4b08 (2026-08-26). Data as JSON: /api/errors/8dd191137b1c77b7. Report an issue: GitHub.

Appendix: source

Thrown at skills/mle-workflow/SKILL.md:277

- Primary metric aligned to product behavior
- Guardrail metrics for latency, calibration, fairness slices, cost, and error concentration
- Slice metrics for important cohorts, geographies, devices, languages, or data sources
- Confidence intervals or repeated-run variance when metrics are noisy
- Failure examples reviewed by a human for high-impact models
- Explicit "do not ship" thresholds

```python
PROMOTION_GATES = {
    "auc": ("min", 0.82),
    "calibration_error": ("max", 0.04),
    "p95_latency_ms": ("max", 80),
}


def assert_promotion_ready(metrics: dict[str, float]) -> None:
    missing = sorted(name for name in PROMOTION_GATES if name not in metrics)
    if missing:
        raise ValueError(f"Model promotion metrics missing required gates: {missing}")

    failures = {
        name: value
        for name, (direction, threshold) in PROMOTION_GATES.items()
        for value in [metrics[name]]
        if (direction == "min" and value < threshold)
        or (direction == "max" and value > threshold)
    }
    if failures:
        raise ValueError(f"Model failed promotion gates: {failures}")
```

Use offline metrics as gates, not guarantees. When the model changes product behavior, plan shadow evaluation, canary rollout, or A/B testing before full rollout.

### 5. Package for Serving

An ML artifact is production-ready only when the serving contract is testable:

View on GitHub (pinned to d8409a4b08)