affaan-m/ECC · error · ValueError

Model promotion metrics missing required gates: {missing}

Error message

Model promotion metrics missing required gates: {missing}

What it means

`assert_promotion_ready(metrics)` enforces that every name in `PROMOTION_GATES` (auc, calibration_error, p95_latency_ms) is present in the supplied `metrics` dict. If any are missing, it raises `ValueError("Model promotion metrics missing required gates: {missing}")` before evaluating thresholds. The missing-key check runs first to distinguish 'not measured' from 'measured and failed.'

Source

Thrown at skills/mle-workflow/SKILL.md:277

- Primary metric aligned to product behavior
- Guardrail metrics for latency, calibration, fairness slices, cost, and error concentration
- Slice metrics for important cohorts, geographies, devices, languages, or data sources
- Confidence intervals or repeated-run variance when metrics are noisy
- Failure examples reviewed by a human for high-impact models
- Explicit "do not ship" thresholds

```python
PROMOTION_GATES = {
    "auc": ("min", 0.82),
    "calibration_error": ("max", 0.04),
    "p95_latency_ms": ("max", 80),
}


def assert_promotion_ready(metrics: dict[str, float]) -> None:
    missing = sorted(name for name in PROMOTION_GATES if name not in metrics)
    if missing:
        raise ValueError(f"Model promotion metrics missing required gates: {missing}")

    failures = {
        name: value
        for name, (direction, threshold) in PROMOTION_GATES.items()
        for value in [metrics[name]]
        if (direction == "min" and value < threshold)
        or (direction == "max" and value > threshold)
    }
    if failures:
        raise ValueError(f"Model failed promotion gates: {failures}")
```

Use offline metrics as gates, not guarantees. When the model changes product behavior, plan shadow evaluation, canary rollout, or A/B testing before full rollout.

### 5. Package for Serving

An ML artifact is production-ready only when the serving contract is testable:

View on GitHub (pinned to 01e15490f0)

Solutions

1. Re-run the full eval suite so the metrics dict contains every gate key.
2. Align key names between `PROMOTION_GATES` and the eval report (case-sensitive).
3. If a gate is not applicable to this model class, formally exclude it (e.g. a per-model gate config) rather than omit the metric.
4. Add a unit test that promotion fails on a partial dict and succeeds on a complete passing dict.

Example fix

# before
missing = sorted(name for name in PROMOTION_GATES if name not in metrics)
if missing:
    raise ValueError(f"Model promotion metrics missing required gates: {missing}")

# after — explicit gate applicability per model kind
APPLICABLE = {"classifier": ["auc","calibration_error","p95_latency_ms"], "regressor": ["p95_latency_ms"]}
gates = {k: PROMOTION_GATES[k] for k in APPLICABLE[model_kind]}
missing = sorted(name for name in gates if name not in metrics)
if missing:
    raise ValueError(f"Missing required gates for {model_kind}: {missing}")

def has_all_gates(metrics: dict) -> bool:
    return all(name in metrics for name in PROMOTION_GATES)

def is_complete_metrics(metrics: dict) -> bool:
    return isinstance(metrics, dict) and all(
        isinstance(metrics.get(k), (int, float)) for k in PROMOTION_GATES
    )

try:
    assert_promotion_ready(metrics)
except ValueError as e:
    if 'missing required gates' in str(e):
        rerun_full_eval_suite()
    raise

When it happens

Trigger: A candidate model's eval report omits one of the gated metrics (e.g. the latency harness was skipped, or `calibration_error` not computed for a regressor). Promotion is attempted with a partial metrics dict.

Common situations: Eval pipeline changed and dropped a metric; threshold renamed but metrics dict still uses the old key; classifier vs regressor eval report reused; CI gates run on a stub metrics dict.

Related errors

• Model failed promotion gates: {failures}
• ECC_PROJECT_DIR must be a child path within /workspace.
• Unknown argument: ${arg}
• Unable to infer ECC repo root from install-state operations
• Invalid ECC repo root: missing package.json at ${packageJson

AI-assisted analysis of affaan-m/ECC@01e15490f0 (2026-08-13). Data as JSON: /api/errors/8dd191137b1c77b7. Report an issue: GitHub.