affaan-m/ECC · error · ValueError

Model failed promotion gates: {failures}

Error message

Model failed promotion gates: {failures}

What it means

After confirming all gate keys exist (597), `assert_promotion_ready` evaluates each gate using its direction (`min`/`max`) and threshold. Any metric on the wrong side of its threshold is collected into `failures`; if non-empty, the function raises `ValueError("Model failed promotion gates: {failures}")`. Promotion is all-or-nothing.

Source

Thrown at skills/mle-workflow/SKILL.md:287

    "calibration_error": ("max", 0.04),
    "p95_latency_ms": ("max", 80),
}


def assert_promotion_ready(metrics: dict[str, float]) -> None:
    missing = sorted(name for name in PROMOTION_GATES if name not in metrics)
    if missing:
        raise ValueError(f"Model promotion metrics missing required gates: {missing}")

    failures = {
        name: value
        for name, (direction, threshold) in PROMOTION_GATES.items()
        for value in [metrics[name]]
        if (direction == "min" and value < threshold)
        or (direction == "max" and value > threshold)
    }
    if failures:
        raise ValueError(f"Model failed promotion gates: {failures}")
```

Use offline metrics as gates, not guarantees. When the model changes product behavior, plan shadow evaluation, canary rollout, or A/B testing before full rollout.

### 5. Package for Serving

An ML artifact is production-ready only when the serving contract is testable:

- Model artifact includes version, training data reference, config, and preprocessing
- Input schema rejects invalid, stale, or out-of-range features
- Output schema includes model version and confidence or explanation fields when useful
- Serving path has timeout, batching, resource limits, and fallback behavior
- CPU/GPU requirements are explicit and tested
- Prediction logs avoid PII and include enough identifiers for debugging and label joins
- Integration tests cover missing features, stale features, bad types, empty batches, and fallback path

Never let training-only feature code diverge from serving feature code without a test that proves equivalence.

View on GitHub (pinned to 01e15490f0)

Solutions

1. Inspect the `failures` dict in the error message — it names the failing metric and its value.
2. For latency failures, re-run on the production-representative hardware profile.
3. For AUC/calibration, revisit features/preprocessing or retrain; do not relax the gate without sign-off.
4. If the trade-off is acceptable, formally revise `PROMOTION_GATES` (with rationale) so the change is auditable.

Example fix

# before
failures = {
    name: value
    for name, (direction, threshold) in PROMOTION_GATES.items()
    for value in [metrics[name]]
    if (direction == "min" and value < threshold)
    or (direction == "max" and value > threshold)
}
if failures:
    raise ValueError(f"Model failed promotion gates: {failures}")

# after — include direction+threshold so the failure is actionable
failures = {}
for name, (direction, threshold) in PROMOTION_GATES.items():
    value = metrics[name]
    ok = value >= threshold if direction == "min" else value <= threshold
    if not ok:
        failures[name] = {"value": value, "direction": direction, "threshold": threshold}
if failures:
    raise ValueError(f"Model failed promotion gates: {failures}")

null

null

try:
    assert_promotion_ready(metrics)
except ValueError as e:
    if 'failed promotion gates' in str(e):
        block_deployment(e)  # do not ship; investigate failures dict
    raise

When it happens

Trigger: A model passes some gates but fails at least one: AUC below 0.82, calibration_error above 0.04, or p95_latency_ms above 80. Any one failure blocks promotion.

Common situations: A new model has better AUC but worse latency (cross-gate trade-off). Thresholds were calibrated on a different dataset. Eval ran on a slower machine inflating latency. Calibration regressed after a preprocessing change.

Related errors

• Model promotion metrics missing required gates: {missing}
• ECC_PROJECT_DIR must be a child path within /workspace.
• Unknown argument: ${arg}
• Unable to infer ECC repo root from install-state operations
• Invalid ECC repo root: missing package.json at ${packageJson

AI-assisted analysis of affaan-m/ECC@01e15490f0 (2026-08-13). Data as JSON: /api/errors/a68b2f3eaa2303ef. Report an issue: GitHub.