Hmbown/CodeWhale · error · RuntimeContractError

identity changed for

Error message

identity changed for {label} [`{'.'.join(path)}`]{detail}

What it means

compare() diffs identity fields (paths of the receipt vs the stored budget that must be byte-identical, e.g. tool surface profile, fixture id, identity hashes). When any identity value differs it raises RuntimeContractError naming the label, the dotted path, and — for list-valued identities — which items were added/removed. Identity drift means the measurement no longer describes the same contract the budget was pinned to, so metric comparison would be meaningless.

Solutions

1. Read the added=/removed= detail and either revert the identity change or intentionally accept it by running with `--update` to re-pin the budget to the new identity.
2. Diff scripts/runtime-contract-budget.json against a freshly generated receipt to see exactly which identity field moved.
3. If the change is legitimate, regenerate the budget in the same commit as the identity change.
4. Ensure you are not comparing a receipt measured on a dirty checkout against a budget from clean CI state.

Example fix

// before
identity changed for tool surface [`tool_surface_profile`] (added=['mcp'] removed=[])
// after: rebuild budget to match, or drop the mcp surface
python3 scripts/check-runtime-contract-budget.py --update

import json
receipt, budget = json.load(open("receipt.json")), json.load(open("scripts/runtime-contract-budget.json"))
for key in ("tool_surface_profile", "fixture_id"):
    assert receipt.get(key) == budget.get(key), f"identity drift on {key}"

def identities_match(receipt: dict, budget: dict, keys: list[str]) -> bool:
    return all(receipt.get(k) == budget.get(k) for k in keys)

try:
    compare_receipt_against_budget(receipt, budget)
except IdentityDrift as e:
    if not user_confirms(e.details):
        raise
    regenerate_budget(receipt)  # --update, after review

When it happens

Trigger: Running the checker (or calling compare from main) where a receipt identity list/string differs from the budget's — e.g. the tool surface profile changed, a stage was added/removed from REPRESENTATIVE_STAGES, or the budget was generated on a different configuration.

Common situations: A PR changed visible modes, tool surfaces, or representative stages without regenerating the budget; a developer merged a budget from another branch; measuring on a checkout with locally modified prompts/tool catalog.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• budget metric `representative_context.fixture_id` must be
• MAX_FRAGMENT_TOKENS must be
• baseline_observation.accepted_requests must equal…
• baseline_observation.
• bounded fragment module must define a matches_text…

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/8171fed962295db4. Report an issue: GitHub.

Appendix: source

Thrown at scripts/check-runtime-contract-budget.py:329

def compare(
    receipt: dict[str, Any], budget: dict[str, Any]
) -> tuple[list[MetricResult], list[MetricResult]]:
    """Return (increases, decreases) as path/label/current/ceiling tuples."""
    validate_receipt(receipt)
    validate_budget(budget)
    for path, label in IDENTITIES:
        receipt_value = required_value(receipt, path, "receipt")
        budget_value = required_value(budget, path, "budget")
        if receipt_value != budget_value:
            detail = ""
            if isinstance(receipt_value, list) and isinstance(budget_value, list):
                added = [str(item) for item in receipt_value if item not in budget_value]
                removed = [
                    str(item) for item in budget_value if item not in receipt_value
                ]
                detail = f" (added={added} removed={removed})"
            raise RuntimeContractError(
                f"identity changed for {label} [`{'.'.join(path)}`]{detail}"
            )
    increases: list[MetricResult] = []
    decreases: list[MetricResult] = []
    for path, label in METRICS:
        current = metric_value(receipt, path, "receipt")
        ceiling = metric_value(budget, path, "budget")
        result = (".".join(path), label, current, ceiling)
        if current > ceiling:
            increases.append(result)
        elif current < ceiling:
            decreases.append(result)
    return increases, decreases


def set_path_value(document: dict[str, Any], path: MetricPath, value: Any) -> None:
    target = document
    for part in path[:-1]:

View on GitHub (pinned to 433685b202)