Hmbown/CodeWhale · error · RuntimeContractError

{kind} tool surface_profile must be `{TOOL_SURFACE_PROFILE}`

Error message

{kind} tool surface_profile must be `{TOOL_SURFACE_PROFILE}`, got {profile!r}

What it means

validate_identity_structure compared tool_catalog.surface_profile against the pinned constant `production-default-builtins-no-mcp-no-host-interpreters-v1` and found a different value. The profile string pins the exact tool surface configuration (default builtins, no MCP servers, no host interpreters) under which all tool-catalog ceilings are meaningful; a receipt measured under any other surface is incomparable. Both receipt and budget must carry the same profile.

Source

Thrown at scripts/check-runtime-contract-budget.py:207

def required_value(document: dict[str, Any], path: MetricPath, kind: str) -> Any:
    value: Any = document
    dotted = ".".join(path)
    for part in path:
        if not isinstance(value, dict) or part not in value:
            raise RuntimeContractError(f"{kind} is missing required field `{dotted}`")
        value = value[part]
    return value


def tool_identity_digest(names: list[str]) -> str:
    return hashlib.sha256("\0".join(names).encode("utf-8")).hexdigest()


def validate_identity_structure(document: dict[str, Any], kind: str) -> None:
    profile = required_value(document, ("tool_catalog", "surface_profile"), kind)
    if profile != TOOL_SURFACE_PROFILE:
        raise RuntimeContractError(
            f"{kind} tool surface_profile must be `{TOOL_SURFACE_PROFILE}`, "
            f"got {profile!r}"
        )

    for mode, _label in VISIBLE_MODES:
        for surface, _surface_label in TOOL_SURFACES:
            base = ("tool_catalog", "modes", mode, surface)
            names = required_value(document, (*base, "tool_names"), kind)
            dotted_names = ".".join((*base, "tool_names"))
            if (
                not isinstance(names, list)
                or any(not isinstance(name, str) or not name for name in names)
                or names != sorted(set(names))
            ):
                raise RuntimeContractError(
                    f"{kind} field `{dotted_names}` must be sorted unique non-empty strings"
                )
            count = metric_value(document, (*base, "tools"), kind)

View on GitHub (pinned to 8880682c63)

Solutions

1. Re-run the measurement in the canonical configuration: default builtins, no MCP servers, no host interpreters
2. Verify tool_catalog.surface_profile in the receipt matches the TOOL_SURFACE_PROFILE constant at the top of scripts/check-runtime-contract-budget.py
3. If the surface definition intentionally changed, update TOOL_SURFACE_PROFILE and regenerate the budget with --update as one reviewed change

Example fix

// before
"tool_catalog": { "surface_profile": "production-default-builtins-with-mcp-v1", ... }
// after
"tool_catalog": { "surface_profile": "production-default-builtins-no-mcp-no-host-interpreters-v1", ... }

expected = 'production-default-builtins-no-mcp-no-host-interpreters-v1'
assert doc['tool_catalog']['surface_profile'] == expected, doc['tool_catalog']['surface_profile']

When it happens

Trigger: Measuring with MCP servers configured or host interpreters enabled; changing the default builtin tool set; using a stale receipt from before a profile rename; hand-editing the profile string in the JSON.

Common situations: Developer machines with local MCP configuration leaking into measurement; tool-surface redefinitions; mixing receipts across versions.

Related errors

• budget document_kind must be {BUDGET_KIND}
• budget fixture no longer matches the frozen workload
• budget baseline_receipt path changed
• budget needs ceilings and baseline_observation objects
• baseline provenance build profile/sample count changed

AI-assisted analysis of Hmbown/CodeWhale@8880682c63 (2026-08-16). Data as JSON: /api/errors/ee4f160edbc2c961. Report an issue: GitHub.