Hmbown/CodeWhale · error · PersistenceBacklogError

baseline provenance build profile/sample count changed

Error message

baseline provenance build profile/sample count changed

What it means

Raised by scripts/check-persistence-backlog-budget.py while validating scripts/persistence-backlog-budget.json: the baseline_observation.provenance block no longer records the frozen measurement methodology — build_profile must be exactly "test" and sample_count must be the integer 1. The gate pins how the macOS baseline was produced so ceiling comparisons stay meaningful across runs. The sample_count check uses exact type equality, so a bool or float also fails.

Source

Thrown at scripts/check-persistence-backlog-budget.py:316

    provenance = baseline.get("provenance")
    if not isinstance(provenance, dict):
        raise PersistenceBacklogError("baseline_observation needs provenance")
    if provenance.get("platform") != "macos":
        raise PersistenceBacklogError("baseline provenance platform must be macos")
    if not isinstance(provenance.get("source_sha"), str) or not SOURCE_SHA_PATTERN.fullmatch(
        provenance["source_sha"]
    ):
        raise PersistenceBacklogError("baseline provenance needs an exact source SHA")
    if provenance.get("source_dirty") is not False:
        raise PersistenceBacklogError("baseline provenance must identify a clean source tree")
    for field, prefix in (("rustc_version", "rustc "), ("cargo_version", "cargo ")):
        if not isinstance(provenance.get(field), str) or not provenance[field].startswith(prefix):
            raise PersistenceBacklogError(f"baseline provenance needs {field}")
    if provenance.get("build_profile") != "test" or not (
        type(provenance.get("sample_count")) is int
        and provenance["sample_count"] == 1
    ):
        raise PersistenceBacklogError("baseline provenance build profile/sample count changed")


def validate_baseline_receipt(
    budget: dict[str, Any], baseline_receipt: dict[str, Any]
) -> None:
    validate_receipt(baseline_receipt, require_clean_source=True)
    baseline = budget["baseline_observation"]
    for field in ("accepted_requests", "applied_version", *CEILING_FIELDS):
        if baseline_receipt[field] != baseline[field]:
            raise PersistenceBacklogError(
                f"baseline receipt {field} does not match baseline_observation"
            )
    provenance = baseline["provenance"]
    for field in (
        "platform",
        "source_sha",
        "source_dirty",
        "rustc_version",

View on GitHub (pinned to 8880682c63)

Solutions

1. Restore "build_profile": "test" and "sample_count": 1 in baseline_observation.provenance of scripts/persistence-backlog-budget.json
2. If the methodology genuinely changed, re-measure the baseline on a clean macOS checkout with scripts/measure-persistence-backlog.py and update both the budget's baseline_observation and scripts/persistence-backlog-baseline-receipt.json in the same change
3. Make sure sample_count is a JSON integer (not 1.0 or true), because the checker compares types strictly

Example fix

// before (scripts/persistence-backlog-budget.json)
"provenance": { "platform": "macos", "source_sha": "<40-hex>", "source_dirty": false, "rustc_version": "rustc 1.79.0", "cargo_version": "cargo 1.79.0", "build_profile": "release", "sample_count": 3 }
// after
"provenance": { "platform": "macos", "source_sha": "<40-hex>", "source_dirty": false, "rustc_version": "rustc 1.79.0", "cargo_version": "cargo 1.79.0", "build_profile": "test", "sample_count": 1 }

prov = budget['baseline_observation']['provenance']
assert prov.get('build_profile') == 'test', 'build_profile drifted'
assert type(prov.get('sample_count')) is int and prov['sample_count'] == 1, 'sample_count drifted'

def is_frozen_provenance(prov):
    return (isinstance(prov, dict)
            and prov.get('build_profile') == 'test'
            and type(prov.get('sample_count')) is int
            and prov['sample_count'] == 1)

When it happens

Trigger: Running `python3 scripts/check-persistence-backlog-budget.py` (or the CI job wrapping it) after baseline_observation.provenance.build_profile was changed from "test" (e.g. to "release" or "bench"), or sample_count from 1 (e.g. to 3 for averaging, to 1.0, or to true).

Common situations: A developer tries to make the baseline "more realistic" with a release-profile or multi-sample measurement; hand-editing the budget JSON in an editor that rewrites 1 as 1.0; pasting a provenance block from a receipt measured under a different configuration.

Related errors

• baseline receipt {field} does not match baseline_observation
• baseline receipt {field} does not match baseline provenance
• {kind} field `{dotted_names}` must be sorted unique non-empt
• invalid {label} {path}: {error}
• {label} must be a JSON object

AI-assisted analysis of Hmbown/CodeWhale@8880682c63 (2026-08-16). Data as JSON: /api/errors/862b5290e7712ea0. Report an issue: GitHub.