Hmbown/CodeWhale · error · PersistenceBacklogError

baseline provenance build profile/sample count changed

Error message

baseline provenance build profile/sample count changed

What it means

The checker freezes build_profile == "test" and sample_count == 1 in the baseline provenance; these are measurement-contract constants, not tunables. Any other profile or sample count means the baseline was gathered under a different methodology than the budget assumes, so the comparison is refused.

Solutions

  1. Regenerate the baseline with build_profile "test" and exactly one sample, matching the frozen contract.
  2. If the methodology intentionally changes, update FIXTURE/the checker expectations and regenerate the budget together.
  3. Verify the measure script's defaults weren't overridden via CLI flags during baseline generation.

Example fix

// before
"build_profile": "release", "sample_count": 5
// after
"build_profile": "test", "sample_count": 1
Defensive patterns

Strategy: validation

Validate before calling

p = budget.get("baseline_observation", {}).get("provenance", {})
if p.get("build_profile") != "test" or p.get("sample_count") != 1:
    raise ValueError("baseline must use build_profile 'test' and sample_count 1")

Try / catch

try:
    compare(budget, baseline_receipt)
except PersistenceBacklogError as e:
    if "build profile/sample count" in str(e): re-measure with test profile and one sample

Prevention

When it happens

Trigger: `compare` with provenance.build_profile != "test" (e.g. "release") or sample_count not the integer 1 — typically a baseline measured with --release or with repeated sampling enabled.

Common situations: Generating the baseline with a different build profile for 'more realistic' numbers, or changing the measure script's sampling loop and reusing the old budget.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/862b5290e7712ea0. Report an issue: GitHub.

Appendix: source

Thrown at scripts/check-persistence-backlog-budget.py:316

    provenance = baseline.get("provenance")
    if not isinstance(provenance, dict):
        raise PersistenceBacklogError("baseline_observation needs provenance")
    if provenance.get("platform") != "macos":
        raise PersistenceBacklogError("baseline provenance platform must be macos")
    if not isinstance(provenance.get("source_sha"), str) or not SOURCE_SHA_PATTERN.fullmatch(
        provenance["source_sha"]
    ):
        raise PersistenceBacklogError("baseline provenance needs an exact source SHA")
    if provenance.get("source_dirty") is not False:
        raise PersistenceBacklogError("baseline provenance must identify a clean source tree")
    for field, prefix in (("rustc_version", "rustc "), ("cargo_version", "cargo ")):
        if not isinstance(provenance.get(field), str) or not provenance[field].startswith(prefix):
            raise PersistenceBacklogError(f"baseline provenance needs {field}")
    if provenance.get("build_profile") != "test" or not (
        type(provenance.get("sample_count")) is int
        and provenance["sample_count"] == 1
    ):
        raise PersistenceBacklogError("baseline provenance build profile/sample count changed")


def validate_baseline_receipt(
    budget: dict[str, Any], baseline_receipt: dict[str, Any]
) -> None:
    validate_receipt(baseline_receipt, require_clean_source=True)
    baseline = budget["baseline_observation"]
    for field in ("accepted_requests", "applied_version", *CEILING_FIELDS):
        if baseline_receipt[field] != baseline[field]:
            raise PersistenceBacklogError(
                f"baseline receipt {field} does not match baseline_observation"
            )
    provenance = baseline["provenance"]
    for field in (
        "platform",
        "source_sha",
        "source_dirty",
        "rustc_version",

View on GitHub (pinned to 433685b202)