paperclipai/paperclip · error · ValueError

Invalid campaign counts

Error message

Invalid campaign counts

What it means

summarize() validates the selected campaign's pass/selected counts: both must be true ints (bool excluded via `type(n) is not int`) and satisfy 0 <= passed <= selected. This guards the published page against nonsensical stats from a malformed feed, raising ValueError when the invariant is broken.

Solutions

1. Fix the campaign publisher so passed/selected are ints with passed <= selected.
2. Check you used the right kind: 'runner' reads campaign['totals'], 'product' reads counts from the campaign itself.
3. Validate/normalize the history feed upstream before building (reject non-int counts at publish time).
4. Correct the fixture counts if this came from offline --history-dir test data.

Example fix

# before
totals = {"passed": 12.0, "selected": 10}
# after
totals = {"passed": 8, "selected": 10}  # ints, passed <= selected

def counts_ok(totals):
    p, s = totals.get("passed"), totals.get("selected")
    return all(type(n) is int for n in (p, s)) and 0 <= p <= s

def has_valid_counts(totals):
    return (isinstance(totals, dict)
            and type(totals.get("passed")) is int
            and type(totals.get("selected")) is int
            and 0 <= totals["passed"] <= totals["selected"])

try:
    values = summarize(history, kind)
except ValueError as e:
    if str(e) == "Invalid campaign counts":
        log.error("campaign counts failed 0 <= passed <= selected int check; refusing to publish")
        raise
    raise

When it happens

Trigger: A campaign (or its `totals` object for kind='runner') where passed/selected are floats, strings, None, booleans, or where passed > selected or a value is negative — e.g. a partially written totals object or a producer bug counting retries incorrectly.

Common situations: Upstream schema change moving totals; JSON producers emitting 3.0 instead of 3; a runner that reported more passes than selected cases due to double counting; kind mismatch causing the code to read counts from the wrong level of the campaign object.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Latest campaign is absent from history
• github_attachment_canonical_response_unavailable
• History date must include a timezone
• History has no latest campaign
• must be a non-empty string.

AI-assisted analysis of paperclipai/paperclip@3f1d897a7c (2026-09-18). Data as JSON: /api/errors/e47aae2f159aa89b. Report an issue: GitHub.

Appendix: source

Thrown at scripts/evals-hub/build.py:56

            or parsed.query or parsed.fragment):
        raise ValueError("Unexpected public report URL")
    return html.escape(url, quote=True)


def display_date(value):
    date = datetime.fromisoformat(value.replace("Z", "+00:00"))
    if date.tzinfo is None:
        raise ValueError("History date must include a timezone")
    return date.astimezone(timezone.utc).strftime("%d %b %Y · %H:%M UTC")


def summarize(history, kind):
    prefix, schema = SYSTEMS[kind]
    campaign = latest(history, schema)
    totals = campaign["totals"] if kind == "runner" else campaign
    passed, selected = totals["passed"], totals["selected"]
    if any(type(n) is not int for n in (passed, selected)) or not 0 <= passed <= selected:
        raise ValueError("Invalid campaign counts")
    if type(campaign.get("complete")) is not bool:
        raise ValueError("Missing campaign coverage flag")
    revision = campaign.get("reportRevision", {})
    if revision and not isinstance(revision.get("sourceGeneratedAt"), str):
        raise ValueError("Report refresh has no reliable measurement date")
    measured = revision.get("sourceGeneratedAt", campaign["generatedAt"])
    return {
        "url": public_report(campaign["publicUrl"], prefix),
        "date": html.escape(display_date(measured)),
        "counts": f"{passed} / {selected} selected cases passed",
        "coverage": "Complete campaign" if campaign["complete"] else "Partial campaign",
        "id": html.escape(revision.get("sourceCampaignId", campaign["campaignId"])),
    }


def main():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--output", type=Path, required=True)

View on GitHub (pinned to 3f1d897a7c)