{"record":{"id":"e47aae2f159aa89b","repo":"paperclipai/paperclip","slug":"invalid-campaign-counts","errorCode":null,"errorMessage":"Invalid campaign counts","messagePattern":"Invalid campaign counts","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/evals-hub/build.py","lineNumber":56,"sourceCode":"            or parsed.query or parsed.fragment):\n        raise ValueError(\"Unexpected public report URL\")\n    return html.escape(url, quote=True)\n\n\ndef display_date(value):\n    date = datetime.fromisoformat(value.replace(\"Z\", \"+00:00\"))\n    if date.tzinfo is None:\n        raise ValueError(\"History date must include a timezone\")\n    return date.astimezone(timezone.utc).strftime(\"%d %b %Y · %H:%M UTC\")\n\n\ndef summarize(history, kind):\n    prefix, schema = SYSTEMS[kind]\n    campaign = latest(history, schema)\n    totals = campaign[\"totals\"] if kind == \"runner\" else campaign\n    passed, selected = totals[\"passed\"], totals[\"selected\"]\n    if any(type(n) is not int for n in (passed, selected)) or not 0 <= passed <= selected:\n        raise ValueError(\"Invalid campaign counts\")\n    if type(campaign.get(\"complete\")) is not bool:\n        raise ValueError(\"Missing campaign coverage flag\")\n    revision = campaign.get(\"reportRevision\", {})\n    if revision and not isinstance(revision.get(\"sourceGeneratedAt\"), str):\n        raise ValueError(\"Report refresh has no reliable measurement date\")\n    measured = revision.get(\"sourceGeneratedAt\", campaign[\"generatedAt\"])\n    return {\n        \"url\": public_report(campaign[\"publicUrl\"], prefix),\n        \"date\": html.escape(display_date(measured)),\n        \"counts\": f\"{passed} / {selected} selected cases passed\",\n        \"coverage\": \"Complete campaign\" if campaign[\"complete\"] else \"Partial campaign\",\n        \"id\": html.escape(revision.get(\"sourceCampaignId\", campaign[\"campaignId\"])),\n    }\n\n\ndef main():\n    parser = argparse.ArgumentParser(description=__doc__)\n    parser.add_argument(\"--output\", type=Path, required=True)","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/scripts/evals-hub/build.py#L38-L74","documentation":"summarize() validates the selected campaign's pass/selected counts: both must be true ints (bool excluded via `type(n) is not int`) and satisfy 0 <= passed <= selected. This guards the published page against nonsensical stats from a malformed feed, raising ValueError when the invariant is broken.","triggerScenarios":"A campaign (or its `totals` object for kind='runner') where passed/selected are floats, strings, None, booleans, or where passed > selected or a value is negative — e.g. a partially written totals object or a producer bug counting retries incorrectly.","commonSituations":"Upstream schema change moving totals; JSON producers emitting 3.0 instead of 3; a runner that reported more passes than selected cases due to double counting; kind mismatch causing the code to read counts from the wrong level of the campaign object.","solutions":["Fix the campaign publisher so passed/selected are ints with passed <= selected.","Check you used the right kind: 'runner' reads campaign['totals'], 'product' reads counts from the campaign itself.","Validate/normalize the history feed upstream before building (reject non-int counts at publish time).","Correct the fixture counts if this came from offline --history-dir test data."],"exampleFix":"# before\ntotals = {\"passed\": 12.0, \"selected\": 10}\n# after\ntotals = {\"passed\": 8, \"selected\": 10}  # ints, passed <= selected","handlingStrategy":"validation","validationCode":"def counts_ok(totals):\n    p, s = totals.get(\"passed\"), totals.get(\"selected\")\n    return all(type(n) is int for n in (p, s)) and 0 <= p <= s","typeGuard":"def has_valid_counts(totals):\n    return (isinstance(totals, dict)\n            and type(totals.get(\"passed\")) is int\n            and type(totals.get(\"selected\")) is int\n            and 0 <= totals[\"passed\"] <= totals[\"selected\"])","tryCatchPattern":"try:\n    values = summarize(history, kind)\nexcept ValueError as e:\n    if str(e) == \"Invalid campaign counts\":\n        log.error(\"campaign counts failed 0 <= passed <= selected int check; refusing to publish\")\n        raise\n    raise","preventionTips":["Serialize counts as JSON ints, never floats (emit 8, not 8.0).","Enforce passed <= selected in the runner before publishing totals.","Check kind before reading counts: campaign['totals'] for runner, campaign itself for product.","Validate totals against the raw result set (count of selected cases) before publish."],"tags":["validation","type-mismatch","python","data-integrity"],"backgroundTag":"value-out-of-range","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-09-18T08:03:59.046Z","contentChangedAt":"2026-09-18T08:03:59.046Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}