{"record":{"id":"a1db7f208941ad18","repo":"paperclipai/paperclip","slug":"report-refresh-has-no-reliable-measurement-date","errorCode":null,"errorMessage":"Report refresh has no reliable measurement date","messagePattern":"Report refresh has no reliable measurement date","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/evals-hub/build.py","lineNumber":61,"sourceCode":"def display_date(value):\n    date = datetime.fromisoformat(value.replace(\"Z\", \"+00:00\"))\n    if date.tzinfo is None:\n        raise ValueError(\"History date must include a timezone\")\n    return date.astimezone(timezone.utc).strftime(\"%d %b %Y · %H:%M UTC\")\n\n\ndef summarize(history, kind):\n    prefix, schema = SYSTEMS[kind]\n    campaign = latest(history, schema)\n    totals = campaign[\"totals\"] if kind == \"runner\" else campaign\n    passed, selected = totals[\"passed\"], totals[\"selected\"]\n    if any(type(n) is not int for n in (passed, selected)) or not 0 <= passed <= selected:\n        raise ValueError(\"Invalid campaign counts\")\n    if type(campaign.get(\"complete\")) is not bool:\n        raise ValueError(\"Missing campaign coverage flag\")\n    revision = campaign.get(\"reportRevision\", {})\n    if revision and not isinstance(revision.get(\"sourceGeneratedAt\"), str):\n        raise ValueError(\"Report refresh has no reliable measurement date\")\n    measured = revision.get(\"sourceGeneratedAt\", campaign[\"generatedAt\"])\n    return {\n        \"url\": public_report(campaign[\"publicUrl\"], prefix),\n        \"date\": html.escape(display_date(measured)),\n        \"counts\": f\"{passed} / {selected} selected cases passed\",\n        \"coverage\": \"Complete campaign\" if campaign[\"complete\"] else \"Partial campaign\",\n        \"id\": html.escape(revision.get(\"sourceCampaignId\", campaign[\"campaignId\"])),\n    }\n\n\ndef main():\n    parser = argparse.ArgumentParser(description=__doc__)\n    parser.add_argument(\"--output\", type=Path, required=True)\n    parser.add_argument(\"--history-dir\", type=Path, help=\"Use saved public history JSON files for offline verification\")\n    parser.add_argument(\"--docs-ref\", default=\"master\", help=\"Git ref containing doc/evals.md\")\n    args = parser.parse_args()\n    values = {}\n    for kind, (prefix, _) in SYSTEMS.items():","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/scripts/evals-hub/build.py#L43-L79","documentation":"When a campaign carries a reportRevision (a presentation-only refresh of an earlier report), the page must still show the original measurement date. summarize() therefore requires revision.sourceGeneratedAt to be a string; a revision without it would silently drift the displayed date to the refresh time, so it raises ValueError.","triggerScenarios":"A campaign entry with a non-empty reportRevision object whose sourceGeneratedAt is missing, null, or a non-string (e.g. a number timestamp) — typically a report-refresh writer that copied the revision block but omitted the original generation timestamp.","commonSituations":"Report refresh tooling upgraded without backfilling sourceGeneratedAt; hand-edited fixtures adding reportRevision with only renderedAt/sourceCampaignId; timestamp serialized as epoch int instead of ISO string.","solutions":["Fix the report-refresh publisher to always include sourceGeneratedAt (ISO string of the original campaign's generatedAt).","Backfill existing history entries: set reportRevision.sourceGeneratedAt from the source campaign's generatedAt.","If a revision is genuinely unreconstructable, drop the reportRevision block so the script falls back to campaign['generatedAt'].","In fixtures, serialize the timestamp as an ISO string, not an epoch number."],"exampleFix":"# before\nreportRevision = {\"renderedAt\": \"2026-09-18T10:00:00Z\", \"sourceCampaignId\": \"c-1\"}\n# after\nreportRevision = {\"renderedAt\": \"2026-09-18T10:00:00Z\", \"sourceCampaignId\": \"c-1\", \"sourceGeneratedAt\": \"2026-09-10T08:00:00Z\"}","handlingStrategy":"validation","validationCode":"def revision_has_source_date(campaign):\n    rev = campaign.get(\"reportRevision\") or {}\n    return (not rev) or isinstance(rev.get(\"sourceGeneratedAt\"), str)","typeGuard":"def refresh_is_traceable(campaign):\n    rev = campaign.get(\"reportRevision\")\n    return not rev or isinstance(rev.get(\"sourceGeneratedAt\"), str)","tryCatchPattern":"try:\n    values = summarize(history, kind)\nexcept ValueError as e:\n    if str(e) == \"Report refresh has no reliable measurement date\":\n        log.error(\"reportRevision without sourceGeneratedAt; backfill before building\")\n        raise\n    raise","preventionTips":["Always copy sourceGeneratedAt (ISO string of the original generatedAt) into reportRevision when refreshing a report.","Serialize timestamps as ISO strings, never epoch numbers.","Backfill existing refreshed entries from their source campaign's generatedAt.","Add a publish-time check: reportRevision present implies sourceGeneratedAt present."],"tags":["validation","missing-field","python","report"],"backgroundTag":"missing-config-key","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-09-18T08:03:59.046Z","contentChangedAt":"2026-09-18T08:03:59.046Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}