{"record":{"id":"d124666be6f40309","repo":"paperclipai/paperclip","slug":"unexpected-public-report-url","errorCode":null,"errorMessage":"Unexpected public report URL","messagePattern":"Unexpected public report URL","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/evals-hub/build.py","lineNumber":39,"sourceCode":"        raise ValueError(\"Unsupported history schema\")\n    campaign_id = history.get(\"latestCampaignId\")\n    if not campaign_id:\n        raise ValueError(\"History has no latest campaign\")\n    candidates = [c for c in history[\"campaigns\"] if\n                  c[\"campaignId\"] == campaign_id or\n                  c.get(\"reportRevision\", {}).get(\"sourceCampaignId\") == campaign_id]\n    if not candidates:\n        raise ValueError(\"Latest campaign is absent from history\")\n    # Report refreshes replace presentation, never the measurement date.\n    return max(candidates, key=lambda c: c.get(\"reportRevision\", {}).get(\"renderedAt\", c[\"generatedAt\"]))\n\n\ndef public_report(url, prefix):\n    parsed = urlsplit(url)\n    if (parsed.scheme != \"https\" or parsed.netloc != urlsplit(ORIGIN).netloc\n            or not parsed.path.startswith(f\"/{prefix}/campaigns/\")\n            or parsed.query or parsed.fragment):\n        raise ValueError(\"Unexpected public report URL\")\n    return html.escape(url, quote=True)\n\n\ndef display_date(value):\n    date = datetime.fromisoformat(value.replace(\"Z\", \"+00:00\"))\n    if date.tzinfo is None:\n        raise ValueError(\"History date must include a timezone\")\n    return date.astimezone(timezone.utc).strftime(\"%d %b %Y · %H:%M UTC\")\n\n\ndef summarize(history, kind):\n    prefix, schema = SYSTEMS[kind]\n    campaign = latest(history, schema)\n    totals = campaign[\"totals\"] if kind == \"runner\" else campaign\n    passed, selected = totals[\"passed\"], totals[\"selected\"]\n    if any(type(n) is not int for n in (passed, selected)) or not 0 <= passed <= selected:\n        raise ValueError(\"Invalid campaign counts\")\n    if type(campaign.get(\"complete\")) is not bool:","sourceCodeStart":21,"sourceCodeEnd":57,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/scripts/evals-hub/build.py#L21-L57","documentation":"public_report() validates that the campaign's publicUrl is an https URL on the expected CloudFront origin (ORIGIN), under /{prefix}/campaigns/, with no query string or fragment, before it is escaped and embedded in the public page. Anything else could point the published eval page at an unintended host, so it raises ValueError.","triggerScenarios":"A campaign whose publicUrl uses http instead of https, a different host than d1p6rlowie26tp.cloudfront.net, a path not starting with /{prefix}/campaigns/ (e.g. wrong prefix for the kind), or URLs carrying ?query or #fragment.","commonSituations":"Switching the page host without updating ORIGIN in build.py; campaigns published to a local/dev origin during testing; wrong prefix passed for the kind (runner-protocol-evals vs runner-e2e); tracking params appended to the URL.","solutions":["Fix the campaign publisher so publicUrl is https on the ORIGIN host under /{prefix}/campaigns/ with no query/fragment.","If the origin legitimately changed, update ORIGIN at the top of build.py to the new host.","Ensure summarize() is called with the kind whose prefix matches the URL's path segment.","Strip query strings/fragments at publish time and store the canonical campaign URL."],"exampleFix":"# before\npublicUrl = \"http://localhost:8000/runner-e2e/campaigns/c-1/?x=1\"\n# after\npublicUrl = \"https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/c-1\"","handlingStrategy":"validation","validationCode":"from urllib.parse import urlsplit\nORIGIN = \"https://d1p6rlowie26tp.cloudfront.net\"\ndef url_is_public_campaign(url, prefix):\n    p = urlsplit(url)\n    return (p.scheme == \"https\" and p.netloc == urlsplit(ORIGIN).netloc\n            and p.path.startswith(f\"/{prefix}/campaigns/\") and not p.query and not p.fragment)","typeGuard":"def is_canonical_campaign_url(url, prefix):\n    p = urlsplit(url)\n    return p.scheme == \"https\" and p.netloc == urlsplit(ORIGIN).netloc and p.path.startswith(f\"/{prefix}/campaigns/\") and not p.query and not p.fragment","tryCatchPattern":"try:\n    values = summarize(history, kind)\nexcept ValueError as e:\n    if str(e) == \"Unexpected public report URL\":\n        log.warning(\"rejecting campaign with non-canonical publicUrl\", e)\n        return None\n    raise","preventionTips":["Store canonical campaign URLs at publish time: https, origin host, /{prefix}/campaigns/ path, no query/fragment.","Update ORIGIN in build.py whenever the page host changes.","Pass the kind whose prefix matches the URL path; don't mix runner/product prefixes.","Strip tracking params before persisting publicUrl."],"tags":["url","validation","security","python"],"backgroundTag":"invalid-url-format","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-09-18T08:03:59.046Z","contentChangedAt":"2026-09-18T08:03:59.046Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}