paperclipai/paperclip · error · ValueError

Unexpected public report URL

Error message

Unexpected public report URL

What it means

public_report() validates that the campaign's publicUrl is an https URL on the expected CloudFront origin (ORIGIN), under /{prefix}/campaigns/, with no query string or fragment, before it is escaped and embedded in the public page. Anything else could point the published eval page at an unintended host, so it raises ValueError.

Solutions

1. Fix the campaign publisher so publicUrl is https on the ORIGIN host under /{prefix}/campaigns/ with no query/fragment.
2. If the origin legitimately changed, update ORIGIN at the top of build.py to the new host.
3. Ensure summarize() is called with the kind whose prefix matches the URL's path segment.
4. Strip query strings/fragments at publish time and store the canonical campaign URL.

Example fix

# before
publicUrl = "http://localhost:8000/runner-e2e/campaigns/c-1/?x=1"
# after
publicUrl = "https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/c-1"

from urllib.parse import urlsplit
ORIGIN = "https://d1p6rlowie26tp.cloudfront.net"
def url_is_public_campaign(url, prefix):
    p = urlsplit(url)
    return (p.scheme == "https" and p.netloc == urlsplit(ORIGIN).netloc
            and p.path.startswith(f"/{prefix}/campaigns/") and not p.query and not p.fragment)

def is_canonical_campaign_url(url, prefix):
    p = urlsplit(url)
    return p.scheme == "https" and p.netloc == urlsplit(ORIGIN).netloc and p.path.startswith(f"/{prefix}/campaigns/") and not p.query and not p.fragment

try:
    values = summarize(history, kind)
except ValueError as e:
    if str(e) == "Unexpected public report URL":
        log.warning("rejecting campaign with non-canonical publicUrl", e)
        return None
    raise

When it happens

Trigger: A campaign whose publicUrl uses http instead of https, a different host than d1p6rlowie26tp.cloudfront.net, a path not starting with /{prefix}/campaigns/ (e.g. wrong prefix for the kind), or URLs carrying ?query or #fragment.

Common situations: Switching the page host without updating ORIGIN in build.py; campaigns published to a local/dev origin during testing; wrong prefix passed for the kind (runner-protocol-evals vs runner-e2e); tracking params appended to the URL.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• Artifact URL does not match its content hash and trusted…
• Cloud control assertion request id is invalid
• Cloud control claims are incomplete
• Codex working directory cannot be a filesystem root
• CreateOS API URL is invalid.

AI-assisted analysis of paperclipai/paperclip@3f1d897a7c (2026-09-18). Data as JSON: /api/errors/d124666be6f40309. Report an issue: GitHub.

Appendix: source

Thrown at scripts/evals-hub/build.py:39

        raise ValueError("Unsupported history schema")
    campaign_id = history.get("latestCampaignId")
    if not campaign_id:
        raise ValueError("History has no latest campaign")
    candidates = [c for c in history["campaigns"] if
                  c["campaignId"] == campaign_id or
                  c.get("reportRevision", {}).get("sourceCampaignId") == campaign_id]
    if not candidates:
        raise ValueError("Latest campaign is absent from history")
    # Report refreshes replace presentation, never the measurement date.
    return max(candidates, key=lambda c: c.get("reportRevision", {}).get("renderedAt", c["generatedAt"]))


def public_report(url, prefix):
    parsed = urlsplit(url)
    if (parsed.scheme != "https" or parsed.netloc != urlsplit(ORIGIN).netloc
            or not parsed.path.startswith(f"/{prefix}/campaigns/")
            or parsed.query or parsed.fragment):
        raise ValueError("Unexpected public report URL")
    return html.escape(url, quote=True)


def display_date(value):
    date = datetime.fromisoformat(value.replace("Z", "+00:00"))
    if date.tzinfo is None:
        raise ValueError("History date must include a timezone")
    return date.astimezone(timezone.utc).strftime("%d %b %Y · %H:%M UTC")


def summarize(history, kind):
    prefix, schema = SYSTEMS[kind]
    campaign = latest(history, schema)
    totals = campaign["totals"] if kind == "runner" else campaign
    passed, selected = totals["passed"], totals["selected"]
    if any(type(n) is not int for n in (passed, selected)) or not 0 <= passed <= selected:
        raise ValueError("Invalid campaign counts")
    if type(campaign.get("complete")) is not bool:

View on GitHub (pinned to 3f1d897a7c)