paperclipai/paperclip · error · ValueError

Unsupported history schema

Error message

Unsupported history schema

What it means

build.py consumes a public history JSON feed and validates its `schema` field against the expected schema string for the system kind (`paperclip.runner-protocol-eval.history/v1` or `paperclip.runner-e2e.history/v1`). A mismatch means the feed was produced by an unknown/newer/older writer, so the script refuses to guess the shape and raises ValueError in latest().

Solutions

1. Regenerate or re-download the history feed so it carries the expected v1 schema string.
2. If the feed intentionally moved to a new schema version, update SYSTEMS in build.py to accept it and adapt the parser.
3. When using --history-dir, refresh the saved {prefix}-history.json fixture from the live origin.
4. Check you are feeding the right file: runner-protocol-evals history for kind 'runner', runner-e2e history for kind 'product'.

Example fix

# before
history = {"schema": "paperclip.runner-e2e.history/v2", ...}
summarize(history, "product")  # ValueError: Unsupported history schema
# after
history = {"schema": "paperclip.runner-e2e.history/v1", ...}  # matches SYSTEMS["product"][1]

def ensure_schema(history, expected):
    if history.get("schema") != expected:
        raise ValueError(f"history schema {history.get('schema')!r} != expected {expected!r}")

def is_v1_history(history, schema):
    return isinstance(history, dict) and history.get("schema") == schema

try:
    values = summarize(history, kind)
except ValueError as e:
    if str(e) == "Unsupported history schema":
        history = revalidate_and_migrate(history)  # re-fetch or migrate to v1
        values = summarize(history, kind)
    else:
        raise

When it happens

Trigger: Calling latest() (via summarize) with a history dict whose `schema` field is missing or does not equal the expected v1 schema for that kind — e.g. hand-edited history JSON, a feed upgraded to /v2, or swapping runner history with product history.

Common situations: Upstream feed format bump after a deploy; offline fixture files in --history-dir saved from an older format; passing the wrong prefix's history file for a kind.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• History has no latest campaign
• Latest campaign is absent from history
• does not carry submitted form data
• : invalid AppDefinition
• Durable PRP run attachment template is invalid.

AI-assisted analysis of paperclipai/paperclip@3f1d897a7c (2026-09-18). Data as JSON: /api/errors/29373b24acd82526. Report an issue: GitHub.

Appendix: source

Thrown at scripts/evals-hub/build.py:21

import argparse
from datetime import datetime, timezone
import html
import json
from pathlib import Path
from string import Template
from urllib.parse import quote, urlsplit
from urllib.request import urlopen

ORIGIN = "https://d1p6rlowie26tp.cloudfront.net"
SYSTEMS = {
    "runner": ("runner-protocol-evals", "paperclip.runner-protocol-eval.history/v1"),
    "product": ("runner-e2e", "paperclip.runner-e2e.history/v1"),
}


def latest(history, schema):
    if history.get("schema") != schema:
        raise ValueError("Unsupported history schema")
    campaign_id = history.get("latestCampaignId")
    if not campaign_id:
        raise ValueError("History has no latest campaign")
    candidates = [c for c in history["campaigns"] if
                  c["campaignId"] == campaign_id or
                  c.get("reportRevision", {}).get("sourceCampaignId") == campaign_id]
    if not candidates:
        raise ValueError("Latest campaign is absent from history")
    # Report refreshes replace presentation, never the measurement date.
    return max(candidates, key=lambda c: c.get("reportRevision", {}).get("renderedAt", c["generatedAt"]))


def public_report(url, prefix):
    parsed = urlsplit(url)
    if (parsed.scheme != "https" or parsed.netloc != urlsplit(ORIGIN).netloc
            or not parsed.path.startswith(f"/{prefix}/campaigns/")
            or parsed.query or parsed.fragment):
        raise ValueError("Unexpected public report URL")

View on GitHub (pinned to 3f1d897a7c)