What causes "Unsupported protocol eval history schema" in paperclipai/paperclip?

Inspect the remote s3:// / /history.json and check its schema field

paperclipai/paperclip · error · Error

Unsupported protocol eval history schema

Error message

Unsupported protocol eval history schema

What it means

mergeProtocolEvalHistory only accepts an existing history document with schema "paperclip.runner-protocol-eval.history/v1". When merging a freshly published campaign into the S3 history.json, an unknown or older schema version is rejected so the immutable history semantics are not applied to an incompatible document shape.

Source

Thrown at packages/paperclip-runner/scripts/publish-runner-protocol-eval-history.mjs:307

export function protocolEvalHistoryRecord(campaign, publicRoot) {
  return {
    campaignId: campaign.campaignId,
    generatedAt: campaign.generatedAt,
    publicUrl: `${publicRoot}/campaigns/${encodeURIComponent(campaign.campaignId)}/`,
    complete: campaign.complete === true,
    allPassed: campaign.allPassed === true,
    totals: campaign.totals,
    rosters: campaign.rosters,
    source: campaign.source,
    ...(campaign.reportRevision
      ? { reportRevision: campaign.reportRevision }
      : {}),
  };
}

export function mergeProtocolEvalHistory(history, record) {
  if (history.schema !== "paperclip.runner-protocol-eval.history/v1") {
    throw new Error("Unsupported protocol eval history schema");
  }
  const existing = history.campaigns.find(
    (item) => item.campaignId === record.campaignId,
  );
  if (existing && JSON.stringify(existing) !== JSON.stringify(record)) {
    throw new Error(
      `Immutable campaign history changed for ${record.campaignId}`,
    );
  }
  const campaigns = existing
    ? [...history.campaigns]
    : [...history.campaigns, record];
  const activityAt = (campaign) =>
    campaign.reportRevision?.renderedAt ?? campaign.generatedAt;
  const activityOrder = (left, right) =>
    activityAt(right).localeCompare(activityAt(left));
  campaigns.sort(activityOrder);
  // Report revisions are discoverable history entries, never qualification runs.

View on GitHub (pinned to 01ad858492)

Solutions

Inspect the remote s3://<bucket>/<prefix>/history.json and check its schema field
If it is a legacy version, migrate or delete the history object so a fresh v1 history is seeded via emptyProtocolEvalHistory()
Ensure the object passed to mergeProtocolEvalHistory is a history document (schema, campaigns array), not a campaign or bundle manifest
Re-run with the current version of this script, which reads and writes v1

Example fix

// before
const history = JSON.parse(await readFile("history.json", "utf8")); // schema: history/v0
mergeProtocolEvalHistory(history, record);
// after
if (history.schema !== "paperclip.runner-protocol-eval.history/v1") {
  history = emptyProtocolEvalHistory(); // or migrate v0 -> v1
}
mergeProtocolEvalHistory(history, record);

Defensive patterns

Strategy: type-guard

Validate before calling

const history = remote ?? emptyProtocolEvalHistory();
if (history.schema !== "paperclip.runner-protocol-eval.history/v1") throw new Error(`unsupported history schema: ${history.schema}`);

Type guard

const isHistoryV1 = (h) =>
  !!h && typeof h === "object" &&
  h.schema === "paperclip.runner-protocol-eval.history/v1" &&
  Array.isArray(h.campaigns);

Try / catch

try {
  merged = mergeProtocolEvalHistory(history, record);
} catch (e) {
  if (String(e.message).includes("Unsupported protocol eval history schema")) {
    console.error("history.json has an incompatible schema; migrate or reset it");
  }
  throw e;
}

Prevention

Check the remote history.json schema before merging; seed with emptyProtocolEvalHistory() when absent
Do not hand-edit history.json in S3
Version-bump and migrate deliberately if the history schema ever changes

When it happens

Trigger: Calling mergeProtocolEvalHistory(history, record) where history came from an old deployment with a different schema string; downloading a history.json that was hand-edited; passing a campaign document (or other object) instead of a history document; constructing a history object without the schema field.

Common situations: A history.json published by a previous major version of the script still living in the S3 bucket; manual JSON editing of the remote history; a bug writing an empty or wrong-typed file to the history key.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/485a077216bcc929. Report an issue: GitHub.