paperclipai/paperclip · error · Error

Public replay contains raw evidence references

Error message

Public replay contains raw evidence references

What it means

For every check in payload.checks, validatePublicChatPayload() verifies the check's shape and then asserts that evidenceRefs is an empty array. Raw evidence references point at internal run-log/evidence stores that are not published, so a public report carrying them is rejected with "Public replay contains raw evidence references".

Source

Thrown at packages/paperclip-runner/scripts/public-eval-viewer.mjs:138

  fields(
    payload.view.issue,
    "identifier title status priority assignee runState scenarioId fixtureProfile",
  );
  fields(payload.view.composer, "state helper reason pendingInteractionId");
  fields(payload.view.connection, "state attempt");
  fields(
    payload.view.evidence,
    "tools calls authorization control_plane runner state traceability parity",
  );
  for (const check of payload.checks) {
    fields(
      check,
      "id kind passed detail evidenceRefs title description definition anchor",
    );
    fields(check.definition, "id kind");
    fields(check.anchor, "kind id");
    if (check.evidenceRefs.length)
      throw new Error("Public replay contains raw evidence references");
  }
  const visit = (value, key = "") => {
    if (typeof value === "string") {
      if (
        /(?:sessionId|providerSessionId)$/i.test(key) &&
        !["public-report", "unknown", "redacted"].includes(value)
      )
        throw new Error("Public replay contains a private session identity");
      for (const pattern of SECRET_TEXT) {
        pattern.lastIndex = 0;
        if (pattern.test(value))
          throw new Error(
            "Public replay contains credential or private reference material",
          );
      }
    } else if (value && typeof value === "object") {
      for (const [name, child] of Object.entries(value)) {
        if (

View on GitHub (pinned to 01ad858492)

Solutions

1. In the projection step, set evidenceRefs: [] on every check before validation.
2. Build public checks from a dedicated projector instead of passing internal check objects through.
3. Audit all check kinds for evidenceRefs population and ensure the public path empties each one.
4. Regenerate and revalidate the payload after fixing the projector.

Example fix

// before
checks: internalChecks,
// after
checks: internalChecks.map((c) => ({ ...c, evidenceRefs: [] })),

const leakingChecks = (payload.checks ?? []).filter((c) => c.evidenceRefs?.length);
if (leakingChecks.length) console.warn("Checks carrying evidenceRefs:", leakingChecks.map((c) => c.id));

const checksArePublic = (checks) => Array.isArray(checks) && checks.every((c) => Array.isArray(c.evidenceRefs) && c.evidenceRefs.length === 0);

try {
  validatePublicChatPayload(payload);
} catch (err) {
  if (err.message === "Public replay contains raw evidence references") {
    payload.checks = payload.checks.map((c) => ({ ...c, evidenceRefs: [] }));
    validatePublicChatPayload(payload);
  } else throw err;
}

When it happens

Trigger: Calling validatePublicChatPayload where any checks[i].evidenceRefs contains one or more entries — typically because the check builder attached evidence IDs (e.g. {kind:"run_event", id:"..."}) that the public projection did not clear.

Common situations: A new check type is added whose builder always populates evidenceRefs and the public projector wasn't updated; reusing internal check objects directly in the public payload instead of projecting them.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Public replay contains private provider metadata
• Public replay contains a private session identity
• Public replay contains a private field
• Public replay contains credential or private reference mater
• sandbox runtime asset key is not a simple path segment: ${ke

AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/8e81461e3eb67988. Report an issue: GitHub.