{"record":{"id":"8e81461e3eb67988","repo":"paperclipai/paperclip","slug":"public-replay-contains-raw-evidence-references","errorCode":null,"errorMessage":"Public replay contains raw evidence references","messagePattern":"Public replay contains raw evidence references","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/paperclip-runner/scripts/public-eval-viewer.mjs","lineNumber":138,"sourceCode":"  fields(\n    payload.view.issue,\n    \"identifier title status priority assignee runState scenarioId fixtureProfile\",\n  );\n  fields(payload.view.composer, \"state helper reason pendingInteractionId\");\n  fields(payload.view.connection, \"state attempt\");\n  fields(\n    payload.view.evidence,\n    \"tools calls authorization control_plane runner state traceability parity\",\n  );\n  for (const check of payload.checks) {\n    fields(\n      check,\n      \"id kind passed detail evidenceRefs title description definition anchor\",\n    );\n    fields(check.definition, \"id kind\");\n    fields(check.anchor, \"kind id\");\n    if (check.evidenceRefs.length)\n      throw new Error(\"Public replay contains raw evidence references\");\n  }\n  const visit = (value, key = \"\") => {\n    if (typeof value === \"string\") {\n      if (\n        /(?:sessionId|providerSessionId)$/i.test(key) &&\n        ![\"public-report\", \"unknown\", \"redacted\"].includes(value)\n      )\n        throw new Error(\"Public replay contains a private session identity\");\n      for (const pattern of SECRET_TEXT) {\n        pattern.lastIndex = 0;\n        if (pattern.test(value))\n          throw new Error(\n            \"Public replay contains credential or private reference material\",\n          );\n      }\n    } else if (value && typeof value === \"object\") {\n      for (const [name, child] of Object.entries(value)) {\n        if (","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/packages/paperclip-runner/scripts/public-eval-viewer.mjs#L120-L156","documentation":"For every check in payload.checks, validatePublicChatPayload() verifies the check's shape and then asserts that evidenceRefs is an empty array. Raw evidence references point at internal run-log/evidence stores that are not published, so a public report carrying them is rejected with \"Public replay contains raw evidence references\".","triggerScenarios":"Calling validatePublicChatPayload where any checks[i].evidenceRefs contains one or more entries — typically because the check builder attached evidence IDs (e.g. {kind:\"run_event\", id:\"...\"}) that the public projection did not clear.","commonSituations":"A new check type is added whose builder always populates evidenceRefs and the public projector wasn't updated; reusing internal check objects directly in the public payload instead of projecting them.","solutions":["In the projection step, set evidenceRefs: [] on every check before validation.","Build public checks from a dedicated projector instead of passing internal check objects through.","Audit all check kinds for evidenceRefs population and ensure the public path empties each one.","Regenerate and revalidate the payload after fixing the projector."],"exampleFix":"// before\nchecks: internalChecks,\n// after\nchecks: internalChecks.map((c) => ({ ...c, evidenceRefs: [] })),","handlingStrategy":"validation","validationCode":"const leakingChecks = (payload.checks ?? []).filter((c) => c.evidenceRefs?.length);\nif (leakingChecks.length) console.warn(\"Checks carrying evidenceRefs:\", leakingChecks.map((c) => c.id));","typeGuard":"const checksArePublic = (checks) => Array.isArray(checks) && checks.every((c) => Array.isArray(c.evidenceRefs) && c.evidenceRefs.length === 0);","tryCatchPattern":"try {\n  validatePublicChatPayload(payload);\n} catch (err) {\n  if (err.message === \"Public replay contains raw evidence references\") {\n    payload.checks = payload.checks.map((c) => ({ ...c, evidenceRefs: [] }));\n    validatePublicChatPayload(payload);\n  } else throw err;\n}","preventionTips":["Empty evidenceRefs for every check kind in the public projector, including newly added kinds","Add a fixture with evidence-backed checks to the validation test suite","Never reuse internal check objects; always map through the public check projector","Review new check builders for evidence attachment and confirm the public path strips it"],"tags":["security","privacy","validation","data-leak"],"backgroundTag":"schema-validation-failed","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-10T03:14:50.855Z","contentChangedAt":"2026-09-10T03:14:50.855Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}