paperclipai/paperclip · critical · Error

Public report contains credential/session material: ${file}

Error message

Public report contains credential/session material: ${file}

What it means

This error is thrown by validatePublicProtocolEvalReport when a non-rich-attempt file in the public protocol eval report directory matches one of the CREDENTIAL_PATTERNS (AWS access keys like AKIA..., sk- API keys, Bearer tokens, or providerSessionId/sessionId JSON keys). The script is about to publish this directory to a public S3 URL, so any embedded credential or session material would leak secrets publicly. The check runs against every file in the report that is not a rich attempt page (attempts/<id>/index.html).

Source

Thrown at packages/paperclip-runner/scripts/publish-runner-protocol-eval-history.mjs:184

      );
    }
    if (file.startsWith("viewer/")) {
      const expected = viewer?.files.get(file);
      if (!expected || !expected.equals(await readFile(absolute)))
        throw new Error(
          `Public viewer asset differs from trusted build: ${file}`,
        );
      continue;
    }
    const content = await readFile(absolute, "utf8");
    const richAttempt = /^attempts\/[^/]+\/index\.html$/.test(file);
    const payload = richAttempt
      ? validatePublicViewerPage(content, viewer.index)
      : null;
    if (!richAttempt) {
      for (const pattern of CREDENTIAL_PATTERNS) {
        if (pattern.test(content))
          throw new Error(
            `Public report contains credential/session material: ${file}`,
          );
      }
      if (extname(file) !== ".html") continue;
      for (const pattern of ACTIVE_HTML_PATTERNS) {
        if (pattern.test(content))
          throw new Error(
            `Public report contains active or remote HTML: ${file}`,
          );
      }
    }
    const navigation = payload
      ? [
          payload.navigation?.suiteHref,
          payload.navigation?.previous?.href,
          payload.navigation?.next?.href,
        ].filter(Boolean)
      : [];

View on GitHub (pinned to 01ad858492)

Solutions

Find the offending file (named in the error) and remove or redact the matching credential/session string, then regenerate the report.
Add redaction at report-generation time (e.g. scrub secrets from agent transcripts before writing HTML/JSON).
If a session ID is legitimate non-secret metadata, rename the JSON key so it does not match the sessionId/providerSessionId pattern.
Regenerate the report from source rather than editing the published copy, then re-run the publish script.

Example fix

// before: report HTML embeds raw transcript
<pre>$ export OPENAI_API_KEY=sk-abc123def456ghijklmno</pre>
// after: redact secrets before rendering
const redacted = transcript.replace(/sk-[A-Za-z0-9_-]{20,}/g, "sk-[REDACTED]");

Defensive patterns

Strategy: validation

Validate before calling

import { readFile } from "node:fs/promises";
const CREDENTIAL_PATTERNS = [
  /\bAKIA[0-9A-Z]{16}\b/u,
  /\bsk-[A-Za-z0-9_-]{20,}\b/u,
  /\bBearer\s+[A-Za-z0-9._~-]{16,}\b/iu,
  /["'](?:providerSessionId|sessionId)["']\s*:/u,
];
export async function assertNoSecrets(file) {
  const content = await readFile(file, "utf8");
  const hit = CREDENTIAL_PATTERNS.findIndex((p) => p.test(content));
  if (hit !== -1) throw new Error(`Secret-like content in ${file} (pattern #${hit})`);
}

Type guard

const containsSecret = (content) => CREDENTIAL_PATTERNS.some((p) => p.test(content));

Try / catch

try {
  await publishProtocolEvalHistory({ reportRoot, destination, viewerRoot });
} catch (err) {
  if (String(err.message).startsWith("Public report contains credential/session material")) {
    console.error("Redact secrets in the named file and regenerate the report before publishing.");
  }
  throw err;
}

Prevention

Redact secrets in agent transcripts at capture time, before any HTML rendering.
Run validatePublicProtocolEvalReport locally in CI before the publish job.
Keep provider session IDs out of serialized report JSON, or rename those keys.
Never publish raw agent environment dumps; scrub env vars from captured output.

When it happens

Trigger: Running publishProtocolEvalHistory (or the script directly) when a report file such as index.html, tests/*.html, attempts/*.html, or campaign.json contains text matching AKIA[0-9A-Z]{16}, sk-[A-Za-z0-9_-]{20,}, `Bearer <token>` (16+ chars), or a JSON key "providerSessionId"/"sessionId".

Common situations: Agents under evaluation echo environment variables or API keys into their transcript output; a captured Bearer token from an HTTP request log is pasted into an attempt page; a sessionId field from a provider response is serialized into campaign.json or a test result page; CI re-publishes an old report directory generated before secret redaction was added.

Related errors

AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/030632bb991a878c. Report an issue: GitHub.