paperclipai/paperclip · error · Error

Public protocol eval report requires index.html and campaign

Error message

Public protocol eval report requires index.html and campaign.json

What it means

After collecting all files under the public report root, validatePublicProtocolEvalReport checks that the report is structurally complete: a top-level index.html and a top-level campaign.json must both exist. If either is missing the script refuses to publish, since a public report without the entry page or campaign metadata is unusable for viewers.

Source

Thrown at packages/paperclip-runner/scripts/publish-runner-protocol-eval-history.mjs:151

function internalHtmlHrefs(content) {
  return [...content.matchAll(/href\s*=\s*["']([^"']+)["']/giu)]
    .map((match) => match[1])
    .filter((href) => href && !href.startsWith("#"));
}

export async function validatePublicProtocolEvalReport(
  reportRoot,
  { viewerRoot } = {},
) {
  const root = resolve(reportRoot);
  const files = await relativeFiles(root);
  const hasChat = files.some((file) =>
    /^attempts\/[^/]+\/index\.html$/.test(file),
  );
  const viewer = hasChat ? await trustedViewerFiles(viewerRoot) : null;
  if (!files.includes("index.html") || !files.includes("campaign.json")) {
    throw new Error(
      "Public protocol eval report requires index.html and campaign.json",
    );
  }
  for (const file of files) {
    if (!isPublicProtocolEvalPath(file)) {
      throw new Error(
        `Refusing non-allowlisted public protocol eval path ${file}`,
      );
    }
    const absolute = resolve(root, ...file.split("/"));
    const metadata = await stat(absolute);
    if (metadata.size === 0 || metadata.size > 12 * 1024 * 1024) {
      throw new Error(
        `Public protocol eval file exceeds its size boundary: ${file}`,
      );
    }
    if (file.startsWith("viewer/")) {
      const expected = viewer?.files.get(file);

View on GitHub (pinned to 01ad858492)

Solutions

Verify you passed the report ROOT directory (the one containing index.html and campaign.json), not a subdirectory.
Regenerate the report so both index.html and campaign.json are produced before publishing.
Check the report generation step's logs for a failure that skipped writing campaign.json or index.html.

Example fix

// before
await publish({ reportDir: 'public-report/attempts' });
// after
await publish({ reportDir: 'public-report' }); // contains index.html + campaign.json

Defensive patterns

Strategy: validation

Validate before calling

import { existsSync } from 'node:fs';
if (!existsSync(`${reportRoot}/index.html`) || !existsSync(`${reportRoot}/campaign.json`)) {
  throw new Error('report root must contain index.html and campaign.json');
}

Type guard

function isCompleteReportRoot(fs, dir) {
  return fs.existsSync(`${dir}/index.html`) && fs.existsSync(`${dir}/campaign.json`);
}

Try / catch

try {
  await publishReport(root);
} catch (err) {
  if (err.message === 'Public protocol eval report requires index.html and campaign.json') {
    console.error('Wrong or incomplete report dir; regenerate or point at the report root.');
    process.exitCode = 1;
  } else throw err;
}

Prevention

Always pass the report root, never a subdirectory like attempts/.
Add a pre-publish check asserting index.html and campaign.json exist.
Verify the report generator completed successfully (exit code, output logs) before publishing.

When it happens

Trigger: Publishing a directory that lacks index.html at its root, or lacks campaign.json at its root (e.g. pointing the script at an attempts/ subdirectory or a partial build output).

Common situations: Passing the wrong directory (the attempts folder instead of the report root); a build step failed before writing index.html/campaign.json; campaign.json was renamed or deleted; a report produced by a different/older generator layout.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/75c768138bb72e2d. Report an issue: GitHub.