can1357/oh-my-pi · error

Verifier script /tests/test.sh is missing

Error message

Verifier script /tests/test.sh is missing

What it means

runTrial copies the task's tests/ directory into the VM at /tests and then verifies that /tests/test.sh exists before executing it. This error means the verifier entrypoint script was not found at the expected path, i.e. the task directory has no tests/test.sh.

Source

Thrown at packages/metaharness/src/tb/trial.ts:221

		await writeArtifact(opts.trialDir, "transcript.json", transcript);
		unsubscribe();
		await client.stop().catch(() => {});
		client = null;
		checkDeadline();

		opts.log?.("verify");
		// Docker's default capability set denies CAP_SYS_TIME, but root inside a
		// microVM can change its clock. Restore host UTC before verification so
		// agent certificate workarounds cannot poison verifier networking.
		const hostEpoch = Math.floor(Date.now() / 1_000);
		await vm.exec(`date -u -s @${hostEpoch} >/dev/null 2>&1 || true`, { timeoutSec: 10 });
		const verifierStartedAt = performance.now();
		let verifierExitCode: number | null = null;
		let verifierFailure: string | null = null;
		try {
			await vm.copyDirectory(path.join(opts.task.dir, "tests"), "/tests");
			const script = await vm.exec("test -f /tests/test.sh");
			if (script.exitCode !== 0) throw new Error("Verifier script /tests/test.sh is missing");
			const chmod = await vm.exec("chmod +x /tests/test.sh");
			if (chmod.exitCode !== 0) throw new Error(`Could not make verifier executable: ${chmod.stderr.trim()}`);
			const verifier = await vm.exec("bash /tests/test.sh > /logs/verifier/test-stdout.txt 2>&1", {
				timeoutSec: opts.task.verifierTimeoutSec,
				env: opts.task.verifierEnv,
				cwd: vm.workdir,
			});
			verifierExitCode = verifier.exitCode;
		} catch (error) {
			verifierFailure = errorMessage(error);
		} finally {
			verifierMs = elapsedMs(verifierStartedAt);
		}
		checkDeadline();

		const rewardText = await vm.readFile("/logs/verifier/reward.txt");
		const parsedReward = rewardText === null ? Number.NaN : Number.parseFloat(rewardText.trim());
		const reward = Number.isFinite(parsedReward) && parsedReward >= 0 && parsedReward <= 1 ? parsedReward : null;

View on GitHub (pinned to 9690622007)

Solutions

1. Create the missing tests/test.sh in the task directory referenced by opts.task.dir
2. Verify opts.task.dir points at the intended task root (it should contain a tests/ subdir)
3. Check for naming typos: file must be exactly test.sh and directory exactly tests

Example fix

// task layout before (broken)
my-task/task.yaml
// after
my-task/task.yaml
my-task/tests/test.sh

const testSh = path.join(opts.task.dir, "tests", "test.sh");
if (!existsSync(testSh)) {
  throw new Error(`Task ${opts.task.dir} lacks tests/test.sh before starting trial`);
}

try {
  await runTrial(opts);
} catch (err) {
  if (err.message === "Verifier script /tests/test.sh is missing") {
    // skip or flag the task fixture instead of crashing the batch
    reportBrokenTask(opts.task.dir);
    return;
  }
  throw err;
}

When it happens

Trigger: Calling runTrial with opts.task.dir containing no `tests` subdirectory, or the tests directory lacking a test.sh file, so the copied /tests tree has no test.sh.

Common situations: Authoring a new task fixture and forgetting the test.sh entrypoint; typo'd directory name (test/ vs tests/); the task dir path pointing to the wrong fixture.

Related errors

• cannot stat {0}: No such file or directory
• File not found: ${pathArg}
• package.json not found at ${absolutePath}
• Marketplace catalog not found at ${tried.map(p => `"${p}"`).
• Missing cmux relay auth metadata for ${endpoint.host}:${endp

AI-assisted analysis of can1357/oh-my-pi@9690622007 (2026-08-31). Data as JSON: /api/errors/6e0ee7ca3a84b328. Report an issue: GitHub.