What causes "openai-eval.mjs exited ${res.status}: ${(res.stderr || '').slice(0, 200)}" in santifer/career-ops?

Run openai-eval.mjs directly with the same --file/--model flags to see the full stderr beyond the 200-char truncation.

openai-eval.mjs exited

Error message

openai-eval.mjs exited ${res.status}: ${(res.stderr || '').slice(0, 200)}

What it means

In live mode, getCompletion shells out to openai-eval.mjs via spawnSync with a 6-minute timeout, writing the JD to a temp file. If the child process exits with a non-zero status, this error propagates its exit code plus up to 200 characters of stderr, surfacing whatever the underlying evaluator failed on (auth, rate limit, bad model name, crash).

Solutions

Run openai-eval.mjs directly with the same --file/--model flags to see the full stderr beyond the 200-char truncation.
Check API credentials (env var set, not expired, has quota) — the most common cause is missing/invalid key.
Retry after a provider 429/5xx; add backoff or run the batch when rate limits reset.
Validate the --model value is supported by openai-eval.mjs (provider rename? typo?).
If the timeout was hit (6 min), check network egress from your environment or use replay mode with a recorded fixture.

Example fix

# before: opaque failure inside a batch run
node eval-golden.mjs --live
# Error: openai-eval.mjs exited 1: Error: 401 Unauthorized...

# after: reproduce directly to see full stderr, fix the key
OPENAI_API_KEY=sk-... node openai-eval.mjs --file jd.txt --model deepseek/deepseek-chat --no-save
node eval-golden.mjs --live

Defensive patterns

Strategy: retry

Validate before calling

import { spawnSync } from 'node:child_process';
const probe = spawnSync(process.execPath, [join(ROOT, 'openai-eval.mjs'), '--help'], { encoding: 'utf8' });
if (probe.error || probe.status !== 0) console.error('openai-eval.mjs not runnable here:', probe.stderr);

Try / catch

try {
  return getCompletion(testCase, model, { mode: 'live' });
} catch (err) {
  if (err.message.includes('openai-eval.mjs exited')) {
    if (/429|rate/i.test(err.message)) { await sleep(backoff); return retry(); }
    if (/401|403/i.test(err.message)) console.error('Check API key before retrying');
  }
  throw err;
}

Prevention

Verify API credentials and quota before starting a live batch run
Prefer replay mode in CI; reserve --live for deliberate fixture recording
Run openai-eval.mjs standalone first when debugging — it prints full stderr without the 200-char truncation
Add retry-with-backoff around live calls to absorb transient 429/5xx
Ensure CI has network egress to the provider before scheduling live evals

When it happens

Trigger: Calling getCompletion (or eval-golden.mjs) with --live when openai-eval.mjs exits non-zero: invalid/missing API key, network failure, rate limit or provider 5xx, unsupported --model value, unhandled exception inside openai-eval.mjs, or the 360000ms timeout being hit (spawnSync reports a non-null/failed status).

Common situations: Expired or unset OPENAI/DEEPSEEK API key in CI secrets; provider outage or 429 during a batch run; model id typo after a provider rename; IPv4/VPN/network restrictions in sandboxed CI blocking outbound API calls; child crashed due to a malformed JD file.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/2e03f0b47b2a0249. Report an issue: GitHub.

Appendix: source

Thrown at eval-golden.mjs:159

    // path separators, or the fixture lands in a phantom subdirectory. Sanitize
    // to a flat filename — record fixtures under the same sanitized name.
    const fixture = join(fixtureDir, `${testCase.id}__${fixtureModelId(model)}.txt`);
    if (!existsSync(fixture)) {
      throw new Error(`missing replay fixture: ${fixture} — record it or run --live`);
    }
    return readFileSync(fixture, 'utf8');
  }

  // live: write the JD to a temp file and run the existing evaluator.
  const dir = mkdtempSync(join(tmpdir(), 'eval-golden-'));
  try {
    const jdFile = join(dir, 'jd.txt');
    writeFileSync(jdFile, testCase.jd);
    const res = spawnSync(process.execPath,
      [join(ROOT, 'openai-eval.mjs'), '--file', jdFile, '--model', model, '--no-save'],
      { encoding: 'utf8', env: process.env, timeout: 360000 });
    if (res.status !== 0) {
      throw new Error(`openai-eval.mjs exited ${res.status}: ${(res.stderr || '').slice(0, 200)}`);
    }
    return res.stdout || '';
  } finally {
    rmSync(dir, { recursive: true, force: true });
  }
}

/**
 * Median of a numeric array (0 for an empty array).
 *
 * @param {number[]} xs - Values to summarize.
 * @returns {number} The median value.
 */
function median(xs) {
  if (xs.length === 0) return 0;
  const s = [...xs].sort((a, b) => a - b);
  const mid = Math.floor(s.length / 2);
  return s.length % 2 ? s[mid] : (s[mid - 1] + s[mid]) / 2;

View on GitHub (pinned to aac998c7ed)