santifer/career-ops · error

invalid golden case : need string id/jd and label.

Error message

invalid golden case ${f}: need string id/jd and label.{archetype:string, score:number}

What it means

Golden case files are JSON documents that must contain a string id, a string jd, and a label object with a string archetype and a numeric score. At load time each file is parsed and shape-checked; any file failing this contract aborts the whole run with exit code 1, naming the offending file. This guards against typo'd or hand-authored case files silently corrupting evaluation results.

Solutions

  1. Open the file named in the error and make id a string, jd a string, label.archetype a string, and label.score a JSON number (no quotes).
  2. Validate the file parses as strict JSON (no trailing commas, no comments) — parse failures surface as this error too.
  3. Remove or relocate any non-golden .json files from the golden directory so they aren't loaded as cases.
  4. Copy an existing valid golden case as a template instead of authoring the shape from scratch.
  5. Optionally run the file through jq or a JSON schema check before committing it.

Example fix

// before: invalid golden case case12.json
{
  "id": "case12",
  "jd": "Senior backend engineer...",
  "label": { "archetype": "platform", "score": "4.5" }
}

// after: score is a number
{
  "id": "case12",
  "jd": "Senior backend engineer...",
  "label": { "archetype": "platform", "score": 4.5 }
}
Defensive patterns

Strategy: type-guard

Validate before calling

function isValidGoldenCase(parsed) {
  return typeof parsed?.id === 'string' &&
         typeof parsed?.jd === 'string' &&
         typeof parsed?.label?.archetype === 'string' &&
         typeof parsed?.label?.score === 'number';
}
// run before committing: for (const f of files) if (!isValidGoldenCase(JSON.parse(readFileSync(f)))) fail(f);

Type guard

const isGoldenCase = (x) =>
  typeof x === 'object' && x !== null &&
  typeof x.id === 'string' &&
  typeof x.jd === 'string' &&
  typeof x.label === 'object' && x.label !== null &&
  typeof x.label.archetype === 'string' &&
  typeof x.label.score === 'number';

Try / catch

try {
  loadGoldenCases(goldenDir);
} catch (err) {
  console.error(`Golden case load failed: ${err.message}`);
  console.error('Fix the named file (string id/jd, label.archetype string, label.score number) or remove non-case JSON.');
  process.exit(1);
}

Prevention

When it happens

Trigger: Placing a JSON file in the golden directory whose shape violates the contract: missing id/jd/label keys, score given as a string ("4.5") instead of a number, label present but archetype missing, a non-JSON or truncated file caught by JSON.parse's throw being wrapped, or an unrelated .json file (e.g. package metadata) dropped into the golden directory.

Common situations: Hand-writing a new golden case with score as a quoted number; saving a JD export that lacks the label block; a JSON file with a trailing comma fails JSON.parse and surfaces here; copy-pasting a case template and forgetting to fill in label.archetype.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/b3a591dcfb18df35. Report an issue: GitHub.

Appendix: source

Thrown at eval-golden.mjs:198

// ---------------------------------------------------------------------------
// Run
// ---------------------------------------------------------------------------
if (!existsSync(goldenDir)) {
  console.error(`❌  golden-set directory not found: ${goldenDir}`);
  process.exit(1);
}

let cases;
try {
  cases = readdirSync(goldenDir)
    .filter((f) => f.endsWith('.json'))
    .map((f) => {
      const parsed = JSON.parse(readFileSync(join(goldenDir, f), 'utf8'));
      if (typeof parsed?.id !== 'string' ||
          typeof parsed?.jd !== 'string' ||
          typeof parsed?.label?.archetype !== 'string' ||
          typeof parsed?.label?.score !== 'number') {
        throw new Error(`invalid golden case ${f}: need string id/jd and label.{archetype:string, score:number}`);
      }
      return parsed;
    });
} catch (err) {
  console.error(`❌  ${err.message || err}`);
  process.exit(1);
}

if (cases.length === 0) {
  console.error(`❌  no golden cases (*.json) in ${goldenDir}`);
  process.exit(1);
}

console.log(`\ngolden-set eval — model "${model}" (${mode}), ${cases.length} case(s)`);
console.log(`(row ✅ needs both archetype + score; the gate counts archetype agreement only)\n`);

let archetypeHits = 0;
const deltas = [];

View on GitHub (pinned to aac998c7ed)