{"record":{"id":"b3a591dcfb18df35","repo":"santifer/career-ops","slug":"invalid-golden-case-f-need-string-id-jd-and-la","errorCode":null,"errorMessage":"invalid golden case ${f}: need string id/jd and label.{archetype:string, score:number}","messagePattern":"invalid golden case (.+?): need string id/jd and label\\.(.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"eval-golden.mjs","lineNumber":198,"sourceCode":"// ---------------------------------------------------------------------------\n// Run\n// ---------------------------------------------------------------------------\nif (!existsSync(goldenDir)) {\n  console.error(`❌  golden-set directory not found: ${goldenDir}`);\n  process.exit(1);\n}\n\nlet cases;\ntry {\n  cases = readdirSync(goldenDir)\n    .filter((f) => f.endsWith('.json'))\n    .map((f) => {\n      const parsed = JSON.parse(readFileSync(join(goldenDir, f), 'utf8'));\n      if (typeof parsed?.id !== 'string' ||\n          typeof parsed?.jd !== 'string' ||\n          typeof parsed?.label?.archetype !== 'string' ||\n          typeof parsed?.label?.score !== 'number') {\n        throw new Error(`invalid golden case ${f}: need string id/jd and label.{archetype:string, score:number}`);\n      }\n      return parsed;\n    });\n} catch (err) {\n  console.error(`❌  ${err.message || err}`);\n  process.exit(1);\n}\n\nif (cases.length === 0) {\n  console.error(`❌  no golden cases (*.json) in ${goldenDir}`);\n  process.exit(1);\n}\n\nconsole.log(`\\ngolden-set eval — model \"${model}\" (${mode}), ${cases.length} case(s)`);\nconsole.log(`(row ✅ needs both archetype + score; the gate counts archetype agreement only)\\n`);\n\nlet archetypeHits = 0;\nconst deltas = [];","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/santifer/career-ops/blob/aac998c7ed7248ea853b720ceeb1fdbeb322fc5d/eval-golden.mjs#L180-L216","documentation":"Golden case files are JSON documents that must contain a string id, a string jd, and a label object with a string archetype and a numeric score. At load time each file is parsed and shape-checked; any file failing this contract aborts the whole run with exit code 1, naming the offending file. This guards against typo'd or hand-authored case files silently corrupting evaluation results.","triggerScenarios":"Placing a JSON file in the golden directory whose shape violates the contract: missing id/jd/label keys, score given as a string (\"4.5\") instead of a number, label present but archetype missing, a non-JSON or truncated file caught by JSON.parse's throw being wrapped, or an unrelated .json file (e.g. package metadata) dropped into the golden directory.","commonSituations":"Hand-writing a new golden case with score as a quoted number; saving a JD export that lacks the label block; a JSON file with a trailing comma fails JSON.parse and surfaces here; copy-pasting a case template and forgetting to fill in label.archetype.","solutions":["Open the file named in the error and make id a string, jd a string, label.archetype a string, and label.score a JSON number (no quotes).","Validate the file parses as strict JSON (no trailing commas, no comments) — parse failures surface as this error too.","Remove or relocate any non-golden .json files from the golden directory so they aren't loaded as cases.","Copy an existing valid golden case as a template instead of authoring the shape from scratch.","Optionally run the file through jq or a JSON schema check before committing it."],"exampleFix":"// before: invalid golden case case12.json\n{\n  \"id\": \"case12\",\n  \"jd\": \"Senior backend engineer...\",\n  \"label\": { \"archetype\": \"platform\", \"score\": \"4.5\" }\n}\n\n// after: score is a number\n{\n  \"id\": \"case12\",\n  \"jd\": \"Senior backend engineer...\",\n  \"label\": { \"archetype\": \"platform\", \"score\": 4.5 }\n}","handlingStrategy":"type-guard","validationCode":"function isValidGoldenCase(parsed) {\n  return typeof parsed?.id === 'string' &&\n         typeof parsed?.jd === 'string' &&\n         typeof parsed?.label?.archetype === 'string' &&\n         typeof parsed?.label?.score === 'number';\n}\n// run before committing: for (const f of files) if (!isValidGoldenCase(JSON.parse(readFileSync(f)))) fail(f);","typeGuard":"const isGoldenCase = (x) =>\n  typeof x === 'object' && x !== null &&\n  typeof x.id === 'string' &&\n  typeof x.jd === 'string' &&\n  typeof x.label === 'object' && x.label !== null &&\n  typeof x.label.archetype === 'string' &&\n  typeof x.label.score === 'number';","tryCatchPattern":"try {\n  loadGoldenCases(goldenDir);\n} catch (err) {\n  console.error(`Golden case load failed: ${err.message}`);\n  console.error('Fix the named file (string id/jd, label.archetype string, label.score number) or remove non-case JSON.');\n  process.exit(1);\n}","preventionTips":["Author new golden cases by copying an existing valid case, never from scratch","Keep score as an unquoted JSON number — \"4.5\" fails the check","Lint golden JSON with jq or a schema check in CI before the eval runs","Keep only case files in the golden directory; store other JSON elsewhere"],"tags":["json","validation","schema","testing"],"backgroundTag":"schema-validation-failed","analyzedSha":"aac998c7ed7248ea853b720ceeb1fdbeb322fc5d","analyzedAt":"2026-09-16T06:35:29.214Z","contentChangedAt":"2026-09-16T06:35:29.214Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}