{"record":{"id":"0840054c80fe6f00","repo":"koala73/worldmonitor","slug":"classify-eval-expected-one-openrouter-classify-provider","errorCode":null,"errorMessage":"classify-eval: expected one openrouter classify provider, found ${entries.length}","messagePattern":"classify-eval: expected one openrouter classify provider, found (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scripts/lib/classify-eval.mjs","lineNumber":34,"sourceCode":"// A prompt is hashed and sent as source text, so it must BE the text production sends:\n// an interpolated template would pin (and send) a literal `${...}`.\nfunction staticTemplate(text, what) {\n  if (text.includes('${')) throw new Error(`classify-eval: ${what} is interpolated; the eval can only read a static template literal`);\n  return text;\n}\n\n// scripts/ais-relay.cjs — the batch prompt (50 titles per completion).\nexport function extractRelayPrompt(relaySrc) {\n  return staticTemplate(mustMatch(relaySrc, /const CLASSIFY_SYSTEM_PROMPT = `([\\s\\S]*?)`;/, 'CLASSIFY_SYSTEM_PROMPT'), 'CLASSIFY_SYSTEM_PROMPT');\n}\n\n// Reads ONLY the `openrouter` entry, comments stripped. A model held in a constant, or\n// missing, throws rather than falling through to a later provider's literal: this value\n// is what `--capture` sends and records, so a wrong read would pin itself.\nexport function extractRelayOpenRouterModel(relaySrc) {\n  const list = mustMatch(relaySrc, /const CLASSIFY_LLM_PROVIDERS = \\[([\\s\\S]*?)\\n\\];/, 'CLASSIFY_LLM_PROVIDERS');\n  const entries = list.split(/(?=\\{\\s*\\n\\s*name: ')/).filter((e) => /^\\{\\s*\\n\\s*name: 'openrouter',/.test(e));\n  if (entries.length !== 1) throw new Error(`classify-eval: expected one openrouter classify provider, found ${entries.length}`);\n  const code = entries[0].replace(/\\/\\/[^\\n]*/g, '');\n  return mustMatch(code, /\\n\\s*model: '([^']+)',/, \"a quoted model on the relay's openrouter classify provider\");\n}\n\n// server/worldmonitor/intelligence/v1/classify-event.ts — one title per completion.\nexport function extractRpcPrompt(rpcSrc) {\n  return staticTemplate(mustMatch(rpcSrc, /const systemPrompt = `([\\s\\S]*?)`;/, 'classify-event systemPrompt'), 'classify-event systemPrompt');\n}\n\nexport function extractRpcOpenRouterModel(rpcSrc) {\n  return mustMatch(rpcSrc, /^const CLASSIFY_OPENROUTER_MODEL = '([^']+)';/m, 'CLASSIFY_OPENROUTER_MODEL');\n}\n\n// The block both prompts must carry verbatim, delimited by its first and last line.\nexport function extractHighBoundaryBlock(prompt) {\n  return mustMatch(prompt, /(Do not under-rate \"high\"\\.[\\s\\S]*?not for the event itself\\.)/, 'the \"high\" boundary block');\n}\n","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/koala73/worldmonitor/blob/e586b8b4b80f595aa7ece295eec10d76f2921240/scripts/lib/classify-eval.mjs#L16-L52","documentation":"extractRelayOpenRouterModel reads only the `openrouter` entry of CLASSIFY_LLM_PROVIDERS from the relay source, splitting the array into entries by name. Because this model value is what --capture sends and records, an ambiguous or missing read would pin the wrong literal, so it throws when the number of matched openrouter entries is not exactly one. The error reports the actual count found.","triggerScenarios":"Running classify-eval when CLASSIFY_LLM_PROVIDERS contains zero openrouter entries (renamed, removed, or the array regex matched the wrong region) or more than one (duplicated provider blocks, e.g. two openrouter entries for different tiers).","commonSituations":"Renaming the provider from 'openrouter' in the relay config; copy-pasting a provider block to add a second model; reformatting the array (extra blank lines/comments) so the split-by-name heuristic miscounts.","solutions":["Ensure CLASSIFY_LLM_PROVIDERS contains exactly one entry with name: 'openrouter' — remove duplicates or restore a removed entry.","If the provider was renamed, update the regex filter (/name: 'openrouter',/) in classify-eval.mjs.","Check that the array literal still matches /const CLASSIFY_LLM_PROVIDERS = \\[...\\n\\];/ and that formatting changes did not confuse entry splitting.","Confirm the quoted model field on that entry so the subsequent mustMatch can read it."],"exampleFix":"// before (duplicated)\nconst CLASSIFY_LLM_PROVIDERS = [{ name: 'openrouter', model: 'a' }, { name: 'openrouter', model: 'b' }];\n// after\nconst CLASSIFY_LLM_PROVIDERS = [{ name: 'openrouter', model: 'a' }];","handlingStrategy":"validation","validationCode":"const count = (relaySrc.match(/name: 'openrouter',/g) || []).length;\nif (count !== 1) throw new Error(`expected exactly one openrouter provider, found ${count}`);","typeGuard":null,"tryCatchPattern":"try { const model = extractRelayOpenRouterModel(src); } catch (e) { if (e.message.includes('openrouter classify provider')) { console.error('Fix CLASSIFY_LLM_PROVIDERS to hold exactly one openrouter entry'); } throw e; }","preventionTips":["Keep exactly one 'openrouter' provider entry in CLASSIFY_LLM_PROVIDERS","If a second openrouter-backed model is needed, use a distinct provider name or extend the eval extractor","Update classify-eval filters in the same PR that renames providers","Run classify-eval --capture in CI to catch extraction drift early"],"tags":["eval-harness","llm","config-extraction","regex"],"backgroundTag":"unexpected-response-shape","analyzedSha":"e586b8b4b80f595aa7ece295eec10d76f2921240","analyzedAt":"2026-09-22T01:50:49.965Z","contentChangedAt":"2026-09-22T01:50:49.965Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}