koala73/worldmonitor · error

classify-eval: could not find

Error message

classify-eval: could not find ${what}

What it means

mustMatch is a helper that extracts a capture group from source text using a regex and throws 'classify-eval: could not find <what>' when the regex does not match. The classify-eval harness uses it to locate exact prompt/provider literals (e.g. CLASSIFY_SYSTEM_PROMPT) in production source files; if the anchor text has drifted, extraction fails loudly rather than pinning the wrong prompt.

Solutions

1. Update the regex in classify-eval.mjs to match the renamed/reformatted literal in the source file.
2. Verify you are pointing classify-eval at the correct current source file (scripts/ais-relay.cjs or the classify-event.ts path).
3. If the constant is now interpolated or dynamic, restore a static template literal so the eval can hash the exact production text.
4. Run the eval again after aligning the source and the extraction patterns.

Example fix

// before (source renamed)
const CLASSIFY_PROMPT = `...`;
// after (align source with eval regex)
const CLASSIFY_SYSTEM_PROMPT = `...`;

if (!/const CLASSIFY_SYSTEM_PROMPT = `[\s\S]*?`;/.test(relaySrc)) throw new Error('relay source missing CLASSIFY_SYSTEM_PROMPT literal');

try { const prompt = extractRelayPrompt(relaySrc); } catch (e) { if (e.message.startsWith('classify-eval: could not find')) { console.error('Source literal drifted; update classify-eval regexes or restore the constant'); } throw e; }

When it happens

Trigger: Running classify-eval when the target source file no longer contains the expected literal, e.g. the regex /const CLASSIFY_SYSTEM_PROMPT = `([\s\S]*?)`;/ no longer matches because the constant was renamed, reformatted, moved, or changed to an interpolated value.

Common situations: Refactoring the relay or classify-event source (renaming constants, changing template syntax) without updating classify-eval's regexes; running the eval against a stale or wrong file path.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• classify-eval: expected one openrouter classify provider…
• classify-eval: is interpolated; the eval can only read a…
• Enter one quantity unit of 1-24 characters.
• INVALID_PREFIX
• no initial JS assets referenced by

AI-assisted analysis of koala73/worldmonitor@e586b8b4b8 (2026-09-22). Data as JSON: /api/errors/bc60bf1757c99a06. Report an issue: GitHub.

Appendix: source

Thrown at scripts/lib/classify-eval.mjs:12

// Pure helpers behind scripts/eval-classify-labels.mjs and its tests: read the two
// headline-classification prompts and their OpenRouter model straight out of the
// source files (so the eval can never drift from what production sends), and score a
// set of labels against blind-judged levels.
import { createHash } from 'node:crypto';

export const ALERT_LEVELS = ['critical', 'high'];
export const isAlertLevel = (level) => ALERT_LEVELS.includes(level);

function mustMatch(src, re, what) {
  const m = src.match(re);
  if (!m) throw new Error(`classify-eval: could not find ${what}`);
  return m[1];
}

// A prompt is hashed and sent as source text, so it must BE the text production sends:
// an interpolated template would pin (and send) a literal `${...}`.
function staticTemplate(text, what) {
  if (text.includes('${')) throw new Error(`classify-eval: ${what} is interpolated; the eval can only read a static template literal`);
  return text;
}

// scripts/ais-relay.cjs — the batch prompt (50 titles per completion).
export function extractRelayPrompt(relaySrc) {
  return staticTemplate(mustMatch(relaySrc, /const CLASSIFY_SYSTEM_PROMPT = `([\s\S]*?)`;/, 'CLASSIFY_SYSTEM_PROMPT'), 'CLASSIFY_SYSTEM_PROMPT');
}

// Reads ONLY the `openrouter` entry, comments stripped. A model held in a constant, or
// missing, throws rather than falling through to a later provider's literal: this value
// is what `--capture` sends and records, so a wrong read would pin itself.

View on GitHub (pinned to e586b8b4b8)