koala73/worldmonitor · error

classify-eval: is interpolated; the eval can only read a…

Error message

classify-eval: ${what} is interpolated; the eval can only read a static template literal

What it means

staticTemplate guards classify-eval's core invariant: prompts are hashed and sent as literal source text, so a template literal containing '${' would hash and send a literal interpolation placeholder instead of the real runtime prompt. If the extracted prompt text includes '${', it throws naming the offending item. This prevents the eval from silently pinning an incorrect prompt.

Solutions

  1. Rewrite the production prompt as a static template literal with no ${...} interpolation.
  2. If runtime values are needed, move them into the completion request as separate parameters/messages, not into the system prompt text.
  3. Concatenate fixed string pieces instead of interpolating, only if the concatenation itself is static at the definition site.
  4. Re-run classify-eval to confirm the extracted prompt is static before trusting captured results.

Example fix

// before
const CLASSIFY_SYSTEM_PROMPT = `Classify for model ${MODEL_NAME}.`;
// after
const CLASSIFY_SYSTEM_PROMPT = `Classify the title.`; // model passed as a request parameter
Defensive patterns

Strategy: validation

Validate before calling

const m = relaySrc.match(/const CLASSIFY_SYSTEM_PROMPT = `([\s\S]*?)`;/);
if (m && m[1].includes('${')) throw new Error('CLASSIFY_SYSTEM_PROMPT is interpolated');

Type guard

const isStaticTemplate = (text) => typeof text === 'string' && !text.includes('${');

Try / catch

try { const prompt = extractRelayPrompt(src); } catch (e) { if (e.message.includes('is interpolated')) { console.error('Remove ${...} from the production prompt; pass dynamic values as request parameters'); } throw e; }

Prevention

When it happens

Trigger: Running classify-eval when the production prompt constant was converted into an interpolated template literal, e.g. `Prompt for ${modelName}` or a template embedding config values, so the extracted source contains '${'.

Common situations: A developer refactoring prompts to interpolate model names, dates, or thresholds; templating added for localization; generated prompt files where an outer template leaks inner '${'.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of koala73/worldmonitor@e586b8b4b8 (2026-09-22). Data as JSON: /api/errors/86274abb5972abad. Report an issue: GitHub.

Appendix: source

Thrown at scripts/lib/classify-eval.mjs:19

// Pure helpers behind scripts/eval-classify-labels.mjs and its tests: read the two
// headline-classification prompts and their OpenRouter model straight out of the
// source files (so the eval can never drift from what production sends), and score a
// set of labels against blind-judged levels.
import { createHash } from 'node:crypto';

export const ALERT_LEVELS = ['critical', 'high'];
export const isAlertLevel = (level) => ALERT_LEVELS.includes(level);

function mustMatch(src, re, what) {
  const m = src.match(re);
  if (!m) throw new Error(`classify-eval: could not find ${what}`);
  return m[1];
}

// A prompt is hashed and sent as source text, so it must BE the text production sends:
// an interpolated template would pin (and send) a literal `${...}`.
function staticTemplate(text, what) {
  if (text.includes('${')) throw new Error(`classify-eval: ${what} is interpolated; the eval can only read a static template literal`);
  return text;
}

// scripts/ais-relay.cjs — the batch prompt (50 titles per completion).
export function extractRelayPrompt(relaySrc) {
  return staticTemplate(mustMatch(relaySrc, /const CLASSIFY_SYSTEM_PROMPT = `([\s\S]*?)`;/, 'CLASSIFY_SYSTEM_PROMPT'), 'CLASSIFY_SYSTEM_PROMPT');
}

// Reads ONLY the `openrouter` entry, comments stripped. A model held in a constant, or
// missing, throws rather than falling through to a later provider's literal: this value
// is what `--capture` sends and records, so a wrong read would pin itself.
export function extractRelayOpenRouterModel(relaySrc) {
  const list = mustMatch(relaySrc, /const CLASSIFY_LLM_PROVIDERS = \[([\s\S]*?)\n\];/, 'CLASSIFY_LLM_PROVIDERS');
  const entries = list.split(/(?=\{\s*\n\s*name: ')/).filter((e) => /^\{\s*\n\s*name: 'openrouter',/.test(e));
  if (entries.length !== 1) throw new Error(`classify-eval: expected one openrouter classify provider, found ${entries.length}`);
  const code = entries[0].replace(/\/\/[^\n]*/g, '');
  return mustMatch(code, /\n\s*model: '([^']+)',/, "a quoted model on the relay's openrouter classify provider");
}

View on GitHub (pinned to e586b8b4b8)