⚠️ No matching evaluation file found in

Error message

⚠️   No matching evaluation file found in ${customModesDir}; using default modes/oferta.md

What it means

When language.modes_dir in config/profile.yml points to a directory that exists and is inside the project root, gemini-eval.mjs looks for one of the known market evaluation files (oferta.md, angebot.md, offre.md, kyujin.md, is-ilani.md, naukri.md). If none exists, it warns and falls back to the default modes/oferta.md. Evaluation still runs, but with the default English evaluation mode instead of the intended market-specific one.

Solutions

Add one of the recognized evaluation files to the configured directory: copy the closest market template (e.g. modes/de/angebot.md) and translate/adapt it.
Or rename your existing evaluation file to one of the candidate names (oferta.md, angebot.md, offre.md, kyujin.md, is-ilani.md, naukri.md).
Or point language.modes_dir at an existing market dir shipped with the project (modes/de, modes/fr, modes/ar, modes/ja, modes/tr, modes/hi).

Example fix

# before
modes/es/
  _shared.md        # only shared file -> warning, falls back to modes/oferta.md

# after
cp modes/de/angebot.md modes/es/angebot.md   # recognized candidate -> used
# (then translate angebot.md content to Spanish)

Defensive patterns

Strategy: validation

Validate before calling

import { existsSync, readdirSync } from 'node:fs';
import { join } from 'node:path';
const CANDIDATES = ['oferta.md','angebot.md','offre.md','kyujin.md','is-ilani.md','naukri.md'];
const dir = resolve('modes', profile?.language?.modes_dir ?? '');
const hasEval = CANDIDATES.some(f => existsSync(join(dir, f)));
if (!hasEval) console.warn(`${dir} has no evaluation file; add one of: ${CANDIDATES.join(', ')}`);

Try / catch

try {
  await runEvaluation();
} catch (err) {
  if (/No matching evaluation file/.test(err.message)) {
    console.warn('Custom modes_dir lacks a recognized eval file; defaulting to modes/oferta.md.');
  } else throw err;
}

Prevention

When creating a custom market dir, copy a shipped market dir wholesale so the recognized eval filename is preserved.
Name your translated evaluation file with one of the recognized candidate names (oferta.md, angebot.md, offre.md, kyujin.md, is-ilani.md, naukri.md).
After adding a market dir, verify it contains both _shared.md and an evaluation file before pointing modes_dir at it.
Prefer the shipped market dirs (modes/de, modes/fr, etc.) over hand-rolled directories unless you maintain the full file set.

When it happens

Trigger: modes_dir is set to a directory that contains no recognized evaluation mode file — e.g. a directory with only _shared.md, a custom modes dir holding files with non-standard names, or an empty/misnamed directory. The existsSync(join(dirPath, file)) check fails for all six candidate filenames.

Common situations: Users create their own market dir (e.g. modes/es) but name the file evaluacion.md instead of a recognized candidate; users point modes_dir at modes/ root where only oferta.md lives in default locations but the candidate search happens in the custom dir; partial manual setups that copied _shared.md but forgot the evaluation file.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/65a9eb9ecc5f8a78. Report an issue: GitHub.

Appendix: source

Thrown at gemini-eval.mjs:122

if (existsSync(PATHS.profileYml)) {
  try {
    const yamlContent = stripBom(readFileSync(PATHS.profileYml, 'utf-8'));
    const profile = yaml.load(yamlContent);
    if (profile && profile.language && profile.language.modes_dir) {
      const customModesDir = profile.language.modes_dir;
      const dirPath = resolve(CODE_ROOT, customModesDir);
      const rel = relative(CODE_ROOT, dirPath);
      if (rel.startsWith('..') || isAbsolute(customModesDir)) {
        console.warn(`⚠️   modes_dir "${customModesDir}" escapes project root; using default modes/`);
      } else {
        if (existsSync(dirPath)) {
          const candidateFiles = ['oferta.md', 'angebot.md', 'offre.md', 'kyujin.md', 'is-ilani.md', 'naukri.md'];
          const found = candidateFiles.find((file) => existsSync(join(dirPath, file)));
          if (found) {
            modesDir = customModesDir;
            evalFilename = found;
          } else {
            console.warn(`⚠️   No matching evaluation file found in ${customModesDir}; using default modes/oferta.md`);
          }
        } else {
          console.warn(`⚠️   modes_dir "${customModesDir}" not found; using default modes/`);
        }
      }
    }
  } catch (err) {
    console.warn(`⚠️   Could not parse config/profile.yml: ${err.message}`);
  }
}

PATHS.shared = join(CODE_ROOT, modesDir, '_shared.md');
PATHS.oferta = join(CODE_ROOT, modesDir, evalFilename);

// ---------------------------------------------------------------------------
// CLI argument parsing
// ---------------------------------------------------------------------------
const args = process.argv.slice(2);

View on GitHub (pinned to aac998c7ed)