What causes "Model config not found: {model_file}" in abhigyanpatwari/GitNexus?

List available models: `ls eval/configs/models/` (or print AVAILABLE_MODELS in the script).

abhigyanpatwari/GitNexus · error · FileNotFoundError

Model config not found: {model_file}

Error message

Model config not found: {model_file}

What it means

Thrown by build_config() in eval/run_eval.py: it looks for eval/configs/models/<model_name>.yaml (MODELS_DIR) and raises FileNotFoundError if it does not exist. AVAILABLE_MODELS is computed at import time by globbing that directory, so the message tells you the resolved path that was missing.

Source

Thrown at eval/run_eval.py:102

def merge_configs(*configs: dict) -> dict:
    """Recursively merge multiple config dicts (later values win)."""
    result = {}
    for config in configs:
        for key, value in config.items():
            if key in result and isinstance(result[key], dict) and isinstance(value, dict):
                result[key] = merge_configs(result[key], value)
            else:
                result[key] = value
    return result


def build_config(model_name: str, mode_name: str) -> dict:
    """Build a complete config from model + mode YAML files."""
    model_file = MODELS_DIR / f"{model_name}.yaml"
    mode_file = MODES_DIR / f"{mode_name}.yaml"

    if not model_file.exists():
        raise FileNotFoundError(f"Model config not found: {model_file}")
    if not mode_file.exists():
        raise FileNotFoundError(f"Mode config not found: {mode_file}")

    model_config = load_yaml_config(model_file)
    mode_config = load_yaml_config(mode_file)

    return merge_configs(mode_config, model_config)


def load_instances(subset: str, split: str, slice_spec: str = "", filter_spec: str = "") -> list[dict]:
    """Load SWE-bench instances."""
    from datasets import load_dataset
    import re

    dataset_path = DATASET_MAPPING.get(subset, subset)
    logger.info(f"Loading dataset: {dataset_path}, split: {split}")
    instances = list(load_dataset(dataset_path, split=split))

View on GitHub (pinned to d540b00184)

Solutions

List available models: `ls eval/configs/models/` (or print AVAILABLE_MODELS in the script).
Create eval/configs/models/<model_name>.yaml with the required model fields (api base, model id, etc.).
Fix the typo / case on the --model argument to match a filename stem exactly.
Run the eval from the eval/ directory (or pass absolute paths) so MODELS_DIR resolves correctly.

Example fix

# before
python run_eval.py --model claude-sonet --mode workflow
# FileNotFoundError: Model config not found: eval/configs/models/claude-sonet.yaml

# after
python run_eval.py --model claude-sonnet --mode workflow
# or add the file: eval/configs/models/claude-sonet.yaml

Defensive patterns

Strategy: validation

Validate before calling

from pathlib import Path
MODELS_DIR = Path(__file__).parent / 'configs' / 'models'
def resolve_model(name: str) -> Path:
    p = (MODELS_DIR / f'{name}.yaml')
    if not p.exists():
        avail = sorted(x.stem for x in MODELS_DIR.glob('*.yaml'))
        raise SystemExit(f'Unknown model {name!r}. Available: {avail}')
    return p
# call before build_config()

Type guard

def model_config_exists(name: str) -> bool:
    return (MODELS_DIR / f'{name}.yaml').exists()

Try / catch

try:
    cfg = build_config(model_name, mode_name)
except FileNotFoundError as e:
    if 'Model config' in str(e):
        print('available:', AVAILABLE_MODELS); raise
    raise

Prevention

Always print AVAILABLE_MODELS in --help output.
Validate --model against the directory listing at CLI parse time, not deep inside build_config.
Run the eval from the eval/ directory so relative CONFIGS_DIR resolves correctly.

When it happens

Trigger: Calling run_eval with --model <name> where <name>.yaml is not present under eval/configs/models/. Examples: typo 'claude-sonet' vs 'claude-sonnet'; a model file that was never added; running from a different working directory so MODELS_DIR resolves elsewhere.

Common situations: Model-name typo on the CLI; new model not yet given a config yaml; running the eval from outside the eval/ directory so the relative CONFIGS_DIR is wrong; case mismatch (Claude-Sonnet vs claude-sonnet).

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/d551b1085de8b284. Report an issue: GitHub.