abhigyanpatwari/GitNexus · error · ValueError

{flag} must name a nonblank, versioned model

Error message

{flag} must name a nonblank, versioned model

What it means

Thrown by normalized_model_identifier when the model string passed via --model (or equivalent) is empty after stripping whitespace. The benchmark requires a pinned, reproducible model identity, so a blank value (None, '', or whitespace-only) is rejected up front rather than silently falling back to a default.

Source

Thrown at eval/workflow_bench/runner_tasks.py:19

"""Model and immutable task-binding validation for workflow benchmarks."""

from __future__ import annotations

import hashlib
import re
from collections.abc import Mapping
from pathlib import Path
from typing import Any

from .oracle_assets import TaskOracleSnapshot, capture_task_oracles, validate_oracle_declaration
from .process_control import run_checked, run_managed
from .task_assets import TaskAssetCache, capture_task_dependency_binding


def normalized_model_identifier(value: str | None, *, flag: str = "--model") -> str:
    model = (value or "").strip()
    if not model:
        raise ValueError(f"{flag} must name a nonblank, versioned model")
    if re.search(r"(?:^|[-/@:])(?:auto|latest)$", model.casefold()):
        raise ValueError(f"{flag} must not use a mutable auto/latest model alias: {model!r}")
    return model


def select_tasks(tasks: list[Any], *, include_expensive: bool) -> tuple[list[dict[str, Any]], list[str]]:
    """Validate task metadata and filter opt-in expensive scenarios."""

    selected: list[dict[str, Any]] = []
    skipped: list[str] = []
    seen: set[str] = set()
    required_strings = ("id", "class", "repo", "prompt", "verify")
    optional_strings = ("ref", "setup")
    for index, raw_task in enumerate(tasks):
        if not isinstance(raw_task, Mapping):
            raise ValueError(f"task {index} must be a mapping")
        task = dict(raw_task)
        for field in required_strings:

View on GitHub (pinned to d540b00184)

Solutions

1. Pass an explicit, versioned model id to --model, e.g. 'claude-sonnet-4-5'.
2. If reading from an env var, fail fast with a clear message when it is unset rather than passing ''.
3. Set a required-model check in your wrapper before invoking the harness.
4. Audit config files for blank/missing model fields.

Example fix

// before
model = os.environ.get('MODEL')  # unset -> None -> raises
normalized_model_identifier(model)

// after
model = os.environ['MODEL']  # KeyError makes the missing var obvious
# or
model = os.environ.get('MODEL') or 'claude-sonnet-4-5'
normalized_model_identifier(model)

def is_nonblank_model(value: str | None) -> bool:
    return isinstance(value, str) and bool(value.strip())

def is_nonblank_model(value: str | None) -> bool:
    return isinstance(value, str) and bool(value.strip())

try:
    model = normalized_model_identifier(value)
except ValueError as e:
    if 'nonblank, versioned model' in str(e):
        # supply an explicit versioned model id
        raise
    raise

When it happens

Trigger: normalized_model_identifier(None) or normalized_model_identifier(' ') — value is None or strips to empty. The caller passed no --model flag or an empty string.

Common situations: The CLI was invoked without --model; the model came from an env var that was unset; a wrapper script forwarded an empty model arg; a config file had a blank model field.

Related errors

• {flag} must not use a mutable auto/latest model alias: {mode
• {label} is unavailable: {path}: {exc}
• {label} must be a real non-symlink directory: {path}
• {label} contains an unsafe path component: {relative}
• {label} is unreadable: {path}: {exc}

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/98f9c28cf3961673. Report an issue: GitHub.