abhigyanpatwari/GitNexus · error · ValueError

task {index} must be a mapping

Error message

task {index} must be a mapping

What it means

Thrown by select_tasks while iterating the raw task list from the benchmark config. Each entry must be a Mapping (dict-like) so the harness can read its required string fields (id, class, repo, prompt, verify); a non-mapping entry (a string, number, list, None) cannot satisfy that contract, so it is rejected with the offending index for easy location.

Source

Thrown at eval/workflow_bench/runner_tasks.py:35

    model = (value or "").strip()
    if not model:
        raise ValueError(f"{flag} must name a nonblank, versioned model")
    if re.search(r"(?:^|[-/@:])(?:auto|latest)$", model.casefold()):
        raise ValueError(f"{flag} must not use a mutable auto/latest model alias: {model!r}")
    return model


def select_tasks(tasks: list[Any], *, include_expensive: bool) -> tuple[list[dict[str, Any]], list[str]]:
    """Validate task metadata and filter opt-in expensive scenarios."""

    selected: list[dict[str, Any]] = []
    skipped: list[str] = []
    seen: set[str] = set()
    required_strings = ("id", "class", "repo", "prompt", "verify")
    optional_strings = ("ref", "setup")
    for index, raw_task in enumerate(tasks):
        if not isinstance(raw_task, Mapping):
            raise ValueError(f"task {index} must be a mapping")
        task = dict(raw_task)
        for field in required_strings:
            if not isinstance(task.get(field), str) or not task[field].strip():
                raise ValueError(f"task {index} requires a nonblank string {field}")
        for field in optional_strings:
            if field in task and not isinstance(task[field], str):
                raise ValueError(f"task {task['id']} field {field} must be a string")
        task_id = task["id"]
        if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}", task_id):
            raise ValueError(f"task id must be a simple artifact-safe slug: {task_id!r}")
        if task_id in seen:
            raise ValueError(f"duplicate task id: {task_id}")
        seen.add(task_id)
        expensive = task.get("expensive", False)
        if not isinstance(expensive, bool):
            raise ValueError(f"task {task_id} expensive metadata must be boolean")
        copies = task.get("sandbox_copy", [])
        if not isinstance(copies, list) or not all(isinstance(path, str) and path for path in copies):

View on GitHub (pinned to d540b00184)

Solutions

1. Make every item in the tasks list a mapping with the required keys: {id, class, repo, prompt, verify}.
2. Validate the parsed config with a schema (jsonschema/pydantic) before passing to select_tasks.
3. Check the reported index in the error and fix that specific list entry.
4. Lint the tasks file to ensure each - item is an object.

Example fix

# before (tasks.yaml)
tasks:
  - my-task-id          # scalar, not a mapping

# after
tasks:
  - id: my-task-id
    class: implementation
    repo: https://github.com/org/repo
    prompt: "Implement X"
    verify: "npm test"

from collections.abc import Mapping

def tasks_are_mappings(tasks: list) -> bool:
    return all(isinstance(t, Mapping) for t in tasks)

from collections.abc import Mapping

def is_task_mapping(value) -> bool:
    return isinstance(value, Mapping)

from collections.abc import Mapping
try:
    selected, skipped = select_tasks(tasks, include_expensive=False)
except ValueError as e:
    if 'must be a mapping' in str(e):
        # the reported index points at a non-dict entry; fix the tasks file
        raise
    raise

When it happens

Trigger: In the tasks loop, isinstance(raw_task, Mapping) is False for the entry at position `index`. The YAML/JSON list contains a scalar or null where a task object was expected.

Common situations: A YAML list item was written as a string instead of a mapping (- task_id instead of - id: task_id); a null entry from a template placeholder; a JSON array mixing objects and scalars; a copy-paste left a bare comment-stripped line.

Related errors

• {label} is unavailable: {path}: {exc}
• {label} must be a real non-symlink directory: {path}
• {label} contains an unsafe path component: {relative}
• {label} is unreadable: {path}: {exc}
• {label} must be a regular non-symlink file: {path}

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/27ca81c882a3d7a8. Report an issue: GitHub.