abhigyanpatwari/GitNexus · error · ValueError

task file must contain a tasks list

Error message

task file must contain a tasks list

What it means

Raised in main() right after YAML-loading the --tasks file: the parsed document must be a Mapping with a top-level 'tasks' key that is a list. The harness selects runs from that list, so any other shape (a bare list, a scalar, a mapping without 'tasks', or 'tasks' that is not a list) is rejected before any task is selected. It is caught alongside YAMLError and routed through parser.error (exit 2).

Source

Thrown at eval/workflow_bench/runner.py:920

    parser.add_argument("--promotion-max-task-regression", type=float, default=20.0)
    parser.add_argument("--task-bindings-json", default=None, help=argparse.SUPPRESS)
    parser.add_argument("--promotion-target-bases-json", default=None, help=argparse.SUPPRESS)
    return parser


def main() -> None:
    parser = build_parser()
    args = parser.parse_args()
    try:
        args.model = normalized_model_identifier(args.model)
        args.proposer_model = (
            normalized_model_identifier(args.proposer_model, flag="--proposer-model")
            if args.proposer_model is not None
            else None
        )
        task_document = yaml.safe_load(args.tasks.read_text())
        if not isinstance(task_document, Mapping) or not isinstance(task_document.get("tasks"), list):
            raise ValueError("task file must contain a tasks list")
        tasks, skipped_expensive = select_tasks(
            task_document["tasks"],
            include_expensive=args.include_expensive,
        )
        oracle_snapshots = capture_task_oracles(tasks)
        expected_task_bindings = json.loads(args.task_bindings_json) if args.task_bindings_json else None
        if expected_task_bindings is not None and not isinstance(expected_task_bindings, list):
            raise ValueError("--task-bindings-json must contain a list")
        supplied_promotion_target_bases = (
            json.loads(args.promotion_target_bases_json) if args.promotion_target_bases_json else {}
        )
        if not isinstance(supplied_promotion_target_bases, dict) or not all(
            isinstance(path, str) and isinstance(digest, str)
            for path, digest in supplied_promotion_target_bases.items()
        ):
            raise ValueError("--promotion-target-bases-json must contain a string mapping")
        ce_plugin_config = validate_ce_plugin_inputs(
            args.arms,

View on GitHub (pinned to d540b00184)

Solutions

1. Open the --tasks file and confirm the top level is a mapping with a list-valued 'tasks' key.
2. Wrap a bare list: prefix the document with `tasks:` and indent the entries.
3. Validate locally before running: `python3 -c "import yaml,sys; d=yaml.safe_load(open(sys.argv[1])); assert isinstance(d,dict) and isinstance(d['tasks'],list)" your_tasks.yaml`.
4. If you passed the wrong file, point --tasks at the actual scenarios file.

Example fix

# before — bare list at top level
- name: task-a
  prompt: ...
# after — object with a tasks list
tasks:
  - name: task-a
    prompt: ...

import yaml

doc = yaml.safe_load(open('tasks.yaml'))
assert isinstance(doc, dict) and isinstance(doc.get('tasks'), list), (
    'task file must be a mapping with a list-valued "tasks" key'
)

from collections.abc import Mapping

def is_task_document(doc: object) -> bool:
    return isinstance(doc, Mapping) and isinstance(doc.get('tasks'), list)

When it happens

Trigger: `yaml.safe_load(args.tasks.read_text())` succeeded but `not isinstance(task_document, Mapping) or not isinstance(task_document.get('tasks'), list)` — e.g. the file is a YAML list of tasks instead of an object with a tasks: key, or tasks: is a mapping/null.

Common situations: Author wrote a bare YAML list `- name: ...` instead of `tasks: [- name: ...]`; mis-indented so tasks became a mapping; wrong file passed to --tasks (a scenario YAML that uses a different schema); copy-paste from docs that showed a fragment without the wrapper.

Related errors

• --task-bindings-json must contain a list
• --promotion-target-bases-json must contain a string mapping
• Malformed --shard arg '${malformed}' — expected --shard=<ind
• ${flag} must be a positive integer
• ${flag} is too large

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/eabbde0a01a7da0c. Report an issue: GitHub.