{"record":{"id":"f28eef4a245c85d9","repo":"abhigyanpatwari/GitNexus","slug":"task-task-id-oracle-requires-exactly-command-and","errorCode":null,"errorMessage":"task {task_id} oracle requires exactly command and files","messagePattern":"task (.+?) oracle requires exactly command and files","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"eval/workflow_bench/oracle_assets.py","lineNumber":89,"sourceCode":"    if not isinstance(value, str) or not value:\n        raise ValueError(f\"{label} must be a nonblank relative path\")\n    if \"\\\\\" in value or \"\\x00\" in value or len(value.encode()) > MAX_ORACLE_PATH_BYTES:\n        raise ValueError(f\"{label} is not a bounded portable path: {value!r}\")\n    relative = PurePosixPath(value)\n    if relative.is_absolute() or not relative.parts or any(part in {\"\", \".\", \"..\"} for part in relative.parts):\n        raise ValueError(f\"{label} must not be absolute or traverse parents: {value!r}\")\n    if relative.parts[0] == \".git\":\n        raise ValueError(f\"{label} cannot target git metadata: {value!r}\")\n    return relative\n\n\ndef validate_oracle_declaration(task: dict[str, Any]) -> None:\n    \"\"\"Validate the declarative shape without reading harness-owned files.\"\"\"\n\n    task_id = str(task.get(\"id\", \"<unknown>\"))\n    oracle = task.get(\"oracle\")\n    if not isinstance(oracle, dict) or set(oracle) != {\"command\", \"files\"}:\n        raise ValueError(f\"task {task_id} oracle requires exactly command and files\")\n    command = oracle.get(\"command\")\n    if (\n        not isinstance(command, str)\n        or not command.strip()\n        or len(command.encode()) > MAX_ORACLE_COMMAND_BYTES\n        or \"\\x00\" in command\n    ):\n        raise ValueError(f\"task {task_id} oracle command must be nonblank and bounded\")\n    files = oracle.get(\"files\")\n    if not isinstance(files, list) or not files or len(files) > MAX_ORACLE_FILES:\n        raise ValueError(f\"task {task_id} oracle files must contain 1..{MAX_ORACLE_FILES} entries\")\n    sources: set[str] = set()\n    targets: set[str] = set()\n    for index, declaration in enumerate(files):\n        if not isinstance(declaration, dict) or set(declaration) != {\"source\", \"target\"}:\n            raise ValueError(f\"task {task_id} oracle file {index} requires exactly source and target\")\n        source = _bounded_relative_path(declaration.get(\"source\"), label=f\"task {task_id} oracle source\")\n        target = _bounded_relative_path(declaration.get(\"target\"), label=f\"task {task_id} oracle target\")","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/oracle_assets.py#L71-L107","documentation":"Raised by validate_oracle_declaration when a benchmark task's 'oracle' field is not a dict whose key set is exactly {\"command\", \"files\"}. The harness requires a precise declarative shape so the hidden behavioral oracle is unambiguous; any missing key, extra key, wrong type, or absent oracle is rejected before any file is read. The task_id in the message comes from task.get(\"id\", \"<unknown>\") so it identifies which task declaration is malformed.","triggerScenarios":"Calling validate_oracle_declaration(task) or capture_task_oracle(task) where task[\"oracle\"] is None, a string, a list, or a dict containing keys other than exactly 'command' and 'files' (e.g. missing 'files', or an extra 'description'/'timeout' key, or a typo like 'file' instead of 'files').","commonSituations":"Authoring a new benchmark task YAML/JSON and forgetting the 'files' array; adding documentation metadata inside the oracle dict; renaming keys during a schema refactor; loading a task fixture that predates this strict schema; typos like 'file' vs 'files' or 'cmd' vs 'command'.","solutions":["Set task[\"oracle\"] to a dict with exactly two keys: 'command' (str) and 'files' (list) — nothing else.","Run validate_oracle_declaration(task) locally before invoking capture_task_oracle to surface schema errors early.","If you added metadata fields, move them to the top-level task dict, not inside 'oracle'.","Check for typos: it is 'files' (plural) and 'command', not 'file'/'cmd'."],"exampleFix":"// before\ntask = {\"id\": \"t1\", \"oracle\": {\"command\": \"pytest\"}}\n// after\ntask = {\"id\": \"t1\", \"oracle\": {\"command\": \"pytest\", \"files\": [{\"source\": \"expect.txt\", \"target\": \"expect.txt\"}]}}","handlingStrategy":"validation","validationCode":"from eval.workflow_bench.oracle_assets import validate_oracle_declaration\n\ndef safe_validate(task):\n    oracle = task.get(\"oracle\")\n    if not isinstance(oracle, dict) or set(oracle) != {\"command\", \"files\"}:\n        raise ValueError(\"oracle must be a dict with exactly keys 'command' and 'files'\")\n    validate_oracle_declaration(task)","typeGuard":"def is_well_shaped_oracle(task: dict) -> bool:\n    oracle = task.get(\"oracle\")\n    return isinstance(oracle, dict) and set(oracle) == {\"command\", \"files\"}","tryCatchPattern":"try:\n    validate_oracle_declaration(task)\nexcept ValueError as exc:\n    raise SystemExit(f\"Bad oracle declaration for task {task.get('id')}: {exc}\") from exc","preventionTips":["Always run validate_oracle_declaration before capture_task_oracle.","Author oracle declarations via a factory/helper that fixes the key set.","Add a unit test asserting every task fixture passes validate_oracle_declaration."],"tags":["python","benchmark","validation","oracle","schema"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}