{"record":{"id":"ab3ccd3412b14a18","repo":"zed-industries/zed","slug":"zed-benchmarks-require-build-id","errorCode":null,"errorMessage":"zed benchmarks require build_id","messagePattern":"zed benchmarks require build_id","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"crates/eval_cli/zed_eval/harness_command.py","lineNumber":69,"sourceCode":"    harness = benchmark.get(\"harness\")\n    if harness not in (benchmarks.HARNESS_HARBOR, benchmarks.HARNESS_PIER):\n        raise ValueError(f\"unsupported harness: {harness}\")\n    return harness\n\n\ndef eval_cli_timeout(run_request: dict[str, Any], benchmark: dict[str, Any]) -> int:\n    return int(\n        run_request.get(\"eval_cli_timeout\")\n        or benchmark.get(\"default_timeout_secs\")\n        or config.DEFAULT_SANDBOX_TIMEOUT_SECS\n    )\n\n\ndef build_harness_command(run_request: dict[str, Any], jobs_dir: str) -> list[str]:\n    benchmark = _benchmark_block(run_request)\n    build_id = run_request.get(\"build_id\")\n    if not build_id:\n        raise ValueError(\"zed benchmarks require build_id\")\n    volume_name = run_request[\"volume_name\"]\n    api_secret_name = run_request[\"api_secret_name\"]\n    run_id = run_request[\"run_id\"]\n    agent_model = run_request.get(\"agent_model\") or config.DEFAULT_MODEL\n    n_concurrent = int(run_request.get(\"n_concurrent\") or config.DEFAULT_N_CONCURRENT)\n    sandbox_timeout_secs = int(\n        run_request.get(\"sandbox_timeout_secs\") or config.DEFAULT_SANDBOX_TIMEOUT_SECS\n    )\n    sandbox_idle_timeout_secs = int(\n        run_request.get(\"sandbox_idle_timeout_secs\")\n        or config.DEFAULT_SANDBOX_IDLE_TIMEOUT_SECS\n    )\n    override_cpus = run_request.get(\"override_cpus\") or config.DEFAULT_OVERRIDE_CPUS\n    override_memory_mb = (\n        run_request.get(\"override_memory_mb\") or config.DEFAULT_OVERRIDE_MEMORY_MB\n    )\n\n    secret_names = [api_secret_name]","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/zed-industries/zed/blob/bc538def4545534201bbfcac4e95ac34ea6501b6/crates/eval_cli/zed_eval/harness_command.py#L51-L87","documentation":"Raised by harness_command.build_harness_command() when run_request has no build_id. Every zed benchmark run executes a compiled eval-cli binary mounted from the Modal volume at /data/builds/{build_id}/eval-cli, so a build id is not optional. The normal launch path fills it in from prepare_shared_build(args) inside build_benchmark_run_request.","triggerScenarios":"Invoking the controller (run_controller) or build_harness_command with a hand-built run request that omits build_id; copying a stored request.json and stripping build fields; scripting a run without going through prepare_benchmark_runs, which is the step that attaches the shared build.","commonSituations":"Tooling that tries to relaunch a stored request while forcing a rebuild, dropping build_id in the process; confusion with rejudge requests (which legitimately carry no build_id) fed into the wrong entry point; assuming the controller will pick a default build.","solutions":["Attach a build: run zed-eval build first and pass the resulting build_id, or go through zed-eval run which prepares the shared build automatically.","When reusing a stored request, keep the original build_id field intact.","Validate the request before spawning the controller: assert run_request.get(\"build_id\")."],"exampleFix":"# before\nrun_request = {\"run_id\": \"r1\", \"benchmark\": {...}}  # no build_id\nbuild_harness_command(run_request, \"/tmp/jobs\")  # -> ValueError\n\n# after\nbuild_id, build_request = prepare_shared_build(args)\nrun_request[\"build_id\"] = build_id\nbuild_harness_command(run_request, \"/tmp/jobs\")","handlingStrategy":"validation","validationCode":"if not run_request.get(\"build_id\"):\n    raise SystemExit(\n        \"run request needs build_id; run `zed-eval build` or launch via `zed-eval run`\"\n    )","typeGuard":"def has_build_id(run_request: dict) -> bool:\n    return bool(run_request.get(\"build_id\"))","tryCatchPattern":"try:\n    command = harness_command.build_harness_command(run_request, jobs_dir)\nexcept ValueError as error:\n    raise SystemExit(f\"cannot build harness command: {error}\")","preventionTips":["Prefer the standard launch path (zed-eval run) which attaches the shared build automatically.","When replaying stored requests, keep the original build_id untouched.","Validate required keys (build_id, volume_name, api_secret_name, run_id) before spawning controllers."],"tags":["validation","build","run-request"],"backgroundTag":null,"analyzedSha":"bc538def4545534201bbfcac4e95ac34ea6501b6","analyzedAt":"2026-08-16T07:30:46.435Z","schemaVersion":2},"datasetVersion":"2026-08-16T08:17:34.114Z"}