zed-industries/zed · error · ValueError

zed benchmarks require build_id

Error message

zed benchmarks require build_id

What it means

Raised by harness_command.build_harness_command() when run_request has no build_id. Every zed benchmark run executes a compiled eval-cli binary mounted from the Modal volume at /data/builds/{build_id}/eval-cli, so a build id is not optional. The normal launch path fills it in from prepare_shared_build(args) inside build_benchmark_run_request.

Source

Thrown at crates/eval_cli/zed_eval/harness_command.py:69

    harness = benchmark.get("harness")
    if harness not in (benchmarks.HARNESS_HARBOR, benchmarks.HARNESS_PIER):
        raise ValueError(f"unsupported harness: {harness}")
    return harness


def eval_cli_timeout(run_request: dict[str, Any], benchmark: dict[str, Any]) -> int:
    return int(
        run_request.get("eval_cli_timeout")
        or benchmark.get("default_timeout_secs")
        or config.DEFAULT_SANDBOX_TIMEOUT_SECS
    )


def build_harness_command(run_request: dict[str, Any], jobs_dir: str) -> list[str]:
    benchmark = _benchmark_block(run_request)
    build_id = run_request.get("build_id")
    if not build_id:
        raise ValueError("zed benchmarks require build_id")
    volume_name = run_request["volume_name"]
    api_secret_name = run_request["api_secret_name"]
    run_id = run_request["run_id"]
    agent_model = run_request.get("agent_model") or config.DEFAULT_MODEL
    n_concurrent = int(run_request.get("n_concurrent") or config.DEFAULT_N_CONCURRENT)
    sandbox_timeout_secs = int(
        run_request.get("sandbox_timeout_secs") or config.DEFAULT_SANDBOX_TIMEOUT_SECS
    )
    sandbox_idle_timeout_secs = int(
        run_request.get("sandbox_idle_timeout_secs")
        or config.DEFAULT_SANDBOX_IDLE_TIMEOUT_SECS
    )
    override_cpus = run_request.get("override_cpus") or config.DEFAULT_OVERRIDE_CPUS
    override_memory_mb = (
        run_request.get("override_memory_mb") or config.DEFAULT_OVERRIDE_MEMORY_MB
    )

    secret_names = [api_secret_name]

View on GitHub (pinned to bc538def45)

Solutions

1. Attach a build: run zed-eval build first and pass the resulting build_id, or go through zed-eval run which prepares the shared build automatically.
2. When reusing a stored request, keep the original build_id field intact.
3. Validate the request before spawning the controller: assert run_request.get("build_id").

Example fix

# before
run_request = {"run_id": "r1", "benchmark": {...}}  # no build_id
build_harness_command(run_request, "/tmp/jobs")  # -> ValueError

# after
build_id, build_request = prepare_shared_build(args)
run_request["build_id"] = build_id
build_harness_command(run_request, "/tmp/jobs")

if not run_request.get("build_id"):
    raise SystemExit(
        "run request needs build_id; run `zed-eval build` or launch via `zed-eval run`"
    )

def has_build_id(run_request: dict) -> bool:
    return bool(run_request.get("build_id"))

try:
    command = harness_command.build_harness_command(run_request, jobs_dir)
except ValueError as error:
    raise SystemExit(f"cannot build harness command: {error}")

When it happens

Trigger: Invoking the controller (run_controller) or build_harness_command with a hand-built run request that omits build_id; copying a stored request.json and stripping build fields; scripting a run without going through prepare_benchmark_runs, which is the step that attaches the shared build.

Common situations: Tooling that tries to relaunch a stored request while forcing a rebuild, dropping build_id in the process; confusion with rejudge requests (which legitimately carry no build_id) fed into the wrong entry point; assuming the controller will pick a default build.

Related errors

• run request is missing a 'benchmark' block
• eval-cli binary not found at {binary}. Build it with: cargo
• unknown benchmark '{benchmark_id}' (valid: {valid})
• unknown benchmark '{selector}' (valid: {valid})
• build ids may only contain lowercase letters, numbers, '.',

AI-assisted analysis of zed-industries/zed@bc538def45 (2026-08-16). Data as JSON: /api/errors/ab3ccd3412b14a18. Report an issue: GitHub.