zed-industries/zed · warning · ValueError

provide a run_id or --job-dir

Error message

provide a run_id or --job-dir

What it means

locate_job_dir resolves which fetched Harbor/Pier job directory a `zed-eval report` invocation should read: it first honors an explicit --job-dir, then falls back to <jobs_dir>/<run_id>. If neither args.job_dir nor args.run_id is present, it raises this ValueError immediately, before anything is fetched or resolved.

Solutions

1. Pass the run id: zed-eval report <run_id>
2. Or point at an already-fetched job: zed-eval report --job-dir ~/.cache/harbor/jobs/<run_id>
3. In scripts, guard against empty $RUN_ID before invoking the CLI
4. Check argparse wiring if you maintain a wrapper, so at least one source is required

Example fix

# before
zed-eval report   # ValueError: provide a run_id or --job-dir

# after
zed-eval report 2026-08-20T10-00-00-abc123
# or
zed-eval report --job-dir ~/.cache/harbor/jobs/2026-08-20T10-00-00-abc123

if not (getattr(args, "run_id", None) or getattr(args, "job_dir", None)):
    raise SystemExit("report needs a run id or --job-dir")

try:
    job_dir = locate_job_dir(args)
except ValueError as err:
    raise SystemExit(f"usage: {err}")

When it happens

Trigger: Running `zed-eval report` with no positional run id and no --job-dir; a wrapper script that drops empty arguments (empty string is falsy, same result); an argparse setup where both options are optional and the user supplied neither.

Common situations: Copy-pasting a report command from docs that still contains a placeholder run id; scripting report over many runs and passing an unset shell variable; interactive use on a machine with no prior runs in the index.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• no run id given and the local run index is empty. Launch a…
• no fetched job at . Run `zed-eval fetch --experiment-name `…
• no split specifications provided
• usage: ep split [input.jsonl] train.jsonl=80%…
• Cargo.toml not found in current directory. Please run this…

AI-assisted analysis of zed-industries/zed@f4178619ac (2026-08-20). Data as JSON: /api/errors/c5f52cfa8c4c4c67. Report an issue: GitHub.

Appendix: source

Thrown at crates/eval_cli/zed_eval/report.py:427

        for entry in report["errored_trials"][:20]:
            lines.append(f"  - {entry['task_name']}: {entry['reason']}")
        if len(report["errored_trials"]) > 20:
            lines.append(f"  ... and {len(report['errored_trials']) - 20} more")
        lines.append("")

    return "\n".join(lines)


def locate_job_dir(args: Any) -> Path:
    """Resolve the fetched Harbor/Pier job directory for a `report` invocation,
    fetching it first when asked."""
    job_dir = getattr(args, "job_dir", None)
    if job_dir:
        return Path(job_dir)

    run_id = getattr(args, "run_id", None)
    if not run_id:
        raise ValueError("provide a run_id or --job-dir")
    jobs_dir = Path(getattr(args, "jobs_dir", Path.home() / ".cache/harbor/jobs"))
    candidate = jobs_dir / run_id

    if getattr(args, "fetch", False) or not candidate.exists():
        # Reuse the existing fetch command to download + extract the archive.
        from .volume import command_fetch

        command_fetch(args)
    if not candidate.exists():
        raise ValueError(
            f"no fetched job at {candidate}. Run `zed-eval fetch {run_id} "
            f"--experiment-name <benchmark>` first, or pass --job-dir."
        )
    return candidate


def _timeout_is_failure(args: Any) -> bool:
    """Agent timeouts count as failures on test-scored benchmarks. Derived from

View on GitHub (pinned to f4178619ac)