mvanhorn/last30days-skill · error · RuntimeError

failed for ' ' with exit \n

Error message

{repo_dir.name} failed for '{topic}' with exit {result.returncode}\n{result.stderr.strip()}

What it means

run_repo() in evaluate_search_quality.py executes the last30days engine as a subprocess in a git worktree and raises RuntimeError when the child exits non-zero. The message includes the worktree name, the topic, the exit code, and the child's captured stderr, so the real cause is always in the appended stderr text.

Solutions

Read the stderr suffix of the message first — it names the engine's own error; fix that (key, flag, or code issue) rather than the harness.
Verify the env dict built for the child includes every credential the engine sources need for the topic's query type.
Reproduce manually: cd into the temp worktree path and run the same cmd printed from the harness.
If the revision is simply broken, pin the evaluator to a known-good rev or fix the commit under test.

Defensive patterns

Strategy: try-catch

Try / catch

try:
    payload = run_repo(repo_dir, topic, mock=mock)
except RuntimeError as exc:
    # the trailing stderr names the engine's real failure
    print(f'engine run failed: {exc}', file=sys.stderr)
    raise SystemExit(1)

Prevention

Always read the stderr suffix — it contains the engine's own error message.
Propagate every required credential env var into the child env the harness builds.
Smoke-test each worktree revision with --mock before live benchmark runs.

When it happens

Trigger: subprocess.run(cmd, cwd=repo_dir, capture_output=True, check=False) returning returncode != 0 — e.g. the engine hit a missing API key, a SystemExit(2) from flag validation (bad --search source, bad plan JSON), a network failure inside the engine, or an unhandled exception in the evaluated revision.

Common situations: Evaluating an old worktree revision that lacks a newer flag the harness passes; env vars (REDDIT credentials etc.) not propagated into the child env; the engine's Python version guard (error [5]) exiting 1 because the child resolves a different interpreter; worktree checkout of a broken commit.

Related errors

AI-assisted analysis of mvanhorn/last30days-skill@c7460f6114 (2026-08-15). Data as JSON: /api/errors/da0f56ad01bee246. Report an issue: GitHub.

Appendix: source

Thrown at skills/last30days/scripts/evaluate_search_quality.py:357

    if not engine.exists() or "--json-profile" in engine.read_text(encoding="utf-8"):
        cmd.append("--json-profile=raw")
    if search:
        cmd.extend(["--search", search])
    if quick:
        cmd.append("--quick")
    if mock:
        cmd.append("--mock")
    result = subprocess.run(
        cmd,
        cwd=repo_dir,
        env=env,
        capture_output=True,
        text=True,
        timeout=timeout_seconds,
        check=False,
    )
    if result.returncode != 0:
        raise RuntimeError(f"{repo_dir.name} failed for '{topic}' with exit {result.returncode}\n{result.stderr.strip()}")
    payload = json.loads(result.stdout)
    # Shape guard: the evaluator compares raw Report fields. If the engine
    # emitted the agent profile anyway (flag detection missed a future
    # spelling), fail loudly instead of scoring empty ranked_candidates.
    if "schema_version" in payload and "ranked_candidates" not in payload:
        raise RuntimeError(
            f"{repo_dir.name} emitted the agent JSON profile; the evaluator "
            "requires the raw Report (--json-profile=raw)."
        )
    return payload


def create_worktree(rev: str) -> Path:
    worktree_dir = Path(tempfile.mkdtemp(prefix="last30days-eval-"))
    subprocess.run(
        ["git", "worktree", "add", "--detach", str(worktree_dir), rev],
        cwd=REPO_ROOT,
        check=True,

View on GitHub (pinned to c7460f6114)