mvanhorn/last30days-skill · error · RuntimeError

{repo_dir.name} failed for '{topic}' with exit {result.retur

Error message

{repo_dir.name} failed for '{topic}' with exit {result.returncode}\n{result.stderr.strip()}

What it means

run_repo() in evaluate_search_quality.py executes the last30days engine as a subprocess in a git worktree and raises RuntimeError when the child exits non-zero. The message includes the worktree name, the topic, the exit code, and the child's captured stderr, so the real cause is always in the appended stderr text.

Source

Thrown at skills/last30days/scripts/evaluate_search_quality.py:357

    if not engine.exists() or "--json-profile" in engine.read_text(encoding="utf-8"):
        cmd.append("--json-profile=raw")
    if search:
        cmd.extend(["--search", search])
    if quick:
        cmd.append("--quick")
    if mock:
        cmd.append("--mock")
    result = subprocess.run(
        cmd,
        cwd=repo_dir,
        env=env,
        capture_output=True,
        text=True,
        timeout=timeout_seconds,
        check=False,
    )
    if result.returncode != 0:
        raise RuntimeError(f"{repo_dir.name} failed for '{topic}' with exit {result.returncode}\n{result.stderr.strip()}")
    payload = json.loads(result.stdout)
    # Shape guard: the evaluator compares raw Report fields. If the engine
    # emitted the agent profile anyway (flag detection missed a future
    # spelling), fail loudly instead of scoring empty ranked_candidates.
    if "schema_version" in payload and "ranked_candidates" not in payload:
        raise RuntimeError(
            f"{repo_dir.name} emitted the agent JSON profile; the evaluator "
            "requires the raw Report (--json-profile=raw)."
        )
    return payload


def create_worktree(rev: str) -> Path:
    worktree_dir = Path(tempfile.mkdtemp(prefix="last30days-eval-"))
    subprocess.run(
        ["git", "worktree", "add", "--detach", str(worktree_dir), rev],
        cwd=REPO_ROOT,
        check=True,

View on GitHub (pinned to c7460f6114)

Solutions

1. Read the stderr suffix of the message first — it names the engine's own error; fix that (key, flag, or code issue) rather than the harness.
2. Verify the env dict built for the child includes every credential the engine sources need for the topic's query type.
3. Reproduce manually: cd into the temp worktree path and run the same cmd printed from the harness.
4. If the revision is simply broken, pin the evaluator to a known-good rev or fix the commit under test.

try:
    payload = run_repo(repo_dir, topic, mock=mock)
except RuntimeError as exc:
    # the trailing stderr names the engine's real failure
    print(f'engine run failed: {exc}', file=sys.stderr)
    raise SystemExit(1)

When it happens

Trigger: subprocess.run(cmd, cwd=repo_dir, capture_output=True, check=False) returning returncode != 0 — e.g. the engine hit a missing API key, a SystemExit(2) from flag validation (bad --search source, bad plan JSON), a network failure inside the engine, or an unhandled exception in the evaluated revision.

Common situations: Evaluating an old worktree revision that lacks a newer flag the harness passes; env vars (REDDIT credentials etc.) not propagated into the child env; the engine's Python version guard (error [5]) exiting 1 because the child resolves a different interpreter; worktree checkout of a broken commit.

Related errors

• {repo_dir.name} emitted the agent JSON profile; the evaluato
• Gemini response did not contain text.

AI-assisted analysis of mvanhorn/last30days-skill@c7460f6114 (2026-08-15). Data as JSON: /api/errors/da0f56ad01bee246. Report an issue: GitHub.