google-gemini/gemini-cli · error · RuntimeError

Triage execution failed: {raw_output}

Error message

Triage execution failed: {raw_output}

What it means

This RuntimeError is raised in eval_issue() when process_issue_triage() returns success=False, meaning the triage orchestrator executed but reported a failure (non-zero exit, timeout, crash, or invalid output). The raw_output string in the message contains the stderr/stdout captured from the worker subprocess, which is the primary diagnostic. It is an execution-failure signal, not a schema error.

Source

Thrown at tools/caretaker-agent/evals/triage/runner.py:63

def eval_issue(golden_issue: Dict[str, Any], worker_id: int) -> Dict[str, Any]:
    """Evaluates a single issue under ThreadPoolExecutor using an isolated Git Worktree."""
    issue_num = golden_issue.get("issue_number")
    title = golden_issue.get("issue_title")
    target_version = golden_issue.get("target_version", "main")
    actual_version = target_version

    payload = prep_payload(golden_issue)

    try:
        worktree_dir, actual_version = add_worktree(worker_id, target_version)
        print(f"[TEST START] Issue #{issue_num} (Version: {actual_version[:10]})")

        start_time = time.time()
        success, raw_output = process_issue_triage(payload, target_cwd=worktree_dir)
        execution_time_seconds = round(time.time() - start_time, 2)
        
        if not success:
            raise RuntimeError(f"Triage execution failed: {raw_output}")
            
        try:
            result = json.loads(raw_output)
        except Exception:
            cleaned_output = raw_output.replace("\\'", "'")
            result = json.loads(cleaned_output)

        metadata = result.get("triage_metadata", {})
        predicted_spec = result.get("workable_spec", {})

        cat_eval = evaluate_categorization(metadata, golden_issue)

        golden_spec = golden_issue.get("expected_workable_spec", {})
        spec_grade = {}
        if golden_issue.get("expected_quality") == "OK" and golden_spec:
            spec_grade = judge_workable_spec(predicted_spec, golden_spec)

        record = {

View on GitHub (pinned to 5024443c72)

Solutions

1. Read the raw_output embedded in the error message — it contains the worker's stderr/traceback which identifies the root cause.
2. If raw_output indicates an auth/API error, verify GEMINI_API_KEY is set and valid for the worker process env.
3. If it indicates a git/checkout error, verify the target_version SHA exists in the repo and that worktree creation (add_worktree) succeeded.
4. Re-run the single failing issue in isolation (python -m evals.triage.runner --issues <n> --concurrency 1) to reproduce and debug.
5. Check LOCAL_LOG_DIR issue JSON files for the captured error record written by the except block in eval_issue.

from triage_orchestrator import process_issue_triage

def dry_run_triage(payload: dict, worktree_dir: str) -> bool:
    success, raw = process_issue_triage(payload, target_cwd=worktree_dir)
    return success

# For a single issue, run with concurrency 1 to isolate failures:
# python -m evals.triage.runner --issues <N> --concurrency 1

try:
    success, raw_output = process_issue_triage(payload, target_cwd=worktree_dir)
    if not success:
        raise RuntimeError(f'Triage execution failed: {raw_output}')
except RuntimeError as e:
    print(f'[EVAL] Issue {issue_num} failed: {e}')
    # raw_output is captured in the exception; inspect it for the worker traceback
    record = {'issue_number': issue_num, 'error': str(e)}
    # persist record and continue to next issue

When it happens

Trigger: process_issue_triage(payload, target_cwd=worktree_dir) runs the triage worker against an isolated git worktree; it returns (False, raw_output) when the worker subprocess exits non-zero, times out, or produces an exception traceback. The raw_output is then wrapped in this RuntimeError and bubbled up to the ProcessPoolExecutor.

Common situations: The triage worker hits an unhandled Python exception (e.g., missing GEMINI_API_KEY, network error to the model API). The target git worktree checkout failed or the target_version SHA is invalid. The model API returned a non-JSON or error response that the worker could not parse. Resource exhaustion (OOM) or timeout killing the subprocess. A bug in triage_orchestrator itself.

Related errors

• Command '{cmd_str}' failed with exit code {returncode}
• Invalid or missing 'issue_id' format: {issue_id}
• EVAL_CONFIG must be a JSON object, got {type(cfg).__name__}
• Missing required environment variable '{name}'. Please ensur
• Expected JSON object from LLM, but got {type(data).__name__}

AI-assisted analysis of google-gemini/gemini-cli@5024443c72 (2026-08-12). Data as JSON: /api/errors/314ed2f9f27037a4. Report an issue: GitHub.