{"record":{"id":"314ed2f9f27037a4","repo":"google-gemini/gemini-cli","slug":"triage-execution-failed-raw-output","errorCode":null,"errorMessage":"Triage execution failed: {raw_output}","messagePattern":"Triage execution failed: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"tools/caretaker-agent/evals/triage/runner.py","lineNumber":63,"sourceCode":"def eval_issue(golden_issue: Dict[str, Any], worker_id: int) -> Dict[str, Any]:\n    \"\"\"Evaluates a single issue under ThreadPoolExecutor using an isolated Git Worktree.\"\"\"\n    issue_num = golden_issue.get(\"issue_number\")\n    title = golden_issue.get(\"issue_title\")\n    target_version = golden_issue.get(\"target_version\", \"main\")\n    actual_version = target_version\n\n    payload = prep_payload(golden_issue)\n\n    try:\n        worktree_dir, actual_version = add_worktree(worker_id, target_version)\n        print(f\"[TEST START] Issue #{issue_num} (Version: {actual_version[:10]})\")\n\n        start_time = time.time()\n        success, raw_output = process_issue_triage(payload, target_cwd=worktree_dir)\n        execution_time_seconds = round(time.time() - start_time, 2)\n        \n        if not success:\n            raise RuntimeError(f\"Triage execution failed: {raw_output}\")\n            \n        try:\n            result = json.loads(raw_output)\n        except Exception:\n            cleaned_output = raw_output.replace(\"\\\\'\", \"'\")\n            result = json.loads(cleaned_output)\n\n        metadata = result.get(\"triage_metadata\", {})\n        predicted_spec = result.get(\"workable_spec\", {})\n\n        cat_eval = evaluate_categorization(metadata, golden_issue)\n\n        golden_spec = golden_issue.get(\"expected_workable_spec\", {})\n        spec_grade = {}\n        if golden_issue.get(\"expected_quality\") == \"OK\" and golden_spec:\n            spec_grade = judge_workable_spec(predicted_spec, golden_spec)\n\n        record = {","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/google-gemini/gemini-cli/blob/5024443c7217464a66e98f80d73172a26440bd8f/tools/caretaker-agent/evals/triage/runner.py#L45-L81","documentation":"This RuntimeError is raised in eval_issue() when process_issue_triage() returns success=False, meaning the triage orchestrator executed but reported a failure (non-zero exit, timeout, crash, or invalid output). The raw_output string in the message contains the stderr/stdout captured from the worker subprocess, which is the primary diagnostic. It is an execution-failure signal, not a schema error.","triggerScenarios":"process_issue_triage(payload, target_cwd=worktree_dir) runs the triage worker against an isolated git worktree; it returns (False, raw_output) when the worker subprocess exits non-zero, times out, or produces an exception traceback. The raw_output is then wrapped in this RuntimeError and bubbled up to the ProcessPoolExecutor.","commonSituations":"The triage worker hits an unhandled Python exception (e.g., missing GEMINI_API_KEY, network error to the model API). The target git worktree checkout failed or the target_version SHA is invalid. The model API returned a non-JSON or error response that the worker could not parse. Resource exhaustion (OOM) or timeout killing the subprocess. A bug in triage_orchestrator itself.","solutions":["Read the raw_output embedded in the error message — it contains the worker's stderr/traceback which identifies the root cause.","If raw_output indicates an auth/API error, verify GEMINI_API_KEY is set and valid for the worker process env.","If it indicates a git/checkout error, verify the target_version SHA exists in the repo and that worktree creation (add_worktree) succeeded.","Re-run the single failing issue in isolation (python -m evals.triage.runner --issues <n> --concurrency 1) to reproduce and debug.","Check LOCAL_LOG_DIR issue JSON files for the captured error record written by the except block in eval_issue."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"from triage_orchestrator import process_issue_triage\n\ndef dry_run_triage(payload: dict, worktree_dir: str) -> bool:\n    success, raw = process_issue_triage(payload, target_cwd=worktree_dir)\n    return success\n\n# For a single issue, run with concurrency 1 to isolate failures:\n# python -m evals.triage.runner --issues <N> --concurrency 1","typeGuard":null,"tryCatchPattern":"try:\n    success, raw_output = process_issue_triage(payload, target_cwd=worktree_dir)\n    if not success:\n        raise RuntimeError(f'Triage execution failed: {raw_output}')\nexcept RuntimeError as e:\n    print(f'[EVAL] Issue {issue_num} failed: {e}')\n    # raw_output is captured in the exception; inspect it for the worker traceback\n    record = {'issue_number': issue_num, 'error': str(e)}\n    # persist record and continue to next issue","preventionTips":["Set LOCAL_LOG_DIR so the eval runner writes per-issue error records for post-mortem.","Verify GEMINI_API_KEY and all worker env vars are present before starting the suite.","Run a single failing issue with --concurrency 1 to reproduce and debug.","Monitor worker stderr in the captured raw_output for tracebacks and act on them."],"tags":["execution","triage-worker","subprocess","evals","runtime","python"],"backgroundTag":null,"analyzedSha":"5024443c7217464a66e98f80d73172a26440bd8f","analyzedAt":"2026-08-12T06:01:53.711Z","schemaVersion":2},"datasetVersion":"2026-08-12T13:17:24.610Z"}