{"record":{"id":"1852166bf83816f4","repo":"zylon-ai/private-gpt","slug":"scrape-script-failed-exit-code-result-exit-code","errorCode":null,"errorMessage":"Scrape script failed (exit_code={result.exit_code}): {error}","messagePattern":"Scrape script failed \\(exit_code=(.+?)\\): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/web/scraper/runner.py","lineNumber":137,"sourceCode":"        f\" {shlex.quote(CONFIG_FILENAME)}\",\n        SandboxExecOptions(\n            cwd=base_dir, timeout=total_timeout + _EXEC_TIMEOUT_MARGIN_SECONDS\n        ),\n    )\n\n    if result.exit_code == 124:\n        raise TimeoutError(\n            f\"Timeout ({config.timeout_seconds}s) scraping \"\n            f\"{[r.url for r in config.requests]}\"\n        )\n    if result.failed:\n        error = result.stderr or result.stdout\n        if _BROWSERS_MISSING_PATTERN.search(error):\n            raise RuntimeError(\n                \"Playwright browsers are not installed. \"\n                \"Run `playwright install` and try again.\"\n            )\n        raise RuntimeError(\n            f\"Scrape script failed (exit_code={result.exit_code}): {error}\"\n        )\n\n    results_raw = json.loads(\n        (await session.read_file(posixpath.join(base_dir, RESULTS_FILENAME))).decode(\n            \"utf-8\"\n        )\n    )\n    by_output_path = {entry[\"output_path\"]: entry for entry in results_raw}\n\n    outputs: list[str | Exception] = []\n    for request in config.requests:\n        entry = by_output_path.get(request.output_path)\n        if entry is None:\n            outputs.append(\n                RuntimeError(f\"Scrape script returned no result for {request.url}\")\n            )\n        elif entry.get(\"error\"):","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/web/scraper/runner.py#L119-L155","documentation":"Catch-all failure from run_scrape_in_session: the scrape script process exited non-zero (and not with timeout code 124). The message embeds the exit code and the script's stderr (falling back to stdout), so the underlying cause — an unhandled Python exception in scrape_script.py, an import error, a corrupt config — is visible in the error text itself.","triggerScenarios":"The sandbox Python missing a dependency the script imports (e.g. playwright not installed for session.python_executable); an unexpected exception in scrape_script.py (bad config JSON, invalid URL shape); write_file of the script/config failing silently then executing a stale or absent file; OSError from disk quota in the session.","commonSituations":"Sandbox venv diverging from the host venv (different python_executable); partial upgrades where the script file was updated but the sandbox image's dependencies were not; encoding issues when non-ASCII URLs land in the config; leftover files from a previous run in base_dir colliding.","solutions":["Read the stderr embedded in the message — it names the actual exception from scrape_script.py; fix that first.","Confirm the sandbox's python_executable has playwright (and its deps) installed; re-run `uv sync` in that environment.","Reproduce manually: session.exec the same command line printed in the runner to see full output.","Check that base_dir is writable and clean between runs, and that the config JSON (config.model_dump_json()) is valid for the script's schema."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"# sanity-check the sandbox interpreter has the script's deps before the run\nres = await session.exec(\n    f'{session.python_executable} -c \"import playwright, lxml\"',\n    SandboxExecOptions(cwd=base_dir, timeout=30),\n)\nif res.failed:\n    raise RuntimeError(f'sandbox env broken: {res.stderr}')","typeGuard":"def is_scrape_script_failure(exc: BaseException) -> bool:\n    return isinstance(exc, RuntimeError) and 'Scrape script failed (exit_code=' in str(exc)","tryCatchPattern":"try:\n    outputs = await run_scrape_in_session(session, base_dir, config)\nexcept RuntimeError as e:\n    logger.error('scrape script failed: %s', e)  # message already embeds stderr\n    raise","preventionTips":["Keep the sandbox environment's dependencies in lockstep with the host (same uv lock).","Run the scrape script manually in a session when onboarding a new sandbox backend.","Treat any stderr text change in playwright exceptions as a canary for version drift."],"tags":["scraper","sandbox","diagnostics","playwright"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}