What causes "Scrape script failed (exit_code={result.exit_code}): {error}" in zylon-ai/private-gpt?

Read the stderr embedded in the message — it names the actual exception from scrape_script.py; fix that first.

zylon-ai/private-gpt · error · RuntimeError

Scrape script failed (exit_code={result.exit_code}): {error}

Error message

Scrape script failed (exit_code={result.exit_code}): {error}

What it means

Catch-all failure from run_scrape_in_session: the scrape script process exited non-zero (and not with timeout code 124). The message embeds the exit code and the script's stderr (falling back to stdout), so the underlying cause — an unhandled Python exception in scrape_script.py, an import error, a corrupt config — is visible in the error text itself.

Source

Thrown at private_gpt/components/web/scraper/runner.py:137

        f" {shlex.quote(CONFIG_FILENAME)}",
        SandboxExecOptions(
            cwd=base_dir, timeout=total_timeout + _EXEC_TIMEOUT_MARGIN_SECONDS
        ),
    )

    if result.exit_code == 124:
        raise TimeoutError(
            f"Timeout ({config.timeout_seconds}s) scraping "
            f"{[r.url for r in config.requests]}"
        )
    if result.failed:
        error = result.stderr or result.stdout
        if _BROWSERS_MISSING_PATTERN.search(error):
            raise RuntimeError(
                "Playwright browsers are not installed. "
                "Run `playwright install` and try again."
            )
        raise RuntimeError(
            f"Scrape script failed (exit_code={result.exit_code}): {error}"
        )

    results_raw = json.loads(
        (await session.read_file(posixpath.join(base_dir, RESULTS_FILENAME))).decode(
            "utf-8"
        )
    )
    by_output_path = {entry["output_path"]: entry for entry in results_raw}

    outputs: list[str | Exception] = []
    for request in config.requests:
        entry = by_output_path.get(request.output_path)
        if entry is None:
            outputs.append(
                RuntimeError(f"Scrape script returned no result for {request.url}")
            )
        elif entry.get("error"):

View on GitHub (pinned to 4a030776a3)

Solutions

Read the stderr embedded in the message — it names the actual exception from scrape_script.py; fix that first.
Confirm the sandbox's python_executable has playwright (and its deps) installed; re-run `uv sync` in that environment.
Reproduce manually: session.exec the same command line printed in the runner to see full output.
Check that base_dir is writable and clean between runs, and that the config JSON (config.model_dump_json()) is valid for the script's schema.

Defensive patterns

Strategy: try-catch

Validate before calling

# sanity-check the sandbox interpreter has the script's deps before the run
res = await session.exec(
    f'{session.python_executable} -c "import playwright, lxml"',
    SandboxExecOptions(cwd=base_dir, timeout=30),
)
if res.failed:
    raise RuntimeError(f'sandbox env broken: {res.stderr}')

Type guard

def is_scrape_script_failure(exc: BaseException) -> bool:
    return isinstance(exc, RuntimeError) and 'Scrape script failed (exit_code=' in str(exc)

Try / catch

try:
    outputs = await run_scrape_in_session(session, base_dir, config)
except RuntimeError as e:
    logger.error('scrape script failed: %s', e)  # message already embeds stderr
    raise

Prevention

Keep the sandbox environment's dependencies in lockstep with the host (same uv lock).
Run the scrape script manually in a session when onboarding a new sandbox backend.
Treat any stderr text change in playwright exceptions as a canary for version drift.

When it happens

Trigger: The sandbox Python missing a dependency the script imports (e.g. playwright not installed for session.python_executable); an unexpected exception in scrape_script.py (bad config JSON, invalid URL shape); write_file of the script/config failing silently then executing a stale or absent file; OSError from disk quota in the session.

Common situations: Sandbox venv diverging from the host venv (different python_executable); partial upgrades where the script file was updated but the sandbox image's dependencies were not; encoding issues when non-ASCII URLs land in the config; leftover files from a previous run in base_dir colliding.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/1852166bf83816f4. Report an issue: GitHub.