run-llama/liteparse · error · ParseError

liteparse worker process died while parsing

Error message

liteparse worker process died while parsing {source}: {e}

What it means

Raised as ParseError when a LiteParse pool worker subprocess exits (crashes, is OOM-killed, or hits EOF on stdout) in the middle of a parse request instead of replying. The pool detects the dead process via the reader thread's EOF sentinel, raises _WorkerCrashed, and this handler converts it to ParseError. The dead worker is retired and a fresh one is spawned automatically, so the pool remains usable for subsequent calls.

Solutions

1. Re-run the parse — the pool already replaced the dead worker, so a retry often succeeds
2. Reduce per-parse memory pressure: split large PDFs or parse page ranges instead of whole documents
3. Raise the container/process memory limit so the worker is not OOM-killed
4. Capture the worker's stderr (it is inherited and printed) to identify a segfault in native PDFium/Tesseract code
5. If crashes are deterministic, parse the document without the pool (pool_size=None) to get the native stack trace directly

Example fix

# before
parser = LiteParse(pool_size=2, parse_timeout=300)
result = parser.parse("huge.pdf")  # ParseError: worker process died

# after
import time
for attempt in range(3):
    try:
        result = parser.parse("huge.pdf")
        break
    except ParseError as e:
        if "worker process died" not in str(e) or attempt == 2:
            raise
        time.sleep(1)

import os
# heuristic: reject inputs wildly larger than the worker's safe budget
MAX_BYTES = 500 * 1024 * 1024
size = os.path.getsize(path)
if size > MAX_BYTES:
    raise ValueError(f"{path} is {size} bytes; split before parsing")

def worker_died(e: Exception) -> bool:
    return isinstance(e, ParseError) and "worker process died" in str(e)

from liteparse.types import ParseError
import time
for attempt in range(3):
    try:
        result = parser.parse(path)
        break
    except ParseError as e:
        if "worker process died" not in str(e):
            raise
        if attempt == 2:
            raise
        time.sleep(0.5 * (attempt + 1))

When it happens

Trigger: Calling LiteParse(pool_size=N).parse(...) (or parse_bytes) when the worker process dies mid-request: the OS OOM-killer terminates the child on a huge PDF, the child segfaults in native code (PDFium/Tesseract), or someone/something kills the subprocess externally. Any of these surface as _WorkerCrashed inside WorkerPool.parse.

Common situations: Parsing very large or malformed PDFs that blow up worker memory in containerized environments with tight memory limits (Kubernetes OOMKill); native-library segfaults on unusual PDF constructs; shared CI runners where processes get killed; misconfigured cgroup limits.

Related errors

• parse of exceeded ms; the worker process was killed
• liteparse worker process died while parsing
• liteparse worker failed to initialize
• {data}
• parseTimeoutMs requires poolSize

AI-assisted analysis of run-llama/liteparse@22d2dd8cd7 (2026-09-08). Data as JSON: /api/errors/d9d493ff61173127. Report an issue: GitHub.

Appendix: source

Thrown at packages/python/liteparse/_pool.py:253

        """
        if self._closed:
            raise ParseError("parser pool is closed")
        worker = self._idle.get()
        replace = False
        try:
            status, data = worker.request(payload, self._timeout)
        except _WorkerTimeout:
            replace = True
            timeout = self._timeout
            raise ParseTimeoutError(
                f"parse of {source} exceeded {timeout}s; "
                "the worker process was killed",
                source=source,
                timeout=timeout,
            ) from None
        except _WorkerCrashed as e:
            replace = True
            raise ParseError(
                f"liteparse worker process died while parsing {source}: {e}"
            ) from None
        except _WorkerInitFailed as e:
            replace = True
            raise ParseError(f"liteparse worker failed to initialize: {e}") from None
        finally:
            if replace:
                self._retire_worker(worker)
                if not self._closed:
                    self._spawn_worker()
            elif self._closed:
                worker.stop()
            else:
                self._idle.put(worker)
        if status == "ok":
            return data
        raise ParseError(data)

View on GitHub (pinned to 22d2dd8cd7)