{"record":{"id":"d9d493ff61173127","repo":"run-llama/liteparse","slug":"liteparse-worker-process-died-while-parsing-sourc","errorCode":null,"errorMessage":"liteparse worker process died while parsing {source}: {e}","messagePattern":"liteparse worker process died while parsing (.+?): (.+?)","errorType":"exception","errorClass":"ParseError","httpStatus":null,"severity":"error","filePath":"packages/python/liteparse/_pool.py","lineNumber":253,"sourceCode":"        \"\"\"\n        if self._closed:\n            raise ParseError(\"parser pool is closed\")\n        worker = self._idle.get()\n        replace = False\n        try:\n            status, data = worker.request(payload, self._timeout)\n        except _WorkerTimeout:\n            replace = True\n            timeout = self._timeout\n            raise ParseTimeoutError(\n                f\"parse of {source} exceeded {timeout}s; \"\n                \"the worker process was killed\",\n                source=source,\n                timeout=timeout,\n            ) from None\n        except _WorkerCrashed as e:\n            replace = True\n            raise ParseError(\n                f\"liteparse worker process died while parsing {source}: {e}\"\n            ) from None\n        except _WorkerInitFailed as e:\n            replace = True\n            raise ParseError(f\"liteparse worker failed to initialize: {e}\") from None\n        finally:\n            if replace:\n                self._retire_worker(worker)\n                if not self._closed:\n                    self._spawn_worker()\n            elif self._closed:\n                worker.stop()\n            else:\n                self._idle.put(worker)\n        if status == \"ok\":\n            return data\n        raise ParseError(data)\n","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/run-llama/liteparse/blob/22d2dd8cd7f7b9320102b57ddaf0e663ff7d15a8/packages/python/liteparse/_pool.py#L235-L271","documentation":"Raised as ParseError when a LiteParse pool worker subprocess exits (crashes, is OOM-killed, or hits EOF on stdout) in the middle of a parse request instead of replying. The pool detects the dead process via the reader thread's EOF sentinel, raises _WorkerCrashed, and this handler converts it to ParseError. The dead worker is retired and a fresh one is spawned automatically, so the pool remains usable for subsequent calls.","triggerScenarios":"Calling LiteParse(pool_size=N).parse(...) (or parse_bytes) when the worker process dies mid-request: the OS OOM-killer terminates the child on a huge PDF, the child segfaults in native code (PDFium/Tesseract), or someone/something kills the subprocess externally. Any of these surface as _WorkerCrashed inside WorkerPool.parse.","commonSituations":"Parsing very large or malformed PDFs that blow up worker memory in containerized environments with tight memory limits (Kubernetes OOMKill); native-library segfaults on unusual PDF constructs; shared CI runners where processes get killed; misconfigured cgroup limits.","solutions":["Re-run the parse — the pool already replaced the dead worker, so a retry often succeeds","Reduce per-parse memory pressure: split large PDFs or parse page ranges instead of whole documents","Raise the container/process memory limit so the worker is not OOM-killed","Capture the worker's stderr (it is inherited and printed) to identify a segfault in native PDFium/Tesseract code","If crashes are deterministic, parse the document without the pool (pool_size=None) to get the native stack trace directly"],"exampleFix":"# before\nparser = LiteParse(pool_size=2, parse_timeout=300)\nresult = parser.parse(\"huge.pdf\")  # ParseError: worker process died\n\n# after\nimport time\nfor attempt in range(3):\n    try:\n        result = parser.parse(\"huge.pdf\")\n        break\n    except ParseError as e:\n        if \"worker process died\" not in str(e) or attempt == 2:\n            raise\n        time.sleep(1)","handlingStrategy":"retry","validationCode":"import os\n# heuristic: reject inputs wildly larger than the worker's safe budget\nMAX_BYTES = 500 * 1024 * 1024\nsize = os.path.getsize(path)\nif size > MAX_BYTES:\n    raise ValueError(f\"{path} is {size} bytes; split before parsing\")","typeGuard":"def worker_died(e: Exception) -> bool:\n    return isinstance(e, ParseError) and \"worker process died\" in str(e)","tryCatchPattern":"from liteparse.types import ParseError\nimport time\nfor attempt in range(3):\n    try:\n        result = parser.parse(path)\n        break\n    except ParseError as e:\n        if \"worker process died\" not in str(e):\n            raise\n        if attempt == 2:\n            raise\n        time.sleep(0.5 * (attempt + 1))","preventionTips":["Set container memory limits above the peak needed for your largest PDFs","Split very large documents and parse page ranges","Watch the inherited worker stderr for native crash signatures (segfault, OOM)","Retry once on this specific error — the pool replaces the worker automatically"],"tags":["subprocess","crash","worker-pool","memory","parse"],"backgroundTag":"worker-process-crashed","analyzedSha":"22d2dd8cd7f7b9320102b57ddaf0e663ff7d15a8","analyzedAt":"2026-09-08T06:09:49.009Z","contentChangedAt":"2026-09-08T06:09:49.009Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}