{"record":{"id":"d39e91eff047b5de","repo":"huggingface/transformers","slug":"cb-worker-is-dead-and-cannot-accept-request-reque","errorCode":null,"errorMessage":"CB worker is dead and cannot accept request {request_id}: {self._cb.fatal_error}","messagePattern":"CB worker is dead and cannot accept request (.+?): (.+?)","errorType":"http","errorClass":"CBWorkerDeadError","httpStatus":503,"severity":"critical","filePath":"src/transformers/cli/serving/utils.py","lineNumber":856,"sourceCode":"            return\n\n        self._cb = model.init_continuous_batching(\n            generation_config=gen_config, continuous_batching_config=self._cb_config\n        )\n        self._cb.start()\n\n    def is_alive(self) -> bool:\n        \"\"\"Whether the CB worker is healthy. ``True`` before ``init_cb()`` is called.\"\"\"\n        return self._cb is None or self._cb.fatal_error is None\n\n    def _check_alive(self, request_id: str) -> None:\n        \"\"\"Raise :class:`CBWorkerDeadError` if the CB worker has died.\n\n        Called at request entry to fail fast — submitting to a dead worker would otherwise\n        enqueue the request into a void where it never gets processed.\n        \"\"\"\n        if self._cb is not None and self._cb.fatal_error is not None:\n            raise CBWorkerDeadError(\n                f\"CB worker is dead and cannot accept request {request_id}: {self._cb.fatal_error}\"\n            )\n\n    def generate_streaming(\n        self,\n        model: \"PreTrainedModel\",\n        processor: \"ProcessorMixin | PreTrainedTokenizerFast\",\n        inputs: dict,\n        gen_config: \"GenerationConfig\",\n        request_id: str,\n        response_parser: \"ResponseParser | None\" = None,\n    ) -> tuple[asyncio.Queue, CBStreamer]:\n        \"\"\"Start streaming CB generation. Registers a per-request output handler.\"\"\"\n        cb = self._cb\n        if cb is None:\n            raise RuntimeError(\"CB manager not initialized. Call `init_cb()` first.\")\n        self._check_alive(request_id)\n","sourceCodeStart":838,"sourceCodeEnd":874,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cli/serving/utils.py#L838-L874","documentation":"CBWorkerDeadError raised at request entry when the continuous-batching worker process has a non-None fatal_error. Once the CB worker dies, every queued or new request would silently hang, so the guard fails fast with (typically) a 503 instead. The message embeds the worker's original fatal error for diagnosis.","triggerScenarios":"Any generation request (e.g. /v1/chat/completions) submitted after the CB worker crashed earlier — OOM kill, CUDA error, or an unhandled exception in the batch processor set fatal_error on the worker handle.","commonSituations":"GPU out-of-memory inside the CB worker killing the process; a driver/runtime crash; requests continuing to arrive after the worker died because the HTTP server itself is still healthy.","solutions":["Inspect the embedded fatal_error in the message to find the root cause (usually OOM or a CUDA fault)","Reduce memory pressure (smaller max batch / max length, smaller model, more GPU memory) and restart the server","Catch CBWorkerDeadError client-side and treat it as 503: stop retrying immediately and alert instead of hammering the server"],"exampleFix":"# before\nresp = client.post('/v1/chat/completions', json=body)  # keeps retrying a dead worker\n# after\ntry:\n    resp = client.post('/v1/chat/completions', json=body)\nexcept HTTPError as e:\n    if e.response.status_code == 503:\n        alert('CB worker dead, restart serving process')\n    raise","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    resp = client.post('/v1/chat/completions', json=body, timeout=30)\nexcept (httpx.HTTPStatusError, requests.HTTPError) as e:\n    if getattr(e.response, 'status_code', None) == 503 and 'CB worker is dead' in e.response.text:\n        alert_and_stop_retries(e.response.text)  # worker crashed; needs restart\n    raise","preventionTips":["Monitor the CB worker process and auto-restart the server on fatal_error","Cap batch size and sequence length to avoid worker OOM","Treat 503 'worker dead' as a terminal condition, not a transient retry candidate"],"tags":["serving","continuous-batching","crash","http-503"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}