{"record":{"id":"1296f67b5f3a2902","repo":"sgl-project/sglang","slug":"server-is-sleeping-call-resume-memory-occupation","errorCode":null,"errorMessage":"Server is sleeping. Call resume_memory_occupation first.","messagePattern":"Server is sleeping\\. Call resume_memory_occupation first\\.","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/managers/scheduler.py","lineNumber":281,"sourceCode":"\n    def _dispatch_items(\n        self, items: list[tuple[bytes | None, Any]]\n    ) -> OutputBatch | list[OutputBatch] | _SequentiallyReturnedOutputs:\n        \"\"\"Dispatch ready queue items; several plain `Req`s form one dynamic batch.\"\"\"\n        reqs = [item[1] for item in items]\n        if len(reqs) > 1 and all(isinstance(req, Req) for req in reqs):\n            return self._handle_generation(reqs, allow_dynamic_batching=True)\n        if len(reqs) > 1:\n            return [self._dispatch_single_request(req) for req in reqs]\n        return self._dispatch_single_request(reqs[0])\n\n    def _handle_generation(\n        self, reqs: list[Any], *, allow_dynamic_batching: bool = True\n    ):\n        \"\"\"Dispatch generation requests, merging compatible requests when allowed.\"\"\"\n        reqs = self._normalize_generation_reqs(reqs)\n        if self.worker.is_sleeping():\n            raise RuntimeError(\n                \"Server is sleeping. Call resume_memory_occupation first.\"\n            )\n        warmup_reqs = [req for req in reqs if req.is_warmup]\n        if warmup_reqs:\n            self._ensure_warmup_progress_bar(warmup_reqs[0])\n\n        # Use the head request trace context for scheduler-side dispatch work.\n        req = reqs[0]\n        req.trace_ctx.rebuild_thread_context()\n        with trace_slice(\n            req.trace_ctx,\n            DiffStage.SCHEDULER_DISPATCH,\n            thread_finish_flag=True,\n        ):\n            if (\n                len(reqs) == 1\n                and self.server_args.pipeline_config.supports_sequential_multi_output_inference()\n                and max(1, int(req.num_outputs_per_prompt or 1)) > 1","sourceCodeStart":263,"sourceCodeEnd":299,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/managers/scheduler.py#L263-L299","documentation":"Raised by Scheduler._handle_generation when generation requests are dispatched while worker.is_sleeping() is true. The model weights are offloaded, so no forward pass can run until the server is woken.","triggerScenarios":"Sending generate/warmup requests to a server that has executed release_memory_occupation (sleep) without a subsequent resume_memory_occupation.","commonSituations":"Sleeping the server to free VRAM for training and then letting clients/inference requests hit it before wake; warmup requests queued during sleep; health-check traffic arriving mid-sleep.","solutions":["Call resume_memory_occupation before sending generation requests","Buffer/queue client requests while sleeping and flush after wake","Make clients retry on this RuntimeError after triggering wake"],"exampleFix":"// before\ncontroller.release_memory_occupation()\nserver.generate(req)  # raises\n// after\ncontroller.resume_memory_occupation()\nserver.generate(req)","handlingStrategy":"validation","validationCode":"assert not worker.is_sleeping(), \"server asleep; resume_memory_occupation first\"","typeGuard":null,"tryCatchPattern":"try:\n    server.generate(req)\nexcept RuntimeError as e:\n    if \"Server is sleeping\" in str(e):\n        controller.resume_memory_occupation(); retry(req)\n    else:\n        raise","preventionTips":["Queue client requests during sleep","Always pair sleep with wake in a try/finally"],"tags":["sleep-wake","generation","invalid-state"],"backgroundTag":"invalid-state-transition","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}