{"record":{"id":"0cf029d7140f6548","repo":"sgl-project/sglang","slug":"weight-cache-op-of-model-weights-is-not-suppor","errorCode":null,"errorMessage":"[weight_cache] {op} of model weights is not supported while the weight cache is active (--weight-cache-mode {mode}): the weights are shared with the daemon via CUDA IPC, so freeing them would corrupt the daemon's master copy and every co-attached engine. Restart with --weight-cache-mode off to use this operation.","messagePattern":"\\[weight_cache\\] (.+?) of model weights is not supported while the weight cache is active \\(--weight-cache-mode (.+?)\\): the weights are shared with the daemon via CUDA IPC, so freeing them would corrupt the daemon's master copy and every co-attached engine\\. Restart with --weight-cache-mode off to use this operation\\.","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/scheduler_components/weight_updater.py","lineNumber":203,"sourceCode":"                self.record_weight_version_after_update(recv_req.weight_version)\n            else:\n                logger.error(message)\n            torch.distributed.barrier(group=self.tp_cpu_group)\n            return UpdateWeightsFromIPCReqOutput(success=success, message=message)\n\n    def get_weights_by_name(self, recv_req: GetWeightsByNameReqInput):\n        parameter = self.tp_worker.get_weights_by_name(recv_req)\n        return GetWeightsByNameReqOutput(parameter=parameter)\n\n    def _assert_weight_cache_inactive(self, op: str) -> None:\n        \"\"\"Reject freeing/restoring model weights while the CUDA IPC weight\n        cache is active: the weights are shared with the daemon via CUDA IPC, so\n        freeing them would leave the daemon and every peer pointing at released\n        memory.\n        \"\"\"\n        mode = self.tp_worker.model_runner.server_args.weight_cache_mode\n        if mode != \"off\":\n            raise RuntimeError(\n                f\"[weight_cache] {op} of model weights is not supported while the \"\n                f\"weight cache is active (--weight-cache-mode {mode}): the weights \"\n                f\"are shared with the daemon via CUDA IPC, so freeing them would \"\n                f\"corrupt the daemon's master copy and every co-attached engine. \"\n                f\"Restart with --weight-cache-mode off to use this operation.\"\n            )\n\n    def release_memory_occupation(self, recv_req: ReleaseMemoryOccupationReqInput):\n        assert (\n            self.is_fully_idle()\n        ), \"release_memory_occupation should be called only when server is idle.\"\n\n        tags = recv_req.tags\n\n        if tags is None or len(tags) == 0:\n            tags = GPU_MEMORY_ALL_TYPES\n\n        for tag in tags:","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/scheduler_components/weight_updater.py#L185-L221","documentation":"With --weight-cache-mode active, model weights are shared with a weight-cache daemon via CUDA IPC, so freeing/re-reserving them would corrupt the daemon's master copy and peer engines. The weight updater blocks release_memory_occupation/resume_memory_occupation in that mode.","triggerScenarios":"Calling release_memory_occupation or resume_memory_occupation (e.g. sleep-mode / memory hibernation flows, some update-weight or restart flows) while weight_cache_mode != 'off'.","commonSituations":"Enabling the weight cache daemon for fast multi-engine startup and then triggering a flow that releases GPU memory (server sleep, weight updates requiring re-init).","solutions":["Restart with --weight-cache-mode off if you need the memory-release flow","Avoid sleep/hibernate or flows that call release_memory_occupation while weight cache is on","Gate your orchestration: skip release operations when weight_cache_mode != 'off'"],"exampleFix":"# before\npython -m sglang.launch_server --weight-cache-mode daemon --enable-sleep-mode\n# after\npython -m sglang.launch_server --weight-cache-mode off --enable-sleep-mode","handlingStrategy":"validation","validationCode":"mode = server_args.weight_cache_mode\nassert mode == 'off' or not needs_memory_release_flow, 'weight cache blocks release_memory_occupation'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never enable sleep-mode/memory-release flows together with --weight-cache-mode","Add a launch-config lint rule rejecting the combination"],"tags":["weight-cache","cuda-ipc","memory-management","feature-incompatibility"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}