sgl-project/sglang · error · RuntimeError

sleep/wake does not support FSDP inference

Error message

sleep/wake does not support FSDP inference

What it means

Raised by release_memory_occupation when sleep/wake (memory offload) is attempted while the controller was configured with use_fsdp_inference=True. FSDP shards/owns parameter placement, so the module-moving offload strategy cannot be applied.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/memory_managers/memory_occupation_controller.py:185

    def _restore_modules_to_original_devices(
        self, module_device_map: dict[str, str]
    ) -> None:
        grouped: dict[str, list[str]] = {}
        for name, device in module_device_map.items():
            grouped.setdefault(device, []).append(name)

        for device, names in grouped.items():
            self._move_modules(names, device)

    def release_memory_occupation(self) -> dict[str, bool | str]:
        logger.info(f"[SLEEP] release_memory_occupation rank={self.rank}")
        if self._sleeping:
            return self._memory_occupation_result(
                success=True,
                message="already sleeping",
            )
        if self.use_fsdp_inference:
            raise RuntimeError("sleep/wake does not support FSDP inference")
        if self.pipeline is None:
            return self._memory_occupation_result(
                success=False,
                message="pipeline not initialized",
            )

        self._sleep_restore_map = self._offload_active_modules_to_cpu()
        self._sleeping = True
        return self._memory_occupation_result(
            success=True,
            message="released GPU memory (moved active modules to CPU)",
        )

    def resume_memory_occupation(self) -> dict[str, bool | str]:
        logger.info(f"[WAKE] resume_memory_occupation rank={self.rank}")
        if not self._sleeping:
            return self._memory_occupation_result(
                success=True,

View on GitHub (pinned to 0132848349)

Solutions

1. Disable FSDP inference if you need sleep/wake offload
2. Keep FSDP and drop sleep/wake from the workflow
3. Gate the code: skip release_memory_occupation when use_fsdp_inference is set

Example fix

// before
controller.release_memory_occupation()  # with use_fsdp_inference=True
// after
if not controller.use_fsdp_inference:
    controller.release_memory_occupation()

if controller.use_fsdp_inference:
    raise RuntimeError("sleep/wake unavailable under FSDP; disable FSDP first")

When it happens

Trigger: Enabling FSDP inference (e.g. fsdp inference config) and then calling release_memory_occupation / sleep on the controller.

Common situations: Enabling FSDP for large-model memory savings and then trying to reuse a sleep/wake RL workflow that expects plain module offload; config drift where both flags end up enabled.

Related errors

• Rank-local FSDP shard produced for non-DTensor parameter {ta
• Rank-local TP shard produced for DTensor parameter {target_p
• GGUF diffusion checkpoints are incompatible with FSDP infere
• Component {component_name!r} resolved to component-offload,
• Cannot update weights while the server is sleeping. Call res

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/176adf35662eec86. Report an issue: GitHub.