sgl-project/sglang · error · ComponentResidencyError

Component {component_name!r} resolved to layerwise-offload,

Error message

Component {component_name!r} resolved to layerwise-offload, but its loaded module did not enable layerwise offload

What it means

build_component_residency_strategy resolves the residency mode for a component: if the loaded module actually supports layerwise offload it returns a LayerwiseOffloadStrategy regardless of config; but if the config explicitly asked for layerwise-offload and the module was NOT loaded with layerwise offload enabled, it raises ComponentResidencyError.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/memory_managers/component_manager.py:86

    ) -> list[ComponentUse]: ...


class ComponentResidencyPipeline(Protocol):
    modules: Mapping[str, object]
    _stage_name_mapping: Mapping[str, ComponentResidencyStage]
    component_residency_strategies: MutableMapping[str, "ComponentResidencyStrategy"]


def build_component_residency_strategy(
    component_name: str,
    module: nn.Module,
    server_args: ServerArgs,
) -> ComponentResidencyStrategy:
    residency_mode = server_args.residency_mode(component_name)
    if is_layerwise_offloaded_module(module):
        return LayerwiseOffloadStrategy()
    if residency_mode == LAYERWISE_OFFLOAD:
        raise ComponentResidencyError(
            f"Component {component_name!r} resolved to layerwise-offload, but its "
            "loaded module did not enable layerwise offload"
        )
    if residency_mode == COMPONENT_OFFLOAD and is_fsdp_managed_module(module):
        raise ComponentResidencyError(
            f"Component {component_name!r} resolved to component-offload, but it "
            "was loaded as an FSDP-managed module"
        )
    if (
        not current_platform.is_mps()
        and not is_fsdp_managed_module(module)
        and residency_mode == COMPONENT_OFFLOAD
    ):
        return ComponentOffloadStrategy()
    return ResidentStrategy()


class ComponentResidencyManager:

View on GitHub (pinned to 0132848349)

Solutions

Remove or change the layerwise-offload assignment for that component in the residency config
Enable layerwise offload at module-load time so is_layerwise_offloaded_module(module) is true, then retry
Use component-offload or gpu mode for components that do not support layerwise loading

Example fix

# before
--component-residency vit=layerwise-offload
# after
--component-residency vit=component-offload

Defensive patterns

Strategy: validation

Validate before calling

from sglang.multimodal_gen.runtime.managers.memory_managers.component_manager import is_layerwise_offloaded_module
if not is_layerwise_offloaded_module(module):
    raise SystemExit("module not layerwise-loaded; remove layerwise-offload residency for it")

Type guard

def can_use_layerwise(module) -> bool:
    return is_layerwise_offloaded_module(module)

Prevention

Match residency mode to how the module was actually loaded, not just to intent
Re-verify residency configs after upgrading, since module loading paths change

When it happens

Trigger: server_args.residency_mode(name) == LAYERWISE_OFFLOAD while is_layerwise_offloaded_module(module) is False — e.g. setting component residency like "vit=layerwise-offload" for a component whose weights were loaded monolithically.

Common situations: Enabling layerwise offload for a component (e.g. a VAE or vision tower) that was loaded without the layerwise loading path; changing residency flags without re-loading/re-configuring the module; copying a residency config from a model that supports layerwise to one that does not.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6a6cc10d3505cf91. Report an issue: GitHub.