sgl-project/sglang · error · ValueError

unknown residency policy {policy!r} for component {component

Error message

unknown residency policy {policy!r} for component {component_name!r}, expected one of {RESIDENCY_POLICIES}

What it means

layerwise_tuning_for() resolves the residency policy for a component from the per-component map plus a leading/default policy, then validates it against the known RESIDENCY_POLICIES set. An unrecognized policy string (typo, wrong casing, unsupported mode) is rejected immediately because downstream offload logic can only handle known policies.

Source

Thrown at python/sglang/multimodal_gen/runtime/server_args/server_args.py:1087

            self.layerwise_residency_policy, option="--layerwise-residency-policy"
        )

        def _pick(mapping: dict[str, str], group_default, aux_default):
            if component_name is not None and component_name in mapping:
                return mapping[component_name]
            return group_default if dit_group else aux_default

        prefetch = float(_pick(prefetch_map, self.dit_offload_prefetch_size, 0.0))
        resident = float(_pick(resident_map, self.dit_layerwise_resident_layers, 0.0))
        policy = str(
            _pick(
                policy_map,
                self.dit_layerwise_residency_policy,
                RESIDENCY_POLICY_LEADING,
            )
        )
        if policy not in RESIDENCY_POLICIES:
            raise ValueError(
                f"unknown residency policy {policy!r} for component "
                f"{component_name!r}, expected one of {RESIDENCY_POLICIES}"
            )
        return prefetch, resident, policy

    @staticmethod
    def _parse_component_attention_backend_map(
        value: dict[str, str] | str | None,
    ) -> dict[str, str]:
        if value is None or value == "":
            return {}
        if isinstance(value, dict):
            return dict(value)
        if not isinstance(value, str):
            raise ValueError(
                "component_attention_backends must be a dict or a comma-separated component=backend string"
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Print RESIDENCY_POLICIES (import from the server_args module) and use an exact member
2. Fix the typo in the component=policy entry or the leading default policy
3. Upgrade/downgrade docs alignment: check the version's supported policy names

Example fix

# before
--dit-layerwise-residency-policy dit=layerwise
# after
--dit-layerwise-residency-policy dit=layerwise_offload

from sglang.multimodal_gen.runtime.server_args.server_args import RESIDENCY_POLICIES
policy = args.layerwise_tuning_for('dit', dit_group=True)
# pre-check any user-supplied policy:
assert user_policy in RESIDENCY_POLICIES, f'policy must be one of {RESIDENCY_POLICIES}'

def is_valid_policy(p: str) -> bool:
    return p in RESIDENCY_POLICIES

try:
    prefetch, resident, policy = args.layerwise_tuning_for(name, dit_group=True)
except ValueError as e:
    if 'unknown residency policy' in str(e):
        policy = 'resident'  # fallback default
    else:
        raise

When it happens

Trigger: Calling server_args.layerwise_tuning_for('dit', dit_group=True) when the effective policy resolves to e.g. 'offload' or 'layer_wise' (misspelled) instead of a member of RESIDENCY_POLICIES; passing --dit-layerwise-residency-policy dit=foo.

Common situations: Policy renamed across versions; copying policy names from older docs or other flags; using underscores/hyphens inconsistently; tests exercising rejection of unknown policies.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• {option} must use component=value entries
• Invalid threshold_type for topk: {threshold_type}. Choose 'q
• Invalid threshold_type: {threshold_type}. Choose 'query_head
• f"Unknown feature map: {feature_map}"
• f"Unsupported patch_size type: {type(patch_size)}"

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4dfdf6d85ecaf364. Report an issue: GitHub.