sgl-project/sglang · error · ValueError

Unsupported encoder folding mode: {mode!r}

Error message

Unsupported encoder folding mode: {mode!r}

What it means

Raised by get_folding_tp_group when the encoder folding mode string is neither 'replica' nor None. Only two modes exist: replica folding (replica TP group) and default TP (None → get_tp_group()).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/encoders/base.py:46

)
from sglang.multimodal_gen.runtime.platforms import AttentionBackendEnum


def get_folding_tp_group(config: EncoderConfig):
    """Return the TP group selected for an encoder."""
    mode = config.parallel_folding_mode
    if mode == "sp":
        return get_sp_group()
    if mode == "world":
        # the whole single-replica DiT (all GPUs), regardless of tp/sp/cfg.
        return get_world_group()
    if mode == "replica":
        # the ranks serving this rank's pipeline replica (== world when
        # dp_size is 1); the shape-independent choice for explicit folding
        return get_replica_group()
    if mode is None:
        return get_tp_group()
    raise ValueError(f"Unsupported encoder folding mode: {mode!r}")


# measured on 2/4xH100: folding wins only for wide encoders (T5-XXL 4096: -20%
# at batch 1, R-insensitive); narrower ones lose to the per-layer all_reduce
# (Qwen3 2560: +35%, CLIP 768: +50%)
FOLD_MIN_HIDDEN_SIZE = 4096
# below this width the encoder stays latency-bound across batch sizes, so
# data-parallel encoding saves no compute and the all_gather is a pure loss
# (CLIP 768: dp slower at every batch/R measured)
DP_MIN_HIDDEN_SIZE = 1024


def _encoder_dims(config: EncoderConfig):
    """Best-effort (hidden, attention_heads, mlp_intermediate) from a config,
    spelled differently across families (hidden_size/d_model, num_heads, d_ff)."""

    def first(names):
        for name in names:

View on GitHub (pinned to 0132848349)

Solutions

1. Use mode='replica' for replica-based folding
2. Use mode=None for the default TP group
3. Check the spelling of the folding mode in your server args / config

Example fix

# before
group = get_folding_tp_group(mode="fold")
# after
group = get_folding_tp_group(mode="replica")  # or None

assert mode in (None, "replica"), f"bad folding mode {mode!r}"

def is_supported_folding_mode(mode) -> bool:
    return mode is None or mode == "replica"

When it happens

Trigger: Passing an unknown mode such as 'fold', 'dp', or a typo to get_folding_tp_group from load_model, finalize_encoder_folding, or encoder __init__.

Common situations: Custom folding configuration strings from server args or a config file that do not match the two accepted values.

Related errors

• bad compress_ratio {compress_ratio}
• The requested FlashAttention forward configuration exceeds S
• flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach
• GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports onl
• kv-canary: RealKvSource.page_size must be >= 1, got {self.pa

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0836cd0e3fae14a7. Report an issue: GitHub.