sgl-project/sglang · error · ValueError

sm_group_num must be >= 3

Error message

sm_group_num must be >= 3

What it means

PD multiplexing divides SMs into at least three groups (prefill, decode, plus control), so sm_group_num values of 1 or 2 are structurally invalid and rejected.

Source

Thrown at python/sglang/srt/multiplex/pdmux_context.py:36

        default_factory=list
    )  # [prefill_sm, decode_sm, decode_bs_threshold]
    split_forward_token_budget: int = 65536
    decode_bs_divisor: int = 36


def load_pdmux_config(config_path: str) -> PDMuxConfig:
    """Load pdmux configuration from YAML file into a dataclass."""
    if not config_path:
        return PDMuxConfig()

    with open(config_path, "r") as f:
        raw = yaml.safe_load(f)

    if "sm_group_num" not in raw:
        raise ValueError("Missing required field: sm_group_num")

    if raw["sm_group_num"] < 3:
        raise ValueError("sm_group_num must be >= 3")

    manual_divisions = raw.get("manual_divisions", [])

    expected = raw["sm_group_num"] - 2
    if manual_divisions and len(manual_divisions) != expected:
        raise ValueError(
            f"manual_divisions must have {expected} entries, "
            f"but got {len(manual_divisions)}"
        )

    return PDMuxConfig(
        sm_group_num=raw["sm_group_num"],
        manual_divisions=manual_divisions,
        split_forward_token_budget=raw.get("split_forward_token_budget", 65536),
        decode_bs_divisor=raw.get("decode_bs_divisor", 36),
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Set sm_group_num to >= 3
2. Read the pdmux docs for recommended counts for your GPU count and workload
3. If you only need 2 workload groups, use the corresponding manual_divisions arrangement with the minimum group count

Example fix

# before
sm_group_num: 2
# after
sm_group_num: 4

assert raw['sm_group_num'] >= 3, 'sm_group_num must be >= 3'

When it happens

Trigger: Setting sm_group_num: 2 (or 1) in the pdmux YAML config.

Common situations: Assuming one group per workload type (prefill+decode=2) without knowing the extra control groups are required; shrinking groups to simplify testing.

Related errors

• Invalid threshold_type for topk: {threshold_type}. Choose 'q
• Invalid threshold_type: {threshold_type}. Choose 'query_head
• f"Unknown feature map: {feature_map}"
• f"Unsupported patch_size type: {type(patch_size)}"
• batching config rule requires max_batch_size

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5d3e6352dbb264e5. Report an issue: GitHub.