sgl-project/sglang · error · ValueError

kv_gather_degree does not compose with ulysses_degree or rin

Error message

kv_gather_degree does not compose with ulysses_degree or ring_degree yet; set exactly one of them above 1

What it means

kv_gather_degree is a parallelism axis that is currently mutually exclusive with ulysses_degree and ring_degree. If kv_gather_degree > 1, both of the others must be 1 (or None, treated as 1).

Source

Thrown at python/sglang/multimodal_gen/runtime/server_args/server_args.py:1365

                self.sp_split_auto = True
                logger.info(
                    "Automatically set kv_gather_degree=sp_degree=2; set "
                    "--ulysses-degree explicitly to keep the Ulysses exchange"
                )
            else:
                self.ulysses_degree = self.sp_degree
                logger.info(
                    "Automatically set ulysses_degree=sp_degree=%d for the "
                    "sequence-parallel process-group layout",
                    self.ulysses_degree,
                )

        if self.kv_gather_degree is None:
            self.kv_gather_degree = 1

        if self.kv_gather_degree > 1:
            if (self.ulysses_degree or 1) != 1 or (self.ring_degree or 1) != 1:
                raise ValueError(
                    "kv_gather_degree does not compose with ulysses_degree or "
                    "ring_degree yet; set exactly one of them above 1"
                )

        if self.ulysses_degree is None:
            self.ulysses_degree = 1
            logger.debug(
                f"Ulysses degree not set, using default value {self.ulysses_degree}"
            )

        if self.ring_degree is None:
            self.ring_degree = 1
            logger.debug(f"Ring degree not set, using default value {self.ring_degree}")

        if self.kv_gather_degree > 1:
            # K/V-gather rows occupy the contiguous inner SP dimension; the
            # process groups are built from ulysses_degree, so alias it until
            # gather gets a first-class dimension (needed only once it

View on GitHub (pinned to 0132848349)

Solutions

1. Keep exactly one of kv_gather_degree / ulysses_degree / ring_degree above 1; set the others to 1
2. If Ulysses parallelism is required, unset --kv-gather-degree (leave it 1)
3. Recompute total parallelism against tp/ep world size after dropping one axis

Example fix

# before
--kv-gather-degree 2 --ulysses-degree 2
# after
--ulysses-degree 2

axes = [kv_gather_degree or 1, ulysses_degree or 1, ring_degree or 1]
assert sum(a > 1 for a in axes) <= 1, 'only one parallelism axis may exceed 1'

When it happens

Trigger: Setting --kv-gather-degree 2 together with --ulysses-degree 2 or --ring-degree > 1 in _adjust_parallelism.

Common situations: Migrating configs from a Ulysses/ring setup and adding kv_gather; enabling all parallelism knobs to maximize sharding; defaults changed so a previously-1 degree is now >1.

Related errors

• Invalid threshold_type for topk: {threshold_type}. Choose 'q
• Invalid threshold_type: {threshold_type}. Choose 'query_head
• f"Unknown feature map: {feature_map}"
• f"Unsupported patch_size type: {type(patch_size)}"
• batching config rule requires max_batch_size

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/311f578512cc299c. Report an issue: GitHub.