sgl-project/sglang · error · ValueError

LingBot causal sequence sharding requires forward_batch.sequ

Error message

LingBot causal sequence sharding requires forward_batch.sequence_shard_splits.

What it means

When sequence sharding is active, LingBot packs Q/K/V into one all-to-all and needs to know how the packed sequence splits per request; it reads forward_batch.sequence_shard_splits. If that attribute is missing, it cannot partition tokens across Ulysses ranks and raises.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py:284

            if sequence_shard_enabled:
                raise NotImplementedError(
                    "LingBot causal sequence sharding currently requires kv_cache-backed inference."
                )
            return super().forward(
                q,
                k,
                v,
                (cos, sin),
                block_mask,
                kv_cache,
                current_start,
                cache_start,
            )

        if sequence_shard_enabled:
            seq_splits = getattr(forward_batch, "sequence_shard_splits", None)
            if seq_splits is None:
                raise ValueError(
                    "LingBot causal sequence sharding requires forward_batch.sequence_shard_splits."
                )
            seq_splits = list(seq_splits)
            uniform_seq_splits = _sequence_splits_are_uniform(seq_splits)
            # Pack Q/K/V to avoid launching three Ulysses all-to-all collectives.
            qkv = torch.cat([roped_query, roped_key, v], dim=-1)
            qkv = (
                _usp_input_all_to_all(qkv, head_dim=2)
                if uniform_seq_splits
                else _usp_input_all_to_all_varlen(qkv, seq_splits, head_dim=2)
            )
            roped_query, roped_key, v = qkv.chunk(3, dim=-1)

        if (
            not sequence_shard_enabled
            and not update_cache_only
            and kv_cache.can_direct_current_attention(roped_key.shape[1])
        ):

View on GitHub (pinned to 0132848349)

Solutions

1. Update the scheduler/batch construction to set forward_batch.sequence_shard_splits (list of per-request split sizes) when enabling sharding
2. Upgrade sglang to a version where the model and batch plumbing are in sync
3. Disable enable_sequence_shard until the splits are provided

Example fix

# before
batch.enable_sequence_shard = True
out = model(x, forward_batch=batch)

# after
batch.enable_sequence_shard = True
batch.sequence_shard_splits = [S1, S2, ...]   # per-request token splits
out = model(x, forward_batch=batch)

if getattr(forward_batch, 'enable_sequence_shard', False):\n    splits = getattr(forward_batch, 'sequence_shard_splits', None)\n    if splits is None:\n        forward_batch.enable_sequence_shard = False  # or populate splits

def shard_splits_present(forward_batch) -> bool:\n    return getattr(forward_batch, 'sequence_shard_splits', None) is not None

When it happens

Trigger: forward_batch.enable_sequence_shard=True with ulysses_degree > 1 and a valid kv_cache, but the batch object was never given sequence_shard_splits (older batch class, manually built batch, or sharding not wired in the scheduler).

Common situations: Upgrading the model code without the matching scheduler change that populates sequence_shard_splits; constructing ForwardBatch manually in tests; version skew between model and runtime batch definitions.

Related errors

• LingBot causal sequence sharding currently requires kv_cache
• LingBot causal sequence sharding currently supports ulysses_
• LingBot causal sequence sharding currently requires ulysses_
• q, k, and v must have the same 3D shape
• Krea-2 sequence parallelism does not support ragged/padded m

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8adf3f08a668f38f. Report an issue: GitHub.