sgl-project/sglang · error · ValueError

perturbation_configs length must match batch size, got {len(

Error message

perturbation_configs length must match batch size, got {len(perturbation_configs)=} {batch_size=}.

What it means

When the optional perturbation_configs kwarg is supplied (flow-matching perturbation schedules per sample), its length must equal the batch size of hidden_states. A mismatch would misalign per-sample perturbation handling, so forward validates it.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py:2000

        video_memory_prefix_len: int = 0,
        late_layer_ratio: float = 1.0,
        late_audio_self_attention_mask: Optional[torch.Tensor] = None,
        **kwargs,
    ) -> tuple[torch.Tensor | None, torch.Tensor | None]:
        batch_size = hidden_states.size(0)
        audio_timestep = audio_timestep if audio_timestep is not None else timestep

        if num_frames is None or height is None or width is None:
            raise ValueError(
                "num_frames/height/width must be provided for RoPE coordinate generation."
            )
        if audio_num_frames is None:
            raise ValueError(
                "audio_num_frames must be provided for RoPE coordinate generation."
            )
        perturbation_configs = kwargs.get("perturbation_configs")
        if perturbation_configs is not None and len(perturbation_configs) != batch_size:
            raise ValueError(
                "perturbation_configs length must match batch size, got "
                f"{len(perturbation_configs)=} {batch_size=}."
            )

        if video_coords is None:
            # Wan-style SP-RoPE: when SP is enabled, each rank runs on its local
            # time shard but RoPE positions must be offset to global time.
            #
            # We assume equal time sharding across SP ranks.
            if model_parallel_is_initialized():
                sp_world_size = get_sp_world_size()
                sp_rank = get_sp_parallel_rank()
            else:
                sp_world_size = 1
                sp_rank = 0

            video_shift = int(sp_rank) * int(num_frames) if sp_world_size > 1 else 0
            video_coords = self.rope.prepare_video_coords(

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate perturbation_configs per batch with len == hidden_states.size(0), or pass None to disable perturbation
2. Slice/expand the config list to the current batch size before forward (e.g. [cfg[0]] * batch_size if the config is batch-homogeneous)
3. Add an assert in the batching layer that config count tracks batch size

Example fix

# before
out = model(hidden_states, ..., perturbation_configs=cfgs)  # len(cfgs)=1, batch=4

# after
cfgs = cfgs * hidden_states.size(0)
out = model(hidden_states, ..., perturbation_configs=cfgs)

if perturbation_configs is not None:
    assert len(perturbation_configs) == hidden_states.size(0), 'config/batch mismatch'
    # or: perturbation_configs = perturbation_configs * hidden_states.size(0)

When it happens

Trigger: Calling forward with hidden_states of batch B but a perturbation_configs list of a different length — e.g. batching multiple requests while reusing a single-sample perturbation config, or vice versa.

Common situations: Continuous batching/scheduler changes that resize hidden_states without regenerating perturbation configs; static single-sample configs reused after batch aggregation; off-by-one when slicing configs per chunk.

Related errors

• num_frames/height/width must be provided for RoPE coordinate
• audio_num_frames must be provided for RoPE coordinate genera
• The number of initial states is expected to be equal to the
• Mismatched batch sizes: mixed_qkv.shape[0]={B}, a.shape[0]={
• Mismatched batch sizes: mixed_qkv.shape[0]={B}, a.shape[0]={

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e4bcdb869c0a4f63. Report an issue: GitHub.