sgl-project/sglang · error · ValueError

Krea-2 sequence parallelism does not support ragged/padded m

Error message

Krea-2 sequence parallelism does not support ragged/padded multi-prompt batches; use a single prompt or --tp-size.

What it means

Krea-2's sequence-parallel (SP) path shards image RoPE positions across SP ranks while keeping the text prefix replicated (via num_replicated_prefix). The masked-text path (used for ragged/padded multi-prompt batches) is incompatible with replicated-prefix all-to-all, so the pipeline rejects any batch where the text mask contains padding/multiple prompts.

Source

Thrown at python/sglang/multimodal_gen/configs/pipeline_configs/krea2.py:89

        patch = self.dit_config.arch_config.patch
        vsf = self.get_vae_scale_factor()
        h_tok = int(batch.height) // vsf // patch
        w_tok = int(batch.width) // vsf // patch

        img_ids = torch.zeros(h_tok, w_tok, 3, device=device)
        img_ids[..., 1] = torch.arange(h_tok, device=device)[:, None]
        img_ids[..., 2] = torch.arange(w_tok, device=device)[None, :]
        img_pos = img_ids.reshape(h_tok * w_tok, 3).unsqueeze(0).expand(b, -1, -1)
        txt_pos = torch.zeros(b, txt_len, 3, device=device)

        sp_world_size = get_sp_world_size()
        if sp_world_size > 1:
            # Shard the image RoPE positions to match the denoise stage's latent
            # sharding; the text prefix stays replicated (kept out of the all-to-all
            # via num_replicated_prefix). The masked path is incompatible with
            # replicated-prefix, so ragged multi-prompt batches aren't supported.
            if text_mask is not None and not bool(text_mask.all()):
                raise ValueError(
                    "Krea-2 sequence parallelism does not support ragged/padded "
                    "multi-prompt batches; use a single prompt or --tp-size."
                )
            img_pos = self._shard_img_pos_for_sp(img_pos, sp_world_size)
            return {"pos": torch.cat([txt_pos, img_pos], dim=1), "mask": None}

        pos = torch.cat([txt_pos, img_pos], dim=1)
        img_mask = torch.ones(b, h_tok * w_tok, dtype=torch.bool, device=device)
        if text_mask is None:
            txt_mask = torch.ones(b, txt_len, dtype=torch.bool, device=device)
        else:
            txt_mask = text_mask.to(device=device).bool()
        mask = torch.cat([txt_mask, img_mask], dim=1)
        return {"pos": pos, "mask": mask}

    @staticmethod
    def _shard_img_pos_for_sp(img_pos, sp_world_size):
        # This rank's contiguous image-position slice, padded to a multiple of

View on GitHub (pinned to 0132848349)

Solutions

1. Use a single prompt per batch when SP is enabled (one prompt per request)
2. Disable/avoid sequence parallelism for multi-prompt batches — use tensor parallelism (--tp-size) instead
3. If batching is required, ensure all prompts tokenize to identical lengths so no padding mask is produced (text_mask.all() is True)
4. Check that the server was not started with an SP size > 1 for this model if you need ragged batches

Example fix

# before
prompts = ["a cat", "a much longer prompt about a dog"]  # padded batch with --sp-size 2

# after
# one prompt per generate call with SP enabled
image = pipe(prompt="a cat")
# or launch server with --tp-size 2 instead of --sp-size 2 for batching

sp = get_sp_world_size()
if sp > 1:
    assert len(prompts) == 1 or all_equal_tokenized_lengths(prompts, tokenizer), \
        "Krea-2 SP does not support ragged multi-prompt batches"

except ValueError as e:
    if "ragged/padded multi-prompt" in str(e):
        # fall back to single-prompt requests or TP
        for p in prompts: run_single(p)

When it happens

Trigger: Running Krea-2 generation with --sp-size > 1 (sp_world_size > 1) and a batch containing multiple prompts or padded text, i.e. text_mask is not None and not all True — for example passing a list of prompts of differing lengths so the tokenizer pads them, hitting _build_pos_and_mask via prepare_pos_cond_kwargs/prepare_neg_cond_kwargs.

Common situations: Switching from single-prompt to multi-prompt or batched generation on a multi-GPU SP deployment; enabling sequence parallelism on an existing batched workload; using reference-image duplication with several prompts while SP is on.

Related errors

• num_inference_steps must be positive, got {steps}
• LTX-2 SP time-sharding for packed token latents currently re
• Expected {seq_len=} > 0 for packed token latents.
• SANA-WM does not support temporal sequence parallelism yet.
• Every extra_key should be a string.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/825774dce0a17a75. Report an issue: GitHub.