sgl-project/sglang · error · ValueError

Unable to infer Z-Image caption length for rotary embeddings

Error message

Unable to infer Z-Image caption length for rotary embeddings

What it means

When building rotary embedding caches, Z-Image needs the caption sequence length. _caption_rope_length tries several inference strategies (stored text seq lens, prompt-embed shapes) and, when none apply, raises this error. It means the batch state carries neither per-request text seq lens nor inspectable prompt embedding tensors with a usable shape.

Source

Thrown at python/sglang/multimodal_gen/configs/pipeline_configs/zimage.py:274

                    negative=negative,
                    expected_batch_size=int(prompt_embeds.shape[0]),
                )
                return max(seq_lens) if seq_lens else int(prompt_embeds.shape[1])

        if isinstance(prompt_embeds, (list, tuple)) and prompt_embeds:
            first = prompt_embeds[0]
            if torch.is_tensor(first):
                if first.ndim == 3:
                    seq_lens = self.require_text_seq_lens(
                        batch,
                        0,
                        negative=negative,
                        expected_batch_size=int(first.shape[0]),
                    )
                    return max(seq_lens) if seq_lens else int(first.shape[1])
                return max(int(item.shape[0]) for item in prompt_embeds)

        raise ValueError("Unable to infer Z-Image caption length for rotary embeddings")

    def get_pos_prompt_embeds(self, batch):
        return self._split_text_embeds_for_dit(batch, negative=False)

    def get_neg_prompt_embeds(self, batch):
        return self._split_text_embeds_for_dit(batch, negative=True)

    def get_latent_dtype(self, prompt_dtype: torch.dtype) -> torch.dtype:
        # Match the official diffusers Z-Image pipeline, which samples latents in fp32
        # and keeps scheduler state in fp32.
        return torch.float32

    def shard_latents_for_sp(self, batch, latents):
        sp_size = get_sp_world_size()
        if sp_size <= 1 or latents.dim() != 5:
            return latents, False

        plan = self._get_zimage_sp_plan(batch)

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the batch includes non-empty text prompt embeds or stored text seq lens (require_text_seq_lens data) before calling get_freqs_cis
2. If constructing batches programmatically, populate the text seq lens field so length inference succeeds
3. Pass prompts/embeddings with valid [seq, dim] or [batch, seq, dim] shapes so the max(len) fallback works
4. Check upstream logs for an earlier failure that left prompt_embeds empty

if not batch.text_prompt_embeds and not getattr(batch, "text_seq_lens", None):
    raise RuntimeError("batch lacks prompt embeds and text seq lens; cannot infer caption length")

def batch_has_caption_length(batch) -> bool:
    pe = getattr(batch, "text_prompt_embeds", None) or getattr(batch, "prompt_embeds", None)
    return bool(getattr(batch, "text_seq_lens", None)) or (pe is not None and len(pe) > 0 and all(hasattr(x, "shape") for x in pe))

When it happens

Trigger: Calling get_freqs_cis (directly or via prepare_pos_cond_kwargs / prepare_neg_cond_kwargs) on a batch whose prompt embeds are empty/None or have unexpected structure, so max seq length cannot be derived from text seq lens or from prompt_embeds[i].shape[0].

Common situations: Running with an empty prompt list; a batch assembled manually without text_seq_lens metadata; a code path that strips or forgets to populate prompt embeds before conditioning kwargs are prepared; regressions after batch-state refactors.

Related errors

• Z-Image text embeddings must have shape [seq, dim] or [batch
• Z-Image transformer has no `rotary_emb`. It likely loaded vi
• Z-Image caption tensor must have rank 2 or 3
• cos/sin shape does not cover image tokens and head_dim
• Unsupported image type: {type(image)}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/10f66cf97a5f654e. Report an issue: GitHub.