{"record":{"id":"10f66cf97a5f654e","repo":"sgl-project/sglang","slug":"unable-to-infer-z-image-caption-length-for-rotary","errorCode":null,"errorMessage":"Unable to infer Z-Image caption length for rotary embeddings","messagePattern":"Unable to infer Z-Image caption length for rotary embeddings","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/configs/pipeline_configs/zimage.py","lineNumber":274,"sourceCode":"                    negative=negative,\n                    expected_batch_size=int(prompt_embeds.shape[0]),\n                )\n                return max(seq_lens) if seq_lens else int(prompt_embeds.shape[1])\n\n        if isinstance(prompt_embeds, (list, tuple)) and prompt_embeds:\n            first = prompt_embeds[0]\n            if torch.is_tensor(first):\n                if first.ndim == 3:\n                    seq_lens = self.require_text_seq_lens(\n                        batch,\n                        0,\n                        negative=negative,\n                        expected_batch_size=int(first.shape[0]),\n                    )\n                    return max(seq_lens) if seq_lens else int(first.shape[1])\n                return max(int(item.shape[0]) for item in prompt_embeds)\n\n        raise ValueError(\"Unable to infer Z-Image caption length for rotary embeddings\")\n\n    def get_pos_prompt_embeds(self, batch):\n        return self._split_text_embeds_for_dit(batch, negative=False)\n\n    def get_neg_prompt_embeds(self, batch):\n        return self._split_text_embeds_for_dit(batch, negative=True)\n\n    def get_latent_dtype(self, prompt_dtype: torch.dtype) -> torch.dtype:\n        # Match the official diffusers Z-Image pipeline, which samples latents in fp32\n        # and keeps scheduler state in fp32.\n        return torch.float32\n\n    def shard_latents_for_sp(self, batch, latents):\n        sp_size = get_sp_world_size()\n        if sp_size <= 1 or latents.dim() != 5:\n            return latents, False\n\n        plan = self._get_zimage_sp_plan(batch)","sourceCodeStart":256,"sourceCodeEnd":292,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/configs/pipeline_configs/zimage.py#L256-L292","documentation":"When building rotary embedding caches, Z-Image needs the caption sequence length. _caption_rope_length tries several inference strategies (stored text seq lens, prompt-embed shapes) and, when none apply, raises this error. It means the batch state carries neither per-request text seq lens nor inspectable prompt embedding tensors with a usable shape.","triggerScenarios":"Calling get_freqs_cis (directly or via prepare_pos_cond_kwargs / prepare_neg_cond_kwargs) on a batch whose prompt embeds are empty/None or have unexpected structure, so max seq length cannot be derived from text seq lens or from prompt_embeds[i].shape[0].","commonSituations":"Running with an empty prompt list; a batch assembled manually without text_seq_lens metadata; a code path that strips or forgets to populate prompt embeds before conditioning kwargs are prepared; regressions after batch-state refactors.","solutions":["Ensure the batch includes non-empty text prompt embeds or stored text seq lens (require_text_seq_lens data) before calling get_freqs_cis","If constructing batches programmatically, populate the text seq lens field so length inference succeeds","Pass prompts/embeddings with valid [seq, dim] or [batch, seq, dim] shapes so the max(len) fallback works","Check upstream logs for an earlier failure that left prompt_embeds empty"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if not batch.text_prompt_embeds and not getattr(batch, \"text_seq_lens\", None):\n    raise RuntimeError(\"batch lacks prompt embeds and text seq lens; cannot infer caption length\")","typeGuard":"def batch_has_caption_length(batch) -> bool:\n    pe = getattr(batch, \"text_prompt_embeds\", None) or getattr(batch, \"prompt_embeds\", None)\n    return bool(getattr(batch, \"text_seq_lens\", None)) or (pe is not None and len(pe) > 0 and all(hasattr(x, \"shape\") for x in pe))","tryCatchPattern":null,"preventionTips":["Populate text seq lens on programmatically built batches","Never call prepare_*_cond_kwargs with empty prompt lists","Add a startup smoke test that runs one real prompt through get_freqs_cis"],"tags":["z-image","rotary-embeddings","batch-state","multimodal"],"backgroundTag":"missing-required-metadata","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}