sgl-project/sglang · error · ValueError

QwenImage RoPE text cache overflow before denoising: require

Error message

QwenImage RoPE text cache overflow before denoising: required_txt_seq_len={max_txt_seq_len}, txt_cache_len={txt_cache_len}, overflow={overflow}. Please reduce the number of input images, shorten the prompt, or lower the requested resolution.

What it means

QwenImage precomputes RoPE frequency tables (freqs_cis) sized for a text cache based on the resolved sequence lengths. If the actual maximum text sequence length exceeds the cached txt_freqs rows (e.g. because embeddings/conditioning expanded the text length beyond what the rotary embedding cache was allocated for), generation aborts before denoising starts.

Source

Thrown at python/sglang/multimodal_gen/configs/pipeline_configs/qwen_image.py:307

            vae_arch_config.latents_std, device=device
        ).view(1, vae_arch_config.z_dim, 1, 1, 1).to(device, dtype)
        shift_factor = (
            torch.tensor(vae_arch_config.latents_mean)
            .view(1, vae_arch_config.z_dim, 1, 1, 1)
            .to(device, dtype)
        )
        return scaling_factor, shift_factor

    @staticmethod
    def get_freqs_cis(img_shapes, txt_seq_lens, rotary_emb, device, dtype):
        # img_shapes: for global entire image
        img_freqs, txt_freqs = rotary_emb(img_shapes, txt_seq_lens, device=device)

        max_txt_seq_len = max(txt_seq_lens) if txt_seq_lens else 0
        txt_cache_len = int(txt_freqs.shape[0])
        if max_txt_seq_len > txt_cache_len:
            overflow = max_txt_seq_len - txt_cache_len
            raise ValueError(
                "QwenImage RoPE text cache overflow before denoising: "
                f"required_txt_seq_len={max_txt_seq_len}, txt_cache_len={txt_cache_len}, "
                f"overflow={overflow}. "
                "Please reduce the number of input images, shorten the prompt, "
                "or lower the requested resolution."
            )

        # flashinfer RoPE expects a float32 cos/sin cache concatenated on the last dim
        img_cos_half = img_freqs.real.to(dtype=torch.float32).contiguous()
        img_sin_half = img_freqs.imag.to(dtype=torch.float32).contiguous()
        txt_cos_half = txt_freqs.real.to(dtype=torch.float32).contiguous()
        txt_sin_half = txt_freqs.imag.to(dtype=torch.float32).contiguous()

        img_cos_sin_cache = torch.cat([img_cos_half, img_sin_half], dim=-1)
        txt_cos_sin_cache = torch.cat([txt_cos_half, txt_sin_half], dim=-1)
        return img_cos_sin_cache, txt_cos_sin_cache

    def _prepare_cond_kwargs(

View on GitHub (pinned to 0132848349)

Solutions

1. Shorten the prompt text and retry
2. Reduce the number of input condition images
3. Lower the requested output resolution
4. Recompute/invalidate the RoPE cache so it is rebuilt for the current sequence lengths before get_freqs_cis is called

Example fix

// before
prompt = "<2000-token detailed editing instruction>"
images = [img1, img2, img3, img4]

// after
prompt = "Make the sky sunset-colored and add reflections"  # shorter
images = [img1]  # fewer condition images

# Estimate text length before generation
est_tokens = len(tokenizer(prompt).input_ids)
assert est_tokens < txt_cache_budget, (
    f"prompt ~{est_tokens} tokens exceeds text cache budget {txt_cache_budget}")

try:
    freqs = pipe.get_freqs_cis(img_shapes, txt_seq_lens)
except ValueError as e:
    if "text cache overflow" in str(e):
        prompt = summarize(prompt)  # shorten and retry once
        freqs = pipe.get_freqs_cis(img_shapes, [len(tokenizer(prompt).input_ids)])
    else:
        raise

When it happens

Trigger: Very long prompts combined with multiple input images in QwenImage/QwenImageEditPlus; requesting high resolution (which reallocates caches for image tokens and squeezes the text cache); reusing a freqs_cis cache created for a shorter prompt batch on a longer one.

Common situations: Sending a long detailed editing instruction; many condition images increasing reserved sequence budget; the rotary-emb cache sized from a stale/shorter shape after a previous smaller request in the same session.

Related errors

• cos/sin shape does not cover image tokens and head_dim
• QwenImageEditPlus expects either one shared condition image
• Fused QK-Norm + RoPE kernel only supports float16/bfloat16,
• txt_freqs_cis must be a 2D cos_sin_cache tensor
• image_rotary_emb must be cos_sin_cache tensors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3da19e4d5d2cc414. Report an issue: GitHub.