sgl-project/sglang · error · ValueError

QwenImage RoPE text cache overflow before denoising…

Error message

QwenImage RoPE text cache overflow before denoising: required_txt_seq_len={max_txt_seq_len}, txt_cache_len={txt_cache_len}, overflow={overflow}. Please reduce the number of input images, shorten the prompt, or lower the requested resolution.

What it means

QwenImage precomputes RoPE frequency tables (freqs_cis) sized for a text cache based on the resolved sequence lengths. If the actual maximum text sequence length exceeds the cached txt_freqs rows (e.g. because embeddings/conditioning expanded the text length beyond what the rotary embedding cache was allocated for), generation aborts before denoising starts.

Solutions

1. Shorten the prompt text and retry
2. Reduce the number of input condition images
3. Lower the requested output resolution
4. Recompute/invalidate the RoPE cache so it is rebuilt for the current sequence lengths before get_freqs_cis is called

Example fix

// before
prompt = "<2000-token detailed editing instruction>"
images = [img1, img2, img3, img4]

// after
prompt = "Make the sky sunset-colored and add reflections"  # shorter
images = [img1]  # fewer condition images

# Estimate text length before generation
est_tokens = len(tokenizer(prompt).input_ids)
assert est_tokens < txt_cache_budget, (
    f"prompt ~{est_tokens} tokens exceeds text cache budget {txt_cache_budget}")

try:
    freqs = pipe.get_freqs_cis(img_shapes, txt_seq_lens)
except ValueError as e:
    if "text cache overflow" in str(e):
        prompt = summarize(prompt)  # shorten and retry once
        freqs = pipe.get_freqs_cis(img_shapes, [len(tokenizer(prompt).input_ids)])
    else:
        raise

When it happens

Trigger: Very long prompts combined with multiple input images in QwenImage/QwenImageEditPlus; requesting high resolution (which reallocates caches for image tokens and squeezes the text cache); reusing a freqs_cis cache created for a shorter prompt batch on a longer one.

Common situations: Sending a long detailed editing instruction; many condition images increasing reserved sequence budget; the rotary-emb cache sized from a stale/shorter shape after a previous smaller request in the same session.

Related errors

• cos/sin shape does not cover image tokens and head_dim
• Fused QK-Norm + RoPE kernel only supports float16/bfloat16…
• image_rotary_emb must be cos_sin_cache tensors
• max_new_tokens must leave room for input: max_new_tokens=
• memory_position_mode must be one of

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3da19e4d5d2cc414. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/multimodal_gen/configs/pipeline_configs/qwen_image.py:307

            vae_arch_config.latents_std, device=device
        ).view(1, vae_arch_config.z_dim, 1, 1, 1).to(device, dtype)
        shift_factor = (
            torch.tensor(vae_arch_config.latents_mean)
            .view(1, vae_arch_config.z_dim, 1, 1, 1)
            .to(device, dtype)
        )
        return scaling_factor, shift_factor

    @staticmethod
    def get_freqs_cis(img_shapes, txt_seq_lens, rotary_emb, device, dtype):
        # img_shapes: for global entire image
        img_freqs, txt_freqs = rotary_emb(img_shapes, txt_seq_lens, device=device)

        max_txt_seq_len = max(txt_seq_lens) if txt_seq_lens else 0
        txt_cache_len = int(txt_freqs.shape[0])
        if max_txt_seq_len > txt_cache_len:
            overflow = max_txt_seq_len - txt_cache_len
            raise ValueError(
                "QwenImage RoPE text cache overflow before denoising: "
                f"required_txt_seq_len={max_txt_seq_len}, txt_cache_len={txt_cache_len}, "
                f"overflow={overflow}. "
                "Please reduce the number of input images, shorten the prompt, "
                "or lower the requested resolution."
            )

        # flashinfer RoPE expects a float32 cos/sin cache concatenated on the last dim
        img_cos_half = img_freqs.real.to(dtype=torch.float32).contiguous()
        img_sin_half = img_freqs.imag.to(dtype=torch.float32).contiguous()
        txt_cos_half = txt_freqs.real.to(dtype=torch.float32).contiguous()
        txt_sin_half = txt_freqs.imag.to(dtype=torch.float32).contiguous()

        img_cos_sin_cache = torch.cat([img_cos_half, img_sin_half], dim=-1)
        txt_cos_sin_cache = torch.cat([txt_cos_half, txt_sin_half], dim=-1)
        return img_cos_sin_cache, txt_cos_sin_cache

    def _prepare_cond_kwargs(

View on GitHub (pinned to 0132848349)