sgl-project/sglang · error · ValueError

refiner cu_seqlens live text length must be in [1, {int(prom

Error message

refiner cu_seqlens live text length must be in [1, {int(prompt_embeds.shape[0])}], got {text_len}

What it means

refine_prompt_embeds reads text_len = refiner_cu_seqlens[1] (the live text length in the cumulative-sequence-lengths tensor) and requires 1 <= text_len <= prompt_embeds.shape[0]. Outside that range the row slice prompt_embeds[:text_len] would be empty or out of bounds, so it raises with the valid range and actual value.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:2143

    def _psp_optional_field(psp: Any, field: str) -> Any:
        if isinstance(psp, dict):
            return psp.get(field)
        return getattr(psp, field, None)

    def refine_prompt_embeds(
        self,
        prompt_embeds: torch.Tensor,
        refiner_cu_seqlens: torch.Tensor,
        *,
        device: torch.device,
    ) -> torch.Tensor:
        """Project and refine request-static text conditioning once."""
        self.materialize_mps_non_layer_weights(
            "condition_proj", "token_refiner.final_norm"
        )
        text_len = int(refiner_cu_seqlens[1].item())
        if text_len <= 0 or text_len > int(prompt_embeds.shape[0]):
            raise ValueError(
                "refiner cu_seqlens live text length must be in "
                f"[1, {int(prompt_embeds.shape[0])}], got {text_len}"
            )
        text_rows = prompt_embeds[:text_len].to(device=device, dtype=_BF16_DTYPE)
        true_refiner_cu = torch.stack(
            (
                refiner_cu_seqlens[0],
                refiner_cu_seqlens[1],
                refiner_cu_seqlens[1],
            )
        )
        text_embed, _ = self.condition_proj(text_rows)
        refined = self.token_refiner(
            text_embed,
            cu_seqlens=true_refiner_cu,
            cu_seqlens_host=(0, text_len, text_len),
            max_seqlen=text_len,
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Recompute refiner_cu_seqlens from the same prompt tokens that produced prompt_embeds
2. Assert 1 <= int(refiner_cu_seqlens[1]) <= prompt_embeds.shape[0] before calling
3. Filter out empty-text requests before they reach the refiner

Example fix

# before
refine_prompt_embeds(prompt_embeds=emb[:10], refiner_cu_seqlens=torch.tensor([0, 25]))
# after
text_len = int(cu[1].item())
assert 1 <= text_len <= emb.shape[0]
refine_prompt_embeds(prompt_embeds=emb, refiner_cu_seqlens=cu)

text_len = int(refiner_cu_seqlens[1].item())
assert 1 <= text_len <= int(prompt_embeds.shape[0]), (text_len, prompt_embeds.shape)

def cu_seqlens_valid(cu: torch.Tensor, n_rows: int) -> bool:
    t = int(cu[1].item())
    return 1 <= t <= n_rows

When it happens

Trigger: Passing refiner_cu_seqlens like [0, 0, ...] (zero live text rows) or [0, N+5] where N is prompt_embeds' row count; mismatch between the cu_seqlens built by the tokenizer/batcher and the truncated prompt_embeds tensor.

Common situations: Truncating or padding prompt_embeds after cu_seqlens was computed; batching bugs that pair a batch's cu_seqlens with the wrong embeds tensor; empty-text requests reaching the refiner.

Related errors

• Every extra_key should be a string.
• The length of cache_salt should be equal to the batch size.
• lora_path list length ({len(self.lora_path)}) must match bat
• unsupported input for causal Conv3D cat/pad CUDA
• unsupported input for usp_merge_heads CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8a9ada989f904f92. Report an issue: GitHub.