sgl-project/sglang · error · ValueError

Unsupported LTX-2 RoPE type: {attn.rope_type}

Error message

Unsupported LTX-2 RoPE type: {attn.rope_type}

What it means

streaming_self_attention applies LTX-2 rotary embeddings only for rope_type values 'interleaved' (via apply_interleaved_rotary_emb — shown as the branch above) and 'split' (apply_split_rotary_emb). Any other attn.rope_type string on the attention config raises this ValueError.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/streaming_refiner.py:166

    gate_logits = (
        _to_gate_logits(hidden_states) if _to_gate_logits is not None else None
    )
    query = attn.to_q(hidden_states)
    key = attn.to_k(hidden_states)
    value = attn.to_v(hidden_states)
    query = attn.norm_q(query)
    key = attn.norm_k(key)
    if getattr(attn, "_kv_cache_capture", False):
        attn._cached_kv_pre = (key.detach().clone(), value.detach().clone())

    if attn.rope_type == "interleaved":
        query = apply_interleaved_rotary_emb(query, query_rotary_emb)
        key = apply_interleaved_rotary_emb(key, query_rotary_emb)
    elif attn.rope_type == "split":
        query = apply_split_rotary_emb(query, query_rotary_emb)
        key = apply_split_rotary_emb(key, query_rotary_emb)
    else:
        raise ValueError(f"Unsupported LTX-2 RoPE type: {attn.rope_type}")
    if getattr(attn, "_tf_capture_kv", False):
        attn._cached_kv_post = (key.detach().clone(), value.detach().clone())

    tf_prefix = getattr(attn, "_tf_kv_prefix", None)
    if isinstance(tf_prefix, dict) and tf_prefix.get("mode") == "rf_shifted_sink":
        prefix_k_parts = []
        prefix_v_parts = []
        sink_k_pre = tf_prefix.get("sink_k_pre")
        sink_v = tf_prefix.get("sink_v")
        if sink_k_pre is not None and sink_v is not None and sink_k_pre.shape[1] > 0:
            sink_pe = tf_prefix.get("sink_pe")
            if sink_pe is None:
                raise RuntimeError("rf_shifted_sink prefix requires sink_pe")
            if attn.rope_type == "interleaved":
                sink_k = apply_interleaved_rotary_emb(sink_k_pre.to(key.dtype), sink_pe)
            else:
                sink_k = apply_split_rotary_emb(sink_k_pre.to(key.dtype), sink_pe)
            prefix_k_parts.append(sink_k)

View on GitHub (pinned to 0132848349)

Solutions

1. Set rope_type to 'interleaved' or 'split' in the transformer attention config to match the checkpoint's actual RoPE layout
2. Use the config shipped with the supported checkpoint unmodified
3. If a genuinely new RoPE variant is needed, implement it in this function and register the mode

Example fix

# before
attn.rope_type = "half"  # unsupported
# after
attn.rope_type = "split"  # or "interleaved", per checkpoint config

assert attn.rope_type in ("interleaved", "split"), attn.rope_type

def supported_rope_type(attn) -> bool:
    return getattr(attn, "rope_type", None) in ("interleaved", "split")

null

When it happens

Trigger: Loading an LTX-2-family transformer whose attention config carries an unrecognized rope_type (e.g. a new checkpoint using 'half_rope', 'none', or a renamed value), or hand-editing the config's rope_type field.

Common situations: New model revision changes rope naming; mixing configs between model variants; config-generation scripts defaulting rope_type to an unsupported value.

Related errors

• Expected x.shape[-1] to be even for split rotary, got {last}
• {rope_type=} not supported. Choose between 'interleaved' and
• Unexpected RoPE rank: {cos.ndim}
• SANA-WM refiner requires a string prompt or one prompt per b
• Stage-1 latent has {z.shape[2]} frames but sink_size={sink_s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0a4617019f232a84. Report an issue: GitHub.