{"record":{"id":"0a4617019f232a84","repo":"sgl-project/sglang","slug":"unsupported-ltx-2-rope-type-attn-rope-type-0a4617","errorCode":null,"errorMessage":"Unsupported LTX-2 RoPE type: {attn.rope_type}","messagePattern":"Unsupported LTX-2 RoPE type: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/streaming_refiner.py","lineNumber":166,"sourceCode":"    gate_logits = (\n        _to_gate_logits(hidden_states) if _to_gate_logits is not None else None\n    )\n    query = attn.to_q(hidden_states)\n    key = attn.to_k(hidden_states)\n    value = attn.to_v(hidden_states)\n    query = attn.norm_q(query)\n    key = attn.norm_k(key)\n    if getattr(attn, \"_kv_cache_capture\", False):\n        attn._cached_kv_pre = (key.detach().clone(), value.detach().clone())\n\n    if attn.rope_type == \"interleaved\":\n        query = apply_interleaved_rotary_emb(query, query_rotary_emb)\n        key = apply_interleaved_rotary_emb(key, query_rotary_emb)\n    elif attn.rope_type == \"split\":\n        query = apply_split_rotary_emb(query, query_rotary_emb)\n        key = apply_split_rotary_emb(key, query_rotary_emb)\n    else:\n        raise ValueError(f\"Unsupported LTX-2 RoPE type: {attn.rope_type}\")\n    if getattr(attn, \"_tf_capture_kv\", False):\n        attn._cached_kv_post = (key.detach().clone(), value.detach().clone())\n\n    tf_prefix = getattr(attn, \"_tf_kv_prefix\", None)\n    if isinstance(tf_prefix, dict) and tf_prefix.get(\"mode\") == \"rf_shifted_sink\":\n        prefix_k_parts = []\n        prefix_v_parts = []\n        sink_k_pre = tf_prefix.get(\"sink_k_pre\")\n        sink_v = tf_prefix.get(\"sink_v\")\n        if sink_k_pre is not None and sink_v is not None and sink_k_pre.shape[1] > 0:\n            sink_pe = tf_prefix.get(\"sink_pe\")\n            if sink_pe is None:\n                raise RuntimeError(\"rf_shifted_sink prefix requires sink_pe\")\n            if attn.rope_type == \"interleaved\":\n                sink_k = apply_interleaved_rotary_emb(sink_k_pre.to(key.dtype), sink_pe)\n            else:\n                sink_k = apply_split_rotary_emb(sink_k_pre.to(key.dtype), sink_pe)\n            prefix_k_parts.append(sink_k)","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/streaming_refiner.py#L148-L184","documentation":"streaming_self_attention applies LTX-2 rotary embeddings only for rope_type values 'interleaved' (via apply_interleaved_rotary_emb — shown as the branch above) and 'split' (apply_split_rotary_emb). Any other attn.rope_type string on the attention config raises this ValueError.","triggerScenarios":"Loading an LTX-2-family transformer whose attention config carries an unrecognized rope_type (e.g. a new checkpoint using 'half_rope', 'none', or a renamed value), or hand-editing the config's rope_type field.","commonSituations":"New model revision changes rope naming; mixing configs between model variants; config-generation scripts defaulting rope_type to an unsupported value.","solutions":["Set rope_type to 'interleaved' or 'split' in the transformer attention config to match the checkpoint's actual RoPE layout","Use the config shipped with the supported checkpoint unmodified","If a genuinely new RoPE variant is needed, implement it in this function and register the mode"],"exampleFix":"# before\nattn.rope_type = \"half\"  # unsupported\n# after\nattn.rope_type = \"split\"  # or \"interleaved\", per checkpoint config","handlingStrategy":"validation","validationCode":"assert attn.rope_type in (\"interleaved\", \"split\"), attn.rope_type","typeGuard":"def supported_rope_type(attn) -> bool:\n    return getattr(attn, \"rope_type\", None) in (\"interleaved\", \"split\")","tryCatchPattern":"null","preventionTips":["Ship and use the checkpoint's original attention config","Validate rope_type at model load for LTX-2-family transformers","Extend streaming_self_attention before introducing new RoPE variants"],"tags":["sana-wm","ltx2","rope","attention-config","valueerror"],"backgroundTag":"unsupported-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}