sgl-project/sglang · error · ValueError

Unsupported Spark2_5 layer_type: {layer_type}

Error message

Unsupported Spark2_5 layer_type: {layer_type}

What it means

Spark2_5 attention layers must be typed 'sliding_attention' or 'full_attention'; anything else in the layer's config raises at init. The type decides whether RadixAttention gets the sliding window or full attention.

Source

Thrown at python/sglang/srt/models/spark2_5.py:172

            hidden_size,
            bias=False,
            quant_config=quant_config,
            tp_rank=attn_tp_rank,
            tp_size=attn_tp_size,
            prefix=add_prefix("out_proj", prefix),
        )

        self.rotary_emb = get_rope(
            self.head_dim,
            rotary_dim=self.head_dim,
            max_position=max_position_embeddings,
            base=rope_theta,
            rope_scaling=None,
            partial_rotary_factor=partial_rotary_factor,
            is_neox_style=True,
        )
        if layer_type not in ("sliding_attention", "full_attention"):
            raise ValueError(f"Unsupported Spark2_5 layer_type: {layer_type}")
        sliding_window_size = (
            sliding_window if layer_type == "sliding_attention" else -1
        )
        self.attn = RadixAttention(
            self.num_heads,
            self.head_dim,
            self.scaling,
            num_kv_heads=self.num_kv_heads,
            sliding_window_size=sliding_window_size,
            layer_id=layer_id,
            quant_config=quant_config,
            prefix=add_prefix("attn", prefix),
        )

    def forward(
        self,
        positions: torch.Tensor,
        hidden_states: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Update sglang to a release supporting the checkpoint's layer scheme
2. Inspect config.json layer_types array and fix invalid entries to sliding_attention/full_attention
3. If it's a genuinely new type, patch the check in spark2_5.py to handle it (mapping to an appropriate window)

Example fix

// config.json
// before: "layer_types": ["sliding", "full_attention"]
// after:  "layer_types": ["sliding_attention", "full_attention"]

valid = {"sliding_attention", "full_attention"}
assert set(config.layer_types) <= valid, set(config.layer_types) - valid

When it happens

Trigger: A layer entry in the Spark2.5 checkpoint whose layer_type string is neither of the two supported values (typo, new interleaving scheme, or malformed config.json).

Common situations: Newer Spark2.5 checkpoints introducing new layer types on an older sglang; hand-edited or merged config.json corrupting layer_type arrays.

Related errors

• This browser cannot encode H.264 MP4
• H.264 encoder did not return MP4 decoder config
• This browser does not support gzip stream decoding
• delta payload size mismatch: expected ${expectedSize}, got $
• Unsupported activation: {self.activation}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/430bd770b200b73c. Report an issue: GitHub.