sgl-project/sglang · error · ValueError

The hpc_ops attention backend does not support logit cap.

Error message

The hpc_ops attention backend does not support logit cap.

What it means

The hpc_ops attention backend raises this when a RadixAttention layer is configured with a positive logit cap (layer.logit_cap > 0). The underlying HPC-Ops attention kernels do not implement logit capping (a soft cap applied to pre-softmax logits), so the backend refuses to run such layers rather than silently producing wrong results. It is thrown from _check_layer_supported on every forward_extend/forward_decode call.

Source

Thrown at python/sglang/srt/layers/attention/hpc_ops_backend.py:347

        )
        if self._decode_task_map is not None:
            # Recorded into the decode graph, so the task map is re-populated
            # from the live seq_lens at every replay.
            metadata.hpc_task_map = self._assign_decode_tasks(
                metadata.cache_seqlens_int32
            )
        self.forward_metadata = metadata

    def get_cuda_graph_seq_len_fill_value(self) -> int:
        return 1

    def _check_layer_supported(self, layer: RadixAttention):
        if layer.sliding_window_size is not None and layer.sliding_window_size > -1:
            raise ValueError(
                "The hpc_ops attention backend does not support sliding window."
            )
        if layer.logit_cap > 0:
            raise ValueError(
                "The hpc_ops attention backend does not support logit cap."
            )
        # The HPC-Ops kernels hard-code the softmax scale to head_dim ** -0.5.
        if abs(layer.scaling - layer.head_dim**-0.5) > 1e-8:
            raise ValueError(
                "The hpc_ops attention backend only supports the default "
                f"softmax scaling head_dim ** -0.5, got {layer.scaling}."
            )

    def _paged_kv_buffers(self, layer: RadixAttention):
        k_cache, v_cache = self.token_to_kv_pool.get_kv_buffer(layer.layer_id)
        k_cache = k_cache.view(-1, self.page_size, layer.tp_k_head_num, layer.head_dim)
        v_cache = v_cache.view(-1, self.page_size, layer.tp_v_head_num, layer.head_dim)
        return k_cache, v_cache

    def _layer_kv_scales(self, layer: RadixAttention):
        """Per-tensor K/V scales as fp32 [1] tensors (ones when absent)."""
        k_scale = (

View on GitHub (pinned to 0132848349)

Solutions

  1. Switch to an attention backend that supports logit cap (e.g. flashinfer or fa3/triton) for this model
  2. Verify the model config (attn_logit_softcapping / logit_cap) before selecting hpc_ops; use hpc_ops only for models without softcapping
  3. If you control the model definition, set logit_cap=0 in the attention layers (only if the model mathematically does not need capping)

Example fix

# before
python -m sglang.launch_server --model gemma-3-27b-it --attention-backend hpc_ops
# after
python -m sglang.launch_server --model gemma-3-27b-it --attention-backend flashinfer
Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.layers.radix_attention import RadixAttention

def hpc_ops_compatible(layer) -> bool:
    return layer.logit_cap <= 0

assert all(hpc_ops_compatible(l) for l in model_layers), 'logit cap unsupported by hpc_ops'

Type guard

def uses_logit_cap(layer: RadixAttention) -> bool:
    return layer.logit_cap is not None and layer.logit_cap > 0

Try / catch

try:
    backend.forward_extend(...)
except ValueError as e:
    if 'logit cap' in str(e):
        switch_attention_backend('flashinfer')
    else:
        raise

Prevention

When it happens

Trigger: Serving a model whose attention layers set logit_cap > 0 (e.g. Gemma-2/Gemma-3 style models with --attention-backend hpc_ops, or any config.json with attn_logit_softcapping). The check fires on the first forward pass of an affected layer.

Common situations: User selects --attention-backend hpc_ops for performance on a model that uses attention logit softcapping; or a new model is added whose config enables softcapping and the backend choice is not updated.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0975bb516b52186c. Report an issue: GitHub.