{"record":{"id":"0975bb516b52186c","repo":"sgl-project/sglang","slug":"the-hpc-ops-attention-backend-does-not-support-log","errorCode":null,"errorMessage":"The hpc_ops attention backend does not support logit cap.","messagePattern":"The hpc_ops attention backend does not support logit cap\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/hpc_ops_backend.py","lineNumber":347,"sourceCode":"        )\n        if self._decode_task_map is not None:\n            # Recorded into the decode graph, so the task map is re-populated\n            # from the live seq_lens at every replay.\n            metadata.hpc_task_map = self._assign_decode_tasks(\n                metadata.cache_seqlens_int32\n            )\n        self.forward_metadata = metadata\n\n    def get_cuda_graph_seq_len_fill_value(self) -> int:\n        return 1\n\n    def _check_layer_supported(self, layer: RadixAttention):\n        if layer.sliding_window_size is not None and layer.sliding_window_size > -1:\n            raise ValueError(\n                \"The hpc_ops attention backend does not support sliding window.\"\n            )\n        if layer.logit_cap > 0:\n            raise ValueError(\n                \"The hpc_ops attention backend does not support logit cap.\"\n            )\n        # The HPC-Ops kernels hard-code the softmax scale to head_dim ** -0.5.\n        if abs(layer.scaling - layer.head_dim**-0.5) > 1e-8:\n            raise ValueError(\n                \"The hpc_ops attention backend only supports the default \"\n                f\"softmax scaling head_dim ** -0.5, got {layer.scaling}.\"\n            )\n\n    def _paged_kv_buffers(self, layer: RadixAttention):\n        k_cache, v_cache = self.token_to_kv_pool.get_kv_buffer(layer.layer_id)\n        k_cache = k_cache.view(-1, self.page_size, layer.tp_k_head_num, layer.head_dim)\n        v_cache = v_cache.view(-1, self.page_size, layer.tp_v_head_num, layer.head_dim)\n        return k_cache, v_cache\n\n    def _layer_kv_scales(self, layer: RadixAttention):\n        \"\"\"Per-tensor K/V scales as fp32 [1] tensors (ones when absent).\"\"\"\n        k_scale = (","sourceCodeStart":329,"sourceCodeEnd":365,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/hpc_ops_backend.py#L329-L365","documentation":"The hpc_ops attention backend raises this when a RadixAttention layer is configured with a positive logit cap (layer.logit_cap > 0). The underlying HPC-Ops attention kernels do not implement logit capping (a soft cap applied to pre-softmax logits), so the backend refuses to run such layers rather than silently producing wrong results. It is thrown from _check_layer_supported on every forward_extend/forward_decode call.","triggerScenarios":"Serving a model whose attention layers set logit_cap > 0 (e.g. Gemma-2/Gemma-3 style models with --attention-backend hpc_ops, or any config.json with attn_logit_softcapping). The check fires on the first forward pass of an affected layer.","commonSituations":"User selects --attention-backend hpc_ops for performance on a model that uses attention logit softcapping; or a new model is added whose config enables softcapping and the backend choice is not updated.","solutions":["Switch to an attention backend that supports logit cap (e.g. flashinfer or fa3/triton) for this model","Verify the model config (attn_logit_softcapping / logit_cap) before selecting hpc_ops; use hpc_ops only for models without softcapping","If you control the model definition, set logit_cap=0 in the attention layers (only if the model mathematically does not need capping)"],"exampleFix":"# before\npython -m sglang.launch_server --model gemma-3-27b-it --attention-backend hpc_ops\n# after\npython -m sglang.launch_server --model gemma-3-27b-it --attention-backend flashinfer","handlingStrategy":"validation","validationCode":"from sglang.srt.layers.radix_attention import RadixAttention\n\ndef hpc_ops_compatible(layer) -> bool:\n    return layer.logit_cap <= 0\n\nassert all(hpc_ops_compatible(l) for l in model_layers), 'logit cap unsupported by hpc_ops'","typeGuard":"def uses_logit_cap(layer: RadixAttention) -> bool:\n    return layer.logit_cap is not None and layer.logit_cap > 0","tryCatchPattern":"try:\n    backend.forward_extend(...)\nexcept ValueError as e:\n    if 'logit cap' in str(e):\n        switch_attention_backend('flashinfer')\n    else:\n        raise","preventionTips":["Check model config for attn_logit_softcapping before choosing --attention-backend hpc_ops","Maintain a per-model supported-backend matrix in CI","Prefer backend auto-selection over hard-coding hpc_ops"],"tags":["attention-backend","hpc-ops","logit-cap","unsupported-feature","sglang"],"backgroundTag":"unsupported-attention-backend-feature","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}