sgl-project/sglang · error · ValueError

Cross attention is not supported in the hpc_ops attention ba

Error message

Cross attention is not supported in the hpc_ops attention backend.

What it means

The hpc_ops backend implements self-attention only; encoder-decoder models (runner.model_config.is_encoder_decoder true) also need cross-attention over encoder KV, which hpc_ops does not provide. The factory fails fast at backend creation rather than crashing mid-forward.

Source

Thrown at python/sglang/srt/layers/attention/attention_registry.py:265

    return CutlassMLABackend(runner)


@register_attention_backend("trtllm_mha")
def create_trtllm_mha_backend(runner):
    if runner.use_mla_backend:
        raise ValueError("trtllm_mha backend can only be used with non-MLA models.")
    from sglang.srt.layers.attention.trtllm_mha_backend import TRTLLMHAAttnBackend

    return TRTLLMHAAttnBackend(runner)


@register_attention_backend("hpc_ops")
def create_hpc_ops_backend(runner):
    if runner.use_mla_backend:
        raise ValueError("hpc_ops backend can only be used with non-MLA models.")
    if runner.model_config.is_encoder_decoder:
        raise ValueError(
            "Cross attention is not supported in the hpc_ops attention backend."
        )
    if get_spec().speculative_algorithm is not None:
        raise ValueError(
            "hpc_ops backend does not support speculative decoding for now."
        )
    from sglang.srt.layers.attention.hpc_ops_backend import HPCOpsAttnBackend

    return HPCOpsAttnBackend(runner)


@register_attention_backend("intel_amx")
def create_intel_amx_backend(runner):
    from sglang.srt.layers.attention.intel_amx_backend import IntelAMXAttnBackend

    return IntelAMXAttnBackend(runner)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the hpc_ops backend override and use a backend that supports cross-attention (e.g. the default/fallback attention backend)
2. Keep encoder-decoder models on their default auto-selected backend

Example fix

# before
--attention-backend hpc_ops --model whisper-large
# after
--model whisper-large  # default backend handles cross-attention

if model_runner.model_config.is_encoder_decoder and server_args.attention_backend == "hpc_ops":
    raise SystemExit("hpc_ops has no cross-attention; use the default backend")

def is_encoder_decoder(cfg) -> bool:
    return bool(getattr(cfg, "is_encoder_decoder", False))

When it happens

Trigger: Launching an encoder-decoder model (Whisper, T5-style, encoder-decoder VLMs) with --attention-backend hpc_ops.

Common situations: Using hpc_ops as a fast NPU backend but pointing the server at an encoder-decoder checkpoint; config templates that set a fixed attention backend across heterogeneous models.

Related errors

• hpc_ops backend can only be used with non-MLA models.
• hpc_ops backend does not support speculative decoding for no
• Short-conv hybrid models (ZAYA1 CCA, LFM2 / LFM2-MoE) are no
• The hpc_ops attention backend does not support logit cap.
• The hpc_ops attention backend only supports the default soft

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b6231b1c37410fa4. Report an issue: GitHub.