sgl-project/sglang · error · NotImplementedError

CP attention for non-FIA path on Ascend is not yet implement

Error message

CP attention for non-FIA path on Ascend is not yet implemented. Set ASCEND_USE_FIA=1 to use FIA-based CP attention.

What it means

On Ascend NPUs, context-parallel (CP) attention in the extend path is only implemented for the FIA (Flash Infer Ascend-style) algorithm. If CP mode is active and ASCEND_USE_FIA is not enabled, the backend raises NotImplementedError with the exact env var to set.

Source

Thrown at python/sglang/srt/hardware_backend/npu/attention/ascend_backend.py:1403

                        v_cache,
                        sinks,
                        self.forward_metadata.extend_seq_lens,
                        block_tables,
                        self.forward_metadata.seq_lens,
                        layer.scaling,
                        layer.sliding_window_size,
                        layer.tp_q_head_num,
                        layer.tp_k_head_num,
                    )
                return attn_out

            if is_cp_mode:
                if self.use_fia:
                    attn_output = self.do_cp_attn_fia(
                        q, k_cache, v_cache, layer, forward_batch
                    )
                else:
                    raise NotImplementedError(
                        "CP attention for non-FIA path on Ascend is not yet implemented. "
                        "Set ASCEND_USE_FIA=1 to use FIA-based CP attention."
                    )
                return attn_output

            if self.use_fia:
                if self._can_use_tnd(layer):
                    """FIA supports multi-bs in the current version of CANN"""
                    q = q.reshape(-1, layer.tp_q_head_num, layer.qk_head_dim)
                    num_token_padding = q.shape[0]
                    if num_token_padding > forward_batch.num_token_non_padded_cpu:
                        q, k, v = [
                            data[: forward_batch.num_token_non_padded_cpu]
                            for data in [q, k, v]
                        ]
                    attn_output, _ = torch_npu.npu_fused_infer_attention_score(
                        query=q,
                        key=k_cache.view(

View on GitHub (pinned to 0132848349)

Solutions

1. Set ASCEND_USE_FIA=1 in the server environment and restart
2. If FIA is unsuitable for your model, disable context parallelism on Ascend until non-FIA CP is implemented
3. Verify the env var reaches the scheduler/worker processes (not stripped by a launcher)

Example fix

# before
ASCEND_USE_FIA=0 python -m sglang.launch_server ... --cp-size 4
# after
ASCEND_USE_FIA=1 python -m sglang.launch_server ... --cp-size 4

import os
if cp_size > 1 and platform is ascend:
    assert os.environ.get("ASCEND_USE_FIA", "0") == "1", (
        "Ascend CP attention requires ASCEND_USE_FIA=1")

try:
    attn_output = backend.forward_extend(...)
except NotImplementedError as e:
    if "ASCEND_USE_FIA" in str(e):
        raise SystemExit("Restart with ASCEND_USE_FIA=1 or disable --cp-size") from e
    raise

When it happens

Trigger: Running an extend forward with context parallelism enabled on Ascend while self.use_fia is False — i.e. ASCEND_USE_FIA unset/0 — so the non-FIA CP branch is reached.

Common situations: Enabling CP (e.g. large-context deepseek/long-context serving with context parallel) on Ascend without ASCEND_USE_FIA=1; a deployment copied from a non-FIA Ascend config then scaled to CP.

Related errors

• The 'enable-mixed-chunk' feature is currently unsupported in
• NPU packed attention does not support a sequence that is emp
• Ascend PD transfer does not support HiSparse destination dev
• Short-conv hybrid models (ZAYA1 CCA, LFM2 / LFM2-MoE) are no
• {name} is required for NPU packed attention

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/38ac38a74e79e826. Report an issue: GitHub.