sgl-project/sglang · critical · RuntimeError

FlashAttention combine kernel cannot be implemented with giv

Error message

FlashAttention combine kernel cannot be implemented with given parameters

What it means

The FlashAttention split-KV combine kernel could not be instantiated because the chosen tile/parameter combination (head_dim, tile_m, k_block_size, log_max_splits, num_threads=256) falls outside the implementable envelope. This is an internal configuration constraint check in _compile_fwd_combine, reached via _flash_attn_fwd_combine.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/interface.py:2341

    fa_combine = FlashAttentionForwardCombine(
        dtype=dtype,
        dtype_partial=dtype_partial,
        head_dim=head_dim,
        tile_m=tile_m,
        k_block_size=k_block_size,
        log_max_splits=log_max_splits,
        use_pdl=use_pdl,
    )
    if not fa_combine.can_implement(
        dtype,
        dtype_partial,
        head_dim,
        tile_m,
        k_block_size,
        log_max_splits,
        num_threads=256,
    ):
        raise RuntimeError(
            "FlashAttention combine kernel cannot be implemented with given parameters"
        )

    if has_cu_seqlens:
        # Varlen: (num_splits, total_q, nheads, headdim)
        num_splits, total_q, nheads = sym(), sym(), sym()
        mO_partial = fake_tensor(
            dtype_partial, (num_splits, total_q, nheads, head_dim), divisibility=div
        )
        mLSE_partial = fake_tensor(
            Float32, (num_splits, total_q, nheads), divisibility=1, leading_dim=1
        )
        mO = fake_tensor(dtype, (total_q, nheads, head_dim), divisibility=div)
        mLSE = (
            fake_tensor(Float32, (total_q, nheads), divisibility=1, leading_dim=0)
            if has_lse
            else None
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Report the exact head_dim, seqlen, and split configuration to maintainers — this is a kernel-coverage bug, not user error in most cases.
2. Reduce sequence length or increase batch so fewer KV splits are needed.
3. If you control the wrapper, cap num_splits / lower log_max_splits so the combine kernel fits the envelope.

try:
    _flash_attn_fwd_combine(...)
except RuntimeError as e:
    if "combine kernel cannot be implemented" in str(e):
        # fall back to a kernel/config with fewer KV splits
        run_with_fewer_splits()
    else:
        raise

When it happens

Trigger: The varlen forward runs with an excessive number of KV splits (large log_max_splits) or an unusual head_dim / tile_m combination such that a single thread block cannot cover the combine work with 256 threads; typically triggered by very long sequences with small batch and many splits, or non-standard head dims.

Common situations: Extremely long context lengths causing num_splits beyond what the combine kernel supports; custom head_dim configurations; downstream of automatic split-kv heuristics picking an oversized log_max_splits.

Related errors

• num_heads must be divisible by num_epi_subtiles
• num_heads // num_epi_subtiles must be divisible by 4 (FMA un
• {name}_block_cnt and {name}_block_idx must be on the same de
• {name}_block tensors must live on CUDA
• SplitKV partial output (mO) must be Float32

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9de6ab54ee468682. Report an issue: GitHub.