sgl-project/sglang · critical · ImportError

flash-attn is not installed. Please install it, e.g., `pip i

Error message

flash-attn is not installed. Please install it, e.g., `pip install flash-attn`.

What it means

Raised by the _unsupported stub that replaces flash_attn_varlen_func / _flash_attn_varlen_forward / _flash_attn_varlen_backward when the flash-attn package fails to import. The vmoba.py module needs flash-attn's varlen kernels to run MoBA attention, so any call to these functions without flash-attn installed raises an ImportError.

Source

Thrown at python/sglang/multimodal_gen/csrc/attn/vmoba_attn/vmoba/vmoba.py:21

import random
import time
from typing import Tuple

import torch

try:
    from flash_attn import (  # Use the new flash attention function
        flash_attn_varlen_func,
    )
    from flash_attn.flash_attn_interface import (
        _flash_attn_varlen_backward,
        _flash_attn_varlen_forward,
    )
except ImportError:

    def _unsupported(*args, **kwargs):
        raise ImportError(
            "flash-attn is not installed. Please install it, e.g., `pip install flash-attn`."
        )

    _flash_attn_varlen_forward = _unsupported
    _flash_attn_varlen_backward = _unsupported
    flash_attn_varlen_func = _unsupported

from functools import lru_cache

from einops import rearrange


@lru_cache(maxsize=16)
def calc_chunks(cu_seqlen, moba_chunk_size):
    """
    Calculate chunk boundaries.

    For vision tasks we include all chunks (even the last one which might be shorter)

View on GitHub (pinned to 0132848349)

Solutions

1. pip install flash-attn --no-build-isolation (match your torch/CUDA version)
2. Verify import works: python -c "from flash_attn import flash_attn_varlen_func"; if it fails, check torch/CUDA ABI compatibility
3. Install a prebuilt wheel matching your torch/CUDA (e.g. from the flash-attn GitHub releases) instead of building from source
4. If flash-attn is unavailable, use a non-MoBA / fallback attention path that does not route through vmoba.py

Example fix

# before: ImportError: flash-attn is not installed...
out = flash_attn_varlen_func(q, k, v, cu_seqlens_q, cu_seqlens_k, max_seqlen_q, max_seqlen_k)
# after: pip install flash-attn --no-build-isolation, then same call works
# or guard:
try:
    out = flash_attn_varlen_func(...)
except ImportError:
    out = fallback_attention(q, k, v)

try:
    from flash_attn import flash_attn_varlen_func  # noqa: F401
    HAS_FLASH_ATTN = True
except ImportError:
    HAS_FLASH_ATTN = False

if not HAS_FLASH_ATTN:
    raise RuntimeError("flash-attn required for MoBA attention; pip install flash-attn --no-build-isolation")

try:
    out = vmoba.flash_attn_varlen_func(...)
except ImportError as e:
    if "flash-attn is not installed" in str(e):
        out = fallback_varlen_attention(q, k, v, cu_seqlens_q, cu_seqlens_k)
    else:
        raise

When it happens

Trigger: Calling moba_attn_varlen or anything that invokes flash_attn_varlen_func, _flash_attn_varlen_forward, or _flash_attn_varlen_backward in python/sglang/multimodal_gen/csrc/attn/vmoba_attn/vmoba/vmoba.py when `from flash_attn import ...` raised ImportError (package missing or broken install, e.g. wrong CUDA/torch ABI).

Common situations: New environment without flash-attn; installing flash-attn compiled against a different torch or CUDA version so import fails; running on CPUs or machines without a compatible GPU toolchain.

Related errors

• {name}_block_cnt and {name}_block_idx must be on the same de
• {name}_block tensors must live on CUDA
• FlashAttention combine kernel cannot be implemented with giv
• hd256 forward varlen expects q rank 3 or 5, got rank {q_rank
• hd256 forward non-varlen expects q rank 4 or 5, got rank {q_

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0d2dfa38d879d430. Report an issue: GitHub.