{"record":{"id":"84fdc7bffde1029a","repo":"sgl-project/sglang","slug":"inkling-relative-attention-requires-the-vendored-f","errorCode":null,"errorMessage":"Inkling relative attention requires the vendored FA4 CUTE interface.","messagePattern":"Inkling relative attention requires the vendored FA4 CUTE interface\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/inkling_common/attn.py","lineNumber":61,"sourceCode":"\ntry:\n    import cutlass.cute as cute\n    from cutlass.cute import Float32\n\n    from sglang.kernels.ops.attention.flash_attn.cute.seqlen_info import SeqlenInfoQK\nexcept Exception as _import_error:\n    cute = None\n    Float32 = None\n    SeqlenInfoQK = None\n    _cute_import_error = _import_error\nelse:\n    _cute_import_error = None\n\n\n@cache\ndef get_inkling_relative_attention_score_mod(rel_extent: int) -> Callable:\n    if cute is None or Float32 is None or SeqlenInfoQK is None:\n        raise ImportError(\n            \"Inkling relative attention requires the vendored FA4 CUTE interface.\"\n        ) from _cute_import_error\n\n    @cute.jit\n    def score_mod_rel_bias(\n        scores: cute.TensorSSA,\n        b_idx: cute.TensorSSA,\n        h_idx: cute.TensorSSA,\n        q_idx: cute.TensorSSA,\n        kv_idx: cute.TensorSSA,\n        seqlen_info: SeqlenInfoQK,\n        aux_tensors: list[cute.Tensor],\n    ) -> cute.TensorSSA:\n        rel_logits = aux_tensors[0]\n\n        seqlen_local_offset = seqlen_info.seqlen_k - seqlen_info.seqlen_q\n        rel_dist = (q_idx + seqlen_local_offset) - kv_idx\n        global_q_idx = seqlen_info.offset_q + q_idx","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/attn.py#L43-L79","documentation":"Raised when the Inkling relative-attention score_mod is requested but the vendored FlashAttention-4 CUTE (CuTe DSL) interface failed to import (cute, Float32, or SeqlenInfoQK is None). The module caches the import error and chains it into this ImportError, so relative attention bias cannot be built. It indicates the FA4/CuTe dependency or its supported platform is missing in this environment.","triggerScenarios":"Calling forward -> get_inkling_relative_attention_score_mod(rel_extent) on a GPU/config that enables Inkling relative attention when the vendored sgl_kernel FA4 CUTE symbols (cute, Float32, SeqlenInfoQK) were not importable (non-CUDA platform, sgl_kernel built without FA4, old wheel).","commonSituations":"Running an Inkling model with relative attention on CPU or macOS; using a stale sgl-kernel/sglang wheel that predates the vendored FA4 CUTE interface; broken CUDA install; importing after a partial install.","solutions":["Upgrade sgl-kernel (and sglang) to a version that ships the vendored FA4 CUTE interface and reinstall: pip install -U sglang[all] sgl-kernel","Verify a CUDA GPU is visible (torch.cuda.is_available()) and you are on Linux x86_64; the CUTE path is CUDA-only","Disable the Inkling relative-attention feature / use an attention backend that does not require the CUTE score_mod","If building from source, rebuild sgl-kernel with FA4 enabled and check the original _cute_import_error chained below this message"],"exampleFix":"# before: run Inkling with rel attention on CPU / old wheel -> ImportError\n# after: ensure CUDA + updated kernel\npip install -U \"sglang[all]\" sgl-kernel\npython -c \"import torch; assert torch.cuda.is_available()\"","handlingStrategy":"fallback","validationCode":"from sglang.srt.models.inkling_common import attn\ndef cute_available() -> bool:\n    return not (attn.cute is None or attn.Float32 is None or attn.SeqlenInfoQK is None)\nif cute_available():\n    mod = attn.get_inkling_relative_attention_score_mod(extent)","typeGuard":null,"tryCatchPattern":"except ImportError as e: log warning; fall back to a non-relative attention backend or abort with a clear 'requires FA4 CUTE / CUDA' message.","preventionTips":["Pin sglang + sgl-kernel versions that ship the vendored FA4 CUTE interface","Gate relative-attention features on torch.cuda.is_available() in launcher code","Smoke-test the CUTE import at startup before serving traffic"],"tags":["sglang","import-error","flashattention","cute","cuda","inkling"],"backgroundTag":"missing-optional-dependency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}