sgl-project/sglang · error · ValueError

f"Unknown feature map: {feature_map}"

Error message

f"Unknown feature map: {feature_map}"

What it means

SparseLinearAttention's constructor accepts only a fixed set of feature map names ('relu' and 'softmax' among others). Any other string passed as feature_map reaches the else branch and raises ValueError naming the unsupported value.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/sparse_linear_attn.py:151

        # Learnable linear projection for combining sparse + linear attention
        self.proj_l = nn.Linear(head_size, head_size, dtype=torch.float32)

        # Feature map for linear attention
        # Type annotation for callables
        self.feature_map_q: Callable[[torch.Tensor], torch.Tensor]
        self.feature_map_k: Callable[[torch.Tensor], torch.Tensor]
        if feature_map == "elu":
            self.feature_map_q = lambda x: F.elu(x) + 1
            self.feature_map_k = lambda x: F.elu(x) + 1
        elif feature_map == "relu":
            self.feature_map_q = F.relu
            self.feature_map_k = F.relu
        elif feature_map == "softmax":
            self.feature_map_q = lambda x: F.softmax(x, dim=-1)
            self.feature_map_k = lambda x: F.softmax(x, dim=-1)
        else:
            raise ValueError(f"Unknown feature map: {feature_map}")

        self._init_weights()

    def _init_weights(self) -> None:
        """Initialize projection weights to zero for residual-like behavior."""
        with torch.no_grad():
            nn.init.zeros_(self.proj_l.weight)
            nn.init.zeros_(self.proj_l.bias)  # type: ignore[arg-type]

    def _calc_linear_attention_with_torch(self, q, k, v):
        kv = torch.matmul(k.transpose(-1, -2), v)
        k_sum = torch.sum(k, dim=-2, keepdim=True)
        return torch.matmul(q, kv) / (1e-5 + torch.matmul(q, k_sum.transpose(-1, -2)))

    def forward(
        self,
        query: torch.Tensor,
        key: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of the supported names: feature_map='relu' or feature_map='softmax' (check the full if/elif chain above the raise for other supported options such as identity/exp).
2. Fix casing/typos in the config string, e.g. 'Relu' -> 'relu'.
3. If you need a different feature map, subclass the layer and override feature_map_q/feature_map_k after construction, or extend the if/elif chain with a PR.

Example fix

# before
attn = SparseLinearAttention(..., feature_map="gelu")

# after
attn = SparseLinearAttention(..., feature_map="relu")
# or set custom maps post-init:
# attn.feature_map_q = torch.nn.functional.gelu
# attn.feature_map_k = torch.nn.functional.gelu

SUPPORTED = {"relu", "softmax"}  # mirror the if/elif chain in __init__
assert feature_map in SUPPORTED, f"feature_map must be one of {SUPPORTED}"
attn = SparseLinearAttention(..., feature_map=feature_map)

def is_valid_feature_map(name: str) -> bool:
    return name in {"relu", "softmax"}

When it happens

Trigger: Constructing the sparse linear attention layer with feature_map set to anything other than the supported names, e.g. feature_map='gelu', feature_map='elu', feature_map='sigmoid', or a typo like 'Relu' / 'soft_max'.

Common situations: Porting a config from another codebase (e.g. a Transformer/linear-attention repo that uses 'gelu' feature maps), a casing mismatch, or copy-pasting a YAML/JSON config with an unsupported feature map name.

Related errors

• f"Unsupported patch_size type: {type(patch_size)}"
• Invalid threshold_type for topk: {threshold_type}. Choose 'q
• Invalid threshold_type: {threshold_type}. Choose 'query_head
• n_q/n_k must be one of {VALID_N}, got n_q={n_q}, n_k={n_k}
• batching config rule requires max_batch_size

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5a29df472df3d463. Report an issue: GitHub.