sgl-project/sglang · error · NotImplementedError

Not support activation_func: {activation_func}

Error message

Not support activation_func: {activation_func}

What it means

The Kimi-K3 vision tower's MLP activation is chosen from config.activation_func with a default of gelu_pytorch_tanh; only "gelu_pytorch_tanh" and "gelu" are implemented (kimi_k3_vl.py:772). Any other activation string raises NotImplementedError when the tower is built.

Source

Thrown at python/sglang/srt/models/kimi_k3_vl.py:772

        self.patch_embed = MoonVision3dPatchEmbed(
            out_dim=hidden_size,
            patch_size=config.patch_size,
            pos_emb_height=config.init_pos_emb_height,
            pos_emb_width=config.init_pos_emb_width,
            pos_emb_time=config.init_pos_emb_time,
            pos_emb_type=config.pos_emb_type,
            pos_emb_interpolation_mode=config.pos_emb_interpolation_mode,
            patch_embed_proj_bias=getattr(config, "patch_embed_proj_bias", True),
        )

        activation_func = getattr(config, "activation_func", "gelu_pytorch_tanh")
        if activation_func == "gelu_pytorch_tanh":
            activation = lambda x: F.gelu(x, approximate="tanh")
        elif activation_func == "gelu":
            activation = F.gelu
        else:
            raise NotImplementedError(f"Not support activation_func: {activation_func}")

        self.encoder = MoonViT3dEncoder(
            hidden_dim=hidden_size,
            num_layers=num_layers,
            block_cfg={
                "num_heads": num_heads,
                "hidden_dim": hidden_size,
                "qkv_hidden_size": getattr(config, "qkv_hidden_size", None),
                "mlp_dim": intermediate_size,
                "norm_type": getattr(config, "norm_type", "layernorm"),
                "activation": activation,
                "attn_bias": getattr(config, "attn_bias", True),
                "linear_bias": getattr(config, "linear_bias", True),
            },
        )
        self.cuda_graph_runner = None

    @property

View on GitHub (pinned to 0132848349)

Solutions

1. Check config.activation_func in the checkpoint and align it to gelu or gelu_pytorch_tanh if the weights really use standard GELU
2. Add an elif branch mapping the needed activation to a torch callable if the checkpoint genuinely uses it
3. Use the official checkpoint config

Example fix

// before
raise NotImplementedError(f"Not support activation_func: {activation_func}")
// after
elif activation_func == "silu":
    activation = F.silu

af = cfg.get("activation_func", "gelu_pytorch_tanh")
assert af in {"gelu", "gelu_pytorch_tanh"}, af

def is_supported_activation(name: str) -> bool:
    return name in {"gelu", "gelu_pytorch_tanh"}

When it happens

Trigger: A checkpoint config with vision activation_func set to e.g. "silu", "gelu_new", or "swiglu" being loaded.

Common situations: Finetuned checkpoints that changed the vision MLP activation; config files copied from a different ViT family; typos in activation_func.

Related errors

• Not support pos_emb_type: {pos_emb_type}
• Not support norm_type: {norm_type}
• Not support merge_type: {self.merge_type}
• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b2c736bc6fde03d3. Report an issue: GitHub.