{"record":{"id":"b2c736bc6fde03d3","repo":"sgl-project/sglang","slug":"not-support-activation-func-activation-func","errorCode":null,"errorMessage":"Not support activation_func: {activation_func}","messagePattern":"Not support activation_func: (.+?)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/kimi_k3_vl.py","lineNumber":772,"sourceCode":"\n        self.patch_embed = MoonVision3dPatchEmbed(\n            out_dim=hidden_size,\n            patch_size=config.patch_size,\n            pos_emb_height=config.init_pos_emb_height,\n            pos_emb_width=config.init_pos_emb_width,\n            pos_emb_time=config.init_pos_emb_time,\n            pos_emb_type=config.pos_emb_type,\n            pos_emb_interpolation_mode=config.pos_emb_interpolation_mode,\n            patch_embed_proj_bias=getattr(config, \"patch_embed_proj_bias\", True),\n        )\n\n        activation_func = getattr(config, \"activation_func\", \"gelu_pytorch_tanh\")\n        if activation_func == \"gelu_pytorch_tanh\":\n            activation = lambda x: F.gelu(x, approximate=\"tanh\")\n        elif activation_func == \"gelu\":\n            activation = F.gelu\n        else:\n            raise NotImplementedError(f\"Not support activation_func: {activation_func}\")\n\n        self.encoder = MoonViT3dEncoder(\n            hidden_dim=hidden_size,\n            num_layers=num_layers,\n            block_cfg={\n                \"num_heads\": num_heads,\n                \"hidden_dim\": hidden_size,\n                \"qkv_hidden_size\": getattr(config, \"qkv_hidden_size\", None),\n                \"mlp_dim\": intermediate_size,\n                \"norm_type\": getattr(config, \"norm_type\", \"layernorm\"),\n                \"activation\": activation,\n                \"attn_bias\": getattr(config, \"attn_bias\", True),\n                \"linear_bias\": getattr(config, \"linear_bias\", True),\n            },\n        )\n        self.cuda_graph_runner = None\n\n    @property","sourceCodeStart":754,"sourceCodeEnd":790,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/kimi_k3_vl.py#L754-L790","documentation":"The Kimi-K3 vision tower's MLP activation is chosen from config.activation_func with a default of gelu_pytorch_tanh; only \"gelu_pytorch_tanh\" and \"gelu\" are implemented (kimi_k3_vl.py:772). Any other activation string raises NotImplementedError when the tower is built.","triggerScenarios":"A checkpoint config with vision activation_func set to e.g. \"silu\", \"gelu_new\", or \"swiglu\" being loaded.","commonSituations":"Finetuned checkpoints that changed the vision MLP activation; config files copied from a different ViT family; typos in activation_func.","solutions":["Check config.activation_func in the checkpoint and align it to gelu or gelu_pytorch_tanh if the weights really use standard GELU","Add an elif branch mapping the needed activation to a torch callable if the checkpoint genuinely uses it","Use the official checkpoint config"],"exampleFix":"// before\nraise NotImplementedError(f\"Not support activation_func: {activation_func}\")\n// after\nelif activation_func == \"silu\":\n    activation = F.silu","handlingStrategy":"validation","validationCode":"af = cfg.get(\"activation_func\", \"gelu_pytorch_tanh\")\nassert af in {\"gelu\", \"gelu_pytorch_tanh\"}, af","typeGuard":"def is_supported_activation(name: str) -> bool:\n    return name in {\"gelu\", \"gelu_pytorch_tanh\"}","tryCatchPattern":null,"preventionTips":["Treat any non-default activation_func as a checkpoint incompatibility signal"],"tags":["kimi-k3","vision","activation","config-validation"],"backgroundTag":"unsupported-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}