{"record":{"id":"d715fc8c86c21110","repo":"2noise/ChatTTS","slug":"unsupported-activation-hidden-act-only-silu-is","errorCode":null,"errorMessage":"Unsupported activation: {hidden_act}. Only silu is supported for now.","messagePattern":"Unsupported activation: (.+?)\\. Only silu is supported for now\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ChatTTS/model/velocity/llama.py","lineNumber":80,"sourceCode":"    def __init__(\n        self,\n        hidden_size: int,\n        intermediate_size: int,\n        hidden_act: str,\n        linear_method: Optional[LinearMethodBase] = None,\n    ) -> None:\n        super().__init__()\n        self.gate_up_proj = MergedColumnParallelLinear(\n            hidden_size,\n            [intermediate_size] * 2,\n            bias=False,\n            linear_method=linear_method,\n        )\n        self.down_proj = RowParallelLinear(\n            intermediate_size, hidden_size, bias=False, linear_method=linear_method\n        )\n        if hidden_act != \"silu\":\n            raise ValueError(\n                f\"Unsupported activation: {hidden_act}. \"\n                \"Only silu is supported for now.\"\n            )\n        self.act_fn = SiluAndMul()\n\n    def forward(self, x):\n        gate_up, _ = self.gate_up_proj(x)\n        x = self.act_fn(gate_up)\n        x, _ = self.down_proj(x)\n        return x\n\n\nclass LlamaAttention(nn.Module):\n\n    def __init__(\n        self,\n        hidden_size: int,\n        num_heads: int,","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/2noise/ChatTTS/blob/77b89ee281cd479f5b1a787ada330dc975ca1f2a/ChatTTS/model/velocity/llama.py#L62-L98","documentation":"This fork's LlamaMLP only implements SiluAndMul as the activation function. If hidden_act in the model's config.json is anything other than 'silu' (gelu, relu, etc.), the MLP constructor raises during model build. The check is a hard gate right before instantiating self.act_fn.","triggerScenarios":"Loading any HF checkpoint whose config.json hidden_act != 'silu' through this velocity engine (e.g. gelu_new LLaMA variants, older GPT-style configs, edited configs).","commonSituations":"Pointing the engine at a non-LLaMA or modified LLaMA checkpoint; merging/quantizing tools rewriting hidden_act; using a fine-tune saved with a different activation string like 'gelu_pytorch_tanh'.","solutions":["Use a checkpoint whose config.json has hidden_act set to silu (standard LLaMA-family models).","If you own the model and it genuinely uses another activation, you must implement it here - add the activation class and relax the check; there is no runtime workaround.","Check for accidental config corruption: diff the checkpoint's config.json against the upstream base model's."],"exampleFix":"// config.json before\n{\"hidden_act\": \"gelu_new\"}\n\n// after\n{\"hidden_act\": \"silu\"}","handlingStrategy":"validation","validationCode":"import json\n\ndef is_silu_model(model_path):\n    cfg = json.load(open(f'{model_path}/config.json'))\n    return cfg.get('hidden_act') == 'silu'","typeGuard":null,"tryCatchPattern":"try:\n    model = get_model(model_config)\nexcept ValueError as e:\n    if 'Unsupported activation' in str(e):\n        raise SystemExit(f'{model_config.model} uses a non-silu activation; this engine only supports silu LLaMA checkpoints')\n    raise","preventionTips":["Pre-flight check config.json hidden_act == 'silu' before loading.","Only feed this engine stock LLaMA-family checkpoints."],"tags":["activation","silu","llama","model-config"],"backgroundTag":"unsupported-model-config-value","analyzedSha":"77b89ee281cd479f5b1a787ada330dc975ca1f2a","analyzedAt":"2026-08-26T17:48:24.233Z","schemaVersion":2},"datasetVersion":"2026-08-26T21:11:00.512Z"}