2noise/ChatTTS · error · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported for now.

What it means

This fork's LlamaMLP only implements SiluAndMul as the activation function. If hidden_act in the model's config.json is anything other than 'silu' (gelu, relu, etc.), the MLP constructor raises during model build. The check is a hard gate right before instantiating self.act_fn.

Source

Thrown at ChatTTS/model/velocity/llama.py:80

    def __init__(
        self,
        hidden_size: int,
        intermediate_size: int,
        hidden_act: str,
        linear_method: Optional[LinearMethodBase] = None,
    ) -> None:
        super().__init__()
        self.gate_up_proj = MergedColumnParallelLinear(
            hidden_size,
            [intermediate_size] * 2,
            bias=False,
            linear_method=linear_method,
        )
        self.down_proj = RowParallelLinear(
            intermediate_size, hidden_size, bias=False, linear_method=linear_method
        )
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only silu is supported for now."
            )
        self.act_fn = SiluAndMul()

    def forward(self, x):
        gate_up, _ = self.gate_up_proj(x)
        x = self.act_fn(gate_up)
        x, _ = self.down_proj(x)
        return x


class LlamaAttention(nn.Module):

    def __init__(
        self,
        hidden_size: int,
        num_heads: int,

View on GitHub (pinned to 77b89ee281)

Solutions

1. Use a checkpoint whose config.json has hidden_act set to silu (standard LLaMA-family models).
2. If you own the model and it genuinely uses another activation, you must implement it here - add the activation class and relax the check; there is no runtime workaround.
3. Check for accidental config corruption: diff the checkpoint's config.json against the upstream base model's.

Example fix

// config.json before
{"hidden_act": "gelu_new"}

// after
{"hidden_act": "silu"}

import json

def is_silu_model(model_path):
    cfg = json.load(open(f'{model_path}/config.json'))
    return cfg.get('hidden_act') == 'silu'

try:
    model = get_model(model_config)
except ValueError as e:
    if 'Unsupported activation' in str(e):
        raise SystemExit(f'{model_config.model} uses a non-silu activation; this engine only supports silu LLaMA checkpoints')
    raise

When it happens

Trigger: Loading any HF checkpoint whose config.json hidden_act != 'silu' through this velocity engine (e.g. gelu_new LLaMA variants, older GPT-style configs, edited configs).

Common situations: Pointing the engine at a non-LLaMA or modified LLaMA checkpoint; merging/quantizing tools rewriting hidden_act; using a fine-tune saved with a different activation string like 'gelu_pytorch_tanh'.

AI-assisted analysis of 2noise/ChatTTS@77b89ee281 (2026-08-26). Data as JSON: /api/errors/d715fc8c86c21110. Report an issue: GitHub.