dotnet/machinelearning · error · NotSupportedException

Activation function {name} not supported.

Error message

Activation function {name} not supported.

What it means

ActivationFunction is a switch over known activation names (relu, gelu, gelu_fast, tanh, linear); any other name falls through to a NotSupportedException. The library only implements these activations for NasBERT modules, so unknown or misspelled configuration values are rejected at module construction time.

Source

Thrown at src/Microsoft.ML.TorchSharp/NasBert/Modules/ActivationFunction.cs:28

namespace Microsoft.ML.TorchSharp.NasBert.Modules
{

    internal sealed class ActivationFunction : torch.nn.Module<torch.Tensor, torch.Tensor>
    {
        private readonly torch.nn.Module<torch.Tensor, torch.Tensor> _function;
        private bool _disposedValue;

        public ActivationFunction(string name) : base(name)
        {
            _function = name?.ToLower() switch
            {
                "relu" => torch.nn.ReLU(),
                "gelu" => torch.nn.GELU(),
                "gelu_fast" => new GeLUFast(),
                "tanh" => torch.nn.Tanh(),
                "linear" => torch.nn.Identity(),
                _ => throw new NotSupportedException($"Activation function {name} not supported.")
            };
        }

        [System.Diagnostics.CodeAnalysis.SuppressMessage("Naming", "MSML_GeneralName:This name should be PascalCased", Justification = "Need to match TorchSharp.")]
        public override torch.Tensor forward(torch.Tensor x)
        {
            return _function.forward(x);
        }

        public override string GetName()
        {
            return _function.GetName();
        }

        protected override void Dispose(bool disposing)
        {
            if (!_disposedValue)
            {

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Set the activation name to one of the supported strings: "relu", "gelu", "gelu_fast", "tanh", "linear".
2. Normalize the incoming config value to lowercase and trim whitespace before constructing the module.
3. If another activation is genuinely needed, add a case to the switch in ActivationFunction's constructor implementing it with TorchSharp primitives.
4. Wrap module construction in try-catch on NotSupportedException to report the unsupported name clearly.

Example fix

// before
options.Activation = "gelu-fast"; // NotSupportedException

// after
options.Activation = "gelu_fast"; // supported name

var allowed = new[] { "relu", "gelu", "gelu_fast", "tanh", "linear" };
if (!allowed.Contains(options.Activation?.Trim().ToLowerInvariant()))
    throw new ArgumentException($"Unsupported activation '{options.Activation}'. Use one of: {string.Join(", ", allowed)}.");

try { var act = new ActivationFunction(name, dropout); }
catch (NotSupportedException ex) { log.LogError(ex, "Unsupported activation: {Name}", name); throw new ConfigValidationException(...); }

When it happens

Trigger: Constructing ActivationFunction (directly or via a NasBERT/Roberta options object where Activation is read from config) with a name other than "relu", "gelu", "gelu_fast", "tanh", or "linear" — e.g. "sigmoid", "GELU" (wrong case), or "gelu-fast".

Common situations: Copy-pasting activation names from other frameworks (PyTorch/tensorflow configs) whose supported sets differ, misspelling the name in an ML.NET options class, or case mismatch since the match is case-sensitive.

Related errors

• num_key_value_heads must be specified
• num_key_value_heads must be specified
• No Onnx Session Options
• {nameof(ChannelMessageKind)}.{e.Kind} is not yet implemented
• Training failed with the exception: {_history.Last().Excepti

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/e4b51061ea868a45. Report an issue: GitHub.