dotnet/machinelearning · error · NotSupportedException

Activation function not supported.

Error message

Activation function {name} not supported.

What it means

ActivationFunction is a switch over known activation names (relu, gelu, gelu_fast, tanh, linear); any other name falls through to a NotSupportedException. The library only implements these activations for NasBERT modules, so unknown or misspelled configuration values are rejected at module construction time.

Solutions

1. Set the activation name to one of the supported strings: "relu", "gelu", "gelu_fast", "tanh", "linear".
2. Normalize the incoming config value to lowercase and trim whitespace before constructing the module.
3. If another activation is genuinely needed, add a case to the switch in ActivationFunction's constructor implementing it with TorchSharp primitives.
4. Wrap module construction in try-catch on NotSupportedException to report the unsupported name clearly.

Example fix

// before
options.Activation = "gelu-fast"; // NotSupportedException

// after
options.Activation = "gelu_fast"; // supported name

var allowed = new[] { "relu", "gelu", "gelu_fast", "tanh", "linear" };
if (!allowed.Contains(options.Activation?.Trim().ToLowerInvariant()))
    throw new ArgumentException($"Unsupported activation '{options.Activation}'. Use one of: {string.Join(", ", allowed)}.");

try { var act = new ActivationFunction(name, dropout); }
catch (NotSupportedException ex) { log.LogError(ex, "Unsupported activation: {Name}", name); throw new ConfigValidationException(...); }

When it happens

Trigger: Constructing ActivationFunction (directly or via a NasBERT/Roberta options object where Activation is read from config) with a name other than "relu", "gelu", "gelu_fast", "tanh", or "linear" — e.g. "sigmoid", "GELU" (wrong case), or "gelu-fast".

Common situations: Copy-pasting activation names from other frameworks (PyTorch/tensorflow configs) whose supported sets differ, misspelling the name in an ML.NET options class, or case mismatch since the match is case-sensitive.

Related errors

• No Onnx Session Options
• num_key_value_heads must be specified
• num_key_value_heads must be specified
• A PrimitiveDataViewType cannot have a disposable RawType
• All cross validation folds have empty train or test data…

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/e4b51061ea868a45. Report an issue: GitHub.

Appendix: source

Thrown at src/Microsoft.ML.TorchSharp/NasBert/Modules/ActivationFunction.cs:28

namespace Microsoft.ML.TorchSharp.NasBert.Modules
{

    internal sealed class ActivationFunction : torch.nn.Module<torch.Tensor, torch.Tensor>
    {
        private readonly torch.nn.Module<torch.Tensor, torch.Tensor> _function;
        private bool _disposedValue;

        public ActivationFunction(string name) : base(name)
        {
            _function = name?.ToLower() switch
            {
                "relu" => torch.nn.ReLU(),
                "gelu" => torch.nn.GELU(),
                "gelu_fast" => new GeLUFast(),
                "tanh" => torch.nn.Tanh(),
                "linear" => torch.nn.Identity(),
                _ => throw new NotSupportedException($"Activation function {name} not supported.")
            };
        }

        [System.Diagnostics.CodeAnalysis.SuppressMessage("Naming", "MSML_GeneralName:This name should be PascalCased", Justification = "Need to match TorchSharp.")]
        public override torch.Tensor forward(torch.Tensor x)
        {
            return _function.forward(x);
        }

        public override string GetName()
        {
            return _function.GetName();
        }

        protected override void Dispose(bool disposing)
        {
            if (!_disposedValue)
            {

View on GitHub (pinned to 7b76e69cf9)