dotnet/machinelearning · error · NotImplementedException

Rope type not implemented

Error message

Rope type not implemented

What it means

RotaryEmbedding's constructor supports only a specific rope type (the linear/default theta scaling branch). Any other configured rope type (e.g. llama3 scaling, yarn variants not implemented) hits the else branch and throws NotImplementedException.

Source

Thrown at src/Microsoft.ML.GenAI.Core/Module/RotaryEmbedding.cs:97

        : this(baseValue, dim, new RopeScalingConfig() { RopeType = "default", OriginalMaxPositionEmbeddings = maxPositionEmbeddings })
    {
    }

    public RotaryEmbedding(double baseValue, int dim, RopeScalingConfig config)
        : base(nameof(RotaryEmbedding))
    {
        _base = baseValue;
        _maxPositionEmbeddings = config.OriginalMaxPositionEmbeddings;
        _dim = dim;

        if (config.RopeType == "default")
        {
            var thetaNumerator = torch.arange(0, _dim, 2, dtype: ScalarType.Int64).to(torch.float32);
            this.register_buffer("inv_freq", torch.pow(baseValue, -1.0f * (thetaNumerator / dim)), persistent: false);
        }
        else
        {
            throw new NotImplementedException("Rope type not implemented");
        }
    }

    public int Dim => _dim;

#pragma warning disable MSML_GeneralName // This name should be PascalCased
    public override RotaryEmbeddingOutput forward(RotaryEmbeddingInput input)
#pragma warning restore MSML_GeneralName // This name should be PascalCased
    {
        var x = input.Input;
        var positionIds = input.PositionIds;
        var seqLen = input.SeqLen;
        // TODO
        // can be calculated once and cached
        var invFreq = this.get_buffer("inv_freq").to(x.device);
        var invFreqExpanded = invFreq.unsqueeze(0).unsqueeze(-1);
        invFreqExpanded = invFreqExpanded.expand(new long[] { positionIds.shape[0], -1, 1 });
        var positionIdsExpanded = positionIds.unsqueeze(1).to(torch.float32);

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Remove or normalize the rope_scaling section in the model config to use the supported type
2. Upgrade Microsoft.ML.GenAI to a version implementing the required rope type
3. Implement the missing rope type in RotaryEmbedding's constructor in a fork

Example fix

// before
config.json: "rope_scaling": { "rope_type": "llama3", "factor": 8.0 }
// after
config.json: "rope_scaling": null  // or omit; use default linear rope

var ropeType = config["rope_scaling"]?["rope_type"]?.ToString() ?? "default";
if (ropeType is not ("default" or "linear")) throw new NotSupportedException($"Rope type {ropeType} not supported");

try { var model = pipeline.Load(modelPath); } catch (NotImplementedException ex) when (ex.Message.Contains("Rope")) { // strip rope_scaling from config.json and retry
}

When it happens

Trigger: Instantiating a model whose config specifies rope_type/rope_scaling other than the implemented linear type, e.g. loading a checkpoint with llama3-style rope scaling or dynamic/none types.

Common situations: Loading a newer model checkpoint (e.g. Llama 3.x with rope_scaling metadata) with an older GenAI.Core that only implements basic rope; porting configs that include rope_scaling blocks.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• {fieldType.Name}
• joinAlgorithm
• The method or operation is not implemented.
• NotImplementedException
• nameof(T)

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/575e78b1ae8a389c. Report an issue: GitHub.