{"record":{"id":"575e78b1ae8a389c","repo":"dotnet/machinelearning","slug":"rope-type-not-implemented","errorCode":null,"errorMessage":"Rope type not implemented","messagePattern":"Rope type not implemented","errorType":"exception","errorClass":"NotImplementedException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Core/Module/RotaryEmbedding.cs","lineNumber":97,"sourceCode":"        : this(baseValue, dim, new RopeScalingConfig() { RopeType = \"default\", OriginalMaxPositionEmbeddings = maxPositionEmbeddings })\n    {\n    }\n\n    public RotaryEmbedding(double baseValue, int dim, RopeScalingConfig config)\n        : base(nameof(RotaryEmbedding))\n    {\n        _base = baseValue;\n        _maxPositionEmbeddings = config.OriginalMaxPositionEmbeddings;\n        _dim = dim;\n\n        if (config.RopeType == \"default\")\n        {\n            var thetaNumerator = torch.arange(0, _dim, 2, dtype: ScalarType.Int64).to(torch.float32);\n            this.register_buffer(\"inv_freq\", torch.pow(baseValue, -1.0f * (thetaNumerator / dim)), persistent: false);\n        }\n        else\n        {\n            throw new NotImplementedException(\"Rope type not implemented\");\n        }\n    }\n\n    public int Dim => _dim;\n\n#pragma warning disable MSML_GeneralName // This name should be PascalCased\n    public override RotaryEmbeddingOutput forward(RotaryEmbeddingInput input)\n#pragma warning restore MSML_GeneralName // This name should be PascalCased\n    {\n        var x = input.Input;\n        var positionIds = input.PositionIds;\n        var seqLen = input.SeqLen;\n        // TODO\n        // can be calculated once and cached\n        var invFreq = this.get_buffer(\"inv_freq\").to(x.device);\n        var invFreqExpanded = invFreq.unsqueeze(0).unsqueeze(-1);\n        invFreqExpanded = invFreqExpanded.expand(new long[] { positionIds.shape[0], -1, 1 });\n        var positionIdsExpanded = positionIds.unsqueeze(1).to(torch.float32);","sourceCodeStart":79,"sourceCodeEnd":115,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Core/Module/RotaryEmbedding.cs#L79-L115","documentation":"RotaryEmbedding's constructor supports only a specific rope type (the linear/default theta scaling branch). Any other configured rope type (e.g. llama3 scaling, yarn variants not implemented) hits the else branch and throws NotImplementedException.","triggerScenarios":"Instantiating a model whose config specifies rope_type/rope_scaling other than the implemented linear type, e.g. loading a checkpoint with llama3-style rope scaling or dynamic/none types.","commonSituations":"Loading a newer model checkpoint (e.g. Llama 3.x with rope_scaling metadata) with an older GenAI.Core that only implements basic rope; porting configs that include rope_scaling blocks.","solutions":["Remove or normalize the rope_scaling section in the model config to use the supported type","Upgrade Microsoft.ML.GenAI to a version implementing the required rope type","Implement the missing rope type in RotaryEmbedding's constructor in a fork"],"exampleFix":"// before\nconfig.json: \"rope_scaling\": { \"rope_type\": \"llama3\", \"factor\": 8.0 }\n// after\nconfig.json: \"rope_scaling\": null  // or omit; use default linear rope","handlingStrategy":"validation","validationCode":"var ropeType = config[\"rope_scaling\"]?[\"rope_type\"]?.ToString() ?? \"default\";\nif (ropeType is not (\"default\" or \"linear\")) throw new NotSupportedException($\"Rope type {ropeType} not supported\");","typeGuard":null,"tryCatchPattern":"try { var model = pipeline.Load(modelPath); } catch (NotImplementedException ex) when (ex.Message.Contains(\"Rope\")) { // strip rope_scaling from config.json and retry\n}","preventionTips":["Inspect config.json rope_scaling before loading a checkpoint","Upgrade to a GenAI version that supports the model's rope type","Normalize configs of newer checkpoints (llama3 scaling) before use"],"tags":["genai","rope","not-implemented"],"backgroundTag":"unsupported-operation","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}