{"record":{"id":"14b8ece9c183bd1b","repo":"dotnet/machinelearning","slug":"pretrainingtp-1-is-not-supported-yet","errorCode":null,"errorMessage":"PretrainingTp > 1 is not supported yet.","messagePattern":"PretrainingTp > 1 is not supported yet\\.","errorType":"exception","errorClass":"NotImplementedException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.LLaMA/Module/LlamaMLP.cs","lineNumber":53,"sourceCode":"        this._hiddenSize = config.HiddenSize;\n        this._intermediateSize = config.IntermediateSize;\n        this._hasBias = config.MlpBias;\n        this._pretrainingTp = config.PretrainingTp;\n        var hiddenAct = config.HiddenAct;\n        this.gate_proj = new QuantizedLinear(this._hiddenSize, this._intermediateSize, hasBias: this._hasBias, dtype: config.DType);\n        this.up_proj = new QuantizedLinear(this._hiddenSize, this._intermediateSize, hasBias: this._hasBias, dtype: config.DType);\n        this.down_proj = new QuantizedLinear(this._intermediateSize, this._hiddenSize, hasBias: this._hasBias, dtype: config.DType);\n        this.RegisterComponents();\n        this.activation_fn = Core.Utils.GetActivation(hiddenAct);\n    }\n\n#pragma warning disable MSML_GeneralName // This name should be PascalCased\n    public override Tensor forward(Tensor input)\n#pragma warning restore MSML_GeneralName // This name should be PascalCased\n    {\n        if (this._pretrainingTp > 1)\n        {\n            throw new NotImplementedException(\"PretrainingTp > 1 is not supported yet.\");\n        }\n\n        using var input1 = this.gate_proj.forward(input);\n        using var input2 = this.activation_fn.forward(input1);\n        using var input3 = input2 * this.up_proj.forward(input);\n        return this.down_proj.forward(input3);\n    }\n}\n","sourceCodeStart":35,"sourceCodeEnd":62,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.LLaMA/Module/LlamaMLP.cs#L35-L62","documentation":"LlamaMLP.forward explicitly rejects tensor-parallel pretraining configurations greater than 1 by throwing NotImplementedException. The MLP only implements the single-GPU gate/up/down projection path; tensor-parallel sharding of the weight matrices was never implemented in this C# port.","triggerScenarios":"Loading a LLaMA model whose config.json has pretraining_tp > 1 (e.g. many released LLaMA-1/2 checkpoints were trained with tp=8) and then calling LlamaMLP.forward via model forward.","commonSituations":"Downloading a Hugging Face LLaMA checkpoint trained with tensor parallelism and running it without overriding pretraining_tp to 1 in the model config.","solutions":["Set pretraining_tp to 1 in the model config before loading (e.g. config.PretrainingTp = 1); logits quality is unaffected for inference purposes.","Check your config.json for \"pretraining_tp\": 8 (or other >1) and edit it to 1.","If tensor parallel inference is truly required, use a different runtime (e.g. vLLM/PyTorch) — this library does not support it."],"exampleFix":"// before\nvar config = LlamaConfig.FromFile(\"config.json\"); // pretraining_tp: 8\nvar model = LlamaForCausalLM.FromModelFolder(path, device);\n// after\nvar config = LlamaConfig.FromFile(\"config.json\");\nconfig.PretrainingTp = 1;\nvar model = LlamaForCausalLM.FromModelFolder(path, device);","handlingStrategy":"validation","validationCode":"if (config.PretrainingTp > 1) throw new InvalidOperationException(\"Set pretraining_tp=1; tensor parallel inference is unsupported.\");","typeGuard":"bool SupportsTensorParallel(LlamaConfig c) => c.PretrainingTp <= 1;","tryCatchPattern":"try { output = model.forward(input); } catch (NotImplementedException) { /* reconfigure PretrainingTp=1 and retry, or surface unsupported-feature error */ }","preventionTips":["Check pretraining_tp in config.json when downloading LLaMA checkpoints","Override config.PretrainingTp = 1 right after loading config","Do not expect tensor-parallel inference from this library"],"tags":["not-implemented","tensor-parallel","llama"],"backgroundTag":"method-not-implemented","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}