{"record":{"id":"41fa5eb916a6ef6d","repo":"dotnet/machinelearning","slug":"the-length-of-inputs-and-outputs-should-be-the-sam","errorCode":null,"errorMessage":"The length of inputs and outputs should be the same.","messagePattern":"The length of inputs and outputs should be the same\\.","errorType":"validation","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Core/Trainer/CausalLMDataset.cs","lineNumber":34,"sourceCode":"\npublic class CausalLMDataset : IEnumerable<CausalLMModelInput>\n{\n    private readonly List<CausalLMModelInput> _data;\n\n    private CausalLMDataset(IEnumerable<CausalLMModelInput> data)\n    {\n        _data = new List<CausalLMModelInput>(data);\n    }\n\n    public static CausalLMDataset Create(IEnumerable<IEnumerable<ChatMessage>> inputs,\n        IEnumerable<ChatMessage> outputs,\n        IMEAIChatTemplateBuilder chatTemplateBuilder,\n        Tokenizer tokenizer)\n    {\n        // the length of inputs and outputs should be the same\n        if (inputs.Count() != outputs.Count())\n        {\n            throw new ArgumentException(\"The length of inputs and outputs should be the same.\");\n        }\n\n        var enumerables = inputs.Zip(outputs, (input, output) =>\n        {\n            var inputPrompt = chatTemplateBuilder.BuildPrompt(input.ToList());\n            var outputPrompt = chatTemplateBuilder.BuildPrompt(input.Concat([output]).ToList(), appendAssistantTag: false);\n            var lengthToKeep = outputPrompt.Length - inputPrompt.Length;\n            outputPrompt = outputPrompt.Substring(inputPrompt.Length, lengthToKeep);\n\n            return (inputPrompt, outputPrompt);\n        });\n\n        return Create(enumerables.Select(x => x.inputPrompt), enumerables.Select(x => x.outputPrompt), tokenizer);\n    }\n\n    public static CausalLMDataset Create(IEnumerable<string> inputs, IEnumerable<string> outputs, Tokenizer tokenizer)\n    {\n        // the length of inputs and outputs should be the same","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Core/Trainer/CausalLMDataset.cs#L16-L52","documentation":"CausalLMDataset.Create(IEnumerable<MEAIChatMessage> inputs, outputs, chatTemplateBuilder, tokenizer) throws ArgumentException when the input and output collections have different lengths. The method zips inputs with outputs pairwise to build the fine-tuning prompts, so a length mismatch is meaningless and rejected up front.","triggerScenarios":"Calling CausalLMDataset.Create with IEnumerable<MEAIChatMessage> inputs and outputs of unequal counts, e.g. a chat fine-tuning dataset where a prompt row has no matching assistant response.","commonSituations":"Loading a fine-tuning JSONL where some rows are missing the assistant turn; filtering one collection but not the other; LINQ lazy enumeration causing counts evaluated differently.","solutions":["Check inputs.Count() == outputs.Count() before calling Create and fix the data source.","Materialize both collections to lists (ToList) and trim to the same length or drop unmatched pairs.","Audit the fine-tuning file for records missing assistant responses and remove or repair them.","If you must keep single-sided rows, build the dataset with the string-based or id-based Create overload after pairing manually."],"exampleFix":"// before\ndataset = CausalLMDataset.Create(chatInputs, chatOutputs, templateBuilder, tokenizer);\n// after\nif (chatInputs.Count() != chatOutputs.Count()) throw new InvalidDataException(\"rows missing assistant output\");\ndataset = CausalLMDataset.Create(chatInputs.ToList(), chatOutputs.ToList(), templateBuilder, tokenizer);","handlingStrategy":"validation","validationCode":"if (inputs?.Count() != outputs?.Count()) throw new InvalidDataException($\"inputs={inputs?.Count()} outputs={outputs?.Count()} must match\");","typeGuard":"bool PairedRows<T>(IEnumerable<T> a, IEnumerable<T> b) => a != null && b != null && a.Count() == b.Count();","tryCatchPattern":"try { var ds = CausalLMDataset.Create(inputs, outputs, builder, tokenizer); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"inputs and outputs\")) { logger.LogError(\"Fine-tuning rows are mispaired: {Msg}\", ex.Message); throw; }","preventionTips":["Store fine-tuning data as paired records, not two parallel collections.","Materialize inputs/outputs with ToList before validation and the call.","Filter both collections with identical predicates.","Log counts of both collections before dataset creation."],"tags":["dataset","training","argument"],"backgroundTag":"schema-validation-failed","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}