{"record":{"id":"e3981a3a6b5d8495","repo":"dotnet/machinelearning","slug":"training-failed-with-the-exception-history-last","errorCode":null,"errorMessage":"Training failed with the exception: {_history.Last().Exception}","messagePattern":"Training failed with the exception: (.+?)","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"critical","filePath":"src/Microsoft.ML.AutoML/Experiment/Experiment.cs","lineNumber":191,"sourceCode":"                    _history.Add(suggestedPipelineRunDetail);\n                    WriteIterationLog(pipeline, suggestedPipelineRunDetail, iterationStopwatch);\n\n                    runDetail.RuntimeInSeconds = iterationStopwatch.Elapsed.TotalSeconds;\n                    runDetail.PipelineInferenceTimeInSeconds = getPipelineStopwatch.Elapsed.TotalSeconds;\n\n                    ReportProgress(runDetail);\n                    iterationResults.Add(runDetail);\n\n                    // if model is perfect, break\n                    if (_metricsAgent.IsModelPerfect(suggestedPipelineRunDetail.Score))\n                    {\n                        break;\n                    }\n\n                    // If after third run, all runs have failed so far, throw exception\n                    if (_history.Count() == 3 && _history.All(r => !r.RunSucceeded))\n                    {\n                        throw new InvalidOperationException($\"Training failed with the exception: {_history.Last().Exception}\");\n                    }\n                }\n                catch (OperationCanceledException e)\n                {\n                    // This exception is thrown when the IHost/MLContext of the trainer is canceled due to\n                    // reaching maximum experiment time. Simply catch this exception and return finished\n                    // iteration results.\n                    _logger.Warning(_operationCancelledMessage, e.Message);\n                    return iterationResults;\n                }\n                catch (AggregateException e)\n                {\n                    // This exception is thrown when the IHost/MLContext of the trainer is canceled due to\n                    // reaching maximum experiment time. Simply catch this exception and return finished\n                    // iteration results. For some trainers, like FastTree, because training is done in parallel\n                    // in can throw multiple OperationCancelledExceptions. This causes them to be returned as an\n                    // AggregateException and misses the first catch block. This is to handle that case.\n                    if (e.InnerExceptions.All(exception => exception is OperationCanceledException))","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.AutoML/Experiment/Experiment.cs#L173-L209","documentation":"Execute() tolerates individual failed training runs, but if the first 3 runs all fail it concludes the problem is systematic and rethrows the last run's exception wrapped in InvalidOperationException instead of returning empty results.","triggerScenarios":"Calling experiment.Execute() (e.g. via experiment.Execute(trainData, ...) helpers) where three consecutive trainer runs throw — bad data schema, incompatible labels, or every trainer in the pipeline erroring.","commonSituations":"Wrong column roles (label/columns not set), featurization failures, dataset too small/empty for any trainer, or unsupported data types causing each trial to fail identically.","solutions":["Inspect the inner Exception in the message — fix the root trainer failure (schema, label column, data types).","Validate input data (IDataView schema, label column exists, enough rows) before calling Execute().","Use UserInputValidationUtil / CrossValidationSplit to confirm folds are non-empty.","Catch InvalidOperationException, read _history-style inner exception, and adjust trainer/settings."],"exampleFix":"// before\nvar result = experiment.Execute(trainData, validationData, labelColumnName: \"Target\");\n// after\nif (!trainData.Schema.TryGetColumnIndex(\"Target\", out _))\n    throw new ArgumentException(\"Label column 'Target' missing from training data.\");\nvar result = experiment.Execute(trainData, validationData, labelColumnName: \"Target\");","handlingStrategy":"validation","validationCode":"if (!trainData.Schema.TryGetColumnIndex(labelColumn, out _)) throw new ArgumentException($\"Label '{labelColumn}' missing\");\nif (rowCount < 3) throw new ArgumentException(\"Too few rows for AutoML training\");","typeGuard":null,"tryCatchPattern":"try { var results = experiment.Execute(trainData, labelColumn); }\ncatch (InvalidOperationException ex) { logger.LogError(ex.InnerException ?? ex, \"AutoML training failed on all trials\"); throw; }","preventionTips":["Validate schema and label column before experiments","Ensure enough rows for at least 3 runs","Log each trial failure to see the first root exception"],"tags":["csharp","training","mlnet"],"backgroundTag":"internal-invariant-violation","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}