dotnet/machinelearning · error · ArgumentException

IDatasetManager must be either ITrainTestDatasetManager or I

Error message

IDatasetManager must be either ITrainTestDatasetManager or ICrossValidationDatasetManager

What it means

SweepablePipelineRunner.Run only knows how to evaluate pipelines when the IDatasetManager is an ITrainTestDatasetManager (single train/test split) or an ICrossValidationDatasetManager (CV folds). Any other IDatasetManager implementation falls through all branches and hits this ArgumentException. It signals an unsupported dataset manager type was plugged into the AutoML experiment.

Source

Thrown at src/Microsoft.ML.AutoML/AutoMLExperiment/Runner/SweepablePipelineRunner.cs:91

            if (_datasetManager is ITrainValidateDatasetManager trainTestDatasetManager)
            {
                var model = mlnetPipeline.Fit(trainTestDatasetManager.LoadTrainDataset(_mLContext!, settings));
                var eval = model.Transform(trainTestDatasetManager.LoadValidateDataset(_mLContext!, settings));
                var metric = _metricManager.Evaluate(_mLContext, eval);
                stopWatch.Stop();
                var loss = _metricManager.IsMaximize ? -metric : metric;

                return new TrialResult
                {
                    Loss = loss,
                    Metric = metric,
                    Model = model,
                    DurationInMilliseconds = stopWatch.ElapsedMilliseconds,
                    TrialSettings = settings,
                };
            }

            throw new ArgumentException("IDatasetManager must be either ITrainTestDatasetManager or ICrossValidationDatasetManager");
        }

        public Task<TrialResult> RunAsync(TrialSettings settings, CancellationToken ct)
        {
            try
            {
                using (var ctRegistration = ct.Register(() =>
                {
                    _mLContext?.CancelExecution();
                }))
                {
                    return Task.FromResult(Run(settings));
                }
            }
            catch (Exception ex) when (ct.IsCancellationRequested)
            {
                throw new OperationCanceledException(ex.Message, ex.InnerException);
            }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Use ITrainTestDatasetManager or ICrossValidationDatasetManager as the experiment's dataset manager
2. If a custom strategy is needed, extend one of the two supported interfaces rather than inventing a new IDatasetManager
3. Add a Run branch in a derived runner for the custom manager type
4. Verify the registered service in AutoMLExperiment is the expected concrete type

Example fix

// before
experiment.SetDatasetManager(myCustomDatasetManager);
// after
var trainTest = TrainTestDatasetManager.CreateTrainTestSplit(data, 0.8);
experiment.SetDatasetManager(trainTest); // or a cross-validation dataset manager

if (datasetManager is not ITrainTestDatasetManager && datasetManager is not ICrossValidationDatasetManager)
    throw new InvalidOperationException("Register a supported dataset manager");

bool isSupported = datasetManager is ITrainTestDatasetManager or ICrossValidationDatasetManager;

try { await runner.RunAsync(settings, ct); }
catch (ArgumentException ex) when (ex.Message.Contains("IDatasetManager")) { /* switch to supported manager */ }

When it happens

Trigger: Registering a custom IDatasetManager (neither train-test nor cross-validation) in the AutoMLExperiment and running a trial; passing the wrong dataset manager type to SweepablePipelineRunner via RunAsync.

Common situations: Custom dataset-splitting strategies that don't implement one of the two supported interfaces; refactors that swapped the dataset manager type but kept an old runner; typo injecting an interface instead of a concrete implementation.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• can't add or update result that already save to csv
• OperationCanceledException
• Unable to split the file provided into multiple, consistent
• Unable to infer column types of the file provided.
• Specified column {columnName} is not found in the dataset.

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/b2b4954dc6b5f066. Report an issue: GitHub.