dotnet/machinelearning · error · ArgumentException

Duplicate column name {duplicateColName} is present in two o

Error message

Duplicate column name {duplicateColName} is present in two or more distinct properties of provided column information

What it means

Thrown by UserInputValidationUtil.ValidateColumnInformation when the same column name appears in two or more distinct ColumnInformation properties (e.g. both LabelColumnName and a text column name, or duplicated across categorical/text/ignored lists). Ambiguous column purposes cannot be resolved, so the input is rejected with the first duplicate name in the message.

Source

Thrown at src/Microsoft.ML.AutoML/Utils/UserInputValidationUtil.cs:144

            ValidateColumnInfoEnumerationProperty(columnInformation.CategoricalColumnNames, CategoricalColumnPurposeName);
            ValidateColumnInfoEnumerationProperty(columnInformation.NumericColumnNames, NumericColumnPurposeName);
            ValidateColumnInfoEnumerationProperty(columnInformation.TextColumnNames, TextColumnPurposeName);
            ValidateColumnInfoEnumerationProperty(columnInformation.IgnoredColumnNames, IgnoredColumnPurposeName);

            // keep a list of all columns, to detect duplicates
            var allColumns = new List<string>();
            allColumns.Add(columnInformation.LabelColumnName);
            if (columnInformation.ExampleWeightColumnName != null) { allColumns.Add(columnInformation.ExampleWeightColumnName); }
            if (columnInformation.CategoricalColumnNames != null) { allColumns.AddRange(columnInformation.CategoricalColumnNames); }
            if (columnInformation.NumericColumnNames != null) { allColumns.AddRange(columnInformation.NumericColumnNames); }
            if (columnInformation.TextColumnNames != null) { allColumns.AddRange(columnInformation.TextColumnNames); }
            if (columnInformation.IgnoredColumnNames != null) { allColumns.AddRange(columnInformation.IgnoredColumnNames); }

            var duplicateColName = FindFirstDuplicate(allColumns);
            if (duplicateColName != null)
            {
                throw new ArgumentException($"Duplicate column name {duplicateColName} is present in two or more distinct properties of provided column information", nameof(columnInformation));
            }
        }

        private static void ValidateColumnInfoEnumerationProperty(IEnumerable<string> columns, string columnPurpose)
        {
            if (columns?.Contains(null) == true)
            {
                throw new ArgumentException($"Null column string was specified as {columnPurpose} in column information");
            }
        }

        private static void ValidateLabelColumn(string labelColumn)
        {
            if (labelColumn == null)
            {
                throw new ArgumentException("Provided label column cannot be null");
            }
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Ensure each column name appears in at most one ColumnInformation property
2. De-duplicate purpose lists before constructing ColumnInformation
3. If a column serves multiple roles intentionally, pick the primary purpose and remove it from other lists

Example fix

// before
var colInfo = new ColumnInformation {
    LabelColumnName = "Label",
    IgnoredColumnNames = new[] { "Id", "Label" } // Label duplicated -> ArgumentException
};

// after
var colInfo = new ColumnInformation {
    LabelColumnName = "Label",
    IgnoredColumnNames = new[] { "Id" }
};

var all = new[] { colInfo.LabelColumnName, colInfo.UserIdColumnName, colInfo.ItemIdColumnName, colInfo.GroupIdColumnName }
    .Concat(colInfo.CategoricalColumnNames ?? Enumerable.Empty<string>())
    .Concat(colInfo.TextColumnNames ?? Enumerable.Empty<string>())
    .Concat(colInfo.IgnoredColumnNames ?? Enumerable.Empty<string>())
    .Where(n => n != null).ToList();
if (all.Count != all.Distinct().Count()) throw new InvalidOperationException("Duplicate column across purposes");

try { var r = experiment.Execute(data, colInfo); }
catch (ArgumentException ex) when (ex.Message.Contains("Duplicate column name")) { /* log offending column from message and rebuild ColumnInformation */ }

When it happens

Trigger: Constructing ColumnInformation where, say, LabelColumnName equals an entry of CategoricalColumnNames, or the same name listed in both TextColumnNames and IgnoredColumnNames, then calling experiment.Execute or the infer-columns API.

Common situations: Programmatically building column lists by concatenating purpose arrays without de-duplication; a config file where one column is listed under two purposes; typos causing an intended column to double as label.

Related errors

• Null column string was specified as {columnPurpose} in colum
• If provided, {nameof(samplingKeyColumnName)} must be the sam
• Training data cannot be null
• Training data has 0 rows
• {DefaultColumnNames.Features} column must be of data type {N

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/f809d51696786782. Report an issue: GitHub.