dotnet/machinelearning · error · ArgumentException

Couldn't identify line breaks. Provided file is not text?

Error message

Couldn't identify line breaks. Provided file is not text?

What it means

TextFileSample.CreateFromFullStream reads the first chunk of the file and counts ' ' bytes to estimate row count and chunk boundaries. If the first chunk contains no newline bytes, the file cannot be treated as line-oriented text, so it throws ArgumentException. The file is likely binary, single-line, or uses non-standard line endings.

Source

Thrown at src/Microsoft.ML.AutoML/ColumnInference/TextFileSample.cs:117

            var fileSize = stream.Length;

            if (fileSize <= 2 * BufferSizeMb * (1 << 20))
            {
                return CreateFromHead(stream);
            }

            var firstChunk = new byte[FirstChunkSizeMb * (1 << 20)];
            int count = stream.Read(firstChunk, 0, firstChunk.Length);
            if (!IsEncodingOkForSampling(firstChunk))
                return CreateFromHead(stream);
            // REVIEW: CreateFromHead still truncates the file before the last 0x0A byte. For multi-byte encoding,
            // this might cause an unfinished string to be present in the buffer. Right now this is considered an acceptable
            // price to pay for parse-free processing.

            var lineCount = firstChunk.Count(x => x == '\n');
            if (lineCount == 0)
            {
                throw new ArgumentException("Couldn't identify line breaks. Provided file is not text?");
            }

            long approximateRowCount = (long)(lineCount * fileSize * 1.0 / firstChunk.Length);
            var firstNewline = Array.FindIndex(firstChunk, x => x == '\n');

            // First line may be header, so we exclude it. The remaining lineCount-1 line breaks are
            // splitting the text into lineCount lines, and the last line is actually half-size.
            Double averageLineLength = 2.0 * (firstChunk.Length - firstNewline) / (lineCount * 2 - 1);
            averageLineLength = Math.Max(averageLineLength, 3);

            int usefulChunkSize = (int)(averageLineLength * LinesPerChunk);
            int chunkSize = (int)(usefulChunkSize + averageLineLength); // assuming that 1 line worth will be trimmed out

            int chunkCount = (int)Math.Ceiling((BufferSizeMb * OversamplingRate - FirstChunkSizeMb) * (1 << 20) / usefulChunkSize);
            int maxChunkCount = (int)Math.Floor((double)(fileSize - firstChunk.Length) / chunkSize);
            chunkCount = Math.Min(chunkCount, maxChunkCount);

            var chunks = new List<byte[]>();

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Verify the file is plain text delimited data; convert/re-export binary formats (e.g. save Excel as CSV)
2. Ensure the file contains ' ' line breaks; convert CR-only files to CRLF/LF
3. Check the file isn't empty or a single line without a trailing newline
4. Confirm the file path points at the actual data file, not a compressed or container file

Example fix

// before
var sample = TextFileSample.CreateFromFullFile("report.xlsx");
// after
// export to CSV first, then:
var sample = TextFileSample.CreateFromFullFile("report.csv");

var bytes = File.ReadAllBytes(path).Take(4096).ToArray();
bool hasNewline = bytes.Contains((byte)'\n');
bool mostlyText = bytes.Count(b => b is >= 32 or 9 or 10 or 13) * 20 > bytes.Length * 19;
if (!hasNewline || !mostlyText) throw new InvalidDataException("File is not line-delimited text");

try { var s = TextFileSample.CreateFromFullFile(path); }
catch (ArgumentException ex) when (ex.Message.Contains("line breaks"))
{ /* convert/re-export the file as delimited text */ }

When it happens

Trigger: Calling CreateFromFullStream/CreateFromFullFile on a binary file (Excel, parquet, images), a file whose entire content is one line with no ' ', or a file using only CR (' ') line endings from legacy Mac formats.

Common situations: Pointing AutoML at a .xlsx or compressed file renamed to .csv; concatenated one-line data exports; old CR-only line-ending files; empty or single-record files.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• File is not text, or couldn't detect line breaks
• can't add or update result that already save to csv
• IDatasetManager must be either ITrainTestDatasetManager or I
• OperationCanceledException
• Unable to split the file provided into multiple, consistent

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/b0f011c7e7983050. Report an issue: GitHub.