dotnet/machinelearning · error · ArgumentException

File is not text, or couldn't detect line breaks

Error message

File is not text, or couldn't detect line breaks

What it means

TextFileSample.StitchChunks reassembles sampled chunk buffers into a single buffer and throws ArgumentException if the stitched result is empty. An empty result means no usable text chunks were collected — the file yielded no line-delimited content. It is raised from both CreateFromFullStream and CreateFromHead paths.

Source

Thrown at src/Microsoft.ML.AutoML/ColumnInference/TextFileSample.cs:206

                {
                    int iMin = (i == 0) ? 0 : Array.FindIndex(chunks[i], x => x == '\n') + 1;
                    int iLim = (wholeFile && i == chunks.Length - 1)
                        ? chunks[i].Length
                        : Array.FindLastIndex(chunks[i], x => x == '\n') + 1;

                    if (iLim == 0)
                    {
                        //entire buffer is one string, skip
                        continue;
                    }

                    resultStream.Write(chunks[i], iMin, iLim - iMin);
                }

                var resultBuffer = resultStream.ToArray();
                if (resultBuffer.Length == 0)
                {
                    throw new ArgumentException("File is not text, or couldn't detect line breaks");
                }

                return resultBuffer;
            }
        }

        /// <summary>
        /// Detect whether we can auto-detect EOL characters without parsing.
        /// If we do, we can cheaply sample from different file locations and trim the partial strings.
        /// The encodings that pass the test are UTF8 and all single-byte encodings.
        /// </summary>
        private static bool IsEncodingOkForSampling(byte[] buffer)
        {
            // First check if a BOM/signature exists (sourced from https://www.unicode.org/faq/utf_bom.html#bom4)
            if (buffer.Length >= 4 && buffer[0] == 0x00 && buffer[1] == 0x00 && buffer[2] == 0xFE && buffer[3] == 0xFF)
            {
                // UTF-32, big-endian
                return false;

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Check the file exists and has non-zero size before sampling (FileInfo.Length > 0)
2. Open the file to confirm it contains line-delimited text; replace corrupt/empty files with valid exports
3. Increase the head/sample size so at least one complete line is retained
4. Verify no upstream step (download/unzip) produced an empty or binary file

Example fix

// before
var sample = TextFileSample.CreateFromHeadStream(stream, sampleSize: 0);
// after
if (new FileInfo(path).Length == 0) throw new InvalidDataException("empty file");
var sample = TextFileSample.CreateFromHeadStream(stream, sampleSize: 64 * 1024);

var fi = new FileInfo(path);
if (!fi.Exists || fi.Length == 0) throw new InvalidDataException("File missing or empty");

try { var s = TextFileSample.CreateFromHeadFile(path, sampleSize); }
catch (ArgumentException ex) when (ex.Message.Contains("not text"))
{ /* verify file content/size and increase sample size */ }

When it happens

Trigger: Stitching chunks produced from a file that is empty, entirely whitespace/filtered out, or binary so all sampled content was discarded; head/tail sample sizes resolving to zero bytes of retained content.

Common situations: Zero-byte files passed to AutoML inference; files whose content was fully trimmed by sampling logic; corrupted downloads producing empty or binary-only files; misconfigured sample sizes excluding all lines.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Couldn't identify line breaks. Provided file is not text?
• can't add or update result that already save to csv
• IDatasetManager must be either ITrainTestDatasetManager or I
• OperationCanceledException
• Unable to split the file provided into multiple, consistent

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/45da588b460c7d65. Report an issue: GitHub.