{"record":{"id":"b0f011c7e7983050","repo":"dotnet/machinelearning","slug":"couldn-t-identify-line-breaks-provided-file-is-no","errorCode":null,"errorMessage":"Couldn't identify line breaks. Provided file is not text?","messagePattern":"Couldn't identify line breaks\\. Provided file is not text\\?","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.AutoML/ColumnInference/TextFileSample.cs","lineNumber":117,"sourceCode":"            var fileSize = stream.Length;\n\n            if (fileSize <= 2 * BufferSizeMb * (1 << 20))\n            {\n                return CreateFromHead(stream);\n            }\n\n            var firstChunk = new byte[FirstChunkSizeMb * (1 << 20)];\n            int count = stream.Read(firstChunk, 0, firstChunk.Length);\n            if (!IsEncodingOkForSampling(firstChunk))\n                return CreateFromHead(stream);\n            // REVIEW: CreateFromHead still truncates the file before the last 0x0A byte. For multi-byte encoding,\n            // this might cause an unfinished string to be present in the buffer. Right now this is considered an acceptable\n            // price to pay for parse-free processing.\n\n            var lineCount = firstChunk.Count(x => x == '\\n');\n            if (lineCount == 0)\n            {\n                throw new ArgumentException(\"Couldn't identify line breaks. Provided file is not text?\");\n            }\n\n            long approximateRowCount = (long)(lineCount * fileSize * 1.0 / firstChunk.Length);\n            var firstNewline = Array.FindIndex(firstChunk, x => x == '\\n');\n\n            // First line may be header, so we exclude it. The remaining lineCount-1 line breaks are\n            // splitting the text into lineCount lines, and the last line is actually half-size.\n            Double averageLineLength = 2.0 * (firstChunk.Length - firstNewline) / (lineCount * 2 - 1);\n            averageLineLength = Math.Max(averageLineLength, 3);\n\n            int usefulChunkSize = (int)(averageLineLength * LinesPerChunk);\n            int chunkSize = (int)(usefulChunkSize + averageLineLength); // assuming that 1 line worth will be trimmed out\n\n            int chunkCount = (int)Math.Ceiling((BufferSizeMb * OversamplingRate - FirstChunkSizeMb) * (1 << 20) / usefulChunkSize);\n            int maxChunkCount = (int)Math.Floor((double)(fileSize - firstChunk.Length) / chunkSize);\n            chunkCount = Math.Min(chunkCount, maxChunkCount);\n\n            var chunks = new List<byte[]>();","sourceCodeStart":99,"sourceCodeEnd":135,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.AutoML/ColumnInference/TextFileSample.cs#L99-L135","documentation":"TextFileSample.CreateFromFullStream reads the first chunk of the file and counts '\n' bytes to estimate row count and chunk boundaries. If the first chunk contains no newline bytes, the file cannot be treated as line-oriented text, so it throws ArgumentException. The file is likely binary, single-line, or uses non-standard line endings.","triggerScenarios":"Calling CreateFromFullStream/CreateFromFullFile on a binary file (Excel, parquet, images), a file whose entire content is one line with no '\n', or a file using only CR ('\r') line endings from legacy Mac formats.","commonSituations":"Pointing AutoML at a .xlsx or compressed file renamed to .csv; concatenated one-line data exports; old CR-only line-ending files; empty or single-record files.","solutions":["Verify the file is plain text delimited data; convert/re-export binary formats (e.g. save Excel as CSV)","Ensure the file contains '\n' line breaks; convert CR-only files to CRLF/LF","Check the file isn't empty or a single line without a trailing newline","Confirm the file path points at the actual data file, not a compressed or container file"],"exampleFix":"// before\nvar sample = TextFileSample.CreateFromFullFile(\"report.xlsx\");\n// after\n// export to CSV first, then:\nvar sample = TextFileSample.CreateFromFullFile(\"report.csv\");","handlingStrategy":"validation","validationCode":"var bytes = File.ReadAllBytes(path).Take(4096).ToArray();\nbool hasNewline = bytes.Contains((byte)'\\n');\nbool mostlyText = bytes.Count(b => b is >= 32 or 9 or 10 or 13) * 20 > bytes.Length * 19;\nif (!hasNewline || !mostlyText) throw new InvalidDataException(\"File is not line-delimited text\");","typeGuard":null,"tryCatchPattern":"try { var s = TextFileSample.CreateFromFullFile(path); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"line breaks\"))\n{ /* convert/re-export the file as delimited text */ }","preventionTips":["Only sample plain-text delimited files","Convert binary formats (xlsx, parquet) to CSV first","Normalize line endings to LF/CRLF"],"tags":["dotnet","ml-automl","file-format","text-file"],"backgroundTag":"incompatible-source-type","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}