{"record":{"id":"45da588b460c7d65","repo":"dotnet/machinelearning","slug":"file-is-not-text-or-couldn-t-detect-line-breaks","errorCode":null,"errorMessage":"File is not text, or couldn't detect line breaks","messagePattern":"File is not text, or couldn't detect line breaks","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.AutoML/ColumnInference/TextFileSample.cs","lineNumber":206,"sourceCode":"                {\n                    int iMin = (i == 0) ? 0 : Array.FindIndex(chunks[i], x => x == '\\n') + 1;\n                    int iLim = (wholeFile && i == chunks.Length - 1)\n                        ? chunks[i].Length\n                        : Array.FindLastIndex(chunks[i], x => x == '\\n') + 1;\n\n                    if (iLim == 0)\n                    {\n                        //entire buffer is one string, skip\n                        continue;\n                    }\n\n                    resultStream.Write(chunks[i], iMin, iLim - iMin);\n                }\n\n                var resultBuffer = resultStream.ToArray();\n                if (resultBuffer.Length == 0)\n                {\n                    throw new ArgumentException(\"File is not text, or couldn't detect line breaks\");\n                }\n\n                return resultBuffer;\n            }\n        }\n\n        /// <summary>\n        /// Detect whether we can auto-detect EOL characters without parsing.\n        /// If we do, we can cheaply sample from different file locations and trim the partial strings.\n        /// The encodings that pass the test are UTF8 and all single-byte encodings.\n        /// </summary>\n        private static bool IsEncodingOkForSampling(byte[] buffer)\n        {\n            // First check if a BOM/signature exists (sourced from https://www.unicode.org/faq/utf_bom.html#bom4)\n            if (buffer.Length >= 4 && buffer[0] == 0x00 && buffer[1] == 0x00 && buffer[2] == 0xFE && buffer[3] == 0xFF)\n            {\n                // UTF-32, big-endian\n                return false;","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.AutoML/ColumnInference/TextFileSample.cs#L188-L224","documentation":"TextFileSample.StitchChunks reassembles sampled chunk buffers into a single buffer and throws ArgumentException if the stitched result is empty. An empty result means no usable text chunks were collected — the file yielded no line-delimited content. It is raised from both CreateFromFullStream and CreateFromHead paths.","triggerScenarios":"Stitching chunks produced from a file that is empty, entirely whitespace/filtered out, or binary so all sampled content was discarded; head/tail sample sizes resolving to zero bytes of retained content.","commonSituations":"Zero-byte files passed to AutoML inference; files whose content was fully trimmed by sampling logic; corrupted downloads producing empty or binary-only files; misconfigured sample sizes excluding all lines.","solutions":["Check the file exists and has non-zero size before sampling (FileInfo.Length > 0)","Open the file to confirm it contains line-delimited text; replace corrupt/empty files with valid exports","Increase the head/sample size so at least one complete line is retained","Verify no upstream step (download/unzip) produced an empty or binary file"],"exampleFix":"// before\nvar sample = TextFileSample.CreateFromHeadStream(stream, sampleSize: 0);\n// after\nif (new FileInfo(path).Length == 0) throw new InvalidDataException(\"empty file\");\nvar sample = TextFileSample.CreateFromHeadStream(stream, sampleSize: 64 * 1024);","handlingStrategy":"validation","validationCode":"var fi = new FileInfo(path);\nif (!fi.Exists || fi.Length == 0) throw new InvalidDataException(\"File missing or empty\");","typeGuard":null,"tryCatchPattern":"try { var s = TextFileSample.CreateFromHeadFile(path, sampleSize); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"not text\"))\n{ /* verify file content/size and increase sample size */ }","preventionTips":["Check file size > 0 before sampling","Verify downloads/unzips produced a complete file","Use a sample size large enough to retain at least one full line"],"tags":["dotnet","ml-automl","empty-file","text-file"],"backgroundTag":"empty-required-field","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}