{"record":{"id":"dce4c6afa17e5f77","repo":"apache/seatunnel","slug":"file-s-has-fewer-lines-than-expected-to-skip","errorCode":null,"errorMessage":"File [%s] has fewer lines than expected to skip.","messagePattern":"File \\[(.+?)\\] has fewer lines than expected to skip\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/CsvReadStrategy.java","lineNumber":116,"sourceCode":"            Map<String, String> partitionsMap,\n            String currentFileName)\n            throws IOException {\n        log.info(\n                \"Start reading CSV file: {}, split start: {}, split length: {}\",\n                currentFileName,\n                split.getStart(),\n                split.getLength());\n        final boolean useSplitRead = isSplitReadEnabled(split);\n        try (BufferedReader reader =\n                        createBomAwareBufferedReader(\n                                wrapInputStream(inputStream, split), encoding);\n                CSVParser csvParser = new CSVParser(reader, getCSVFormat(split))) {\n            // skip lines\n            // if split range is used, no need to skip\n            if (!useSplitRead) {\n                for (int i = 0; i < skipHeaderNumber; i++) {\n                    if (reader.readLine() == null) {\n                        throw new IOException(\n                                String.format(\n                                        \"File [%s] has fewer lines than expected to skip.\",\n                                        currentFileName));\n                    }\n                }\n            }\n            // read header lines\n            List<String> headers = getHeaders(csvParser, split);\n            // Clean up BOM characters (\\uFEFF) in the header to solve occasional BOM residue\n            // issues\n            List<String> cleanedHeaders =\n                    headers.stream()\n                            .map(header -> header.replace(\"\\uFEFF\", \"\"))\n                            .collect(Collectors.toList());\n            for (CSVRecord csvRecord : csvParser) {\n                HashMap<Integer, String> fieldIdValueMap = new HashMap<>();\n                for (int i = 0; i < cleanedHeaders.size(); i++) {\n                    // the user input schema may not contain all the columns in the csv header","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/CsvReadStrategy.java#L98-L134","documentation":"CsvReadStrategy.readProcess skips skip_header_number lines before parsing. If the reader hits EOF before all requested header lines are skipped (the file has fewer lines than skip_header_number), a plain IOException is thrown stating the file lacks enough lines. Split-range reads bypass this skip logic entirely.","triggerScenarios":"Reading a CSV with skip_header_number = N where the split's file contains fewer than N lines, and useSplitRead is false (no split-range read).","commonSituations":"Header count copied from another file; empty or truncated CSV files; miscomputed skip value such as 3 when the file has only a 1-line header; reading many small files where some are headerless.","solutions":["Lower skip_header_number to at most (file line count).","Remove or fix skip_header_number if files have no header (use 0).","Verify the source files are not truncated/empty and check currentFileName in the message."],"exampleFix":"// before\nskip_header_number = 5\n// after\nskip_header_number = 1","handlingStrategy":"validation","validationCode":"// Java, before reading: ensure file has at least skipHeaderNumber lines\nlong lines = countLines(file);\nif (skipHeaderNumber > lines) {\n    throw new IllegalArgumentException(\"skip_header_number=\" + skipHeaderNumber + \" exceeds file lines=\" + lines);\n}","typeGuard":null,"tryCatchPattern":"try {\n    rows = readProcess(inputStream, partitionsMap, fileName);\n} catch (IOException e) {\n    if (e.getMessage().contains(\"fewer lines than expected to skip\")) {\n        log.error(\"Header skip exceeds file length for {}\", fileName);\n    }\n    throw e;\n}","preventionTips":["Match skip_header_number to the actual header line count of every file in the directory.","Validate that source files are non-empty and not truncated.","Use 0 when files have no header."],"tags":["csv","config-validation","skip-headers"],"backgroundTag":"invalid-config-value","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}