{"record":{"id":"75a334c9d21427e9","repo":"apache/flink","slug":"line-could-not-be-parsed-expect-field-types","errorCode":null,"errorMessage":"Line could not be parsed: '{}'\nExpect field types: {} \nin file: {}","messagePattern":"Line could not be parsed: '(.+?)'\nExpect field types: (.+?) \nin file: (.+?)","errorType":"exception","errorClass":"ParseException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/common/io/GenericCsvInputFormat.java","lineNumber":444,"sourceCode":"                        && field != fieldIncluded.length - 1\n                        && !FieldParser.endsWithDelimiter(bytes, startPos - 1, fieldDelim)) {\n                    // We are at the end of the record, but not all fields have been read\n                    // and the end is not a field delimiter indicating an empty last field.\n                    if (lenient) {\n                        return false;\n                    } else {\n                        throw new ParseException(\n                                \"Row too short: \" + new String(bytes, offset, numBytes));\n                    }\n                }\n                output++;\n            } else {\n                // skip field\n                startPos = skipFields(bytes, startPos, limit, this.fieldDelim);\n                if (startPos < 0) {\n                    if (!lenient) {\n                        String lineAsString = new String(bytes, offset, numBytes, getCharset());\n                        throw new ParseException(\n                                \"Line could not be parsed: '\"\n                                        + lineAsString\n                                        + \"'\\n\"\n                                        + \"Expect field types: \"\n                                        + fieldTypesToString()\n                                        + \" \\n\"\n                                        + \"in file: \"\n                                        + currentSplit.getPath());\n                    } else {\n                        return false;\n                    }\n                } else if (startPos == limit\n                        && field != fieldIncluded.length - 1\n                        && !FieldParser.endsWithDelimiter(bytes, startPos - 1, fieldDelim)) {\n                    // We are at the end of the record, but not all fields have been read\n                    // and the end is not a field delimiter indicating an empty last field.\n                    if (lenient) {\n                        return false;","sourceCodeStart":426,"sourceCodeEnd":462,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/io/GenericCsvInputFormat.java#L426-L462","documentation":"Thrown in the branch that SKIPS a field (fieldIncluded[field] == false) when skipFields returns a negative cursor, meaning the skipper could not advance past the field delimiter boundary. It reports the offending line, the declared field types, and the source file path. Only thrown in non-lenient mode; lenient mode returns false instead.","triggerScenarios":"GenericCsvInputFormat is configured to include only some columns (the rest are skipped via the includedMask/index array), but a skipped field cannot be traversed because the delimiter sequence does not appear where expected, e.g. the file uses a different delimiter than configured or the row is malformed.","commonSituations":"Selecting a subset of columns (sparse projection) but specifying the wrong fieldDelimiter; delimiter set to multi-char string that does not actually occur in the data; quoting that swallows the delimiter; row shorter than the largest declared field index.","solutions":["Confirm fieldDelimiter matches the file exactly (including multi-char delimiters).","Set quotedStringParsing/quoteCharacter correctly so quoted fields don't swallow delimiters.","Enable lenient mode (format.setLenient(true)) to drop problematic rows.","Reduce the largest field index to match the actual row width."],"exampleFix":"// before: file is comma-delimited but format expects ';'\nformat.setFieldDelimiter(';');\n// after\nformat.setFieldDelimiter(',');","handlingStrategy":"validation","validationCode":"// Confirm the configured delimiter actually occurs in the file before sparse projection\nbyte[] delim = format.getFieldDelimiter();\ntry (InputStream in = fs.open(path)) {\n    byte[] buf = in.readNBytes(8192);\n    if (!contains(buf, delim)) {\n        throw new IllegalStateException(\"Field delimiter not found in sample of \" + path);\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    return format.nextRecord(reuse);\n} catch (ParseException e) {\n    log.warn(\"Skip failed for row in {}: {}\", currentSplit, e.getMessage());\n    return null;\n}","preventionTips":["Set the exact fieldDelimiter (including multi-char) that the file uses.","Configure quotedStringParsing/quoteCharacter when fields may contain delimiters.","Enable lenient mode for tolerant row skipping.","Ensure the largest field index is within the real row width."],"tags":["csv","input-format","data-quality","parsing"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}