apache/flink · error · ParseException

Line could not be parsed: '{}' Expect field types: {} in fi

Error message

Line could not be parsed: '{}'
Expect field types: {} 
in file: {}

What it means

Thrown in the branch that SKIPS a field (fieldIncluded[field] == false) when skipFields returns a negative cursor, meaning the skipper could not advance past the field delimiter boundary. It reports the offending line, the declared field types, and the source file path. Only thrown in non-lenient mode; lenient mode returns false instead.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/GenericCsvInputFormat.java:444

                        && field != fieldIncluded.length - 1
                        && !FieldParser.endsWithDelimiter(bytes, startPos - 1, fieldDelim)) {
                    // We are at the end of the record, but not all fields have been read
                    // and the end is not a field delimiter indicating an empty last field.
                    if (lenient) {
                        return false;
                    } else {
                        throw new ParseException(
                                "Row too short: " + new String(bytes, offset, numBytes));
                    }
                }
                output++;
            } else {
                // skip field
                startPos = skipFields(bytes, startPos, limit, this.fieldDelim);
                if (startPos < 0) {
                    if (!lenient) {
                        String lineAsString = new String(bytes, offset, numBytes, getCharset());
                        throw new ParseException(
                                "Line could not be parsed: '"
                                        + lineAsString
                                        + "'\n"
                                        + "Expect field types: "
                                        + fieldTypesToString()
                                        + " \n"
                                        + "in file: "
                                        + currentSplit.getPath());
                    } else {
                        return false;
                    }
                } else if (startPos == limit
                        && field != fieldIncluded.length - 1
                        && !FieldParser.endsWithDelimiter(bytes, startPos - 1, fieldDelim)) {
                    // We are at the end of the record, but not all fields have been read
                    // and the end is not a field delimiter indicating an empty last field.
                    if (lenient) {
                        return false;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Confirm fieldDelimiter matches the file exactly (including multi-char delimiters).
2. Set quotedStringParsing/quoteCharacter correctly so quoted fields don't swallow delimiters.
3. Enable lenient mode (format.setLenient(true)) to drop problematic rows.
4. Reduce the largest field index to match the actual row width.

Example fix

// before: file is comma-delimited but format expects ';'
format.setFieldDelimiter(';');
// after
format.setFieldDelimiter(',');

// Confirm the configured delimiter actually occurs in the file before sparse projection
byte[] delim = format.getFieldDelimiter();
try (InputStream in = fs.open(path)) {
    byte[] buf = in.readNBytes(8192);
    if (!contains(buf, delim)) {
        throw new IllegalStateException("Field delimiter not found in sample of " + path);
    }
}

try {
    return format.nextRecord(reuse);
} catch (ParseException e) {
    log.warn("Skip failed for row in {}: {}", currentSplit, e.getMessage());
    return null;
}

When it happens

Trigger: GenericCsvInputFormat is configured to include only some columns (the rest are skipped via the includedMask/index array), but a skipped field cannot be traversed because the delimiter sequence does not appear where expected, e.g. the file uses a different delimiter than configured or the row is malformed.

Common situations: Selecting a subset of columns (sparse projection) but specifying the wrong fieldDelimiter; delimiter set to multi-char string that does not actually occur in the data; quoting that swallows the delimiter; row shorter than the largest declared field index.

Related errors

• Row too short: {}
• Line could not be parsed: '{}' ParserError {} Expect field
• Orphaned minus sign.
• The record length exceeded the maximum record length (${line
• Delimiter must not be null

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/75a334c9d21427e9. Report an issue: GitHub.