apache/flink · error · RuntimeException

Fail to deserialize at field: %s.

Error message

Fail to deserialize at field: %s.

What it means

Per-field error context in CsvToRowDataConverters.createRowConverter: when converting one field of a CSV row, any Throwable from the field's converter (bad number, bad date/time string, wrong node type) is caught and re-thrown as RuntimeException naming the failing field ('Fail to deserialize at field: %s.'). This pinpoints which column of the CSV row is malformed; the cause holds the precise parse failure. Typically it then bubbles up to error 1331's 'Failed to deserialize CSV row' handler.

Source

Thrown at flink-formats/flink-csv/src/main/java/org/apache/flink/formats/csv/CsvToRowDataConverters.java:112

            }

            GenericRowData row = new GenericRowData(arity);
            for (int i = 0; i < arity; i++) {
                JsonNode field;
                // Jackson only supports mapping by name in the first level
                if (isTopLevel) {
                    field = jsonNode.get(fieldNames[i]);
                } else {
                    field = jsonNode.get(i);
                }
                try {
                    if (field == null) {
                        row.setField(i, null);
                    } else {
                        row.setField(i, fieldConverters[i].convert(field));
                    }
                } catch (Throwable t) {
                    throw new RuntimeException(
                            String.format("Fail to deserialize at field: %s.", fieldNames[i]), t);
                }
            }
            return row;
        };
    }

    /** Creates a runtime converter which is null safe. */
    private CsvToRowDataConverter createNullableConverter(LogicalType type) {
        final CsvToRowDataConverter converter = createConverter(type);
        return jsonNode -> {
            if (jsonNode == null || jsonNode.isNull()) {
                return null;
            }
            try {
                return converter.convert(jsonNode);
            } catch (Throwable t) {
                if (!ignoreParseErrors) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Read the field name in the message and the chained cause, then fix that column's value or producer formatting.
2. For tolerant pipelines set 'csv.ignore-parse-errors'='true' so bad rows are skipped instead of failing the job.
3. Declare nullable columns where empty strings appear, or pre-emptively CAST to STRING and validate downstream.
4. Match 'csv.timestamp-format'/'csv.date-format'/'csv.time-format' options to the actual string shapes.

Example fix

-- before
'format'='csv'  -- job dies on '2023-13-45' in dt column

-- after
'format'='csv',
'csv.ignore-parse-errors'='true'

-- Sample-based validation before production:
-- SELECT COUNT(*) FROM csv_t WHERE NOT <type predicate per column> (e.g. ts REGEXP '^\\d{4}-\\d{2}-\\d{2}$');
-- Configure explicit formats when the shape is known:
-- 'csv.date-format'='yyyy-MM-dd', 'csv.time-format'='HH:mm:ss'

catch (RuntimeException e) { if (e.getMessage().startsWith("Fail to deserialize at field:")) { /* extract field name, route row to DLQ or skip when ignore-parse-errors=true */ } throw e; }

When it happens

Trigger: A specific column value incompatible with its declared type: '2023-13-45' for DATE, 'abc' for INT, '1,234' (locale thousands separator) for DOUBLE, an empty string for a non-nullable primitive parsed strictly, or a quoted fragment breaking array element parsing.

Common situations: Mixed-quality producer data; timezone/locale formatting of timestamps; CSV files where nulls are encoded as empty strings but the column is declared NOT NULL primitive.

Related errors

• Failed to deserialize CSV row '%s'.
• Row too short: {}
• Line could not be parsed: '{}' ParserError {} Expect field
• Line could not be parsed: '{}' Expect field types: {} in fi
• value overflow

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/36a94227c7f8f3d1. Report an issue: GitHub.