apache/flink · error · JsonParseException

Could not find field with name '%s'.

Error message

Could not find field with name '%s'.

What it means

Thrown by JsonToRowDataConverters.convertField when a field declared in the ROW schema is absent from an incoming JSON object and the failOnMissingField flag is true. With the flag false (the default) a missing field simply becomes null. The flag maps to the table option 'json.fail-on-missing-field'.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonToRowDataConverters.java:377

                String fieldName = fieldNames[i];
                JsonNode field = node.get(fieldName);
                try {
                    Object convertedField = convertField(fieldConverters[i], fieldName, field);
                    row.setField(i, convertedField);
                } catch (Throwable t) {
                    throw new JsonParseException(
                            String.format("Fail to deserialize at field: %s.", fieldName), t);
                }
            }
            return row;
        };
    }

    private Object convertField(
            JsonToRowDataConverter fieldConverter, String fieldName, JsonNode field) {
        if (field == null) {
            if (failOnMissingField) {
                throw new JsonParseException("Could not find field with name '" + fieldName + "'.");
            } else {
                return null;
            }
        } else {
            return fieldConverter.convert(field);
        }
    }

    private JsonToRowDataConverter wrapIntoNullableConverter(JsonToRowDataConverter converter) {
        return jsonNode -> {
            if (jsonNode == null || jsonNode.isNull() || jsonNode.isMissingNode()) {
                return null;
            }
            try {
                return converter.convert(jsonNode);
            } catch (Throwable t) {
                if (!ignoreParseErrors) {
                    throw t;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set 'json.fail-on-missing-field' = 'false' (default) so absent fields deserialize to null
2. Ensure the producer includes all declared fields, or align the DDL to the actual JSON payload
3. If both strict and lenient consumers are needed, split the topic or preprocess records to fill defaults before ingestion

Example fix

// before
WITH ('connector'='kafka', 'format'='json', 'json.fail-on-missing-field'='true')

// after
WITH ('connector'='kafka', 'format'='json', 'json.fail-on-missing-field'='false')

// If unsure the producer always emits all fields, do not enable strict mode:
// 'json.fail-on-missing-field' defaults to false; leave it false and null-check downstream
if (row.getField(i) == null) { /* handle absent column explicitly */ }

catch (JsonParseException e) on 'Could not find field' — either disable fail-on-missing-field or fix the producer; retrying cannot help.

When it happens

Trigger: DDL/table option 'json.fail-on-missing-field' = 'true' while the consumed JSON message does not contain one or more declared columns. Triggered per record by JsonRowDataDeserializationSchema whenever node.get(fieldName) returns null.

Common situations: Schema declared with more columns than the producer emits; upstream schema evolution dropping a field; enabling fail-on-missing-field defensively and then feeding historical/sparse records; mixing producers with different record shapes into one topic.

Related errors

• Some field is missing in the JSON data.
• Unsupported type: {}
• Numeric value (%s) out of range of Java byte.
• JSON format doesn't support non-string as key type of map. T
• Some field is missing in the Json data.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/3819cbc6bf9dcf14. Report an issue: GitHub.