apache/flink · error · IllegalArgumentException

JSON format doesn't support failOnMissingField and…

Error message

JSON format doesn't support failOnMissingField and ignoreParseErrors are both enabled.

What it means

IllegalArgumentException from the AbstractJsonDeserializationSchema constructor when both failOnMissingField=true and ignoreParseErrors=true. These options are contradictory: one demands strict schema adherence (fail on a missing field), the other demands leniency (skip bad records). The JSON format refuses to construct the deserializer in that state.

Solutions

1. Set ignoreParseErrors=false if you want strict missing-field enforcement
2. Or set failOnMissingField=false if you want malformed rows skipped instead of failing the job
3. Prefer going through the JSON format factory (DDL WITH options) so validateDecodingFormatOptions rejects the combo early with a clearer message

Example fix

// before
new JsonRowDataDeserializationSchema(rowType, typeInfo, true /*failOnMissingField*/, true /*ignoreParseErrors*/, TimestampFormat.SQL);

// after: pick one strictness mode
new JsonRowDataDeserializationSchema(rowType, typeInfo, true, false, TimestampFormat.SQL);

if (failOnMissingField && ignoreParseErrors) {
    throw new IllegalArgumentException(
        "Choose one: failOnMissingField (strict) OR ignoreParseErrors (lenient)");
}

catch (IllegalArgumentException e) {
    // surface config conflict to operator with actionable message
}

When it happens

Trigger: Programmatically building JsonRowDataDeserializationSchema (or a subclass) with failOnMissingField=true and ignoreParseErrors=true; constructing the schema directly bypasses the Discovery/Factory validation that would normally reject this earlier at DDL time.

Common situations: Custom connector code creating the deserializer by hand; copying test code that sets both flags; upgrading where defaults changed and a hardcoded option set now conflicts.

Related errors

• Corrupt Maxwell JSON message
• Corrupt Ogg JSON message
• Debezium JSON serialization doesn't support
• fail-on-missing-field and ignore-parse-errors shouldn't…
• Failed to deserialize JSON

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1bee04c7664230dc. Report an issue: GitHub.

Appendix: source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/AbstractJsonDeserializationSchema.java:90

    protected transient ObjectMapper objectMapper;

    /** Timestamp format specification which is used to parse timestamp. */
    private final TimestampFormat timestampFormat;

    private final boolean hasDecimalType;

    private transient Collector<RowData> collector;

    private transient List<RowData> reusableCollectList;

    public AbstractJsonDeserializationSchema(
            RowType rowType,
            TypeInformation<RowData> resultTypeInfo,
            boolean failOnMissingField,
            boolean ignoreParseErrors,
            TimestampFormat timestampFormat) {
        if (ignoreParseErrors && failOnMissingField) {
            throw new IllegalArgumentException(
                    "JSON format doesn't support failOnMissingField and ignoreParseErrors are both enabled.");
        }
        this.resultTypeInfo = checkNotNull(resultTypeInfo);
        this.failOnMissingField = failOnMissingField;
        this.ignoreParseErrors = ignoreParseErrors;
        this.timestampFormat = timestampFormat;
        this.hasDecimalType = LogicalTypeChecks.hasNested(rowType, t -> t instanceof DecimalType);
    }

    @Override
    public void open(InitializationContext context) throws Exception {
        objectMapper =
                JacksonMapperFactory.createObjectMapper()
                        .configure(
                                JsonReadFeature.ALLOW_UNESCAPED_CONTROL_CHARS.mappedFeature(),
                                true);
        if (hasDecimalType) {
            objectMapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);

View on GitHub (pinned to 2f3c205e92)