apache/flink · error · IllegalArgumentException

JSON format doesn't support failOnMissingField and ignorePar

Error message

JSON format doesn't support failOnMissingField and ignoreParseErrors are both enabled.

What it means

IllegalArgumentException from the AbstractJsonDeserializationSchema constructor when both failOnMissingField=true and ignoreParseErrors=true. These options are contradictory: one demands strict schema adherence (fail on a missing field), the other demands leniency (skip bad records). The JSON format refuses to construct the deserializer in that state.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/AbstractJsonDeserializationSchema.java:90

    protected transient ObjectMapper objectMapper;

    /** Timestamp format specification which is used to parse timestamp. */
    private final TimestampFormat timestampFormat;

    private final boolean hasDecimalType;

    private transient Collector<RowData> collector;

    private transient List<RowData> reusableCollectList;

    public AbstractJsonDeserializationSchema(
            RowType rowType,
            TypeInformation<RowData> resultTypeInfo,
            boolean failOnMissingField,
            boolean ignoreParseErrors,
            TimestampFormat timestampFormat) {
        if (ignoreParseErrors && failOnMissingField) {
            throw new IllegalArgumentException(
                    "JSON format doesn't support failOnMissingField and ignoreParseErrors are both enabled.");
        }
        this.resultTypeInfo = checkNotNull(resultTypeInfo);
        this.failOnMissingField = failOnMissingField;
        this.ignoreParseErrors = ignoreParseErrors;
        this.timestampFormat = timestampFormat;
        this.hasDecimalType = LogicalTypeChecks.hasNested(rowType, t -> t instanceof DecimalType);
    }

    @Override
    public void open(InitializationContext context) throws Exception {
        objectMapper =
                JacksonMapperFactory.createObjectMapper()
                        .configure(
                                JsonReadFeature.ALLOW_UNESCAPED_CONTROL_CHARS.mappedFeature(),
                                true);
        if (hasDecimalType) {
            objectMapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set ignoreParseErrors=false if you want strict missing-field enforcement
2. Or set failOnMissingField=false if you want malformed rows skipped instead of failing the job
3. Prefer going through the JSON format factory (DDL WITH options) so validateDecodingFormatOptions rejects the combo early with a clearer message

Example fix

// before
new JsonRowDataDeserializationSchema(rowType, typeInfo, true /*failOnMissingField*/, true /*ignoreParseErrors*/, TimestampFormat.SQL);

// after: pick one strictness mode
new JsonRowDataDeserializationSchema(rowType, typeInfo, true, false, TimestampFormat.SQL);

if (failOnMissingField && ignoreParseErrors) {
    throw new IllegalArgumentException(
        "Choose one: failOnMissingField (strict) OR ignoreParseErrors (lenient)");
}

catch (IllegalArgumentException e) {
    // surface config conflict to operator with actionable message
}

When it happens

Trigger: Programmatically building JsonRowDataDeserializationSchema (or a subclass) with failOnMissingField=true and ignoreParseErrors=true; constructing the schema directly bypasses the Discovery/Factory validation that would normally reject this earlier at DDL time.

Common situations: Custom connector code creating the deserializer by hand; copying test code that sets both flags; upgrading where defaults changed and a hardcoded option set now conflicts.

Related errors

• Option %s.%s is required for serialization
• Please invoke DeserializationSchema#deserialize(byte[], Coll
• Unsupported timestamp format '%s'. Validator should have che
• Unsupported map null key handling mode '%s'. Validator shoul
• fail-on-missing-field and ignore-parse-errors shouldn't both

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1bee04c7664230dc. Report an issue: GitHub.