apache/flink · error · java.lang.IllegalStateException

Illegal JSON array data...

Error message

Illegal JSON array data...

What it means

IllegalStateException from createArrayConverter's lambda when the current token is not START_ARRAY while converting an ARRAY column — i.e., the JSON value at that position is a scalar/object where the schema expects '[' ... ']'. It indicates a data-shape mismatch, not parser corruption.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonParserToRowDataConverters.java:354

        final int scale = decimalType.getScale();
        return jp -> {
            BigDecimal bigDecimal;
            if (jp.currentToken() == JsonToken.VALUE_STRING) {
                bigDecimal = new BigDecimal(jp.getText().trim());
            } else {
                bigDecimal = jp.getDecimalValue();
            }
            return DecimalData.fromBigDecimal(bigDecimal, precision, scale);
        };
    }

    private JsonParserToRowDataConverter createArrayConverter(ArrayType arrayType) {
        JsonParserToRowDataConverter elementConverter = createConverter(arrayType.getElementType());
        final Class<?> elementClass =
                LogicalTypeUtils.toInternalConversionClass(arrayType.getElementType());
        return jp -> {
            if (jp.currentToken() != JsonToken.START_ARRAY) {
                throw new IllegalStateException("Illegal JSON array data...");
            }
            List<Object> result = new ArrayList<>();
            while (jp.nextToken() != JsonToken.END_ARRAY) {
                Object convertField = elementConverter.convert(jp);
                result.add(convertField);
            }
            final Object[] array = (Object[]) Array.newInstance(elementClass, result.size());
            return new GenericArrayData(result.toArray(array));
        };
    }

    private JsonParserToRowDataConverter createMapConverter(
            String typeSummary, LogicalType keyType, LogicalType valueType) {
        if (!keyType.is(LogicalTypeFamily.CHARACTER_STRING)) {
            throw new UnsupportedOperationException(
                    "JSON format doesn't support non-string as key type of map. "
                            + "The type is: "
                            + typeSummary);

View on GitHub (pinned to 2f3c205e92)

Solutions

  1. Align the DDL with the real payload shape (STRING instead of ARRAY, or ARRAY if data is consistently a list)
  2. Fix the producer to always emit an array, even for single elements
  3. Sample the topic/files with the actual JSON to confirm the field's real shape before fixing the schema

Example fix

-- before
tags ARRAY<STRING>   -- data is "tags": "a,b"

-- after
tags STRING
Defensive patterns

Strategy: validation

Validate before calling

// sample check: field declared ARRAY must always be a JSON array
JsonNode n = mapper.readTree(sample).get("tags");
if (!n.isArray()) throw new IllegalStateException("tags must be an array in every record");

Try / catch

catch (IllegalStateException e) {
    if (e.getMessage().contains("Illegal JSON array data")) {
        // payload shape != ARRAY<...> in DDL; align schema or producer
    }
}

Prevention

When it happens

Trigger: Schema says ARRAY<...> but the payload has "tags": "a,b" or "tags": {"a":1}; also nested arrays where the element converter hits a non-array token.

Common situations: Producers sometimes emitting a single value instead of a list for one-element arrays; schema evolution from scalar to array; hand-written DDL guessing the wrong shape.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/bce885e21eb2210c. Report an issue: GitHub.