apache/flink · error · org.apache.flink.formats.json.JsonParseException

Unable to deserialize byte array.

Error message

Unable to deserialize byte array.

What it means

Thrown by JsonToRowDataConverters.convertToBytes when JsonNode.binaryValue() raises an IOException. binaryValue() only succeeds on textual nodes holding valid Base64 (or binary nodes); any other shape (numbers, objects, malformed Base64 text) fails. The target Flink field is declared BINARY/VARBINARY, so the deserializer tries to Base64-decode the JSON value into a byte array.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonToRowDataConverters.java:289

            return StringData.fromString(jsonNode.toString());
        } else {
            return StringData.fromString(jsonNode.asText());
        }
    }

    private BinaryVariant convertToVariant(JsonNode jsonNode) {
        try {
            return BinaryVariantInternalBuilder.parseJson(jsonNode.toString(), false);
        } catch (IOException e) {
            throw new JsonParseException("Unable to deserialize VARIANT value.", e);
        }
    }

    private byte[] convertToBytes(JsonNode jsonNode) {
        try {
            return jsonNode.binaryValue();
        } catch (IOException e) {
            throw new JsonParseException("Unable to deserialize byte array.", e);
        }
    }

    private JsonToRowDataConverter createDecimalConverter(DecimalType decimalType) {
        final int precision = decimalType.getPrecision();
        final int scale = decimalType.getScale();
        return jsonNode -> {
            BigDecimal bigDecimal;
            if (jsonNode.isBigDecimal()) {
                bigDecimal = jsonNode.decimalValue();
            } else {
                bigDecimal = new BigDecimal(jsonNode.asText());
            }
            return DecimalData.fromBigDecimal(bigDecimal, precision, scale);
        };
    }

    private JsonToRowDataConverter createArrayConverter(ArrayType arrayType) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the JSON value for the BINARY column is a Base64-encoded string (e.g. via an online decoder or jq)
2. If the payload is plain text, change the column type from BINARY/VARBINARY to STRING/VARCHAR in the DDL
3. If the payload is hex-encoded, pre-process upstream, or deserialize as STRING and convert with hex decoding in a UDF
4. Add sample-record validation before deploying the schema (consume one message and check field shapes)

Example fix

// before
`data` BINARY,   -- JSON field "data": "aGVsbG8=" ok; "data": 123 fails

// after
`data` STRING    -- when the field carries non-Base64 text

// Verify the JSON field is Base64 text before using BINARY in the DDL
String s = node.get("data").asText();
try { Base64.getDecoder().decode(s); } catch (IllegalArgumentException e) { /* not base64: use STRING or fix upstream */ }

catch (JsonParseException e) for 'Unable to deserialize byte array' — log the field value, treat as data error; do not retry (deterministic).

When it happens

Trigger: A column declared BINARY or VARBINARY in the JSON-format table schema while the corresponding JSON field is a number, object/array, or a string that is not valid Base64 (e.g. "hello world!!"). Reached via JsonRowDataDeserializationSchema on any source using format 'json'.

Common situations: Schema declared BINARY but upstream sends raw non-Base64 strings; column-type mix-ups when hand-writing DDL against an existing JSON stream; producers that emit binary data hex-encoded instead of Base64.

Related errors

• JSON format doesn't support non-string as key type of map. T
• JSON format doesn't support failOnMissingField and ignorePar
• Please invoke DeserializationSchema#deserialize(byte[], Coll
• Failed to deserialize JSON '%s'.
• Unsupported type: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1007377aa77f68cb. Report an issue: GitHub.