apache/flink · error · IllegalArgumentException

The record must not be null.

Error message

The record must not be null.

What it means

BytePrimitiveArraySerializer writes the array length then the raw bytes. serialize() rejects null because a null array has no length to emit and primitive-array type information is non-nullable by contract.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/typeutils/base/array/BytePrimitiveArraySerializer.java:70

        byte[] copy = new byte[from.length];
        System.arraycopy(from, 0, copy, 0, from.length);
        return copy;
    }

    @Override
    public byte[] copy(byte[] from, byte[] reuse) {
        return copy(from);
    }

    @Override
    public int getLength() {
        return -1;
    }

    @Override
    public void serialize(byte[] record, DataOutputView target) throws IOException {
        if (record == null) {
            throw new IllegalArgumentException("The record must not be null.");
        }

        final int len = record.length;
        target.writeInt(len);
        target.write(record);
    }

    @Override
    public byte[] deserialize(DataInputView source) throws IOException {
        final int len = source.readInt();
        byte[] result = new byte[len];
        source.readFully(result);
        return result;
    }

    @Override
    public byte[] deserialize(byte[] reuse, DataInputView source) throws IOException {
        return deserialize(source);

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Initialize byte[] fields to an empty array instead of null before they reach the sink/state.
2. Filter out records with null arrays upstream.
3. If nullability is legitimate, use a nullable/object type (e.g., Byte[] via ObjectArraySerializer) or a Row with null handling.
4. Add a null check in your mapper to replace null arrays with empty arrays at the serialization boundary.

Example fix

// before: out.payload may be null -> serializer.serialize(payload) throws
// after: data.map(r -> { if (r.payload == null) r.payload = new byte[0]; return r; })

// Validate before serialize
byte[] safe = record == null ? new byte[0] : record;
serializer.serialize(safe, target);

try {
    serializer.serialize(record, target);
} catch (IllegalArgumentException e) {
    if (e.getMessage().equals("The record must not be null.")) {
        record = new byte[0];
        serializer.serialize(record, target);
    } else throw e;
}

When it happens

Trigger: Calling serialize(null, target) on the byte[] serializer — a field typed byte[] that resolved to null at runtime.

Common situations: A binary field typed byte[] that is null due to a null source value; a UDF returning null where a byte array is expected; a nullable SQL BINARY/VARBINARY column mapped to a primitive byte[].

Related errors

• The record must not be null.
• The record must not be null.
• The record must not be null.
• The record must not be null.
• The record must not be null.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/34984eabdc8d0051. Report an issue: GitHub.