apache/flink · error · RuntimeException

Unable to deserialize message

Error message

Unable to deserialize message

What it means

Thrown by TypeInformationSerializationSchema.deserialize when the underlying TypeSerializer.deserialize throws an IOException. This schema wraps a TypeSerializer to convert byte[] to T; if the bytes are malformed, truncated, or were serialized with an incompatible serializer, deserialization fails. The IOException is wrapped in a RuntimeException because the DeserializationSchema.deserialize contract does not declare checked exceptions.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/serialization/TypeInformationSerializationSchema.java:94

            TypeInformation<T> typeInfo, TypeSerializer<T> serializer) {
        this.typeInfo = checkNotNull(typeInfo, "typeInfo");
        this.serializer = checkNotNull(serializer, "serializer");
    }

    // ------------------------------------------------------------------------

    @Override
    public T deserialize(byte[] message) {
        if (dis != null) {
            dis.setBuffer(message);
        } else {
            dis = new DataInputDeserializer(message);
        }

        try {
            return serializer.deserialize(dis);
        } catch (IOException e) {
            throw new RuntimeException("Unable to deserialize message", e);
        }
    }

    /**
     * This schema never considers an element to signal end-of-stream, so this method returns always
     * false.
     *
     * @param nextElement The element to test for the end-of-stream signal.
     * @return Returns false.
     */
    @Override
    public boolean isEndOfStream(T nextElement) {
        return false;
    }

    @Override
    public byte[] serialize(T element) {
        if (dos == null) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the byte[] was produced by a serializer compatible with the TypeInformation used to construct the schema.
2. If the schema changed, implement a TypeSerializerSnapshot for migration and ensure the source produces the new format.
3. Catch the RuntimeException at the operator level and route bad messages to a dead-letter side output instead of failing the job.

Example fix

// before: raw deserialize can crash the job
@Override
public T deserialize(byte[] message) {
    return schema.deserialize(message); // throws RuntimeException on bad bytes
}

// after: catch and route to side output
try {
    return schema.deserialize(message);
} catch (RuntimeException e) {
    ctx.output(deadLetterTag, message);
    return null;
}

// Validate message is non-null and non-empty before deserializing
if (message == null || message.length == 0) {
    return null; // or route to error handling
}

try {
    return schema.deserialize(message);
} catch (RuntimeException e) {
    // route to dead-letter / side output instead of failing the pipeline
    return null;
}

When it happens

Trigger: Feeding byte[] that was not produced by the same TypeSerializer (schema mismatch), truncated messages, or corrupted bytes to the deserialize method. The serializer attempts to read fields and hits an IOException (e.g. EOFException, unexpected data).

Common situations: Changing the type schema or serializer version without a migration snapshot; consuming messages from a source serialized by a different producer; network corruption; partial messages from a misbehaving source.

Related errors

• Failed to serialize ExecutionPlan.
• Failed to deserialize an element from the source. If you are
• Reached the end of the collection. This could be caused by i
• Failed to deserialize an element from the source. If you are
• Unable to serialize record

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/a1b632657c80b608. Report an issue: GitHub.