apache/flink · error · CloneFailedException

Could not clone serializer instance of class

Error message

Could not clone serializer instance of class {className}

What it means

KryoSerializer must be duplicated per thread; duplicating is done by deep-copying any SerializableSerializer instances held in Kryo registrations via InstantiationUtil.clone (Java serialization clone with the context classloader). If that Java-serialization clone fails with IOException or ClassNotFoundException, Flink throws CloneFailedException 'Could not clone serializer instance of class <className>'. The root failure is almost always that the custom Kryo serializer class is not serializable or not loadable.

Solutions

1. Make the custom Kryo serializer class implement java.io.Serializable and mark any non-serializable fields transient (re-create them in readObject).
2. Prefer registering the serializer CLASS (registerTypeWithKryoSerializer(Class type, Class<? extends Serializer> serializerClass)) instead of an instance — class-based registration avoids the deep copy entirely.
3. Ensure the jar containing the serializer class is included in the job submission (flink run --jar / uber-jar).
4. Remove captured references: make the serializer a static top-level class rather than an inner class or lambda.

Example fix

// before
env.registerTypeWithKryoSerializer(MyType.class, new MyTypeSerializer(conn)); // not serializable

// after (class-based registration, no instance to clone)
env.registerTypeWithKryoSerializer(MyType.class, MyTypeSerializer.class);

// or make the instance serializable:
public class MyTypeSerializer extends Serializer<MyType> implements java.io.Serializable {
    private static final long serialVersionUID = 1L;
    private transient Connection conn;
}

static <T extends com.esotericsoftware.kryo.Serializer<?>> T checkCloneable(T ser) {
    try {
        org.apache.flink.util.InstantiationUtil.clone(
            new org.apache.flink.api.common.typeutils.base.SerializableSerializer<>(ser),
            Thread.currentThread().getContextClassLoader());
        return ser;
    } catch (Exception e) {
        throw new IllegalArgumentException("Serializer " + ser.getClass() + " must be Java-serializable", e);
    }
}

When it happens

Trigger: Registering a Kryo serializer instance (registeredTypesWithSerializers / registerTypeWithKryoSerializer) whose class does not implement java.io.Serializable; the serializer instance holding non-serializable fields; the serializer class not being on the user code classloader when the clone happens.

Common situations: Users pass a custom 'new MyKryoSerializer()' carrying a connection or configuration object; serializer classes defined in a jar not shipped with the job; serializer written as a lambda or inner class capturing an enclosing non-serializable object.

Related errors

• Error during Java deserialization.
• Error during Java serialization.
• The Kryo Output still contains data from a previous…
• Buffer too small: capacity
• Buffer underflow.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/000d525cd1e49b5a. Report an issue: GitHub.

Appendix: source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java:687

        // kryoRegistrations may be null if this Kryo serializer is deserialized from an old version
        if (kryoRegistrations == null) {
            this.kryoRegistrations =
                    buildKryoRegistrations(
                            type,
                            registeredTypes,
                            registeredTypesWithSerializerClasses,
                            registeredTypesWithSerializers,
                            TernaryBoolean.UNDEFINED);
        }
    }

    private SerializableSerializer<? extends Serializer<?>> deepCopySerializer(
            SerializableSerializer<? extends Serializer<?>> original) {
        try {
            return InstantiationUtil.clone(
                    original, Thread.currentThread().getContextClassLoader());
        } catch (IOException | ClassNotFoundException ex) {
            throw new CloneFailedException(
                    "Could not clone serializer instance of class " + original.getClass(), ex);
        }
    }

    // --------------------------------------------------------------------------------------------
    // For testing
    // --------------------------------------------------------------------------------------------

    private void enterExclusiveThread() {
        // we use simple get, check, set here, rather than CAS
        // we don't need lock-style correctness, this is only a sanity-check and we thus
        // favor speed at the cost of some false negatives in this check
        Thread previous = currentThread;
        Thread thisThread = Thread.currentThread();

        if (previous == null) {
            currentThread = thisThread;
        } else if (previous != thisThread) {

View on GitHub (pinned to 2f3c205e92)