apache/flink · error · CloneFailedException

Could not clone serializer instance of class {className}

Error message

Could not clone serializer instance of class {className}

What it means

KryoSerializer must be duplicated per thread; duplicating is done by deep-copying any SerializableSerializer instances held in Kryo registrations via InstantiationUtil.clone (Java serialization clone with the context classloader). If that Java-serialization clone fails with IOException or ClassNotFoundException, Flink throws CloneFailedException 'Could not clone serializer instance of class <className>'. The root failure is almost always that the custom Kryo serializer class is not serializable or not loadable.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/kryo/KryoSerializer.java:687

        // kryoRegistrations may be null if this Kryo serializer is deserialized from an old version
        if (kryoRegistrations == null) {
            this.kryoRegistrations =
                    buildKryoRegistrations(
                            type,
                            registeredTypes,
                            registeredTypesWithSerializerClasses,
                            registeredTypesWithSerializers,
                            TernaryBoolean.UNDEFINED);
        }
    }

    private SerializableSerializer<? extends Serializer<?>> deepCopySerializer(
            SerializableSerializer<? extends Serializer<?>> original) {
        try {
            return InstantiationUtil.clone(
                    original, Thread.currentThread().getContextClassLoader());
        } catch (IOException | ClassNotFoundException ex) {
            throw new CloneFailedException(
                    "Could not clone serializer instance of class " + original.getClass(), ex);
        }
    }

    // --------------------------------------------------------------------------------------------
    // For testing
    // --------------------------------------------------------------------------------------------

    private void enterExclusiveThread() {
        // we use simple get, check, set here, rather than CAS
        // we don't need lock-style correctness, this is only a sanity-check and we thus
        // favor speed at the cost of some false negatives in this check
        Thread previous = currentThread;
        Thread thisThread = Thread.currentThread();

        if (previous == null) {
            currentThread = thisThread;
        } else if (previous != thisThread) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Make the custom Kryo serializer class implement java.io.Serializable and mark any non-serializable fields transient (re-create them in readObject).
2. Prefer registering the serializer CLASS (registerTypeWithKryoSerializer(Class type, Class<? extends Serializer> serializerClass)) instead of an instance — class-based registration avoids the deep copy entirely.
3. Ensure the jar containing the serializer class is included in the job submission (flink run --jar / uber-jar).
4. Remove captured references: make the serializer a static top-level class rather than an inner class or lambda.

Example fix

// before
env.registerTypeWithKryoSerializer(MyType.class, new MyTypeSerializer(conn)); // not serializable

// after (class-based registration, no instance to clone)
env.registerTypeWithKryoSerializer(MyType.class, MyTypeSerializer.class);

// or make the instance serializable:
public class MyTypeSerializer extends Serializer<MyType> implements java.io.Serializable {
    private static final long serialVersionUID = 1L;
    private transient Connection conn;
}

static <T extends com.esotericsoftware.kryo.Serializer<?>> T checkCloneable(T ser) {
    try {
        org.apache.flink.util.InstantiationUtil.clone(
            new org.apache.flink.api.common.typeutils.base.SerializableSerializer<>(ser),
            Thread.currentThread().getContextClassLoader());
        return ser;
    } catch (Exception e) {
        throw new IllegalArgumentException("Serializer " + ser.getClass() + " must be Java-serializable", e);
    }
}

When it happens

Trigger: Registering a Kryo serializer instance (registeredTypesWithSerializers / registerTypeWithKryoSerializer) whose class does not implement java.io.Serializable; the serializer instance holding non-serializable fields; the serializer class not being on the user code classloader when the clone happens.

Common situations: Users pass a custom 'new MyKryoSerializer()' carrying a connection or configuration object; serializer classes defined in a jar not shipped with the job; serializer written as a lambda or inner class capturing an enclosing non-serializable object.

Related errors

• Error during Java serialization.
• Error during Java deserialization.
• The Kryo Output still contains data from a previous serializ
• Failed to serialize value '{value}'
• Cannot register null class or serializer.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/000d525cd1e49b5a. Report an issue: GitHub.