apache/flink · error · RuntimeException

Could not load deserializer from the configuration.

Error message

Could not load deserializer from the configuration.

What it means

RuntimeSerializerFactory.readParametersFromConfig re-reads the stored class and serializer objects from the Configuration on the cluster side. This error covers any deserialization failure other than ClassNotFoundException (which is rethrown as-is): corrupted byte stream, incompatible class versions, or a failing custom readObject in the serializer.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/RuntimeSerializerFactory.java:76

        } catch (Exception e) {
            throw new RuntimeException("Could not serialize serializer into the configuration.", e);
        }
    }

    @Override
    public void readParametersFromConfig(Configuration config, ClassLoader cl)
            throws ClassNotFoundException {
        if (config == null || cl == null) {
            throw new NullPointerException();
        }

        try {
            this.clazz = InstantiationUtil.readObjectFromConfig(config, CONFIG_KEY_CLASS, cl);
            this.serializer = InstantiationUtil.readObjectFromConfig(config, CONFIG_KEY_SER, cl);
        } catch (ClassNotFoundException e) {
            throw e;
        } catch (Exception e) {
            throw new RuntimeException("Could not load deserializer from the configuration.", e);
        }
    }

    @Override
    public TypeSerializer<T> getSerializer() {
        if (this.serializer != null) {
            return this.serializer.duplicate();
        } else {
            throw new RuntimeException(
                    "SerializerFactory has not been initialized from configuration.");
        }
    }

    @Override
    public Class<T> getDataType() {
        return clazz;
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Read the 'Caused by' chain: InvalidClassException(local class incompatible) means version skew -> align user jars on client and all TaskManagers.
2. Add an explicit private static final long serialVersionUID to the custom serializer so benign refactors do not break compatibility.
3. Make custom readObject defensive: no I/O or external resource access during deserialization; lazy-initialize instead.
4. Do not persist/reuse Configuration blobs across jobs; write and read them within one submission cycle.

Example fix

// before
public class MySer extends TypeSerializer<MyPojo> implements Serializable {
    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        in.defaultReadObject();
        this.schema = RegistryClient.load(); // throws on TM -> 684
    }
}

// after
public class MySer extends TypeSerializer<MyPojo> implements Serializable {
    private static final long serialVersionUID = 1L;
    private transient Schema schema;
    private Schema schema() { if (schema == null) schema = RegistryClient.load(); return schema; }
}

try {
    factory.readParametersFromConfig(config, cl);
} catch (ClassNotFoundException e) {
    // rethrown as-is: missing class in user jar
} catch (RuntimeException e) {
    // InvalidClassException/StreamCorruptedException in cause: version skew or corruption
    if (e.getCause() instanceof java.io.InvalidClassException) { /* realign user jars */ }
    throw e;
}

When it happens

Trigger: TaskManager reads CONFIG_KEY_CLASS/CONFIG_KEY_SER and ObjectInputStream fails with InvalidClassException (serialVersionUID mismatch), StreamCorruptedException, or the serializer's readObject throws a RuntimeException. Happens when the user jar on the TaskManager differs from the client's, or the Configuration was carried over from an incompatible run.

Common situations: Cluster running an older user jar while the client submits with a newer serializer class (or vice versa). Adding serialVersionUID after jobs were serialized. Serializer state whose custom readObject depends on resources unavailable at deserialization time (e.g. opens a file).

Related errors

• Could not serialize serializer into the configuration.
• Cannot deserialize and unwrap accumulators properly.
• Failed to deserialize coordination response
• Cannot deserialize and unwrap accumulators properly.
• Invalid version %d

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/a315e8e38cefbeed. Report an issue: GitHub.