apache/flink · error · IllegalArgumentException

Duplicated serializer for the same class.

Error message

Duplicated serializer for the same class.

What it means

Thrown inside the Collectors.toMap merge function during parseSerializationConfig when two or more entries in the serialization-config map target the same class. The merge function is only invoked on key collisions, so this means the configured class appears more than once with different serializer definitions. The system cannot decide which serializer wins, so it fails fast.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/serialization/SerializerConfigImpl.java:398

        }
    }

    private void parseSerializationConfig(
            ClassLoader classLoader, List<String> serializationConfigs) {
        final LinkedHashMap<Class<?>, Map<String, String>> serializationConfigByClass =
                serializationConfigs.stream()
                        .map(ConfigurationUtils::parseStringToMap)
                        .flatMap(m -> m.entrySet().stream())
                        .collect(
                                Collectors.toMap(
                                        e ->
                                                loadClass(
                                                        e.getKey(),
                                                        classLoader,
                                                        "Could not load class for serialization config"),
                                        e -> ConfigurationUtils.parseStringToMap(e.getValue()),
                                        (v1, v2) -> {
                                            throw new IllegalArgumentException(
                                                    "Duplicated serializer for the same class.");
                                        },
                                        LinkedHashMap::new));
        for (Map.Entry<Class<?>, Map<String, String>> entry :
                serializationConfigByClass.entrySet()) {
            Class<?> type = entry.getKey();
            Map<String, String> config = entry.getValue();
            String configType = config.get("type");
            if (configType == null) {
                throw new IllegalArgumentException("Serializer type not specified for " + type);
            }
            switch (configType) {
                case "pojo":
                    registerPojoType(type);
                    break;
                case "kryo":
                    parseAndRegisterKryoType(classLoader, type, config);
                    break;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Deduplicate the serialization-config entries so each class appears exactly once.
2. If merging configs programmatically, use a Map to ensure last-write-wins or detect conflicts before passing to Flink.
3. Audit the full config string for repeated class names.

Example fix

# before (duplicate class)
pipeline.serialization-config: "class:com.example.MyType{type:kryo};class:com.example.MyType{type:pojo}"

# after (single entry)
pipeline.serialization-config: "class:com.example.MyType{type:kryo}"

// Deduplicate config entries before passing to Flink
Map<String, Map<String, String>> deduped = new LinkedHashMap<>();
for (Map.Entry<String, Map<String, String>> e : rawEntries.entrySet()) {
    if (deduped.containsKey(e.getKey())) {
        throw new IllegalStateException("Duplicate serializer entry for: " + e.getKey());
    }
    deduped.put(e.getKey(), e.getValue());
}

When it happens

Trigger: Setting pipeline.serialization-config with the same fully-qualified class name appearing as a key in two separate config entries. For example, listing 'com.example.MyType' twice in the config string.

Common situations: Merging serialization configs from multiple sources (default + override) without deduplication; copy-paste errors in config files; YAML anchors producing duplicate keys after expansion.

Related errors

• Could not configure serializers from %s.
• Serializer type not specified for {}
• Unsupported serializer type %s for %s
• A TypeInfoFactory for type '{}' is already registered.
• Cannot register null class or serializer.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d98582a5b2fc69a0. Report an issue: GitHub.