apache/kafka · error · ConfigException

Invalid value null for configuration key.deserializer: must

Error message

Invalid value null for configuration key.deserializer: must be non-null.

What it means

Thrown by ConsumerConfig.appendDeserializerToConfig when neither a key Deserializer instance nor the key.deserializer configuration property is provided. Every KafkaConsumer must know how to turn byte[] keys into objects; the client refuses to construct without one. It is a ConfigException raised during KafkaConsumer initialization.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java:798

            String groupInstanceId = this.getString(GROUP_INSTANCE_ID_CONFIG);
            if (groupInstanceId != null)
                JoinGroupRequest.validateGroupInstanceId(groupInstanceId);

            String groupInstanceIdPart = groupInstanceId != null ? groupInstanceId : CONSUMER_CLIENT_ID_SEQUENCE.getAndIncrement() + "";
            String generatedClientId = String.format("consumer-%s-%s", groupId, groupInstanceIdPart);
            configs.put(CLIENT_ID_CONFIG, generatedClientId);
        }
    }

    public static Map<String, Object> appendDeserializerToConfig(Map<String, Object> configs,
                                                                 Deserializer<?> keyDeserializer,
                                                                 Deserializer<?> valueDeserializer) {
        // validate deserializer configuration, if the passed deserializer instance is null, the user must explicitly set a valid deserializer configuration value
        Map<String, Object> newConfigs = new HashMap<>(configs);
        if (keyDeserializer != null)
            newConfigs.put(KEY_DESERIALIZER_CLASS_CONFIG, keyDeserializer.getClass());
        else if (newConfigs.get(KEY_DESERIALIZER_CLASS_CONFIG) == null)
            throw new ConfigException(KEY_DESERIALIZER_CLASS_CONFIG, null, "must be non-null.");
        if (valueDeserializer != null)
            newConfigs.put(VALUE_DESERIALIZER_CLASS_CONFIG, valueDeserializer.getClass());
        else if (newConfigs.get(VALUE_DESERIALIZER_CLASS_CONFIG) == null)
            throw new ConfigException(VALUE_DESERIALIZER_CLASS_CONFIG, null, "must be non-null.");
        return newConfigs;
    }

    private void maybeOverrideEnableAutoCommit(Map<String, Object> configs) {
        Optional<String> groupId = Optional.ofNullable(getString(CommonClientConfigs.GROUP_ID_CONFIG));
        Map<String, Object> originals = originals();
        boolean enableAutoCommit = originals.containsKey(ENABLE_AUTO_COMMIT_CONFIG) ? getBoolean(ENABLE_AUTO_COMMIT_CONFIG) : false;
        if (groupId.isEmpty()) { // overwrite in case of default group id where the config is not explicitly provided
            if (!originals.containsKey(ENABLE_AUTO_COMMIT_CONFIG)) {
                configs.put(ENABLE_AUTO_COMMIT_CONFIG, false);
            } else if (enableAutoCommit) {
                throw new InvalidConfigurationException(ENABLE_AUTO_COMMIT_CONFIG + " cannot be set to true when default group id (null) is used.");
            }
        }

View on GitHub (pinned to c31c9215e1)

Solutions

Set props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); before constructing the consumer.
Or pass a non-null key Deserializer instance to the KafkaConsumer(Map, Deserializer, Deserializer) constructor.
Validate config keys at startup (log ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG) to catch typos like key.deserializer.class.
If keys are intentionally unused, still provide a ByteArrayDeserializer for the key.

Example fix

// before
Properties props = new Properties();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
new KafkaConsumer<>(props); // throws: key.deserializer null

// after
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
new KafkaConsumer<>(props);

Defensive patterns

Strategy: validation

Validate before calling

// Ensure key.deserializer is set before constructing the consumer:
Properties props = new Properties();
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
// or pass a Deserializer instance to the KafkaConsumer constructor
if (props.get(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG) == null && keyDeserializer == null) {
    throw new ConfigException(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, null, "must be non-null");
}

Type guard

static boolean hasKeyDeserializer(Map<String, Object> configs, Deserializer<?> instance) {
    return instance != null || configs.get(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG) != null;
}

Try / catch

try {
    new KafkaConsumer<>(props);
} catch (ConfigException e) {
    if (e.getMessage().contains("key.deserializer")) {
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        new KafkaConsumer<>(props);
    } else throw e;
}

Prevention

Always set key.deserializer (either the config property or pass a Deserializer instance to the KafkaConsumer constructor).
Build a helper or config builder that injects default deserializers so a missing one is impossible.
Use the KafkaConsumer(Map, Deserializer<K>, Deserializer<V>) constructor when you already have instances - it also covers the value side.

When it happens

Trigger: new KafkaConsumer<>(props) (or KafkaConsumer(Map, Deserializer, Deserializer) with keyDeserializer=null) where props does not contain key.deserializer, or where key.deserializer is explicitly set to null.

Common situations: Forgetting to set key.deserializer when building a Properties/Map consumer; passing a non-null value Deserializer but a null key Deserializer to the typed constructor without also setting the config; mis-spelling the property name (e.g. key.deserializer.class); reading config from a file that silently dropped the key.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/d505eacb19bee447.json. Report an issue: GitHub.