apache/kafka · error · ConfigException

Invalid value null for configuration key.serializer: must be

Error message

Invalid value null for configuration key.serializer: must be non-null.

What it means

Thrown by ProducerConfig.appendSerializerToConfig when no key serializer can be determined. The producer needs a Serializer for every record key; if neither a keySerializer instance was passed to the KafkaProducer constructor nor a `key.serializer` class is present in the config map, this ConfigException is raised naming the KEY_SERIALIZER_CLASS_CONFIG with value null.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java:721

    }

    private static String parseAcks(String acksString) {
        try {
            return acksString.trim().equalsIgnoreCase("all") ? "-1" : Short.parseShort(acksString.trim()) + "";
        } catch (NumberFormatException e) {
            throw new ConfigException("Invalid configuration value for 'acks': " + acksString);
        }
    }

    static Map<String, Object> appendSerializerToConfig(Map<String, Object> configs,
            Serializer<?> keySerializer,
            Serializer<?> valueSerializer) {
        // validate serializer configuration, if the passed serializer instance is null, the user must explicitly set a valid serializer configuration value
        Map<String, Object> newConfigs = new HashMap<>(configs);
        if (keySerializer != null)
            newConfigs.put(KEY_SERIALIZER_CLASS_CONFIG, keySerializer.getClass());
        else if (newConfigs.get(KEY_SERIALIZER_CLASS_CONFIG) == null)
            throw new ConfigException(KEY_SERIALIZER_CLASS_CONFIG, null, "must be non-null.");
        if (valueSerializer != null)
            newConfigs.put(VALUE_SERIALIZER_CLASS_CONFIG, valueSerializer.getClass());
        else if (newConfigs.get(VALUE_SERIALIZER_CLASS_CONFIG) == null)
            throw new ConfigException(VALUE_SERIALIZER_CLASS_CONFIG, null, "must be non-null.");
        return newConfigs;
    }

    /**
     * Constructs a new ProducerConfig with the given properties.
     *
     * @param props The producer configuration properties
     */
    public ProducerConfig(Properties props) {
        super(CONFIG, props);
    }

    /**
     * Constructs a new ProducerConfig with the given configuration map.

View on GitHub (pinned to c31c9215e1)

Solutions

Set `key.serializer` to the fully-qualified name of an appropriate Serializer, e.g. org.apache.kafka.common.serialization.StringSerializer.
Alternatively pass a Serializer<?> instance as the second argument to the KafkaProducer constructor.
For Avro/Protobuf, use the corresponding Confluent or framework serializer class.

Example fix

// before
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
new KafkaProducer<String,String>(props);

// after
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
new KafkaProducer<String,String>(props);

Defensive patterns

Strategy: type-guard

Validate before calling

// Ensure a key serializer is present BEFORE constructing KafkaProducer.
static void ensureKeySerializer(Map<String, Object> cfg, Serializer<?> keySer) {
    if (keySer != null) return;
    Object cls = cfg.get(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG);
    if (cls == null || (cls instanceof String && ((String) cls).isEmpty()))
        throw new IllegalArgumentException("key.serializer must be set (class or instance)");
}
// Caller:
ensureKeySerializer(cfg, keySerializer);

Type guard

// Narrow to a non-null serializer guarantee.
static boolean hasKeySerializer(Map<String, Object> cfg, Serializer<?> instance) {
    if (instance != null) return true;
    Object v = cfg.get(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG);
    return v instanceof Class || (v instanceof String && !((String) v).isEmpty());
}
// Use: assert hasKeySerializer(cfg, keySerializer);

Try / catch

try {
    producer = new KafkaProducer<>(cfg, keySerializer, valueSerializer);
} catch (ConfigException e) {
    if (e.getMessage().contains("key.serializer")) {
        // Pick a serializer matching your key type, e.g. StringSerializer.
        keySerializer = new org.apache.kafka.common.serialization.StringSerializer();
        producer = new KafkaProducer<>(cfg, keySerializer, valueSerializer);
    } else throw e;
}

Prevention

Always pass an explicit Serializer<KeyType> instance, or set key.serializer.class, keyed to your actual key type.
Match serializer to key type at compile time with generics: new KafkaProducer<K,V>(cfg, Serdes, Serdes).
Beware copy-pasting a value-only config template — key.serializer is the most commonly omitted sibling.

When it happens

Trigger: Constructing `new KafkaProducer<>(props)` where props has no `key.serializer` entry AND no key Serializer instance is passed; or constructing `new KafkaProducer<>(props, null, valueSerializer)`.

Common situations: First-time producer setup where the developer forgot the serializer property; using a properties file that failed to load; or programmatically building a Properties object and forgetting to call props.put("key.serializer", ...). Common with custom key types like UUID or Avro where the serializer class name must be spelled fully.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/dfcdb46ba9d20427.json. Report an issue: GitHub.