apache/kafka · error · KafkaException

Failed to construct Kafka consumer

Error message

Failed to construct Kafka consumer

What it means

Thrown from the public ConsumerDelegateCreator.create when constructing the chosen consumer delegate (AsyncKafkaConsumer for group.protocol=consumer, or ClassicKafkaConsumer otherwise) raises any Throwable that is not already a KafkaException. KafkaException subclasses are rethrown unchanged (line 67-68) so callers see the original config error; everything else (NullPointerException, IllegalArgumentException, reflective invocation failures, deserializer construction errors) is wrapped in this KafkaException with the original as the cause. This is the top-level failure surfaced from new KafkaConsumer(...).

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerDelegateCreator.java:70

 * not attempt to determine the underlying implementation to avoid coding to an unstable interface. Rather, it is
 * the {@link Consumer} API contract that should serve as the caller's interface.
 */
public class ConsumerDelegateCreator {

    public <K, V> ConsumerDelegate<K, V> create(ConsumerConfig config,
                                                Deserializer<K> keyDeserializer,
                                                Deserializer<V> valueDeserializer) {
        try {
            GroupProtocol groupProtocol = GroupProtocol.valueOf(config.getString(ConsumerConfig.GROUP_PROTOCOL_CONFIG).toUpperCase(Locale.ROOT));

            if (groupProtocol == GroupProtocol.CONSUMER)
                return new AsyncKafkaConsumer<>(config, keyDeserializer, valueDeserializer, Optional.empty());
            else
                return new ClassicKafkaConsumer<>(config, keyDeserializer, valueDeserializer);
        } catch (KafkaException e) {
            throw e;
        } catch (Throwable t) {
            throw new KafkaException("Failed to construct Kafka consumer", t);
        }
    }

    public <K, V> ConsumerDelegate<K, V> create(LogContext logContext,
                                                Time time,
                                                ConsumerConfig config,
                                                Deserializer<K> keyDeserializer,
                                                Deserializer<V> valueDeserializer,
                                                KafkaClient client,
                                                SubscriptionState subscriptions,
                                                ConsumerMetadata metadata,
                                                List<ConsumerPartitionAssignor> assignors) {
        try {
            GroupProtocol groupProtocol = GroupProtocol.valueOf(config.getString(ConsumerConfig.GROUP_PROTOCOL_CONFIG).toUpperCase(Locale.ROOT));

            if (groupProtocol == GroupProtocol.CONSUMER)
                return new AsyncKafkaConsumer<>(
                    logContext,

View on GitHub (pinned to c31c9215e1)

Solutions

1. Read KafkaException#getCause to see the real class and message of the underlying failure.
2. If using custom Deserializers, ensure they have a public no-arg constructor or are passed as instances (not class names).
3. Validate all config values are non-null and correctly typed before constructing the consumer.
4. Verify group.protocol is one of the supported values (classic / consumer) and that partition.assignment.strategy / interceptor.classes entries exist on the classpath.

Example fix

// before
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, MyJsonDeserializer.class.getName());
// MyJsonDeserializer has no no-arg ctor -> Throwable wrapped here
new KafkaConsumer<>(props);

// after
// give the deserializer a no-arg constructor, OR pass an instance:
new KafkaConsumer<>(props, new MyJsonDeserializer(objectMapper), new MyJsonDeserializer(objectMapper));

for (String k : java.util.List.of("bootstrap.servers", "key.deserializer", "value.deserializer")) {
    if (configs.get(k) == null) {
        throw new IllegalArgumentException("missing required consumer config: " + k);
    }
}
// also reject unknown config keys via ConsumerConfig.postProcessAndFetchUid();
// ensure deserializer classes have a public no-arg constructor.

static boolean isConstructionFailure(Throwable t) {
    return t instanceof KafkaException
        && "Failed to construct Kafka consumer".equals(t.getMessage());
}

try {
    consumer = new KafkaConsumer<>(configs);
} catch (KafkaException e) {
    // e.getCause() holds the real reason (bad deserializer, unknown config, etc.)
    log.error("Consumer construction failed", e.getCause());
    throw e;
}

When it happens

Trigger: An application constructs a new KafkaConsumer(Map/Versions props, Deserializer, Deserializer); the resolved group.protocol picks a delegate whose constructor throws a non-Kafka Throwable (bad deserializer with no default ctor, null config value causing NPE, malformed numeric config, missing required property not covered by KafkaException).

Common situations: Custom Deserializer/Serializer class without a no-arg constructor (used when only the class name is configured); passing null for a required config key; a property value that fails Number.valueOf or enum parsing outside KafkaException; reflective instantiation of a pluggable class (partition.assignment.strategy, interceptor.classes) failing; providing an invalid group.protocol value that surfaces as a non-Kafka exception.

Related errors

• User configured {} to empty while trying to subscribe for gr
• Get fenced exception for group.instance.id {}, current membe
• Heartbeat must be set lower than the session timeout
• Invalid value null for configuration key.deserializer: must
• Invalid value null for configuration value.deserializer: mus

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/cc67df331267c219.json. Report an issue: GitHub.