apache/kafka · critical · org.apache.kafka.common.KafkaException

Failed to construct kafka consumer

Error message

Failed to construct kafka consumer

What it means

Thrown by the ClassicKafkaConsumer constructor as a KafkaException wrapping any Throwable raised during initialization. The constructor builds many components (deserializers, metrics, network client, coordinator, fetcher); if any of them throws, the catch-all closes any partially-built internals (see KAFKA-2121) and rethrows under this generic message with the real cause attached. The original exception is always available via getCause().

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ClassicKafkaConsumer.java:289

                    retryBackoffMs,
                    retryBackoffMaxMs);

            this.kafkaConsumerMetrics = new KafkaConsumerMetrics(metrics);

            config.logUnused();
            AppInfoParser.registerAppInfo(CONSUMER_JMX_PREFIX, clientId, metrics, time.milliseconds());
            log.debug("Kafka consumer initialized");
        } catch (Throwable t) {
            // call close methods if internal objects are already constructed; this is to prevent resource leak. see KAFKA-2121
            // we do not need to call `close` at all when `log` is null, which means no internal objects were initialized.
            if (this.log != null) {
                // If a consumer fails during initialization, it means it hasn't joined the group yet.
                // Since it's not a group member, we use REMAIN_IN_GROUP option when closing
                // to prevent sending an unnecessary leave request to the coordinator.
                close(Duration.ZERO, CloseOptions.GroupMembershipOperation.REMAIN_IN_GROUP, true);
            }
            // now propagate the exception
            throw new KafkaException("Failed to construct kafka consumer", t);
        }
    }

    // visible for testing
    ClassicKafkaConsumer(LogContext logContext,
                         Time time,
                         ConsumerConfig config,
                         Deserializer<K> keyDeserializer,
                         Deserializer<V> valueDeserializer,
                         KafkaClient client,
                         SubscriptionState subscriptions,
                         ConsumerMetadata metadata,
                         List<ConsumerPartitionAssignor> assignors) {
        this.log = logContext.logger(getClass());
        this.time = time;
        this.subscriptions = subscriptions;
        this.metadata = metadata;
        this.metrics = new Metrics(time);

View on GitHub (pinned to c31c9215e1)

Solutions

1. Read the wrapped cause: call exception.getCause() (and getCause().getCause() if needed) to find the real class, config key, or network error.
2. If the cause is ConfigException, fix the named config property exactly as the message states.
3. If the cause is a ClassNotFoundException/NoSuchMethodError, fix the classpath/dependency (add the deserializer/assignor module, fix fat-jar shading filters).
4. If the cause is network-related (ConnectException, SSLHandshakeException, SaslAuthenticationException), validate bootstrap.servers reachability, truststore/keystore paths, and JAAS config from the runtime environment.
5. Reproduce with a minimal main() using the same properties to isolate framework-induced config mutations.

Example fix

// before
try {
    new KafkaConsumer<>(props);
} catch (KafkaException e) {
    log.error("consumer init failed", e);
}

// after
try {
    new KafkaConsumer<>(props);
} catch (KafkaException e) {
    Throwable cause = e.getCause() != null ? e.getCause() : e;
    log.error("consumer init failed: {}", cause.getMessage(), cause);
}

// Pre-flight check of required config keys before construction.
Map<String,Object> required = Map.of(
    ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, props.get(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG),
    ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, props.get(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG),
    ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, props.get(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG));
required.forEach((k,v) -> {
    if (v == null) throw new ConfigException("Missing required consumer config: " + k);
});

// KafkaException wraps the real cause; inspect getCause().
try {
    consumer = new KafkaConsumer<K,V>(props);
} catch (KafkaException ke) {
    Throwable cause = ke.getCause() != null ? ke.getCause() : ke;
    if (cause instanceof ConfigException || cause instanceof DeserializationException) {
        log.error("Consumer misconfigured; cannot start", cause);
        throw new FatalStartupException(cause);
    }
    throw ke;
}

When it happens

Trigger: Any constructor failure: bad deserializer class (ClassNotFoundException/IllegalAccessException); invalid config value (e.g. request.timeout.ms not a positive int); missing or unreachable bootstrap broker during initial metadata fetch triggered at construction; SSL/TLS misconfiguration; SASL JAAS errors; unknown partition.assignment.strategy class name; security.provider failures.

Common situations: First-time wiring of a consumer where a class is not on the classpath (custom deserializer, custom assignor); environment differences between dev and prod (TLS truststore path, JAAS config); fat-jar shading stripping broker provider classes; typo in a fully-qualified class name in config; Kerberos/SCRAM credentials not resolvable in the runtime environment.

Related errors

• The configured group.id should not be an empty string or whi
• Topic collection to subscribe to cannot contain null or empt
• No org.apache.kafka:* dependencies found on the configured k
• No project JARs configured on kafkaPublicApiChecker.projectJ
• No org.apache.kafka:* dependencies found on the project clas

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/790dac68ffe4816e.json. Report an issue: GitHub.