What causes "Failed to close Kafka share consumer" in apache/kafka?

Inspect the chained cause (ex.getCause()) — it identifies which sub-component failed (network, deserializer, telemetry reporter).

apache/kafka · error · KafkaException

Failed to close Kafka share consumer

Error message

Failed to close Kafka share consumer

What it means

Thrown by ShareConsumerImpl.close(...) (line 1058) as the wrapper exception when one of the sub-components throws during shutdown and swallowException is false. The close sequence aggregates firstException across metrics, fetch metrics, deserializers, telemetry reporter, and the network client via closeQuietly; if any fails, the original is chained as the cause of this KafkaException. It signals that shutdown did not complete cleanly even though the consumer is functionally closed.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumerImpl.java:1058

        // the reaper nor the background event queue were constructed, so check them first to avoid NPE.
        if (backgroundEventReaper != null && backgroundEventQueue != null)
            backgroundEventReaper.reap(backgroundEventQueue);

        closeQuietly(kafkaShareConsumerMetrics, "kafka share consumer metrics", firstException);
        closeQuietly(asyncConsumerMetrics, "kafka async consumer metrics", firstException);
        closeQuietly(shareFetchMetricsManager, "kafka share consumer fetch metrics", firstException);
        closeQuietly(metrics, "consumer metrics", firstException);
        closeQuietly(deserializers, "consumer deserializers", firstException);
        clientTelemetryReporter.ifPresent(reporter -> closeQuietly(reporter, "consumer telemetry reporter", firstException));

        AppInfoParser.unregisterAppInfo(CONSUMER_JMX_PREFIX, clientId, metrics);
        log.debug("Kafka share consumer has been closed");
        Throwable exception = firstException.get();
        if (exception != null && !swallowException) {
            if (exception instanceof InterruptException) {
                throw (InterruptException) exception;
            }
            throw new KafkaException("Failed to close Kafka share consumer", exception);
        }
    }

    private void stopFindCoordinatorOnClose() {
        if (applicationEventHandler == null) {
            return;
        }
        log.debug("Stop finding coordinator during consumer close");
        applicationEventHandler.add(new StopFindCoordinatorOnCloseEvent());
    }

    private Timer createTimerForCloseRequests(Duration timeout) {
        // this.time could be null if an exception occurs in constructor prior to setting the this.time field
        final Time time = (this.time == null) ? Time.SYSTEM : this.time;
        return time.timer(Math.min(timeout.toMillis(), requestTimeoutMs));
    }

    /**

View on GitHub (pinned to c31c9215e1)

Solutions

Inspect the chained cause (ex.getCause()) — it identifies which sub-component failed (network, deserializer, telemetry reporter).
Ensure graceful shutdown: stop the poll loop and call close() with an adequate timeout before the JVM/ container sends SIGTERM/SIGINT.
Fix or replace custom Deserializer/Metric/Telemetry components whose close() throws.
If the interrupt during close is expected in your runtime, catch InterruptException and KafkaException at the call site and log, but do not ignore the cause.

Example fix

// before
consumer.close();

// after
try {
    consumer.close(Duration.ofSeconds(30));
} catch (org.apache.kafka.common.KafkaException e) {
    log.warn("Share consumer close failed: {}", e.getCause().getMessage());
}

Defensive patterns

Strategy: try-catch

Validate before calling

null

Type guard

null

Try / catch

// close() is best-effort: capture but never let close failures mask the primary exception.
Throwable primary = null;
try {
    /* main work */
} catch (Throwable t) {
    primary = t;
    throw t;
} finally {
    try {
        consumer.close(java.time.Duration.ofSeconds(10));
    } catch (org.apache.kafka.common.KafkaException e) {
        log.warn("Error closing share consumer", e);
        if (primary == null) throw e;
        primary.addSuppressed(e);
    }
}

Prevention

Wrap close() in try/finally and log the wrapped exception, but never let it shadow the primary exception from the try block.
Use a generous close timeout so pending acknowledgements can be flushed; tight timeouts increase the chance of this exception.
Ensure the broker/coordinator is reachable during shutdown; network errors during close are a common root cause.

When it happens

Trigger: Closing a KafkaShareConsumer while the network thread or background event queue raises an exception (e.g. InterruptException from a shutdown hook interrupting mid-flush, or an acknowledgement commit still in flight). Also triggered when a deserializer.close() or metrics reporter.close() throws, and the close path is not allowed to swallow it.

Common situations: JVM shutdown hooks interrupting the consumer during graceful close; broker disconnection while the consumer is flushing acknowledgements on close; misbehaving custom Deserializer or MeterRegistry integration whose close() throws; running under a framework that wraps close() in a timeout and forcibly interrupts the thread.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/da95db245a9de06f.json. Report an issue: GitHub.