{"id":"3f17945038ad99b8","repo":"apache/kafka","slug":"failed-to-close-kafka-consumer","errorCode":null,"errorMessage":"Failed to close kafka consumer","messagePattern":"Failed to close kafka consumer","errorType":"exception","errorClass":"org.apache.kafka.common.KafkaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java","lineNumber":1687,"sourceCode":"            backgroundEventReaper.reap(backgroundEventQueue);\n\n        closeQuietly(interceptors, \"consumer interceptors\", firstException);\n        closeQuietly(kafkaConsumerMetrics, \"kafka consumer metrics\", firstException);\n        closeQuietly(asyncConsumerMetrics, \"async consumer metrics\", firstException);\n        closeQuietly(fetchMetricsManager, \"consumer fetch metrics\", firstException);\n        closeQuietly(rebalanceCallbackMetricsManager, \"consumer rebalance callback metrics\");\n        closeQuietly(metrics, \"consumer metrics\", firstException);\n        closeQuietly(deserializers, \"consumer deserializers\", firstException);\n        clientTelemetryReporter.ifPresent(reporter -> closeQuietly(reporter, \"async consumer telemetry reporter\", firstException));\n\n        AppInfoParser.unregisterAppInfo(CONSUMER_JMX_PREFIX, clientId, metrics);\n        log.debug(\"Kafka consumer has been closed\");\n        Throwable exception = firstException.get();\n        if (exception != null && !swallowException) {\n            if (exception instanceof InterruptException) {\n                throw (InterruptException) exception;\n            }\n            throw new KafkaException(\"Failed to close kafka consumer\", exception);\n        }\n    }\n\n    private Timer createTimerForCloseRequests(Duration timeout) {\n        // this.time could be null if an exception occurs in constructor prior to setting the this.time field\n        final Time time = (this.time == null) ? Time.SYSTEM : this.time;\n        return time.timer(Math.min(timeout.toMillis(), requestTimeoutMs));\n    }\n\n    private void autoCommitOnClose(final Timer timer) {\n        if (groupMetadata.get().isEmpty() || applicationEventHandler == null)\n            return;\n\n        if (autoCommitEnabled)\n            commitSyncAllConsumed(timer);\n\n        applicationEventHandler.add(new CommitOnCloseEvent());\n    }","sourceCodeStart":1669,"sourceCodeEnd":1705,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1669-L1705","documentation":"A wrapping KafkaException ('Failed to close kafka consumer') thrown at the end of the internal close flow when one of the sub-components (metrics, deserializers, telemetry reporter, rebalance callbacks, etc.) threw during shutdown and the error is not being intentionally swallowed. It collects the first exception via a firstException reference and rethrows it as a KafkaException so callers see a single, coherent failure rather than swallowing resource-cleanup errors. InterruptException is rethrown unwrapped; everything else is wrapped here.","triggerScenarios":"Close-time cleanup of an AsyncKafkaConsumer where closeQuietly captured a non-interrupt exception: e.g. the ClientTelemetryReporter failed to deregister, AppInfoParser.unregisterAppInfo threw, or a rebalance callback running on close raised. The firstException is then wrapped and rethrown.","commonSituations":"JVM shutdown hooks racing with the consumer close; metrics registry/MBean server MBean unregistration failures under security managers or duplicate registrations; deserializer close() throwing (e.g. Avro/Protobuf serializer closing a pooled registry client); interrupted threads during shutdown that surface as non-InterruptException in a callback; partial constructor failure leaving some fields null so close partially fails.","solutions":["Inspect the wrapped cause (KafkaException.getCause()) — the real failure is the cause, not the wrapper; fix that root component.","For deserializer/telemetry close errors, ensure the underlying client (schema registry, metrics backend) is still alive when the consumer closes; close consumers before shutting down shared registries.","If shutting down the whole JVM, prefer closing in a controlled order (consumer -> producer -> registry -> broker client) rather than concurrent shutdown hooks.","If the cause is an interrupted close, handle InterruptException at the call site and decide whether to suppress or propagate."],"exampleFix":"// before\ntry {\n    consumer.close();\n} catch (KafkaException e) {\n    log.error(\"close failed\", e); // only sees the wrapper\n}\n\n// after\ntry {\n    consumer.close();\n} catch (KafkaException e) {\n    Throwable cause = e.getCause() != null ? e.getCause() : e;\n    log.error(\"close failed: {}\", cause.toString(), cause);\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    consumer.close(Duration.ofSeconds(30));\n} catch (InterruptException e) {\n    Thread.currentThread().interrupt(); // restore flag, then suppress further close work\n} catch (KafkaException e) {\n    // Wraps the first exception raised during shutdown (network, deserializer close, etc.).\n    log.error(\"Consumer close failed; resources may leak\", e);\n} catch (RuntimeException e) {\n    log.error(\"Unexpected error closing consumer\", e);\n}","preventionTips":["Always wrap consumer.close() in try/catch during shutdown so one failing component does not abort the whole teardown.","Restore the interrupt status when InterruptException is caught, so higher-level shutdown logic still works.","Idempotently attempt close in a finally block and guard against double-close (the consumer marks itself closed).","Investigate the wrapped cause — it usually points to a network issue or a misbehaving close callback."],"tags":["kafka","consumer","lifecycle","resource-cleanup"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}