{"id":"a5ffffc0fd91fc73","repo":"apache/kafka","slug":"failed-to-close-kafka-consumer-a5ffff","errorCode":null,"errorMessage":"Failed to close kafka consumer","messagePattern":"Failed to close kafka consumer","errorType":"exception","errorClass":"KafkaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/ClassicKafkaConsumer.java","lineNumber":1190,"sourceCode":"            // This is a blocking call bound by the time remaining in closeTimer\n            swallow(log, Level.ERROR, \"Failed to close fetcher with a timeout(ms)=\" + closeTimer.timeoutMs(), () -> fetcher.close(closeTimer), firstException);\n        }\n\n        closeQuietly(interceptors, \"consumer interceptors\", firstException);\n        closeQuietly(kafkaConsumerMetrics, \"kafka consumer metrics\", firstException);\n        closeQuietly(fetchMetricsManager, \"kafka fetch metrics\", firstException);\n        closeQuietly(metrics, \"consumer metrics\", firstException);\n        closeQuietly(client, \"consumer network client\", firstException);\n        closeQuietly(deserializers, \"consumer deserializers\", firstException);\n        clientTelemetryReporter.ifPresent(reporter -> closeQuietly(reporter, \"consumer telemetry reporter\", firstException));\n        AppInfoParser.unregisterAppInfo(CONSUMER_JMX_PREFIX, clientId, metrics);\n        log.debug(\"Kafka consumer has been closed\");\n        Throwable exception = firstException.get();\n        if (exception != null && !swallowException) {\n            if (exception instanceof InterruptException) {\n                throw (InterruptException) exception;\n            }\n            throw new KafkaException(\"Failed to close kafka consumer\", exception);\n        }\n    }\n\n    /**\n     * Set the fetch position to the committed position (if there is one)\n     * or reset it using the offset reset policy the user has configured.\n     *\n     * @throws org.apache.kafka.common.errors.AuthenticationException if authentication fails. See the exception for more details\n     * @throws NoOffsetForPartitionException If no offset is stored for a given partition and no offset reset policy is\n     *             defined\n     * @return true iff the operation completed without timing out\n     */\n    private boolean updateFetchPositions(final Timer timer) {\n        // If any partitions have been truncated due to a leader change, we need to validate the offsets\n        offsetFetcher.validatePositionsIfNeeded();\n\n        cachedSubscriptionHasAllFetchPositions = subscriptions.hasAllFetchPositions();\n        if (cachedSubscriptionHasAllFetchPositions) return true;","sourceCodeStart":1172,"sourceCodeEnd":1208,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ClassicKafkaConsumer.java#L1172-L1208","documentation":"Thrown from KafkaConsumer.close when one of the components being closed (coordinator, fetcher, network client, interceptors, metrics, etc.) throws an exception that is not an InterruptException and is not being swallowed. The message wraps the first captured exception in a KafkaException so callers see that shutdown failed, with the cause holding the real error. It surfaces resource cleanup failures during teardown.","triggerScenarios":"Coordinator.close() failing because a LeaveGroup request could not complete; fetcher.close() failing on a network error; metrics/network client close throwing an IOException; deserializer or interceptor close throwing.","commonSituations":"Closing the consumer during broker outage or network partition; close() invoked from a shutdown hook racing with JVM teardown; partial outage leaving sockets half-open; interceptor bugs surfacing only on close; calling close() after an InterruptException interrupted the consumer thread.","solutions":["Inspect the cause of the KafkaException to find which component failed and address that (broker reachability, interceptor bug, deserializer).","Use close(CloseOptions) with an adequate timeout and group membership operation appropriate to your shutdown path.","Ensure the consumer thread is not interrupted during close; catch InterruptException separately and re-interrupt cleanly.","Log and continue on shutdown paths where full cleanup is best-effort, after fixing the root cause."],"exampleFix":"// before\nconsumer.close(); // surfaces KafkaException(\"Failed to close kafka consumer\", cause)\n\n// after\ntry {\n    consumer.close(new CloseOptions()\n        .timeout(Duration.ofSeconds(10))\n        .groupMembershipOperation(GroupMembershipOperation.LEAVE_GROUP));\n} catch (KafkaException e) {\n    log.warn(\"Consumer close failed; cause={}\", e.getCause());\n}","handlingStrategy":"try-catch","validationCode":"// No pre-call validation prevents this; the failure is during close itself.\n// Mitigate by passing a generous, explicit close timeout:\nconsumer.close(CloseOptions.timeout(Duration.ofSeconds(30))\n    .groupMembershipOperation(GroupMembershipOperation.LEAVE));","typeGuard":null,"tryCatchPattern":"// Closing must always succeed from the caller's perspective; capture, log, swallow:\ntry {\n    consumer.close(Duration.ofSeconds(30));\n} catch (KafkaException e) {\n    if (e.getMessage().contains(\"Failed to close kafka consumer\")) {\n        log.error(\"Consumer close failed (suppressed); cause: {}\", e.getCause());\n        // Optionally call wakeup() and re-attempt once, then give up — never let close abort shutdown.\n    } else throw e;\n} finally {\n    consumer = null;\n}","preventionTips":["Always close in a try/finally with a bounded timeout; an unbounded close() is what produces confusing wrap failures during shutdown.","Separate the leave-group step from resource release so a coordinator RPC failure doesn't prevent network/IO cleanup.","Log the cause chain (getCause()) — 'Failed to close' is the wrapper; the real reason (InterruptException, WakeupException, network) is underneath."],"tags":["consumer","shutdown","network","resource-cleanup","java"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}