{"id":"ccd4a23532252ee4","repo":"apache/kafka","slug":"user-rebalance-callback-throws-an-error","errorCode":null,"errorMessage":"User rebalance callback throws an error","messagePattern":"User rebalance callback throws an error","errorType":"exception","errorClass":"KafkaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java","lineNumber":474,"sourceCode":"        maybeUpdateJoinedSubscription(assignedPartitions);\n\n        // Catch any exception here to make sure we could complete the user callback.\n        firstException.compareAndSet(null, invokeOnAssignment(assignor, assignment));\n\n        // Reschedule the auto commit starting from now\n        if (autoCommitEnabled)\n            this.nextAutoCommitTimer.updateAndReset(autoCommitIntervalMs);\n\n        subscriptions.assignFromSubscribed(assignedPartitions);\n\n        // Add partitions that were not previously owned but are now assigned\n        firstException.compareAndSet(null, rebalanceListenerInvoker.invokePartitionsAssigned(addedPartitions));\n\n        if (firstException.get() != null) {\n            if (firstException.get() instanceof KafkaException) {\n                throw (KafkaException) firstException.get();\n            } else {\n                throw new KafkaException(\"User rebalance callback throws an error\", firstException.get());\n            }\n        }\n    }\n\n    void maybeUpdateSubscriptionMetadata() {\n        int version = metadata.updateVersion();\n        if (version > metadataSnapshot.version) {\n            Cluster cluster = metadata.fetch();\n\n            if (subscriptions.hasPatternSubscription())\n                updatePatternSubscription(cluster);\n\n            // Update the current snapshot, which will be used to check for subscription\n            // changes that would require a rebalance (e.g. new partitions).\n            metadataSnapshot = new MetadataSnapshot(rackId, subscriptions, cluster, version);\n        }\n    }\n","sourceCodeStart":456,"sourceCodeEnd":492,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L456-L492","documentation":"Thrown at the end of ConsumerCoordinator.onJoinComplete when one of the user-invoked rebalance callbacks (ConsumerRebalanceListener.onPartitionsRevoked / onPartitionsAssigned, or a custom ConsumerPartitionAssignor.onAssignment) raised an exception that is not a KafkaException. KafkaException subclasses are rethrown unchanged (line 472); any other Throwable is wrapped in this KafkaException with the original as the cause. The library does this so that a misbehaving callback surfaces to the caller of poll() rather than silently corrupting the assignment state.","triggerScenarios":"A consumer using subscribe() with a ConsumerRebalanceListener completes a group join; during onJoinComplete the COOPERATIVE-path invokePartitionsRevoked (line 445), invokeOnAssignment (line 459), or invokePartitionsAssigned (line 468) throws a checked/non-Kafka exception (e.g. IOException, NullPointerException, IllegalStateException). The first such exception is captured in firstException and rethrown at line 474 on the next poll()/commit that triggers the join completion.","commonSituations":"Listener callbacks performing I/O (closing files, calling external cleanup HTTP APIs, DB deletes) that throw IOException; deserializers/assignors with NPEs on null userdata; callbacks that throw RuntimeExceptions from third-party libraries; stateful cleanup logic that fails because resources were already closed.","solutions":["Inspect the wrapped cause (KafkaException#getCause) to identify which callback and which exception type actually failed.","Wrap the body of onPartitionsRevoked / onPartitionsAssigned in try/catch, log the failure, and never let a non-Kafka exception escape the listener.","Keep rebalance callbacks short and side-effect free; move expensive/cleanup work off the rebalance thread.","If the failure is transient (network to cleanup service), catch and retry outside the callback rather than rethrowing."],"exampleFix":"// before\nconsumer.subscribe(topics, new ConsumerRebalanceListener() {\n    public void onPartitionsRevoked(Collection<TopicPartition> p) {\n        Files.delete(cleanupPath); // throws IOException -> wrapped here\n    }\n    public void onPartitionsAssigned(Collection<TopicPartition> p) {}\n});\n\n// after\nconsumer.subscribe(topics, new ConsumerRebalanceListener() {\n    public void onPartitionsRevoked(Collection<TopicPartition> p) {\n        try { Files.deleteIfExists(cleanupPath); }\n        catch (IOException e) { log.warn(\"cleanup failed for {}\", p, e); }\n    }\n    public void onPartitionsAssigned(Collection<TopicPartition> p) {}\n});","handlingStrategy":"try-catch","validationCode":null,"typeGuard":"static boolean isUserCallbackFailure(KafkaException e) {\n    return e.getCause() != null\n        && e.getMessage() != null\n        && e.getMessage().startsWith(\"User rebalance callback throws an error\");\n}","tryCatchPattern":"try {\n    consumer.poll(Duration.ofMillis(1000));\n} catch (KafkaException e) {\n    if (isUserCallbackFailure(e)) {\n        // e.getCause() is the real exception your ConsumerRebalanceListener.onPartitionsAssigned threw\n        log.error(\"Rebalance callback failed\", e.getCause());\n        // repair local state, optionally rethrow if fatal\n    } else {\n        throw e;\n    }\n}","preventionTips":["Wrap the body of every ConsumerRebalanceListener method in try/catch and never propagate","Keep onPartitionsAssigned fast, idempotent, and side-effect-free","Log inside callbacks but swallow non-fatal errors so a rebalance can complete","Avoid blocking I/O or long computation inside rebalance callbacks"],"tags":["kafka","consumer","rebalance","consumer-group","callback"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}