apache/kafka · error · FencedInstanceIdException

Get fenced exception for group.instance.id {}, current membe

Error message

Get fenced exception for group.instance.id {}, current member.id is {}

What it means

Thrown as FencedInstanceIdException from ConsumerCoordinator.invokeCompletedOffsetCommitCallbacks when the flag asyncCommitFenced is set. That flag is set (line 1128) when an asynchronous offset commit completed with a FENCED_INSTANCE_ID error from the broker, meaning another consumer instance using the same group.instance.id (static membership) joined the group and displaced this one. The exception is deferred until the next poll/commit invokes completed commit callbacks, so the app sees it on a normal API call.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java:1039

        client.disableWakeups();
        try {
            maybeAutoCommitOffsetsSync(timer);
            while (pendingAsyncCommits.get() > 0 && timer.notExpired()) {
                ensureCoordinatorReady(timer);
                client.poll(timer);
                invokeCompletedOffsetCommitCallbacks();
            }
        } finally {
            super.close(timer, membershipOperation);
            Utils.closeQuietly(coordinatorMetrics, "consumer coordinator metrics");
            Utils.closeQuietly(rebalanceCallbackMetricsManager, "consumer rebalance callback metrics");
        }
    }

    // visible for testing
    void invokeCompletedOffsetCommitCallbacks() {
        if (asyncCommitFenced.get()) {
            throw new FencedInstanceIdException("Get fenced exception for group.instance.id "
                + rebalanceConfig.groupInstanceId.orElse("unset_instance_id")
                + ", current member.id is " + memberId());
        }
        while (true) {
            OffsetCommitCompletion completion = completedOffsetCommits.poll();
            if (completion == null) {
                break;
            }
            completion.invoke();
        }
    }

    public RequestFuture<Void> commitOffsetsAsync(final Map<TopicPartition, OffsetAndMetadata> offsets, final OffsetCommitCallback callback) {
        invokeCompletedOffsetCommitCallbacks();

        RequestFuture<Void> future = null;
        if (offsets.isEmpty()) {
            // No need to check coordinator if offsets is empty since commit of empty offsets is completed locally.

View on GitHub (pinned to c31c9215e1)

Solutions

Give every live consumer instance a unique group.instance.id (derive it from pod name / hostname / instance id); never share it across concurrently running processes.
If this instance is legitimately the one that should be active, stop the duplicate instance and restart this consumer so it rejoins unfenced.
On catching FencedInstanceIdException, treat the instance as fenced: stop consuming and re-initialize with a fresh, unique group.instance.id (or no static membership).
Audit deployment templates (Helm/K8s) to ensure group.instance.id is templated per-pod, not hardcoded.

Example fix

// before (shared static id across replicas)
props.put(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG, "order-consumer");

// after (unique per instance)
props.put(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG,
    "order-consumer-" + InetAddress.getLocalHost().getHostName());

Defensive patterns

Strategy: try-catch

Validate before calling

String gid = (String) configs.get(ConsumerConfig.GROUP_INSTANCE_ID_CONFIG);
if (gid != null && !gid.isEmpty()) {
    // static membership: ensure no other process advertises the same group.instance.id.
    // Acquire a distributed lock / lease keyed on gid before starting the consumer,
    // or generate gid from a uniquely-assigned pod identity.
}

Type guard

static boolean isFenced(Throwable t) {
    return t instanceof org.apache.kafka.common.errors.FencedInstanceIdException;
}

Try / catch

try {
    consumer.poll(Duration.ofMillis(1000));
} catch (org.apache.kafka.common.errors.FencedInstanceIdException e) {
    // another instance with the same group.instance.id took over this membership
    Utils.closeQuietly(consumer, "fenced consumer");
    // recreate the consumer, ideally after ensuring the old instance is gone
    consumer = new KafkaConsumer<>(configs);
}

Prevention

Ensure group.instance.id is unique per running process
Never run two consumers with the same static member id concurrently
On fencing, close the consumer and recreate it rather than retrying in place
Use dynamic membership (omit group.instance.id) unless static identity is required

When it happens

Trigger: Two consumer processes share the same group.instance.id in the same consumer group; the second one joins and the broker fences the first. A subsequent async offset commit by the fenced instance fails with FENCED_INSTANCE_ID, setting asyncCommitFenced=true; on the next poll()/commitOffsetsAsync() invokeCompletedOffsetCommitCallbacks sees the flag and throws FencedInstanceIdException at line 1039.

Common situations: Static membership (group.instance.id set) deployed with duplicate ids across replicas/pods (e.g. a Deployment with replicaCount>1 reusing a fixed id, or two instances started from the same config); a process restart where the old instance did not shut down before the new one registered with the same id; misconfigured sidecar/daemon spawning duplicates.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/025083d2f8df8830.json. Report an issue: GitHub.