apache/kafka · critical · ProducerFencedException

Producer with transactionalId '{transactionalId}' and {produ

Error message

Producer with transactionalId '{transactionalId}' and {producerIdAndEpoch} has been fenced by another producer with the same transactionalId

What it means

Thrown as ProducerFencedException by TransactionManager.maybeFailWithError when lastError is a ProducerFencedException — the broker returned a fencing error because another producer registered the same transactional.id with a higher (newer) epoch. Kafka allows only one active producer per transactional.id; the older epoch is fenced off to enforce exactly-once. After this, the producer is in FATAL_ERROR and cannot recover — it must be discarded.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java:1187

        else
            log.debug("Transition from state {} to {}", currentState, target);

        currentState = target;
    }

    private void ensureTransactional() {
        if (!isTransactional())
            throw new IllegalStateException("Transactional method invoked on a non-transactional producer.");
    }

    private void maybeFailWithError() {
        if (!hasError()) {
            return;
        }
        // for ProducerFencedException, do not wrap it as a KafkaException
        // but create a new instance without the call trace since it was not thrown because of the current call
        if (lastError instanceof ProducerFencedException) {
            throw new ProducerFencedException("Producer with transactionalId '" + transactionalId
                    + "' and " + producerIdAndEpoch + " has been fenced by another producer " +
                    "with the same transactionalId");
        }
        if (lastError instanceof InvalidProducerEpochException) {
            throw new InvalidProducerEpochException("Producer with transactionalId '" + transactionalId
                    + "' and " + producerIdAndEpoch + " attempted to produce with an old epoch");
        }
        if (lastError instanceof IllegalStateException) {
            throw new IllegalStateException("Producer with transactionalId '" + transactionalId
                    + "' and " + producerIdAndEpoch + " cannot execute transactional method because of previous invalid state transition attempt", lastError);
        }
        throw new KafkaException("Cannot execute transactional method because we are in an error state", lastError);
    }

    private boolean maybeTerminateRequestWithError(TxnRequestHandler requestHandler) {
        if (hasError()) {
            if (hasAbortableError() && requestHandler instanceof FindCoordinatorHandler)
                // No harm letting the FindCoordinator request go through if we're expecting to abort

View on GitHub (pinned to c31c9215e1)

Solutions

Ensure each producer instance has a globally unique transactional.id (or accept fencing as the normal hand-off signal and create a fresh producer).
Treat ProducerFencedException as fatal for this instance: close it and create a new KafkaProducer + initTransactions().
In consume-transform-produce, derive transactional.id from group membership (group.instance.id / member id) per KIP-447 so fencing is the expected rebalance mechanism.
Raise transaction.timeout.ms (producer) and transaction.max.timeout.ms (broker) if legitimate long transactions are being fenced, and tune GC to avoid long pauses.
Confirm no zombie process holds the same transactional.id (check running instances after deploys).

Example fix

// before
props.put("transactional.id", "fixed-tx-id"); // shared across replicas -> fencing

// after (singleton producer)
try {
    producer.beginTransaction();
    ...
} catch (ProducerFencedException e) {
    // fatal: another instance took over
    producer.close();
    producer = createAndInitProducer(); // fresh id or epoch
}

// or for EOS-Streams style, use unique id per member:
props.put("transactional.id", "tx-" + groupInstanceId);

Defensive patterns

Strategy: try-catch

Validate before calling

// No caller-side validation can prevent fencing; it is a server-side coordination outcome.
// Prevention is operational: ensure only one instance owns each transactional.id at a time.

Try / catch

try {
    producer.send(record).get();
} catch (java.util.concurrent.ExecutionException ee) {
    if (ee.getCause() instanceof org.apache.kafka.common.errors.ProducerFencedException) {
        // another producer with the same transactional.id took over
        try { producer.close(Duration.ZERO); } catch (Exception ignore) {}
        // STOP this instance; do not retry on the same transactional.id
    }
} catch (org.apache.kafka.common.errors.ProducerFencedException pfe) {
    // same handling: close and stop
}

Prevention

Assign a unique transactional.id per application instance (or use transactional.id suffixing on consumer-partition rebalances).
Treat ProducerFencedException as terminal for the current producer: close and exit, never retry in place.
Coordinate instance lifecycle (deployment, leader election) so only one process owns a given transactional.id at a time.

When it happens

Trigger: Two KafkaProducer instances with the same transactional.id running concurrently (e.g. app restart before the old process fully died, or horizontal scale-out with a hardcoded id); the new instance called initTransactions() and bumped the epoch, so the old instance's next transactional operation hits maybeFailWithError and throws. Also triggered by transaction timeout on the broker causing epoch bump.

Common situations: Rolling deploys where old and new pods briefly coexist with the same transactional.id; sticky/non-unique transactional.id shared across replicas of a stateful service; long GC pauses on the producer exceeding transaction.max.timeout.ms, after which the broker fences it; manual failover tests leaving a zombie producer alive.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/78ae840ec3d52551.json. Report an issue: GitHub.