{"id":"78ae840ec3d52551","repo":"apache/kafka","slug":"producer-with-transactionalid-transactionalid","errorCode":null,"errorMessage":"Producer with transactionalId '{transactionalId}' and {producerIdAndEpoch} has been fenced by another producer with the same transactionalId","messagePattern":"Producer with transactionalId '(.+?)' and (.+?) has been fenced by another producer with the same transactionalId","errorType":"exception","errorClass":"ProducerFencedException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java","lineNumber":1187,"sourceCode":"        else\n            log.debug(\"Transition from state {} to {}\", currentState, target);\n\n        currentState = target;\n    }\n\n    private void ensureTransactional() {\n        if (!isTransactional())\n            throw new IllegalStateException(\"Transactional method invoked on a non-transactional producer.\");\n    }\n\n    private void maybeFailWithError() {\n        if (!hasError()) {\n            return;\n        }\n        // for ProducerFencedException, do not wrap it as a KafkaException\n        // but create a new instance without the call trace since it was not thrown because of the current call\n        if (lastError instanceof ProducerFencedException) {\n            throw new ProducerFencedException(\"Producer with transactionalId '\" + transactionalId\n                    + \"' and \" + producerIdAndEpoch + \" has been fenced by another producer \" +\n                    \"with the same transactionalId\");\n        }\n        if (lastError instanceof InvalidProducerEpochException) {\n            throw new InvalidProducerEpochException(\"Producer with transactionalId '\" + transactionalId\n                    + \"' and \" + producerIdAndEpoch + \" attempted to produce with an old epoch\");\n        }\n        if (lastError instanceof IllegalStateException) {\n            throw new IllegalStateException(\"Producer with transactionalId '\" + transactionalId\n                    + \"' and \" + producerIdAndEpoch + \" cannot execute transactional method because of previous invalid state transition attempt\", lastError);\n        }\n        throw new KafkaException(\"Cannot execute transactional method because we are in an error state\", lastError);\n    }\n\n    private boolean maybeTerminateRequestWithError(TxnRequestHandler requestHandler) {\n        if (hasError()) {\n            if (hasAbortableError() && requestHandler instanceof FindCoordinatorHandler)\n                // No harm letting the FindCoordinator request go through if we're expecting to abort","sourceCodeStart":1169,"sourceCodeEnd":1205,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java#L1169-L1205","documentation":"Thrown as ProducerFencedException by TransactionManager.maybeFailWithError when lastError is a ProducerFencedException — the broker returned a fencing error because another producer registered the same transactional.id with a higher (newer) epoch. Kafka allows only one active producer per transactional.id; the older epoch is fenced off to enforce exactly-once. After this, the producer is in FATAL_ERROR and cannot recover — it must be discarded.","triggerScenarios":"Two KafkaProducer instances with the same transactional.id running concurrently (e.g. app restart before the old process fully died, or horizontal scale-out with a hardcoded id); the new instance called initTransactions() and bumped the epoch, so the old instance's next transactional operation hits maybeFailWithError and throws. Also triggered by transaction timeout on the broker causing epoch bump.","commonSituations":"Rolling deploys where old and new pods briefly coexist with the same transactional.id; sticky/non-unique transactional.id shared across replicas of a stateful service; long GC pauses on the producer exceeding transaction.max.timeout.ms, after which the broker fences it; manual failover tests leaving a zombie producer alive.","solutions":["Ensure each producer instance has a globally unique transactional.id (or accept fencing as the normal hand-off signal and create a fresh producer).","Treat ProducerFencedException as fatal for this instance: close it and create a new KafkaProducer + initTransactions().","In consume-transform-produce, derive transactional.id from group membership (group.instance.id / member id) per KIP-447 so fencing is the expected rebalance mechanism.","Raise transaction.timeout.ms (producer) and transaction.max.timeout.ms (broker) if legitimate long transactions are being fenced, and tune GC to avoid long pauses.","Confirm no zombie process holds the same transactional.id (check running instances after deploys)."],"exampleFix":"// before\nprops.put(\"transactional.id\", \"fixed-tx-id\"); // shared across replicas -> fencing\n\n// after (singleton producer)\ntry {\n    producer.beginTransaction();\n    ...\n} catch (ProducerFencedException e) {\n    // fatal: another instance took over\n    producer.close();\n    producer = createAndInitProducer(); // fresh id or epoch\n}\n\n// or for EOS-Streams style, use unique id per member:\nprops.put(\"transactional.id\", \"tx-\" + groupInstanceId);","handlingStrategy":"try-catch","validationCode":"// No caller-side validation can prevent fencing; it is a server-side coordination outcome.\n// Prevention is operational: ensure only one instance owns each transactional.id at a time.","typeGuard":null,"tryCatchPattern":"try {\n    producer.send(record).get();\n} catch (java.util.concurrent.ExecutionException ee) {\n    if (ee.getCause() instanceof org.apache.kafka.common.errors.ProducerFencedException) {\n        // another producer with the same transactional.id took over\n        try { producer.close(Duration.ZERO); } catch (Exception ignore) {}\n        // STOP this instance; do not retry on the same transactional.id\n    }\n} catch (org.apache.kafka.common.errors.ProducerFencedException pfe) {\n    // same handling: close and stop\n}","preventionTips":["Assign a unique transactional.id per application instance (or use transactional.id suffixing on consumer-partition rebalances).","Treat ProducerFencedException as terminal for the current producer: close and exit, never retry in place.","Coordinate instance lifecycle (deployment, leader election) so only one process owns a given transactional.id at a time."],"tags":["producer","transactions","eos","fencing","concurrency","java"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}