{"id":"33218d2b8e7227d2","repo":"apache/kafka","slug":"producer-with-transactionalid-transactionalid-33218d","errorCode":null,"errorMessage":"Producer with transactionalId '{transactionalId}' and {producerIdAndEpoch} cannot execute transactional method because of previous invalid state transition attempt","messagePattern":"Producer with transactionalId '(.+?)' and (.+?) cannot execute transactional method because of previous invalid state transition attempt","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java","lineNumber":1196,"sourceCode":"    }\n\n    private void maybeFailWithError() {\n        if (!hasError()) {\n            return;\n        }\n        // for ProducerFencedException, do not wrap it as a KafkaException\n        // but create a new instance without the call trace since it was not thrown because of the current call\n        if (lastError instanceof ProducerFencedException) {\n            throw new ProducerFencedException(\"Producer with transactionalId '\" + transactionalId\n                    + \"' and \" + producerIdAndEpoch + \" has been fenced by another producer \" +\n                    \"with the same transactionalId\");\n        }\n        if (lastError instanceof InvalidProducerEpochException) {\n            throw new InvalidProducerEpochException(\"Producer with transactionalId '\" + transactionalId\n                    + \"' and \" + producerIdAndEpoch + \" attempted to produce with an old epoch\");\n        }\n        if (lastError instanceof IllegalStateException) {\n            throw new IllegalStateException(\"Producer with transactionalId '\" + transactionalId\n                    + \"' and \" + producerIdAndEpoch + \" cannot execute transactional method because of previous invalid state transition attempt\", lastError);\n        }\n        throw new KafkaException(\"Cannot execute transactional method because we are in an error state\", lastError);\n    }\n\n    private boolean maybeTerminateRequestWithError(TxnRequestHandler requestHandler) {\n        if (hasError()) {\n            if (hasAbortableError() && requestHandler instanceof FindCoordinatorHandler)\n                // No harm letting the FindCoordinator request go through if we're expecting to abort\n                return false;\n\n            requestHandler.fail(lastError);\n            return true;\n        }\n        return false;\n    }\n\n    private void enqueueRequest(TxnRequestHandler requestHandler) {","sourceCodeStart":1178,"sourceCodeEnd":1214,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java#L1178-L1214","documentation":"Thrown by TransactionManager.maybeFailWithError when the cached lastError is an IllegalStateException, i.e. an earlier call caused an invalid state-machine transition and poisoned the producer into State.FATAL_ERROR (see shouldPoisonStateOnInvalidTransition). Any later transactional method invokes maybeFailOnError and rewraps the original failure with this message, identifying the transactionalId and producerId/epoch. The producer is permanently unusable because the state machine is in FATAL_ERROR and cannot recover without a new producer instance.","triggerScenarios":"Calling beginTransaction/commitTransaction/abortTransaction/sendOffsetsToTransaction after a prior call to the same or another transactional method performed an illegal transition (e.g. beginTransaction called twice without commit, commitTransaction when not in a transaction, or send before beginTransaction triggered transition to an error state). The first illegal transition set lastError to IllegalStateException and currentState to FATAL_ERROR; this exception surfaces on the next transactional call.","commonSituations":"Application logic bug in transaction sequencing (calling commit twice, aborting after commit, forgetting beginTransaction); library/framework wrapping KafkaProducer that retries the same operation after a partial failure without resetting state; migration from non-EOS to EOS producer without updating call ordering; concurrent threads invoking transactional methods on a shared non-thread-safe producer.","solutions":["Audit the call sequence around the first failure: ensure the pattern is strictly initTransactions -> beginTransaction -> send/sendOffsetsToTransaction -> commitTransaction (or abortTransaction) with no out-of-order or duplicate transitions.","Once the producer is poisoned to FATAL_ERROR it cannot be recovered — close() it and create a new KafkaProducer with initTransactions().","Make the producer access single-threaded or serialize transactional calls through the Sender thread; KafkaProducer is not safe for concurrent transactional calls.","Add unit/integration tests covering the exact transaction lifecycle used by your code path to catch sequencing bugs at build time."],"exampleFix":"// before\nproducer.initTransactions();\nproducer.beginTransaction();\nproducer.commitTransaction();\nproducer.commitTransaction();  // illegal transition -> FATAL_ERROR, next call throws 291\n\n// after\nproducer.initTransactions();\nproducer.beginTransaction();\nproducer.send(record);\nproducer.commitTransaction();\n// start a new transaction explicitly before committing again\nproducer.beginTransaction();\nproducer.send(record2);\nproducer.commitTransaction();","handlingStrategy":"try-catch","validationCode":"// No public API exposes the internal state machine. The only safe pre-check is to\n// ensure the producer is not in an error state by never ignoring prior exceptions.\n// Maintain your own flag and set it whenever any transactional call throws:\nclass TxGuard {\n    private volatile boolean poisoned = false;\n    void markFailed() { poisoned = true; }\n    boolean isUsable(KafkaProducer<?,?> p) { return !poisoned; }\n}","typeGuard":"public static boolean isProducerInCleanState(KafkaProducer<?,?> p) {\n    // Reflectively read hasError(); not part of the public contract, use cautiously.\n    try {\n        java.lang.reflect.Field tm = p.getClass().getDeclaredField(\"transactionManager\");\n        tm.setAccessible(true);\n        Object manager = tm.get(p);\n        java.lang.reflect.Method hasError = manager.getClass().getDeclaredMethod(\"hasError\");\n        hasError.setAccessible(true);\n        return !((Boolean) hasError.invoke(manager));\n    } catch (Exception ex) { return true; }\n}","tryCatchPattern":"try {\n    producer.commitTransaction();\n} catch (IllegalStateException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"invalid state transition\")) {\n        // Producer was driven through an illegal sequence (e.g. commit without begin).\n        // The producer is poisoned; abandon and recreate it.\n        producer.close(Duration.ZERO);\n        throw new IllegalStateException(\"Producer mis-sequenced and is now poisoned; recreate it\", e);\n    }\n    throw e;\n}","preventionTips":["Always follow the exact lifecycle: initTransactions() once, then beginTransaction() before every send/commit/abort, and never call commit/abort twice in a row.","Treat ANY exception during a transactional call as fatal for that producer instance: abort, close, and instantiate a new one rather than retrying on the same object.","Avoid sending offset commits (sendOffsetsToTransaction) before beginTransaction(); verify current state in your own wrapper before each transition."],"tags":["transactions","eos","state-machine","illegal-state"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}