apache/kafka · error · IllegalStateException

Cannot add partition {topicPartition} to transaction while i

Error message

Cannot add partition {topicPartition} to transaction while in state  {currentState}

What it means

Thrown by TransactionManager.maybeAddPartition when the producer is transactional and has a producerId, but currentState is not IN_TRANSACTION. A partition can only be added to an open transaction; calling send() outside beginTransaction()..commit/abort would produce records outside any transaction. The guard preserves EOS guarantees and, because it throws on the application thread, leaves the state machine recoverable.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java:474

                            .setGroupId(groupMetadata.groupId())
            );
            handler = new AddOffsetsToTxnHandler(builder, offsets, groupMetadata);
        }

        enqueueRequest(handler);
        return handler.result;
    }

    public synchronized void maybeAddPartition(TopicPartition topicPartition) {
        maybeFailWithError();
        throwIfPendingState(TransactionOperation.SEND);

        if (isTransactional()) {
            if (!hasProducerId()) {
                throw new IllegalStateException("Cannot add partition " + topicPartition +
                    " to transaction before completing a call to initTransactions");
            } else if (currentState != State.IN_TRANSACTION) {
                throw new IllegalStateException("Cannot add partition " + topicPartition +
                    " to transaction while in state  " + currentState);
            } else if (isTransactionV2Enabled()) {
                txnPartitionMap.getOrCreate(topicPartition);
                partitionsInTransaction.add(topicPartition);
                transactionStarted = true;
            } else if (transactionContainsPartition(topicPartition) || isPartitionPendingAdd(topicPartition)) {
                return;
            } else {
                log.debug("Begin adding new partition {} to transaction", topicPartition);
                txnPartitionMap.getOrCreate(topicPartition);
                newPartitionsInTransaction.add(topicPartition);
            }
        }
    }

    RuntimeException lastError() {
        return lastError;
    }

View on GitHub (pinned to c31c9215e1)

Solutions

Wrap every send() in a beginTransaction()..commitTransaction() block.
After abortTransaction() returns, call beginTransaction() again before any further send — the producer is back in READY.
Serialise all transactional lifecycle calls on a single thread or guard them with a lock to avoid a background sender escaping the bracket.
Inspect currentState reported in the message — it pinpoints which lifecycle step was missed or repeated.

Example fix

// before
producer.initTransactions();
producer.send(new ProducerRecord<>("t", "k", "v")); // throws: state READY
producer.commitTransaction();

// after
producer.initTransactions();
producer.beginTransaction();
producer.send(new ProducerRecord<>("t", "k", "v"));
producer.commitTransaction();

Defensive patterns

Strategy: validation

Validate before calling

// Track whether a transaction is open; only allow send() while it is.
private volatile boolean inTransaction = false; // true between begin and commit/abort
...
if (!inTransaction) throw new IllegalStateException("begin a transaction before sending");
producer.send(record, callback);

Try / catch

try {
    producer.send(record, callback);
} catch (IllegalStateException ise) {
    // not in IN_TRANSACTION state; call beginTransaction() (or abort a bad txn) and retry
}

Prevention

Call producer.beginTransaction() exactly once per transaction before any send().
Commit or abort before beginning the next transaction; overlapping transactions are not supported.
Mirror transaction state in your own variable since KafkaProducer does not expose it.

When it happens

Trigger: Calling producer.send() in the READY state (after init but before beginTransaction), or in COMMITTING/ABORTING/ABORTABLE_ERROR/FATAL_ERROR. The state machine check at line 473-475 fires and reports the offending state in the message.

Common situations: Mis-ordered transaction lifecycle (send before beginTransaction or after commit); code paths that skip beginTransaction on retry; an aborted transaction followed by sends without a new beginTransaction; background senders that race with the transaction boundary.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/2accaeaff19c3c16.json. Report an issue: GitHub.