{"id":"9c29a117102b7daa","repo":"apache/kafka","slug":"invalid-version-found-for-end-transaction-marker","errorCode":null,"errorMessage":"Invalid version found for end transaction marker: {}. May indicate data corruption","messagePattern":"Invalid version found for end transaction marker: (.+?)\\. May indicate data corruption","errorType":"exception","errorClass":"InvalidRecordException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/EndTransactionMarker.java","lineNumber":93,"sourceCode":"    }\n\n    private static void ensureTransactionMarkerControlType(ControlRecordType type) {\n        if (type != ControlRecordType.COMMIT && type != ControlRecordType.ABORT)\n            throw new IllegalArgumentException(\"Invalid control record type for end transaction marker \" + type);\n    }\n\n    public static EndTransactionMarker deserialize(Record record) {\n        ControlRecordType type = ControlRecordType.parse(record.key());\n        return deserializeValue(type, record.value());\n    }\n\n    // Visible for testing\n    static EndTransactionMarker deserializeValue(ControlRecordType type, ByteBuffer value) {\n        ensureTransactionMarkerControlType(type);\n\n        short version = value.getShort();\n        if (version < EndTxnMarker.LOWEST_SUPPORTED_VERSION)\n            throw new InvalidRecordException(\"Invalid version found for end transaction marker: \" + version +\n                    \". May indicate data corruption\");\n\n        if (version > EndTxnMarker.HIGHEST_SUPPORTED_VERSION) {\n            log.debug(\"Received end transaction marker value version {}. Parsing as version {}\", version,\n                    EndTxnMarker.HIGHEST_SUPPORTED_VERSION);\n            version = EndTxnMarker.HIGHEST_SUPPORTED_VERSION;\n        }\n        EndTxnMarker marker = new EndTxnMarker(new ByteBufferAccessor(value), version);\n        return new EndTransactionMarker(type, marker.coordinatorEpoch());\n    }\n\n    public int endTxnMarkerValueSize() {\n        return DefaultRecord.sizeInBytes(0, 0L,\n                type.controlRecordKeySize(),\n                buffer.remaining(),\n                Record.EMPTY_HEADERS);\n    }\n","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/EndTransactionMarker.java#L75-L111","documentation":"Thrown by EndTransactionMarker.deserializeValue() when the version prefix read from the marker value is below EndTxnMarker.LOWEST_SUPPORTED_VERSION. The version field is the first short in the value ByteBuffer; an out-of-range low version means the bytes do not represent a valid end-txn marker value, which the message explicitly flags as a possible sign of data corruption. It is an InvalidRecordException so it is treated as a corrupt-record condition by the broker/consumer.","triggerScenarios":"Calling EndTransactionMarker.deserialize(record) where record.value() begins with a short less than EndTxnMarker.LOWEST_SUPPORTED_VERSION. Happens when the value ByteBuffer is too short, misaligned, or contains garbage where the version short is expected. Triggered during transaction marker processing in the transaction coordinator, log validation, or consumer reading control batches.","commonSituations":"Disk/log corruption that altered the marker value bytes, a segment partially overwritten, a manual or buggy tool that rewrote control records without preserving the version-prefixed value format, a wire-level or serialization bug that truncated the value below 2 bytes, or a client build so old it predates the versioned EndTxnMarker schema. Note: versions above HIGHEST_SUPPORTED_VERSION are tolerated (downgraded with a debug log), so this only fires for too-low or garbage versions.","solutions":["Dump the affected control batch with kafka-dump-log.sh --deep-iteration --print-data-log to inspect the raw marker value bytes and confirm the version short is invalid.","Recover a clean copy of the segment from an in-sync replica; for RF=1, truncate past the corrupt marker.","If a tool/serializer rewrote the value, fix it to preserve the version-prefixed ByteBuffer produced by EndTransactionMarker.serializeValue.","Add broker-side CRC validation logging (log.message.format.version, message.timestamp.type) and watch CorruptRecordException rates to catch the underlying writer bug."],"exampleFix":"// before: writing a control record value without the version prefix\nbuffer.putShort((short) 0).put(coordinatorEpoch);\n// after: use the provided serializer so the version prefix is always correct\nEndTransactionMarker m = new EndTransactionMarker(ControlRecordType.COMMIT, epoch);\nByteBuffer value = m.serializeValue();","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  EndTransactionMarker m = EndTransactionMarker.deserialize(record);\n} catch (org.apache.kafka.common.InvalidRecordException e) {\n  // Marker version below LOWEST_SUPPORTED_VERSION: the control-record value is corrupt.\n  // Re-fetch the control record or treat the transaction as indeterminate.\n  log.warn(\"Corrupt end-txn marker at offset {}: {}\", record.offset(), e.getMessage());\n}","preventionTips":["Cannot be pre-validated without deserializing — wrap deserialize() in try/catch at every call site.","Distinguish 'version too low' (corruption, this error) from 'version too high' (forward-compat, silently downgraded by the library) — only the former throws.","On corruption, do not guess commit/abort; surface as indeterminate transaction state to the application.","Ensure brokers and clients run compatible broker/protocol versions so produced markers carry a supported version.","Audit control-record segments with kafka-dump-log --deep-iteration to catch version corruption early."],"tags":["kafka","records","transactions","control-record","version","corruption"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}