apache/kafka · error · UnsupportedOperationException

Cannot set timestamp for a record with magic = 0

Error message

Cannot set timestamp for a record with magic = 0

What it means

Thrown by AbstractLegacyRecordBatch.setMaxTimestamp when the wrapped LegacyRecord has magic value 0 (the original Kafka v0 message format). V0 records carry no timestamp field at all (timestamps were introduced in v1), so any attempt to mutate the max timestamp is structurally impossible. The check guards the buffer writes in setTimestampAndUpdateCrc, which would otherwise corrupt the layout by writing a timestamp into bytes that do not exist in the v0 on-disk format.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/AbstractLegacyRecordBatch.java:488

        @Override
        public long offset() {
            return buffer.getLong(OFFSET_OFFSET);
        }

        @Override
        public LegacyRecord outerRecord() {
            return record;
        }

        @Override
        public void setLastOffset(long offset) {
            buffer.putLong(OFFSET_OFFSET, offset);
        }

        @Override
        public void setMaxTimestamp(TimestampType timestampType, long timestamp) {
            if (record.magic() == RecordBatch.MAGIC_VALUE_V0)
                throw new UnsupportedOperationException("Cannot set timestamp for a record with magic = 0");

            long currentTimestamp = record.timestamp();
            // We don't need to recompute crc if the timestamp is not updated.
            if (record.timestampType() == timestampType && currentTimestamp == timestamp)
                return;

            setTimestampAndUpdateCrc(timestampType, timestamp);
        }

        @Override
        public void setPartitionLeaderEpoch(int epoch) {
            throw new UnsupportedOperationException("Magic versions prior to 2 do not support partition leader epoch");
        }

        private void setTimestampAndUpdateCrc(TimestampType timestampType, long timestamp) {
            byte attributes = LegacyRecord.computeAttributes(magic(), compressionType(), timestampType);
            buffer.put(LOG_OVERHEAD + LegacyRecord.ATTRIBUTES_OFFSET, attributes);
            buffer.putLong(LOG_OVERHEAD + LegacyRecord.TIMESTAMP_OFFSET, timestamp);

View on GitHub (pinned to c31c9215e1)

Solutions

1. Upgrade the topic's message format to v1 or v2 (kafka-topics --alter --config message.format.version=1.0.0 or higher) so records carry timestamps.
2. Filter or skip v0 batches before attempting setMaxTimestamp; gate on record.magic() >= RecordBatch.MAGIC_VALUE_V1 before invoking it.
3. Reproduce the segment on a newer broker by re-producing the v0 data into a topic whose message.format.version is >= 0.10.0, then retire the legacy segment.

Example fix

// before
batch.setMaxTimestamp(TimestampType.CREATE_TIME, now);

// after
if (batch.magic() >= RecordBatch.MAGIC_VALUE_V1) {
    batch.setMaxTimestamp(TimestampType.CREATE_TIME, now);
}

// Before calling setMaxTimestamp on a LegacyRecordBatch:
byte magic = batch.magic();
if (magic == RecordBatch.MAGIC_VALUE_V0) {
    // V0 records have no timestamp field; skip timestamp assignment
    return;
}

// Guard: only V1+ legacy batches carry timestamps
static boolean supportsTimestamp(RecordBatch b) {
    return b.magic() >= RecordBatch.MAGIC_VALUE_V1;
}

try {
    batch.setMaxTimestamp(timestampType, ts);
} catch (UnsupportedOperationException e) {
    // expected for magic=0 batches; timestamp is not applicable
}

When it happens

Trigger: Calling RecordBatch.setMaxTimestamp(TimestampType, long) on an AbstractLegacyRecordBatch whose underlying LegacyRecord.magic() == RecordBatch.MAGIC_VALUE_V0 (0). This happens when broker/replication or log-recovery code attempts to apply a new max timestamp (e.g., LogValidator.assignTimestamps or rolling logic) onto a segment still containing v0 messages.

Common situations: Reading or rewriting very old log segments from a Kafka cluster that produced v0 messages (pre-0.10). Migration/upgrade tooling, log format conversion, or an inter-broker protocol downgrade where a newer broker is forced to handle legacy v0 segments.

Related errors

• Magic versions prior to 2 do not support partition leader ep
• Timestamp type must be provided to compute attributes for me
• Invalid message timestamp {}
• Invalid negative offset
• Invalid negative timestamp

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/563eaa2f7a3dc423.json. Report an issue: GitHub.