apache/kafka · error · KafkaException

Failed to load record batch at position {} from {}

Error message

Failed to load record batch at position {} from {}

What it means

Thrown by FileChannelRecordBatch.loadBatchWithSize() when Utils.readFullyOrFail() fails with IOException while loading a batch (or its header) fully into a heap ByteBuffer for in-memory access. This is the path behind loadFullBatch() / loadBatchHeader(), used by iterator(), streamingIterator(), isValid(), ensureValid(), and the accessor methods (compressionType, timestampType, etc.). Unlike writeTo (error 528), this materializes the batch into memory; the IOException is wrapped as KafkaException.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/FileLogInputStream.java:220

        protected RecordBatch loadBatchHeader() {
            if (fullBatch != null)
                return fullBatch;

            if (batchHeader == null)
                batchHeader = loadBatchWithSize(headerSize(), "record batch header");

            return batchHeader;
        }

        private RecordBatch loadBatchWithSize(int size, String description) {
            FileChannel channel = fileRecords.channel();
            try {
                ByteBuffer buffer = ByteBuffer.allocate(size);
                Utils.readFullyOrFail(channel, buffer, position, description);
                buffer.rewind();
                return toMemoryRecordBatch(buffer);
            } catch (IOException e) {
                throw new KafkaException("Failed to load record batch at position " + position + " from " + fileRecords, e);
            }
        }

        @Override
        public boolean equals(Object o) {
            if (this == o)
                return true;
            if (o == null || getClass() != o.getClass())
                return false;

            FileChannelRecordBatch that = (FileChannelRecordBatch) o;

            FileChannel channel = fileRecords == null ? null : fileRecords.channel();
            FileChannel thatChannel = that.fileRecords == null ? null : that.fileRecords.channel();

            return offset == that.offset &&
                    position == that.position &&
                    batchSize == that.batchSize &&

View on GitHub (pinned to c31c9215e1)

Solutions

1. Confirm the segment still exists and is open at access time; avoid retaining FileChannelRecordBatch references across operations that may roll/delete segments (retention, compaction).
2. Run kafka-dump-log.sh on the segment to verify the batch at the recorded position is fully readable and not in a truncated tail.
3. If the file was truncated, fix the recovery-point / last-stable-offset metadata so loaders do not attempt to read past the valid region.
4. Check disk/FS health (dmesg, SMART) for low-level I/O failures; recover from ISR if data is genuinely unreadable.

Example fix

// before: lazy access after the segment may be gone
FileChannelRecordBatch b = input.nextBatch();
// ... broker rolls/deletes segment ...
b.ensureValid(); // IOException -> KafkaException
// after: force full materialization while the channel is live
RecordBatch mem = b.loadFullBatch();  // or iterate immediately
// then operate on `mem`, drop the file-backed reference

try {
  // Triggered lazily by iterator(), isValid(), ensureValid(), compressionType(), etc.
  Iterator<Record> it = batch.iterator();
} catch (org.apache.kafka.common.KafkaException e) {
  // loadBatchWithSize() read failed via Utils.readFullyOrFail — IOException on the channel.
  java.io.IOException cause = (java.io.IOException) e.getCause();
  log.warn("Failed to load batch at position {} from {}: {}", batch.position(), batch, cause.getMessage());
}

When it happens

Trigger: Calling any method on a FileChannelRecordBatch that triggers loadFullBatch() or loadBatchHeader() (iterator, streamingIterator, isValid, ensureValid, compressionType, timestampType, checksum, maxTimestamp) when Utils.readFullyOrFail(channel, buffer, position, description) throws IOException. readFullyOrFail additionally throws if fewer than the requested bytes could be read (short read), so a truncated segment tail also surfaces here.

Common situations: Segment file closed or deleted (retention, roll, broker shutdown) while a lazy batch reference is still being accessed; the recorded position+size extends past the actual file length due to truncation or a torn write; disk/FS I/O errors; or a stale batch reference retained across a log cleaning/compaction cycle on a compacted topic.

Related errors

• Failed to decompress record stream
• Error checking for remaining bytes after reading batch
• Found record size %d smaller than minimum record overhead (%
• Failed to read record batch at position {} from {}
• Incorrect declared batch size, records still remaining in fi

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/90d255a9975fc808.json. Report an issue: GitHub.