{"id":"5dc69a16e1ae9a03","repo":"apache/kafka","slug":"failed-to-read-record-batch-at-position-from","errorCode":null,"errorMessage":"Failed to read record batch at position {} from {}","messagePattern":"Failed to read record batch at position (.+?) from (.+?)","errorType":"exception","errorClass":"KafkaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/FileLogInputStream.java","lineNumber":186,"sourceCode":"        public void ensureValid() {\n            loadFullBatch().ensureValid();\n        }\n\n        @Override\n        public int sizeInBytes() {\n            return LOG_OVERHEAD + batchSize;\n        }\n\n        @Override\n        public void writeTo(ByteBuffer buffer) {\n            FileChannel channel = fileRecords.channel();\n            try {\n                int limit = buffer.limit();\n                buffer.limit(buffer.position() + sizeInBytes());\n                Utils.readFully(channel, buffer, position);\n                buffer.limit(limit);\n            } catch (IOException e) {\n                throw new KafkaException(\"Failed to read record batch at position \" + position + \" from \" + fileRecords, e);\n            }\n        }\n\n        protected abstract RecordBatch toMemoryRecordBatch(ByteBuffer buffer);\n\n        protected abstract int headerSize();\n\n        protected RecordBatch loadFullBatch() {\n            if (fullBatch == null) {\n                batchHeader = null;\n                fullBatch = loadBatchWithSize(sizeInBytes(), \"full record batch\");\n            }\n            return fullBatch;\n        }\n\n        protected RecordBatch loadBatchHeader() {\n            if (fullBatch != null)\n                return fullBatch;","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/FileLogInputStream.java#L168-L204","documentation":"Thrown by FileChannelRecordBatch.writeTo() when Utils.readFully() fails with IOException while copying the batch's bytes from the underlying FileChannel into a target ByteBuffer. This path is used when materializing a batch for transmission (e.g. fetch response assembly, replication) without going through the full in-memory load. It wraps the IOException as KafkaException because the failure happened during a channel read that the caller has no direct handle to.","triggerScenarios":"Calling batch.writeTo(buffer) on a FileChannelRecordBatch (obtained from FileLogInputStream.nextBatch) where Utils.readFully(channel, buffer, position) raises IOException. The read covers sizeInBytes() bytes (LOG_OVERHEAD + batchSize) starting at the batch's recorded position. Common in fetch request handling, replication fetchers, and mirror makers.","commonSituations":"The underlying FileChannel/FileRecords was closed before the batch was written out, the segment file was deleted (log retention/roll) while a reference to a batch was still held, a disk/FS error during the read, or the batch's recorded position/size no longer points at valid data (e.g. segment was truncated underneath the holder). Also seen when a stale batch reference is retained past its segment's lifetime.","solutions":["Check that the FileRecords/segment is still open and not deleted; do not hold FileChannelRecordBatch references after the source segment may have been rolled/deleted by retention.","Inspect the broker log for concurrent segment deletion / log rolling around the time of the failure and correlate with retention/log.roll.ms settings.","Verify disk health (dmesg, SMART, FS errors) if the channel read fails with a low-level I/O error.","Re-fetch the batch from an in-sync replica or trigger a leader election to recover a clean copy for the consumer."],"exampleFix":"// before: holding a batch past its segment lifetime\nFileChannelRecordBatch b = input.nextBatch();\n// ... segment rolled/deleted by retention ...\nb.writeTo(out); // IOException\n// after: copy bytes out while the segment is still open\nb.writeTo(out);\n// then drop the reference; never retain across retention operations","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  batch.writeTo(buffer);\n} catch (org.apache.kafka.common.KafkaException e) {\n  // writeTo() does Utils.readFully(channel, ...) — IOException means the FileChannel read failed.\n  java.io.IOException cause = (java.io.IOException) e.getCause();\n  log.warn(\"Failed to read batch at position {} from {}: {}\", batch.position(), batch, cause.getMessage());\n}","preventionTips":["writeTo reads live from the FileChannel — ensure the channel is still open and the segment file still exists at call time (log cleanup can delete it).","Verify the destination ByteBuffer has remaining() >= batch.sizeInBytes() before calling writeTo; an undersized buffer causes a short/partial read path.","Do not hold FileChannelRecordBatch references past segment lifetime — load into memory (loadFullBatch) if you need to outlive the channel.","Treat as I/O-transient first: reopen FileRecords and retry once; on second failure, quarantine the segment.","Guard against concurrent segment compaction deleting the file mid-read by pinning/holding the segment reference."],"tags":["kafka","records","file-records","io-exception","replication","fetch"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}