{"id":"3586f7bd651be573","repo":"apache/kafka","slug":"size-of-filerecords-s-has-been-truncated-during-w","errorCode":null,"errorMessage":"Size of FileRecords %s has been truncated during write: old size %d, new size %d","messagePattern":"Size of FileRecords (.+?) has been truncated during write: old size (.+?), new size (.+?)","errorType":"exception","errorClass":"KafkaException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java","lineNumber":295,"sourceCode":"     */\n    public int truncateTo(int targetSize) throws IOException {\n        int originalSize = sizeInBytes();\n        if (targetSize > originalSize || targetSize < 0)\n            throw new KafkaException(\"Attempt to truncate log segment \" + file + \" to \" + targetSize + \" bytes failed, \" +\n                    \" size of this log segment is \" + originalSize + \" bytes.\");\n        if (targetSize < (int) channel.size()) {\n            channel.truncate(targetSize);\n            size.set(targetSize);\n        }\n        return originalSize - targetSize;\n    }\n\n    @Override\n    public int writeTo(TransferableChannel destChannel, int offset, int length) throws IOException {\n        long newSize = Math.min(channel.size(), end) - start;\n        int oldSize = sizeInBytes();\n        if (newSize < oldSize)\n            throw new KafkaException(String.format(\n                    \"Size of FileRecords %s has been truncated during write: old size %d, new size %d\",\n                    file.getAbsolutePath(), oldSize, newSize));\n\n        long position = start + offset;\n        int count = Math.min(length, oldSize - offset);\n        // safe to cast to int since `count` is an int\n        return (int) destChannel.transferFrom(channel, position, count);\n    }\n\n    /**\n     * Search forward for the file position of the message batch whose last offset that is greater\n     * than or equal to the target offset. If no such batch is found, return null.\n     *\n     * @param targetOffset The offset to search for.\n     * @param startingPosition The starting position in the file to begin searching from.\n     * @return the batch's base offset, its physical position, and its size (including log overhead)\n     */\n    public LogOffsetPosition searchForOffsetFromPosition(long targetOffset, int startingPosition) {","sourceCodeStart":277,"sourceCodeEnd":313,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java#L277-L313","documentation":"Thrown by FileRecords.writeTo when the channel's current size (min'd with end, minus start) is smaller than the cached sizeInBytes at the start of the call. That means the underlying file shrank between the size snapshot and the transferFrom, so the bytes the caller expects to send no longer exist. This is a defensive check against concurrent truncation during a network send.","triggerScenarios":"Calling FileRecords.writeTo(destChannel, offset, length) — used by fetch response transfer to a SocketChannel / TransferableChannel — while another thread truncates the segment (Log.truncate, recovery, leader epoch change). newSize computed from channel.size() comes back smaller than oldSize from sizeInBytes().","commonSituations":"A leader demotion or partition reassignment triggers truncation concurrently with an in-flight produce/fetch response that is streaming the segment to a replica or client. Also seen if an operator runs offline truncation while the broker is serving fetches, or after a disk error caused the file to shrink.","solutions":["Serialize fetch/send paths against the log lock so truncation cannot race with writeTo.","On a leader epoch change, cancel in-flight fetch sessions before truncating so writeTo is not mid-transfer.","Investigate disk/filesystem health if the channel size changed without an explicit truncate (possible ENOSPC or fs bug).","If using tiered storage, ensure the local segment is not evicted under an active fetch transfer."],"exampleFix":"// before: writeTo races with concurrent truncateTo on another thread\nexecutors.submit(() -> segment.writeTo(socketChannel, offset, length));\n// ... elsewhere: segment.truncateTo(target);\n\n// after: guard sends with the log lock so truncation cannot interleave\nlogLock.lock();\ntry {\n    segment.writeTo(socketChannel, offset, length);\n} finally {\n    logLock.unlock();\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    fileRecords.writeTo(destChannel, offset, length);\n} catch (KafkaException e) {\n    // The backing file shrank under us (concurrent truncate or external process).\n    // Reopen the segment or abort the replication/fetch; do not silently retry the same transfer.\n    log.error(\"FileRecords {} truncated during write (was {}, now smaller)\",\n              fileRecords.file().getAbsolutePath(), sizeBefore, e);\n    throw e;\n}","preventionTips":["Enforce single-writer ownership of each segment; never truncate a FileRecords that may be mid-transfer.","Do not run external tools (dd, truncate, log compaction scripts) against live segment files.","Synchronize truncation (leader epoch change, log cleanup) with in-flight writeTo calls."],"tags":["file-records","write","concurrency","truncation","fetch"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}