apache/kafka · error · KafkaException

Attempt to truncate log segment {} to {} bytes failed, size

Error message

Attempt to truncate log segment {} to {} bytes failed,  size of this log segment is {} bytes.

What it means

Thrown by FileRecords.truncateTo when targetSize is greater than the segment's current sizeInBytes or negative. Truncation can only shrink a segment, so growing or truncating to a negative offset is rejected before FileChannel.truncate is called. This is the contract documented on truncateTo.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/FileRecords.java:281

        } finally {
            this.file = f;
        }
    }

    /**
     * Truncate this file message set to the given size in bytes. Note that this API does no checking that the
     * given size falls on a valid message boundary.
     * In some versions of the JDK truncating to the same size as the file message set will cause an
     * update of the files mtime, so truncate is only performed if the targetSize is smaller than the
     * size of the underlying FileChannel.
     * It is expected that no other threads will do writes to the log when this function is called.
     * @param targetSize The size to truncate to. Must be between 0 and sizeInBytes.
     * @return The number of bytes truncated off
     */
    public int truncateTo(int targetSize) throws IOException {
        int originalSize = sizeInBytes();
        if (targetSize > originalSize || targetSize < 0)
            throw new KafkaException("Attempt to truncate log segment " + file + " to " + targetSize + " bytes failed, " +
                    " size of this log segment is " + originalSize + " bytes.");
        if (targetSize < (int) channel.size()) {
            channel.truncate(targetSize);
            size.set(targetSize);
        }
        return originalSize - targetSize;
    }

    @Override
    public int writeTo(TransferableChannel destChannel, int offset, int length) throws IOException {
        long newSize = Math.min(channel.size(), end) - start;
        int oldSize = sizeInBytes();
        if (newSize < oldSize)
            throw new KafkaException(String.format(
                    "Size of FileRecords %s has been truncated during write: old size %d, new size %d",
                    file.getAbsolutePath(), oldSize, newSize));

        long position = start + offset;

View on GitHub (pinned to c31c9215e1)

Solutions

1. Re-read sizeInBytes() at the truncation call site and clamp targetSize to [0, sizeInBytes()] before invoking truncateTo.
2. Serialize truncations against the segment lock to prevent racing appends from changing size mid-call.
3. Validate the offset-index and time-index against the .log file size; rebuild them if they point past the real end.
4. If the target came from controller/KRaft metadata, ensure the broker applied metadata in order so the truncation target reflects the latest leader epoch.

Example fix

// before: targetSize from a stale index entry, can exceed live size
segment.truncateTo(targetSize);

// after: clamp to the segment's real size first
int bounded = Math.max(0, Math.min(targetSize, segment.sizeInBytes()));
if (bounded != targetSize) log.warn("Clamped truncate target from {} to {}", targetSize, bounded);
segment.truncateTo(bounded);

int currentSize = fileRecords.sizeInBytes();
if (targetSize < 0 || targetSize > currentSize) {
    throw new IllegalArgumentException(
        "truncateTo target " + targetSize + " outside [0, " + currentSize + "]");
}
fileRecords.truncateTo(targetSize);

When it happens

Trigger: Calling FileRecords.truncateTo(targetSize) with targetSize > sizeInBytes() or targetSize < 0. Reachable from Log.truncateTo, LogSegment.truncateTo, recovery-point checkpointing, and KRaft snapshot truncation.

Common situations: A recovery or truncation routine computed a target size from a stale offset index (so targetSize points past the real end), a partition leadership handoff raced with a truncation, or an operator invoked kafka-dump-log/--truncate with a bad size. Also seen when log-start offset logic computes a negative target after metadata corruption.

Related errors

• The size of segment {} ({}) is larger than the maximum allow
• Slice from position {} exceeds end position of {}
• Append of size {} bytes is too large for segment with curren
• Found record size %d smaller than minimum record overhead (%
• Failed to read record batch at position {} from {}

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/b657ede20ac93821.json. Report an issue: GitHub.