apache/kafka · warning · IOException

Unable to skip exactly

Error message

Unable to skip exactly

What it means

Thrown by skipBytes when InputStream.skip() returns a value that is either negative or larger than the requested skip amount. This is a violation of the InputStream#skip contract by the underlying stream implementation, not corrupt record data; the library cannot make progress safely and refuses to continue.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java:453

    private static void skipBytes(InputStream in, int bytesToSkip) throws IOException {
        if (bytesToSkip <= 0) return;

        // Starting JDK 12, this implementation could be replaced by InputStream#skipNBytes
        while (bytesToSkip > 0) {
            int ns = (int) in.skip(bytesToSkip);
            if (ns > 0 && ns <= bytesToSkip) {
                // adjust number to skip
                bytesToSkip -= ns;
            } else if (ns == 0) { // no bytes skipped
                // read one byte to check for EOS
                if (in.read() == -1) {
                    throw new InvalidRecordException("Reached end of input stream before skipping all bytes. " +
                        "Remaining bytes:" + bytesToSkip);
                }
                // one byte read so decrement number to skip
                bytesToSkip--;
            } else { // skipped negative or too many bytes
                throw new IOException("Unable to skip exactly");
            }
        }
    }

    private static Header[] readHeaders(ByteBuffer buffer, int numHeaders) {
        Header[] headers = new Header[numHeaders];
        for (int i = 0; i < numHeaders; i++) {
            int headerKeySize = ByteUtils.readVarint(buffer);
            if (headerKeySize < 0)
                throw new InvalidRecordException("Invalid negative header key size " + headerKeySize);

            ByteBuffer headerKeyBuffer = Utils.readBytes(buffer, headerKeySize);

            int headerValueSize = ByteUtils.readVarint(buffer);
            ByteBuffer headerValue = Utils.readBytes(buffer, headerValueSize);

            headers[i] = new RecordHeader(headerKeyBuffer, headerValue);
        }

View on GitHub (pinned to c31c9215e1)

Solutions

  1. Identify the concrete InputStream subclass passed to readPartiallyFrom and audit its skip(long) implementation against the java.io.InputStream contract (0 <= returned <= n).
  2. If you control the stream, fix skip() to return only the bytes actually skipped, or wrap it in a conformant adapter that loops on read() when skip() misbehaves.
  3. If the bug is in a compression codec (snappy/lz4/zstd), upgrade the codec library to a version with a spec-compliant skip().
  4. As a workaround, switch the affected path to the ByteBuffer readFrom overload which does not rely on InputStream#skip.

Example fix

// before: custom stream whose skip() may return more than requested
public class MyInputStream extends InputStream {
    @Override public long skip(long n) throws IOException {
        return in.available(); // WRONG: can exceed n
    }
}

// after: honour the InputStream#skip contract
public class MyInputStream extends InputStream {
    @Override public long skip(long n) throws IOException {
        long skipped = 0;
        while (skipped < n) {
            int r = in.read();
            if (r == -1) break;
            skipped++;
        }
        return skipped;
    }
}
Defensive patterns

Strategy: try-catch

Validate before calling

// InputStream.skip() returned negative or more than requested — a
// misbehaving stream implementation. Wrap the stream so skip() is sane:
public static InputStream safeSkip(InputStream in) {
    return new FilterInputStream(in) {
        public long skip(long n) throws IOException {
            long s = super.skip(n);
            return (s < 0 || s > n) ? 0 : s; // clamp; let the loop retry
        }
    };
}

Try / catch

try {
    PartialDefaultRecord r = DefaultRecord.readPartiallyFrom(input, baseOffset, baseTimestamp, baseSequence, logAppendTime);
} catch (IOException e) {
    // InputStream.skip() violated its contract
    LOG.warn("Misbehaving InputStream.skip() near offset {}", baseOffset, e);
}

Prevention

When it happens

Trigger: Raised at DefaultRecord.java:452-454 in skipBytes(InputStream, int) when InputStream.skip(n) returns ns < 0 or ns > n. Triggered by a custom or wrapped InputStream (for example a decompression stream, a buggy network buffer, or a third-party InputFilterStream) whose skip(N) over-reports or under-reports the bytes consumed.

Common situations: Custom InputStream passed into a record-deserialization path (e.g. a decompressing wrapper around a compressed batch), a JDBC/byte-array adapter whose skip() is incorrectly implemented, an old or buggy JDK stream implementation, or a third-party compression library (snappy/lz4/zstd) with a non-conformant skip.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/52da8c9fb2c69102.json. Report an issue: GitHub.