{"id":"80acd67e667522de","repo":"apache/kafka","slug":"reached-end-of-input-stream-before-skipping-all-by","errorCode":null,"errorMessage":"Reached end of input stream before skipping all bytes. Remaining bytes:{}","messagePattern":"Reached end of input stream before skipping all bytes\\. Remaining bytes:(.+?)","errorType":"exception","errorClass":"InvalidRecordException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java","lineNumber":447,"sourceCode":"     * No-op for case where bytesToSkip <= 0. This could occur for cases where field is expected to be null.\n     * @throws InvalidRecordException if end of stream is encountered before we could skip required bytes.\n     * @throws IOException is an I/O error occurs while trying to skip from InputStream.\n     * \n     * @see java.io.InputStream#skip(long)\n     */\n    private static void skipBytes(InputStream in, int bytesToSkip) throws IOException {\n        if (bytesToSkip <= 0) return;\n\n        // Starting JDK 12, this implementation could be replaced by InputStream#skipNBytes\n        while (bytesToSkip > 0) {\n            int ns = (int) in.skip(bytesToSkip);\n            if (ns > 0 && ns <= bytesToSkip) {\n                // adjust number to skip\n                bytesToSkip -= ns;\n            } else if (ns == 0) { // no bytes skipped\n                // read one byte to check for EOS\n                if (in.read() == -1) {\n                    throw new InvalidRecordException(\"Reached end of input stream before skipping all bytes. \" +\n                        \"Remaining bytes:\" + bytesToSkip);\n                }\n                // one byte read so decrement number to skip\n                bytesToSkip--;\n            } else { // skipped negative or too many bytes\n                throw new IOException(\"Unable to skip exactly\");\n            }\n        }\n    }\n\n    private static Header[] readHeaders(ByteBuffer buffer, int numHeaders) {\n        Header[] headers = new Header[numHeaders];\n        for (int i = 0; i < numHeaders; i++) {\n            int headerKeySize = ByteUtils.readVarint(buffer);\n            if (headerKeySize < 0)\n                throw new InvalidRecordException(\"Invalid negative header key size \" + headerKeySize);\n\n            ByteBuffer headerKeyBuffer = Utils.readBytes(buffer, headerKeySize);","sourceCodeStart":429,"sourceCodeEnd":465,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java#L429-L465","documentation":"Thrown by the private skipBytes helper when InputStream.skip() returns 0 and the following read() returns -1, i.e. the stream hit end-of-stream before all the key/value/header-value bytes of a record could be skipped. It indicates a truncated record body on the streaming partial-read path.","triggerScenarios":"Raised at DefaultRecord.java:446-448 inside skipBytes(InputStream, int) during readPartiallyFrom, when the declared keySize/valueSize/headerKeySize/headerValueSize exceeds the bytes actually available in the stream. Reached when skip() makes no progress for one of those fields and a probe read confirms EOF.","commonSituations":"Network read truncated mid-batch (broker closed the connection, fetch.message.maxBytes cut a batch in two, TLS hiccup), a log segment truncated by an unclean leader election or disk-full shutdown, or a custom InputStream that signals EOF early. Distinct from corruption: the bytes simply are not there.","solutions":["Raise fetch.message.maxBytes (consumer) and message.max.bytes (broker) so single batches are not split across fetches.","Check broker logs for 'Premature end of stream' or connection resets on the affected client; review network/MTU and TLS configuration.","If reading from a local segment, verify the file length on disk matches the log segment metadata; recover from an in-sync replica if truncated.","Reproduce with kafka-dump-log on the same segment; if it succeeds the issue is in the transport, if it fails the segment itself is short."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// If you control the InputStream, you can probe with available() before\n// skipBytes — but available() is advisory. Safer: read into a bounded buffer\n// whose length you have already validated against sizeOfBodyInBytes.\nif (input.available() < bytesStillToSkip) { /* likely EOF; abort gracefully */ }","typeGuard":null,"tryCatchPattern":"try {\n    PartialDefaultRecord r = DefaultRecord.readPartiallyFrom(input, baseOffset, baseTimestamp, baseSequence, logAppendTime);\n} catch (InvalidRecordException | IOException e) {\n    // reached EOS while skipping key/value/header bytes\n    LOG.warn(\"Truncated record stream near offset {}\", baseOffset, e);\n}","preventionTips":["Ensure the InputStream is fully buffered before partial parsing; copy network streams into a ByteArrayInputStream of known length first.","Validate the declared record body size against the remaining batch bytes before descending into records.","Treat early EOF as a sign of network truncation or on-disk corruption; quarantine the batch rather than retrying blindly."],"tags":["kafka","record-format","deserialization","truncation","input-stream"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}