apache/pulsar · error · EOFException

Header was too small

Error message

Header was too small

What it means

Offload deserialization guard in StreamingDataBlockHeaderImpl.fromStream: after the magic word check passes, the header length read from the stream is smaller than the minimum block-header size, so the block header cannot be fully constructed — indicates a corrupt or truncated data block in tiered storage.

Source

Thrown at tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/StreamingDataBlockHeaderImpl.java:108

    }

    // Construct DataBlockHeader from InputStream, which contains `HEADER_MAX_SIZE` bytes readable.
    public static StreamingDataBlockHeaderImpl fromStream(InputStream stream) throws IOException {
        CountingInputStream countingStream = new CountingInputStream(stream);
        DataInputStream dis = new DataInputStream(countingStream);
        int magic = dis.readInt();
        if (magic != MAGIC_WORD) {
            throw new IOException("Data block header magic word not match. read: " + magic
                    + " expected: " + MAGIC_WORD);
        }

        long headerLen = dis.readLong();
        long blockLen = dis.readLong();
        long firstEntryId = dis.readLong();
        long ledgerId = dis.readLong();
        long toSkip = headerLen - countingStream.getCount();
        if (dis.skip(toSkip) != toSkip) {
            throw new EOFException("Header was too small");
        }

        return new StreamingDataBlockHeaderImpl(headerLen, blockLen, ledgerId, firstEntryId);
    }

    /**
     * Get the content of the data block header as InputStream.
     * Read out in format:
     *   [ magic_word -- int ][ block_len -- int ][ first_entry_id  -- long] [padding zeros]
     */
    @Override
    public InputStream toStream() {
        ByteBuf out = PulsarByteBufAllocator.DEFAULT.buffer(HEADER_MAX_SIZE, HEADER_MAX_SIZE);
        out.writeInt(MAGIC_WORD)
                .writeLong(headerLength)
                .writeLong(blockLength)
                .writeLong(firstEntryId)
                .writeLong(ledgerId)

View on GitHub (pinned to 820761864e)

Solutions

1. Check for truncated object storage files
2. Verify the read offset and object size in the offload driver configuration

When it happens

Trigger: Thrown at tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/StreamingDataBlockHeaderImpl.java:108 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/eb7b63eea6799f3c. Report an issue: GitHub.