apache/pulsar · error · IOException

Data block header magic word not match. read: ${magic} expec

Error message

Data block header magic word not match. read: ${magic} expected: ${MAGIC_WORD}

What it means

StreamingDataBlockHeaderImpl.fromStream read a 4-byte magic word that does not match the expected offload data-block magic; the stream does not start with a valid tiered-storage block header (corrupt or misaligned data).

Source

Thrown at tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/StreamingDataBlockHeaderImpl.java:98

    @Override
    public long getFirstEntryId() {
        return this.firstEntryId;
    }

    public StreamingDataBlockHeaderImpl(long headerLength, long blockLength, long ledgerId, long firstEntryId) {
        this.headerLength = headerLength;
        this.blockLength = blockLength;
        this.firstEntryId = firstEntryId;
        this.ledgerId = ledgerId;
    }

    // Construct DataBlockHeader from InputStream, which contains `HEADER_MAX_SIZE` bytes readable.
    public static StreamingDataBlockHeaderImpl fromStream(InputStream stream) throws IOException {
        CountingInputStream countingStream = new CountingInputStream(stream);
        DataInputStream dis = new DataInputStream(countingStream);
        int magic = dis.readInt();
        if (magic != MAGIC_WORD) {
            throw new IOException("Data block header magic word not match. read: " + magic
                    + " expected: " + MAGIC_WORD);
        }

        long headerLen = dis.readLong();
        long blockLen = dis.readLong();
        long firstEntryId = dis.readLong();
        long ledgerId = dis.readLong();
        long toSkip = headerLen - countingStream.getCount();
        if (dis.skip(toSkip) != toSkip) {
            throw new EOFException("Header was too small");
        }

        return new StreamingDataBlockHeaderImpl(headerLen, blockLen, ledgerId, firstEntryId);
    }

    /**
     * Get the content of the data block header as InputStream.
     * Read out in format:

View on GitHub (pinned to 820761864e)

Solutions

1. Verify the offloaded data block integrity and offset alignment
2. Re-offload the data if the ledger was corrupted

When it happens

Trigger: Thrown at tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/StreamingDataBlockHeaderImpl.java:98 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/07531f98dfa6d87e. Report an issue: GitHub.