{"record":{"id":"c73cf6db325cf16d","repo":"apache/pulsar","slug":"header-was-too-small","errorCode":null,"errorMessage":"Header was too small","messagePattern":"Header was too small","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/DataBlockHeaderImpl.java","lineNumber":66,"sourceCode":"        return new DataBlockHeaderImpl(HEADER_MAX_SIZE, blockLength, firstEntryId);\n    }\n\n    // Construct DataBlockHeader from InputStream, which contains `HEADER_MAX_SIZE` bytes readable.\n    public static DataBlockHeader fromStream(InputStream stream) throws IOException {\n        CountingInputStream countingStream = new CountingInputStream(stream);\n        DataInputStream dis = new DataInputStream(countingStream);\n        int magic = dis.readInt();\n        if (magic != MAGIC_WORD) {\n            throw new IOException(\"Data block header magic word not match. read: \" + magic\n                    + \" expected: \" + MAGIC_WORD);\n        }\n\n        long headerLen = dis.readLong();\n        long blockLen = dis.readLong();\n        long firstEntryId = dis.readLong();\n        long toSkip = headerLen - countingStream.getCount();\n        if (dis.skip(toSkip) != toSkip) {\n            throw new EOFException(\"Header was too small\");\n        }\n\n        return new DataBlockHeaderImpl(headerLen, blockLen, firstEntryId);\n    }\n\n    private final long headerLength;\n    private final long blockLength;\n    private final long firstEntryId;\n\n    public static int getBlockMagicWord() {\n        return MAGIC_WORD;\n    }\n\n    public static int getDataStartOffset() {\n        return HEADER_MAX_SIZE;\n    }\n\n    @Override","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/DataBlockHeaderImpl.java#L48-L84","documentation":"After reading the fixed header fields, fromStream skips forward headerLen bytes to reach the payload. If the underlying stream cannot skip that many bytes (dis.skip returns a short count), the stream is shorter than the declared header length, so an EOFException('Header was too small') is thrown. This indicates a truncated or malformed data block.","triggerScenarios":"DataBlockHeaderImpl.fromStream is given a stream containing fewer bytes than the headerLength value declared in the block header itself — e.g. a partially uploaded object, a network read cut short, or a header whose headerLen field was corrupted.","commonSituations":"Interrupted offload writes leaving partial objects; storage provider truncating large blobs; manual copy of an offloaded block that stopped mid-transfer; restored backup missing trailing bytes.","solutions":["Re-offload the ledger segment; the offloaded object is truncated and cannot be repaired in place","Verify the object's size in blob storage matches the expected block length and re-download if it differs","Check storage provider logs for incomplete multipart uploads or failed writes around the offload time","If reading over a network stream, buffer the full block locally before parsing the header"],"exampleFix":"// before: parsing directly from a streaming HTTP payload\nDataBlockHeader h = DataBlockHeaderImpl.fromStream(httpResponse.getEntity().getContent());\n// after: read fully into memory first so skip cannot fall short\nbyte[] all = ByteStreams.toByteArray(httpResponse.getEntity().getContent());\nDataBlockHeader h = DataBlockHeaderImpl.fromStream(new ByteArrayInputStream(all));","handlingStrategy":"validation","validationCode":"// check the object is at least as big as the header claims\nlong objectSize = blob.getMetadata().getContentLength();\nif (objectSize < DataBlockHeaderImpl.HEADER_MAX_SIZE) {\n    throw new IOException(\"Offload block truncated: \" + objectSize + \" bytes\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    DataBlockHeader h = DataBlockHeaderImpl.fromStream(stream);\n} catch (EOFException e) {\n    // stream shorter than declared headerLen: object truncated, re-offload\n    log.warn(\"Offloaded data block truncated\", e);\n}","preventionTips":["Compare blob content-length against expected block size before reading","Buffer network streams fully before parsing headers","Avoid re-uploading offload objects with tools that may truncate","Monitor storage provider for failed/incomplete uploads"],"tags":["io","truncated-data","eof","tiered-storage"],"backgroundTag":"truncated-stream","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}