{"record":{"id":"c6291f535c8926bc","repo":"apache/pulsar","slug":"invalid-magicword-read-0x-x-expected-0x-x","errorCode":null,"errorMessage":"Invalid MagicWord. read: 0x%x  expected: 0x%x","messagePattern":"Invalid MagicWord\\. read: 0x%x  expected: 0x%x","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/OffloadIndexBlockImpl.java","lineNumber":82,"sourceCode":"        this.recyclerHandle = recyclerHandle;\n    }\n\n    public static OffloadIndexBlockImpl get(LedgerMetadata metadata, long dataObjectLength,\n                                            long dataHeaderLength,\n                                            List<OffloadIndexEntryImpl> entries) {\n        OffloadIndexBlockImpl block = RECYCLER.get();\n        block.indexEntries = Maps.newTreeMap();\n        entries.forEach(entry -> block.indexEntries.putIfAbsent(entry.getEntryId(), entry));\n        checkState(entries.size() == block.indexEntries.size());\n        block.segmentMetadata = metadata;\n        block.dataObjectLength = dataObjectLength;\n        block.dataHeaderLength = dataHeaderLength;\n        return block;\n    }\n\n    public static OffloadIndexBlockImpl get(int magic, DataInputStream stream) throws IOException {\n        if (magic != INDEX_MAGIC_WORD) {\n            throw new IOException(String.format(\"Invalid MagicWord. read: 0x%x  expected: 0x%x\",\n                    magic, INDEX_MAGIC_WORD));\n        }\n        OffloadIndexBlockImpl block = RECYCLER.get();\n        block.indexEntries = Maps.newTreeMap();\n        block.fromStream(stream);\n        return block;\n    }\n\n    public void recycle() {\n        dataObjectLength = -1;\n        dataHeaderLength = -1;\n        segmentMetadata = null;\n        indexEntries.clear();\n        indexEntries = null;\n        if (recyclerHandle != null) {\n            recyclerHandle.recycle(this);\n        }\n    }","sourceCodeStart":64,"sourceCodeEnd":100,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/OffloadIndexBlockImpl.java#L64-L100","documentation":"OffloadIndexBlockImpl.get is the deserialization entry point for the v1 offload index block. The first int read from the stream must equal INDEX_MAGIC_WORD; anything else means the stream is not a v1 offload index (or is corrupt), so an IOException with the read and expected magic words is thrown. It is also dispatched to by OffloadIndexBlockV2BuilderImpl.fromStream when sniffing the format.","triggerScenarios":"Calling OffloadIndexBlockImpl.get (directly or via fromStream) on a stream whose leading int is not INDEX_MAGIC_WORD: reading a data block instead of the index object, a corrupted index blob, a v2 index handled by a v1-only code path, or reading from the wrong byte offset.","commonSituations":"Bucket object naming/mapping confusion so the reader fetches payload instead of index; index blob corrupted or truncated to a few bytes; restoring offload data from an incompatible backup or older format.","solutions":["Confirm the object being read is the offload index (offload index key naming convention) not the data block","Re-offload the ledger so a fresh index is written if the index blob is corrupted","Use OffloadIndexBlockV2BuilderImpl.fromStream, which auto-detects v1 vs v2 index magic instead of forcing v1 parsing","Verify the InputStream starts at offset 0 of the index object"],"exampleFix":"// before: assume v1\nOffloadIndexBlock idx = OffloadIndexBlockImpl.get(magic, stream);\n// after: sniff the magic and dispatch\nint magic = stream.readInt();\nOffloadIndexBlock idx = OffloadIndexBlockBuilderImpl.newBuilder()\n    .fromStream(magic, stream);","handlingStrategy":"try-catch","validationCode":"DataInputStream in = new DataInputStream(new BufferedInputStream(stream));\nin.mark(4);\nint magic = in.readInt();\nin.reset();\nif (magic != OffloadIndexBlockImpl.getIndexMagicWord()) {\n    throw new IOException(\"Not a v1 offload index, magic=0x\" + Integer.toHexString(magic));\n}","typeGuard":null,"tryCatchPattern":"try {\n    OffloadIndexBlock idx = OffloadIndexBlockImpl.get(magic, stream);\n} catch (IOException e) {\n    if (e.getMessage().startsWith(\"Invalid MagicWord\")) {\n        // try v2 / auto-detect via OffloadIndexBlockV2BuilderImpl.fromStream\n    } else { throw e; }\n}","preventionTips":["Use OffloadIndexBlockV2BuilderImpl.fromStream for format-agnostic parsing","Confirm the bucket key resolves to the index object, not the data block","Enable object-store versioning/checksums to detect corrupted index blobs","Verify stream position is 0 before deserializing"],"tags":["io","corruption","index-block","tiered-storage"],"backgroundTag":"magic-word-mismatch","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}