{"record":{"id":"1ffd969819885922","repo":"apache/flink","slug":"corrupt-data-magic-number-mismatch-expected-8x-1ffd96","errorCode":null,"errorMessage":"Corrupt data, magic number mismatch. Expected %8x, found %8x","messagePattern":"Corrupt data, magic number mismatch\\. Expected %8x, found %8x","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"flink-core/src/main/java/org/apache/flink/api/common/typeutils/NestedSerializersSnapshotDelegate.java","lineNumber":120,"sourceCode":"    // ------------------------------------------------------------------------\n\n    /** Writes the composite snapshot of all the contained serializers. */\n    public final void writeNestedSerializerSnapshots(DataOutputView out) throws IOException {\n        out.writeInt(MAGIC_NUMBER);\n        out.writeInt(VERSION);\n\n        out.writeInt(nestedSnapshots.length);\n        for (TypeSerializerSnapshot<?> snap : nestedSnapshots) {\n            TypeSerializerSnapshot.writeVersionedSnapshot(out, snap);\n        }\n    }\n\n    /** Reads the composite snapshot of all the contained serializers. */\n    public static NestedSerializersSnapshotDelegate readNestedSerializerSnapshots(\n            DataInputView in, ClassLoader cl) throws IOException {\n        final int magicNumber = in.readInt();\n        if (magicNumber != MAGIC_NUMBER) {\n            throw new IOException(\n                    String.format(\n                            \"Corrupt data, magic number mismatch. Expected %8x, found %8x\",\n                            MAGIC_NUMBER, magicNumber));\n        }\n\n        final int version = in.readInt();\n        if (version != VERSION) {\n            throw new IOException(\"Unrecognized version: \" + version);\n        }\n\n        final int numSnapshots = in.readInt();\n        final TypeSerializerSnapshot<?>[] nestedSnapshots =\n                new TypeSerializerSnapshot<?>[numSnapshots];\n\n        for (int i = 0; i < numSnapshots; i++) {\n            nestedSnapshots[i] = TypeSerializerSnapshot.readVersionedSnapshot(in, cl);\n        }\n","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/typeutils/NestedSerializersSnapshotDelegate.java#L102-L138","documentation":"Thrown by NestedSerializersSnapshotDelegate.readNestedSerializerSnapshots when the magic-number integrity check fails — the first integer read from the stream does not equal MAGIC_NUMBER (1333245). This guards the nested-serializer-snapshots binary segment of a checkpoint/savepoint against corruption or misaligned reads. It is the integrity counterpart to the write path in writeNestedSerializerSnapshots.","triggerScenarios":"Reading a NestedSerializersSnapshotDelegate segment from a corrupt, truncated, or misaligned data stream. Attempting to read a stream whose position is wrong (e.g. reading nested snapshots where outer snapshot data is expected). A checkpoint file damaged by storage failure or incomplete write.","commonSituations":"Corrupted or partially-written checkpoint/savepoint files in HDFS/S3/local disk. Restoring state from a file that was not fully flushed. A custom CompositeTypeSerializerSnapshot that reads the nested delegate at the wrong offset. Version skew causing the reader to expect the magic number where different data sits.","solutions":["Verify the checkpoint/savepoint file integrity (size, checksums, completeness).","Ensure the restoring Flink version is compatible with the version that wrote the checkpoint.","If using a custom CompositeTypeSerializerSnapshot, confirm the order of internalReadOuterSnapshot and readNestedSerializerSnapshots calls matches the write order.","Re-create the checkpoint from a known-good savepoint if the file is confirmed corrupt."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Before restoring, verify checkpoint integrity (size, metadata completeness)\nPath file = Path.of(\"/state/chk-42/db/\");\nif (!Files.exists(file)) {\n    throw new IllegalStateException(\"State file not found: \" + file);\n}","typeGuard":null,"tryCatchPattern":"try {\n    backend.restore(checkpointPath);\n} catch (IOException e) {\n    if (e.getMessage().contains(\"magic number mismatch\")) {\n        log.error(\"Nested serializer snapshot data is corrupt or version-incompatible.\");\n    }\n    throw e;\n}","preventionTips":["Confirm checkpoint/savepoint files are fully written and not truncated before restore.","Restore with the same or compatible Flink version that wrote the checkpoint.","Audit custom CompositeTypeSerializerSnapshot read/write order for the nested delegate."],"tags":["serialization","checkpoint","savepoint","data-corruption","state","io"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}