{"record":{"id":"a31f3ae609faa985","repo":"apache/flink","slug":"corrupt-data-unexpected-magic-number-bytebuffer","errorCode":null,"errorMessage":"Corrupt data: Unexpected magic number ${byteBuffer.getInt()}","messagePattern":"Corrupt data: Unexpected magic number (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-oss-fs-hadoop/src/main/java/org/apache/flink/fs/osshadoop/writer/OSSRecoverableSerializer.java","lineNumber":130,"sourceCode":"\n        return resultBytes;\n    }\n\n    @Override\n    public OSSRecoverable deserialize(int version, byte[] serialized) throws IOException {\n        switch (version) {\n            case 1:\n                return deserializeV1(serialized);\n            default:\n                throw new IOException(\"Unrecognized version or corrupt state: \" + version);\n        }\n    }\n\n    private OSSRecoverable deserializeV1(byte[] serialized) throws IOException {\n        final ByteBuffer byteBuffer = ByteBuffer.wrap(serialized).order(ByteOrder.LITTLE_ENDIAN);\n\n        if (byteBuffer.getInt() != MAGIC_NUMBER) {\n            throw new IOException(\"Corrupt data: Unexpected magic number \" + byteBuffer.getInt());\n        }\n\n        final byte[] objectBytes = new byte[byteBuffer.getInt()];\n        byteBuffer.get(objectBytes);\n\n        final byte[] uploadIdBytes = new byte[byteBuffer.getInt()];\n        byteBuffer.get(uploadIdBytes);\n\n        final int numParts = byteBuffer.getInt();\n        final ArrayList<PartETag> parts = new ArrayList<>(numParts);\n        for (int i = 0; i < numParts; i++) {\n            final int partNum = byteBuffer.getInt();\n            final byte[] buffer = new byte[byteBuffer.getInt()];\n            byteBuffer.get(buffer);\n            parts.add(new PartETag(partNum, new String(buffer, CHARSET)));\n        }\n\n        final long numBytes = byteBuffer.getLong();","sourceCodeStart":112,"sourceCodeEnd":148,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-oss-fs-hadoop/src/main/java/org/apache/flink/fs/osshadoop/writer/OSSRecoverableSerializer.java#L112-L148","documentation":"The v1 OSSRecoverable payload begins with MAGIC_NUMBER. deserializeV1 reads the first int (little-endian); if it differs, the bytes are not a OSSRecoverable v1 body and this IOException with the (already-consumed, hence unreliable) value is thrown. Same corruption signature as 1168 but for the OSS serializer.","triggerScenarios":"Feeding bytes of a HadoopFsRecoverable or S3 recoverable (or arbitrary corrupt bytes) into OSSRecoverableSerializer.deserialize; truncated state; mixed plugin versions writing/reading the same sink state.","commonSituations":"Flink downgrade with reused savepoints; switching filesystem plugins under the same sink state; corrupted state backend bytes on OSS/HDFS.","solutions":["Ensure the checkpoint's sink state was produced by the OSS plugin being used at restore (same scheme throughout)","Restore from an earlier retained checkpoint if the latest state bytes are corrupt","Do not transcode or manually edit serialized recoverable state"],"exampleFix":null,"handlingStrategy":"type-guard","validationCode":"null","typeGuard":"boolean looksLikeOssRecoverableV1(byte[] bytes) {\n    return bytes != null && bytes.length >= 4\n        && ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).getInt() == /* MAGIC_NUMBER */ 0x7bab5dea;\n}","tryCatchPattern":"catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"magic number\")) {\n        // wrong serializer or corrupt bytes: fail fast; restore an earlier checkpoint if available\n    } else { throw e; }\n}","preventionTips":["Never mix recoverable bytes between filesystem plugins","Verify state backend integrity (checksums) where supported"],"tags":["oss","serialization","corrupt-state","magic-number"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}