{"record":{"id":"80d87cf7c4ea19eb","repo":"alibaba/canal","slug":"reading-json-diff","errorCode":null,"errorMessage":"reading json diff","messagePattern":"reading json diff","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/JsonDiffConversion.java","lineNumber":83,"sourceCode":"            buffer.forward((int) path_length);\n            // skip value\n            if (operation_int != DIFF_OPERATION_REMOVE) {\n                long value_length = buffer.getPackedLong();\n                if (value_length > len) {\n                    throw new IllegalArgumentException(\"skipping path\");\n                }\n\n                buffer.forward((int) value_length);\n            }\n\n            // see https://github.com/alibaba/canal/pull/5018\n            if (buffer.position() - position >= len) {\n                break;\n            }\n        }\n\n        if (buffer.position() - position != len) {\n            throw new IllegalArgumentException(\"reading json diff\");\n        }\n\n        // Print function names in reverse order.\n        StringBuilder builder = new StringBuilder();\n        for (int i = operation_names.size() - 1; i >= 0; i--) {\n            if (i == 0 || !Objects.equals(operation_names.get(i - 1), operation_names.get(i))) {\n                builder.append(operation_names.get(i)).append(\"(\");\n            }\n        }\n\n        // Print column id\n        if (columnName != null) {\n            builder.append(columnName);\n        } else {\n            builder.append(\"@\").append(columnIndex);\n        }\n\n        // In case this vector is empty (a no-op), make an early return","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/alibaba/canal/blob/87be50e87686a3e8af08c368d0e1ffd1f59eb04a/dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/JsonDiffConversion.java#L65-L101","documentation":"Thrown at the end of the first pass (skip/scan pass) of JSON diff parsing when the total bytes consumed (buffer.position() - position) does not equal the expected diff length (len). This means the operations consumed more or fewer bytes than declared, indicating the diff payload is internally inconsistent.","triggerScenarios":"After the while loop in print_json_diff's first pass processes all operations (the loop breaks when position - position >= len), a final check compares position - position against len. If they differ, the diff structure is corrupt: the sum of all operation/path/value sizes does not match the declared total.","commonSituations":"A partial-update event where one operation's path or value length was off-by-one, binlog corruption altering a length byte, buffer misalignment, or a MySQL 8.0 minor version change in the diff encoding format.","solutions":["Compare the actual consumed bytes (buffer.position() - position) with len to understand the magnitude of the discrepancy.","Hex-dump the full diff payload and manually verify each operation's path_length and value_length fields.","Verify the canal/dbsync version supports the specific MySQL 8.0 minor version generating these partial updates.","Re-synchronize replication from a known-good position if the binlog is genuinely corrupt."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    JsonDiffConversion.print_json_diff(buffer, len, columnName, columnIndex, charset);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().equals(\"reading json diff\")) {\n        long consumed = buffer.position() - startPos;\n        logger.warn(\"JSON diff consumed {} bytes but expected {}, column={}\", consumed, len, columnName);\n    }\n    throw e;\n}","preventionTips":["Capture the starting buffer position before calling print_json_diff to log consumed-vs-expected on failure.","Verify the binlog event containing the diff is not truncated or corrupted.","Keep the parser version aligned with the MySQL 8.0 server version."],"tags":["json-diff","binlog","mysql","length-mismatch","consistency-check"],"backgroundTag":null,"analyzedSha":"87be50e87686a3e8af08c368d0e1ffd1f59eb04a","analyzedAt":"2026-08-14T04:30:11.918Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}