{"record":{"id":"984f56ee15254ebe","repo":"alibaba/canal","slug":"unknow-compress-type-for","errorCode":null,"errorMessage":"unknow compress type for :","messagePattern":"unknow compress type for :","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/LogDecoder.java","lineNumber":137,"sourceCode":"     * @return\r\n     * @throws IOException\r\n     */\r\n    public List<LogEvent> processIterateDecode(LogEvent event, LogContext context) throws IOException {\r\n        List<LogEvent> events = Lists.newArrayList();\r\n        if (event.getHeader().getType() == LogEvent.TRANSACTION_PAYLOAD_EVENT) {\r\n            // iterate for compresss payload\r\n            TransactionPayloadLogEvent compressEvent = ((TransactionPayloadLogEvent) event);\r\n            LogBuffer iterateBuffer = null;\r\n            if (compressEvent.isCompressByZstd()) {\r\n                try (ZstdCompressorInputStream in = new ZstdCompressorInputStream(\r\n                    new ByteArrayInputStream(compressEvent.getPayload()))) {\r\n                    byte[] decodeBytes = IOUtils.toByteArray(in);\r\n                    iterateBuffer = new LogBuffer(decodeBytes, 0, decodeBytes.length);\r\n                }\r\n            } else if (compressEvent.isCompressByNone()) {\r\n                iterateBuffer = new LogBuffer(compressEvent.getPayload(), 0, compressEvent.getPayload().length);\r\n            } else {\r\n                throw new IllegalArgumentException(\"unknow compress type for \" + event.getHeader().getLogFileName()\r\n                                                   + \":\" + event.getHeader().getLogPos());\r\n            }\r\n\r\n            try {\r\n                context.setIterateDecode(true);\r\n                while (iterateBuffer.hasRemaining()) {// iterate\r\n                    LogEvent deEvent = decode(iterateBuffer, context);\r\n                    if (deEvent == null) {\r\n                        break;\r\n                    }\r\n\r\n                    // compress event logPos = 0\r\n                    deEvent.getHeader().setLogFileName(event.getHeader().getLogFileName());\r\n                    deEvent.getHeader().setLogPos(event.getHeader().getLogPos());\r\n                    // 需要重置payload每个event的eventLen , ack位点更新依赖logPos - eventLen,\r\n                    // 原因:每个payload都是uncompress的eventLen,无法对应物理binlog的eventLen\r\n                    // 隐患:memory计算空间大小时会出现放大的情况,影响getBatch的数量\r\n                    deEvent.getHeader().setEventLen(event.getHeader().getEventLen());\r","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/alibaba/canal/blob/87be50e87686a3e8af08c368d0e1ffd1f59eb04a/dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/LogDecoder.java#L119-L155","documentation":"Thrown by LogDecoder.processIterateDecode() when processing a TRANSACTION_PAYLOAD_EVENT (compressed transaction payload, used by MySQL 8.0+ binlog transaction compression) and the event reports neither Zstd compression nor uncompressed (None). The compression type is determined by TransactionPayloadLogEvent.isCompressByZstd() and isCompressByNone().","triggerScenarios":"processIterateDecode() receives a TransactionPayloadLogEvent. It checks isCompressByZstd() and isCompressByNone(); if neither returns true, the compression type is unrecognized. This means the payload's compression-type field has a value other than the known constants (Zstd or None).","commonSituations":"A new MySQL version introduces a new compression algorithm (e.g., LZ4, Brotli) for binlog transaction compression that the parser does not yet support. The TransactionPayloadLogEvent header parsing is incorrect, producing a wrong compression type value. Or the payload is corrupt.","solutions":["Check the MySQL server version — newer versions may add compression algorithms beyond Zstd.","Upgrade canal/dbsync to a version that supports the compression algorithm used by the MySQL server.","Inspect the raw TransactionPayloadLogEvent header to read the compression-type byte value.","Configure the MySQL server to use Zstd or no compression for binlog transactions if the parser version does not support the new algorithm."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Check compression type before calling processIterateDecode\nTransactionPayloadLogEvent compressEvent = (TransactionPayloadLogEvent) event;\nif (!compressEvent.isCompressByZstd() && !compressEvent.isCompressByNone()) {\n    logger.warn(\"Unknown binlog transaction compression type in {}, logPos={}\",\n        event.getHeader().getLogFileName(), event.getHeader().getLogPos());\n    // return empty list or throw domain exception\n    return Collections.emptyList();\n}","typeGuard":null,"tryCatchPattern":"try {\n    List<LogEvent> events = logDecoder.processIterateDecode(event, context);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().startsWith(\"unknow compress type\")) {\n        logger.error(\"Unsupported binlog compression algorithm in {}:{}, upgrade canal/dbsync\",\n            event.getHeader().getLogFileName(), event.getHeader().getLogPos());\n    }\n    throw e;\n}","preventionTips":["Check the MySQL server's binlog_transaction_compression setting — only Zstd and None are supported.","Upgrade canal/dbsync when the MySQL server adds a new compression algorithm.","Log the raw compression type byte for diagnosis when this fires."],"tags":["compression","binlog","mysql","mysql-8","transaction-payload","version-compat"],"backgroundTag":null,"analyzedSha":"87be50e87686a3e8af08c368d0e1ffd1f59eb04a","analyzedAt":"2026-08-14T04:30:11.918Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}