alibaba/canal · error · IllegalArgumentException

unknow compress type for :

Error message

unknow compress type for :

What it means

Thrown by LogDecoder.processIterateDecode() when processing a TRANSACTION_PAYLOAD_EVENT (compressed transaction payload, used by MySQL 8.0+ binlog transaction compression) and the event reports neither Zstd compression nor uncompressed (None). The compression type is determined by TransactionPayloadLogEvent.isCompressByZstd() and isCompressByNone().

Source

Thrown at dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/LogDecoder.java:137

     * @return
     * @throws IOException
     */
    public List<LogEvent> processIterateDecode(LogEvent event, LogContext context) throws IOException {
        List<LogEvent> events = Lists.newArrayList();
        if (event.getHeader().getType() == LogEvent.TRANSACTION_PAYLOAD_EVENT) {
            // iterate for compresss payload
            TransactionPayloadLogEvent compressEvent = ((TransactionPayloadLogEvent) event);
            LogBuffer iterateBuffer = null;
            if (compressEvent.isCompressByZstd()) {
                try (ZstdCompressorInputStream in = new ZstdCompressorInputStream(
                    new ByteArrayInputStream(compressEvent.getPayload()))) {
                    byte[] decodeBytes = IOUtils.toByteArray(in);
                    iterateBuffer = new LogBuffer(decodeBytes, 0, decodeBytes.length);
                }
            } else if (compressEvent.isCompressByNone()) {
                iterateBuffer = new LogBuffer(compressEvent.getPayload(), 0, compressEvent.getPayload().length);
            } else {
                throw new IllegalArgumentException("unknow compress type for " + event.getHeader().getLogFileName()
                                                   + ":" + event.getHeader().getLogPos());
            }

            try {
                context.setIterateDecode(true);
                while (iterateBuffer.hasRemaining()) {// iterate
                    LogEvent deEvent = decode(iterateBuffer, context);
                    if (deEvent == null) {
                        break;
                    }

                    // compress event logPos = 0
                    deEvent.getHeader().setLogFileName(event.getHeader().getLogFileName());
                    deEvent.getHeader().setLogPos(event.getHeader().getLogPos());
                    // 需要重置payload每个event的eventLen , ack位点更新依赖logPos - eventLen,
                    // 原因:每个payload都是uncompress的eventLen,无法对应物理binlog的eventLen
                    // 隐患:memory计算空间大小时会出现放大的情况,影响getBatch的数量
                    deEvent.getHeader().setEventLen(event.getHeader().getEventLen());

View on GitHub (pinned to 87be50e876)

Solutions

1. Check the MySQL server version — newer versions may add compression algorithms beyond Zstd.
2. Upgrade canal/dbsync to a version that supports the compression algorithm used by the MySQL server.
3. Inspect the raw TransactionPayloadLogEvent header to read the compression-type byte value.
4. Configure the MySQL server to use Zstd or no compression for binlog transactions if the parser version does not support the new algorithm.

// Check compression type before calling processIterateDecode
TransactionPayloadLogEvent compressEvent = (TransactionPayloadLogEvent) event;
if (!compressEvent.isCompressByZstd() && !compressEvent.isCompressByNone()) {
    logger.warn("Unknown binlog transaction compression type in {}, logPos={}",
        event.getHeader().getLogFileName(), event.getHeader().getLogPos());
    // return empty list or throw domain exception
    return Collections.emptyList();
}

try {
    List<LogEvent> events = logDecoder.processIterateDecode(event, context);
} catch (IllegalArgumentException e) {
    if (e.getMessage().startsWith("unknow compress type")) {
        logger.error("Unsupported binlog compression algorithm in {}:{}, upgrade canal/dbsync",
            event.getHeader().getLogFileName(), event.getHeader().getLogPos());
    }
    throw e;
}

When it happens

Trigger: processIterateDecode() receives a TransactionPayloadLogEvent. It checks isCompressByZstd() and isCompressByNone(); if neither returns true, the compression type is unrecognized. This means the payload's compression-type field has a value other than the known constants (Zstd or None).

Common situations: A new MySQL version introduces a new compression algorithm (e.g., LZ4, Brotli) for binlog transaction compression that the parser does not yet support. The TransactionPayloadLogEvent header parsing is incorrect, producing a wrong compression type value. Or the payload is corrupt.

Related errors

• reading operation type (invalid operation code)
• Format Description event header length is too short
• Unknown binlog version:
• No such method: '{}' @ {}
• No such field: '{}' @ {}

AI-assisted analysis of alibaba/canal@87be50e876 (2026-08-14). Data as JSON: /api/errors/984f56ee15254ebe. Report an issue: GitHub.