alibaba/canal · error · CanalParseException

preTransactionStartPosition greater than startPosition from

Error message

preTransactionStartPosition greater than startPosition from zk or localconf, maybe lost data

What it means

findTransactionBeginPosition() scans backward to find the transaction (BEGIN) header preceding the target start position, to avoid losing the first half of a transaction. If the discovered preTransactionStartPosition is greater than the requested entryPosition, it means the located transaction begin sits AFTER where we intend to start — implying data before it would be skipped (lost). Canal throws CanalParseException to refuse silent data loss.

Source

Thrown at parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/MysqlEventParser.java:580

                    if (entry.getHeader().getLogfileOffset() >= entryPosition.getPosition()) {
                        return false;// 退出
                    }

                    lastPosition = buildLastPosition(entry);
                } catch (Exception e) {
                    processSinkError(e, lastPosition, entryPosition.getJournalName(), entryPosition.getPosition());
                    return false;
                }

                return running;
            }
        });

        // 判断一下找到的最接近position的事务头的位置
        if (preTransactionStartPosition.get() > entryPosition.getPosition()) {
            logger.error("preTransactionEndPosition greater than startPosition from zk or localconf, maybe lost data");
            throw new CanalParseException("preTransactionStartPosition greater than startPosition from zk or localconf, maybe lost data");
        }
        return preTransactionStartPosition.get();
    }

    // 根据时间查找binlog位置
    private EntryPosition findByStartTimeStamp(MysqlConnection mysqlConnection, Long startTimestamp) {
        EntryPosition endPosition = findEndPosition(mysqlConnection);
        EntryPosition startPosition = findStartPosition(mysqlConnection);
        String maxBinlogFileName = endPosition.getJournalName();
        String minBinlogFileName = startPosition.getJournalName();
        logger.info("show master status to set search end condition:{} ", endPosition);
        String startSearchBinlogFile = endPosition.getJournalName();
        boolean shouldBreak = false;
        while (running && !shouldBreak) {
            try {
                EntryPosition entryPosition = findAsPerTimestampInSpecificLogFile(mysqlConnection,
                    startTimestamp,
                    endPosition,

View on GitHub (pinned to 87be50e876)

Solutions

1. Reset the stored start position to a known transaction boundary (BEGIN event offset) and restart.
2. Use GTID mode so transaction boundaries are tracked atomically.
3. If the data loss window is acceptable, clear the persisted position and let canal resume from the current master end position (autoResetLatestPosMode).
4. Verify the binlog position manager (zk/memory/file) has a consistent, committed position before restart.

Example fix

# reset the persisted position to a safe transaction boundary
canal.instance.master.position=mysql-bin.000010:4  # start of a transaction
# or enable auto-reset to latest if some loss is acceptable
canal.auto.reset.latest.pos.mode=true

// Sanity-check transaction boundary before trusting it
long preTx = findTransactionBeginPosition(connection, entryPosition);
if (preTx > entryPosition.getPosition()) {
    throw new IllegalStateException(
        "Resolved transaction begin " + preTx + " is after start " + entryPosition.getPosition() +
        "; reset persisted position to a transaction boundary or enable GTID.");
}

When it happens

Trigger: needTransactionPosition is set, the binlog scan finds a transaction BEGIN whose offset exceeds the configured start position, indicating the start position lands mid-transaction and the located begin is downstream.

Common situations: Start position recorded/stored points into the middle of a transaction after a restart or HA failover; binlog position managers (zk/local) out of sync with the actual transaction boundaries; clock/offset skew after a master switch.

Related errors

• [fixed timestamp] can't found begin/commit position before w
• Not implement yet
• command : ' + showSql + ' has an error! pls check. you need
• command : ' + showSql + ' has an error!
• position not found!

AI-assisted analysis of alibaba/canal@87be50e876 (2026-08-14). Data as JSON: /api/errors/d6c0fb1c1492fc27. Report an issue: GitHub.