{"record":{"id":"d6c0fb1c1492fc27","repo":"alibaba/canal","slug":"pretransactionstartposition-greater-than-startposi","errorCode":null,"errorMessage":"preTransactionStartPosition greater than startPosition from zk or localconf, maybe lost data","messagePattern":"preTransactionStartPosition greater than startPosition from zk or localconf, maybe lost data","errorType":"exception","errorClass":"CanalParseException","httpStatus":null,"severity":"error","filePath":"parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/MysqlEventParser.java","lineNumber":580,"sourceCode":"\n                    if (entry.getHeader().getLogfileOffset() >= entryPosition.getPosition()) {\n                        return false;// 退出\n                    }\n\n                    lastPosition = buildLastPosition(entry);\n                } catch (Exception e) {\n                    processSinkError(e, lastPosition, entryPosition.getJournalName(), entryPosition.getPosition());\n                    return false;\n                }\n\n                return running;\n            }\n        });\n\n        // 判断一下找到的最接近position的事务头的位置\n        if (preTransactionStartPosition.get() > entryPosition.getPosition()) {\n            logger.error(\"preTransactionEndPosition greater than startPosition from zk or localconf, maybe lost data\");\n            throw new CanalParseException(\"preTransactionStartPosition greater than startPosition from zk or localconf, maybe lost data\");\n        }\n        return preTransactionStartPosition.get();\n    }\n\n    // 根据时间查找binlog位置\n    private EntryPosition findByStartTimeStamp(MysqlConnection mysqlConnection, Long startTimestamp) {\n        EntryPosition endPosition = findEndPosition(mysqlConnection);\n        EntryPosition startPosition = findStartPosition(mysqlConnection);\n        String maxBinlogFileName = endPosition.getJournalName();\n        String minBinlogFileName = startPosition.getJournalName();\n        logger.info(\"show master status to set search end condition:{} \", endPosition);\n        String startSearchBinlogFile = endPosition.getJournalName();\n        boolean shouldBreak = false;\n        while (running && !shouldBreak) {\n            try {\n                EntryPosition entryPosition = findAsPerTimestampInSpecificLogFile(mysqlConnection,\n                    startTimestamp,\n                    endPosition,","sourceCodeStart":562,"sourceCodeEnd":598,"githubUrl":"https://github.com/alibaba/canal/blob/87be50e87686a3e8af08c368d0e1ffd1f59eb04a/parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/MysqlEventParser.java#L562-L598","documentation":"findTransactionBeginPosition() scans backward to find the transaction (BEGIN) header preceding the target start position, to avoid losing the first half of a transaction. If the discovered preTransactionStartPosition is greater than the requested entryPosition, it means the located transaction begin sits AFTER where we intend to start — implying data before it would be skipped (lost). Canal throws CanalParseException to refuse silent data loss.","triggerScenarios":"needTransactionPosition is set, the binlog scan finds a transaction BEGIN whose offset exceeds the configured start position, indicating the start position lands mid-transaction and the located begin is downstream.","commonSituations":"Start position recorded/stored points into the middle of a transaction after a restart or HA failover; binlog position managers (zk/local) out of sync with the actual transaction boundaries; clock/offset skew after a master switch.","solutions":["Reset the stored start position to a known transaction boundary (BEGIN event offset) and restart.","Use GTID mode so transaction boundaries are tracked atomically.","If the data loss window is acceptable, clear the persisted position and let canal resume from the current master end position (autoResetLatestPosMode).","Verify the binlog position manager (zk/memory/file) has a consistent, committed position before restart."],"exampleFix":"# reset the persisted position to a safe transaction boundary\ncanal.instance.master.position=mysql-bin.000010:4  # start of a transaction\n# or enable auto-reset to latest if some loss is acceptable\ncanal.auto.reset.latest.pos.mode=true","handlingStrategy":"validation","validationCode":"// Sanity-check transaction boundary before trusting it\nlong preTx = findTransactionBeginPosition(connection, entryPosition);\nif (preTx > entryPosition.getPosition()) {\n    throw new IllegalStateException(\n        \"Resolved transaction begin \" + preTx + \" is after start \" + entryPosition.getPosition() +\n        \"; reset persisted position to a transaction boundary or enable GTID.\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use GTID mode so transaction boundaries are tracked atomically.","Reset persisted positions to known BEGIN offsets after HA failover.","Keep the position manager (zk/file) consistent and committed before restart."],"tags":["mysql","binlog","transaction","data-loss","position","canal-parse"],"backgroundTag":null,"analyzedSha":"87be50e87686a3e8af08c368d0e1ffd1f59eb04a","analyzedAt":"2026-08-14T04:30:11.918Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}