{"record":{"id":"7ce778a87040c60b","repo":"apache/seatunnel","slug":"some-of-the-gtids-needed-to-replicate-have-been-al","errorCode":null,"errorMessage":"Some of the GTIDs needed to replicate have been already purged","messagePattern":"Some of the GTIDs needed to replicate have been already purged","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/reader/fetch/MySqlSourceFetchTaskContext.java","lineNumber":407,"sourceCode":"\n        if (gtidSet.isContainedWithin(availableGtidSet)) {\n            LOG.info(\n                    \"MySQL current GTID set {} does contain the GTID set {} required by the connector.\",\n                    availableGtidSet,\n                    gtidSet);\n            // The replication is concept of mysql master-slave replication protocol ...\n            final GtidSet gtidSetToReplicate =\n                    connection.subtractGtidSet(availableGtidSet, gtidSet);\n            final GtidSet purgedGtidSet = connection.purgedGtidSet();\n            LOG.info(\"Server has already purged {} GTIDs\", purgedGtidSet);\n            final GtidSet nonPurgedGtidSetToReplicate =\n                    connection.subtractGtidSet(gtidSetToReplicate, purgedGtidSet);\n            LOG.info(\n                    \"GTID set {} known by the server but not processed yet, for replication are available only GTID set {}\",\n                    gtidSetToReplicate,\n                    nonPurgedGtidSetToReplicate);\n            if (!gtidSetToReplicate.equals(nonPurgedGtidSetToReplicate)) {\n                LOG.warn(\"Some of the GTIDs needed to replicate have been already purged\");\n                logGtidNotAvailableDiagnostics(offset);\n                return false;\n            }\n            return true;\n        }\n        LOG.info(\"Connector last known GTIDs are {}, but MySQL has {}\", gtidSet, availableGtidSet);\n        logGtidNotAvailableDiagnostics(offset);\n        return false;\n    }\n\n    private void logBinlogNotAvailableDiagnostics(\n            String requiredBinlogFilename,\n            List<String> availableBinlogFiles,\n            MySqlOffsetContext offset) {\n        LOG.warn(\n                \"MySQL-CDC diagnostic: requested starting offset sourceInfo={}\",\n                offset.getSourceInfo());\n        LOG.warn(","sourceCodeStart":389,"sourceCodeEnd":425,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/reader/fetch/MySqlSourceFetchTaskContext.java#L389-L425","documentation":"checkGtidSet() computes the GTID set the connector must replicate and subtracts the server's purged GTID set. If the difference shows that some GTIDs required by the saved offset have already been purged from the server's binlogs (gtidSetToReplicate != nonPurgedGtidSetToReplicate), the required transactions no longer exist on the server; the warning is logged and the reader cannot resume.","triggerScenarios":"checkGtidSet(): after subtractGtidSet(gtidSetToReplicate, purgedGtidSet), the result differs from gtidSetToReplicate — the server's PURGED GTID set includes GTIDs still needed by the connector offset, typically because binlogs were purged while the job was down.","commonSituations":"Long stoppage exceeding binlog retention; explicit PURGE BINARY LOGS executed; restoring a checkpoint against a server that already purged the relevant transactions.","solutions":["Take a new snapshot: reset the job state / start offsets so the connector snapshots the tables again.","Increase retention (binlog_expire_logs_seconds) and monitor disk so binlogs survive job downtime.","If the transactions exist on another server in the topology, fail over or clone from a donor that still has them (e.g. provision the server via clone/backup).","Use MySQL backup + PITR to restore the purged binlogs, then resume the job."],"exampleFix":"-- before (server)\nbinlog_expire_logs_seconds = 3600 -- purged while job paused\n-- after\nSET PERSIST binlog_expire_logs_seconds = 604800;\n-- and re-snapshot the job:\n-- restart SeaTunnel CDC job with empty/earliest state","handlingStrategy":"validation","validationCode":"-- Before resuming, ensure the needed GTIDs are not purged:\nSELECT @@global.gtid_purged;\n-- The checkpoint's GTID set must not be a subset overlap of gtid_purged beyond executed set.","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Compare job downtime against binlog retention before restarting","Avoid PURGE BINARY LOGS while CDC jobs are stopped","Monitor gtid_purged growth vs the job's checkpoint GTID set","Re-snapshot the job if required GTIDs are gone","Provision replacement servers via backup/clone instead of empty replicas"],"tags":["mysql","gtid","binlog","retention","checkpoint"],"backgroundTag":"resource-not-found","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}