apache/seatunnel · error

Some of the GTIDs needed to replicate have been already purg

Error message

Some of the GTIDs needed to replicate have been already purged

What it means

checkGtidSet() computes the GTID set the connector must replicate and subtracts the server's purged GTID set. If the difference shows that some GTIDs required by the saved offset have already been purged from the server's binlogs (gtidSetToReplicate != nonPurgedGtidSetToReplicate), the required transactions no longer exist on the server; the warning is logged and the reader cannot resume.

Source

Thrown at seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/reader/fetch/MySqlSourceFetchTaskContext.java:407

        if (gtidSet.isContainedWithin(availableGtidSet)) {
            LOG.info(
                    "MySQL current GTID set {} does contain the GTID set {} required by the connector.",
                    availableGtidSet,
                    gtidSet);
            // The replication is concept of mysql master-slave replication protocol ...
            final GtidSet gtidSetToReplicate =
                    connection.subtractGtidSet(availableGtidSet, gtidSet);
            final GtidSet purgedGtidSet = connection.purgedGtidSet();
            LOG.info("Server has already purged {} GTIDs", purgedGtidSet);
            final GtidSet nonPurgedGtidSetToReplicate =
                    connection.subtractGtidSet(gtidSetToReplicate, purgedGtidSet);
            LOG.info(
                    "GTID set {} known by the server but not processed yet, for replication are available only GTID set {}",
                    gtidSetToReplicate,
                    nonPurgedGtidSetToReplicate);
            if (!gtidSetToReplicate.equals(nonPurgedGtidSetToReplicate)) {
                LOG.warn("Some of the GTIDs needed to replicate have been already purged");
                logGtidNotAvailableDiagnostics(offset);
                return false;
            }
            return true;
        }
        LOG.info("Connector last known GTIDs are {}, but MySQL has {}", gtidSet, availableGtidSet);
        logGtidNotAvailableDiagnostics(offset);
        return false;
    }

    private void logBinlogNotAvailableDiagnostics(
            String requiredBinlogFilename,
            List<String> availableBinlogFiles,
            MySqlOffsetContext offset) {
        LOG.warn(
                "MySQL-CDC diagnostic: requested starting offset sourceInfo={}",
                offset.getSourceInfo());
        LOG.warn(

View on GitHub (pinned to cf67b549a7)

Solutions

1. Take a new snapshot: reset the job state / start offsets so the connector snapshots the tables again.
2. Increase retention (binlog_expire_logs_seconds) and monitor disk so binlogs survive job downtime.
3. If the transactions exist on another server in the topology, fail over or clone from a donor that still has them (e.g. provision the server via clone/backup).
4. Use MySQL backup + PITR to restore the purged binlogs, then resume the job.

Example fix

-- before (server)
binlog_expire_logs_seconds = 3600 -- purged while job paused
-- after
SET PERSIST binlog_expire_logs_seconds = 604800;
-- and re-snapshot the job:
-- restart SeaTunnel CDC job with empty/earliest state

-- Before resuming, ensure the needed GTIDs are not purged:
SELECT @@global.gtid_purged;
-- The checkpoint's GTID set must not be a subset overlap of gtid_purged beyond executed set.

When it happens

Trigger: checkGtidSet(): after subtractGtidSet(gtidSetToReplicate, purgedGtidSet), the result differs from gtidSetToReplicate — the server's PURGED GTID set includes GTIDs still needed by the connector offset, typically because binlogs were purged while the job was down.

Common situations: Long stoppage exceeding binlog retention; explicit PURGE BINARY LOGS executed; restoring a checkpoint against a server that already purged the relevant transactions.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Connector requires binlog file '{}', but it is not available
• Unexpected error while connecting to MySQL and looking at gt
• Connector used GTIDs previously, but MySQL does not know of
• MySQL-CDC diagnostic: variable {}={}
• MySQL-CDC diagnostic: master status file={}, position={}, ex

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/7ce778a87040c60b. Report an issue: GitHub.