{"record":{"id":"469062f16b320ce9","repo":"apache/seatunnel","slug":"error-during-binlog-processing-last-offset-stored","errorCode":null,"errorMessage":"Error during binlog processing. Last offset stored = {}, binlog reader near position = {}","messagePattern":"Error during binlog processing\\. Last offset stored = (.+?), binlog reader near position = (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/io/debezium/connector/mysql/MySqlStreamingChangeEventSource.java","lineNumber":1369,"sourceCode":"\n    private void logStreamingSourceState() {\n        logStreamingSourceState(Level.ERROR);\n    }\n\n    protected void logEvent(MySqlOffsetContext offsetContext, Event event) {\n        LOGGER.trace(\"Received event: {}\", event);\n    }\n\n    private void logStreamingSourceState(Level severity) {\n        final Object position =\n                client == null\n                        ? \"N/A\"\n                        : client.getBinlogFilename() + \"/\" + client.getBinlogPosition();\n        final String message =\n                \"Error during binlog processing. Last offset stored = {}, binlog reader near position = {}\";\n        switch (severity) {\n            case WARN:\n                LOGGER.warn(message, lastOffset, position);\n                break;\n            case DEBUG:\n                LOGGER.debug(message, lastOffset, position);\n                break;\n            default:\n                LOGGER.error(message, lastOffset, position);\n        }\n    }\n\n    /**\n     * Apply the include/exclude GTID source filters to the current {@link #source() GTID set} and\n     * merge them onto the currently available GTID set from a MySQL server.\n     *\n     * <p>The merging behavior of this method might seem a bit strange at first. It's required in\n     * order for Debezium to consume a MySQL binlog that has multi-source replication enabled, if a\n     * failover has to occur. In such a case, the server that Debezium is failed over to might have\n     * a different set of sources, but still include the sources required for Debezium to continue\n     * to function. MySQL does not allow downstream replicas to connect if the GTID set does not","sourceCodeStart":1351,"sourceCodeEnd":1387,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/io/debezium/connector/mysql/MySqlStreamingChangeEventSource.java#L1351-L1387","documentation":"A generic error report emitted when binlog processing throws an exception. The connector builds a message containing the last stored offset and the approximate binlog file/position the reader had reached, then logs it at a level chosen by the incident severity (WARN/DEBUG/ERROR as part of the failure-handling policy) and may propagate the error to the error handler.","triggerScenarios":"The public handleServerIncident-style path in MySqlStreamingChangeEventSource catches any exception thrown while reading/decoding binlog events — e.g. deserialization failures, I/O errors on the binlog connection, or downstream handler exceptions — and calls this logging/reporting routine.","commonSituations":"Network interruption between connector and MySQL during streaming; MySQL server restart closing the binlog connection; corrupt or unsupported binlog events; offsets pointing at a purged or rotated binlog file.","solutions":["Read the underlying exception logged alongside this message to find the root cause (I/O vs deserialization)","Check binlog retention: if the last offset's binlog file was purged (SHOW BINARY LOGS / binlog_expire_logs_seconds), take a new snapshot","For transient network/server issues, let the connector's restart backoff retry; ensure MySQL connectivity and keepalives are stable","Increase binlog retention (binlog_expire_logs_seconds) so offsets remain valid across connector downtime"],"exampleFix":"// before\nSET GLOBAL binlog_expire_logs_seconds = 86400;\n// after (retain >= expected downtime window)\nSET GLOBAL binlog_expire_logs_seconds = 604800;","handlingStrategy":"retry","validationCode":"// Preflight source stability checks:\n// SHOW STATUS LIKE 'Uptime'; -- server not flapping\n// SHOW BINARY LOGS; -- last offset's file still present (not purged)\n// SELECT 1; over the same network path used by the connector","typeGuard":null,"tryCatchPattern":"try {\n    streamBinlogFrom(lastOffset);\n} catch (IOException | SQLException e) {\n    logLastOffsetAndPosition(lastOffset);\n    if (isTransient(e)) {\n        retryWithBackoff(lastOffset);\n    } else if (isBinlogPurged(e)) {\n        triggerReSnapshot();\n    } else {\n        failConnector(e);\n    }\n}","preventionTips":["Set binlog_expire_logs_seconds comfortably above maximum connector downtime","Harden network path to MySQL (timeouts, keepalives, LB idle settings)","Ensure server version compatibility with the connector's Debezium","Alert on this message and inspect the root-cause exception, not just the summary"],"tags":["mysql","cdc","binlog","streaming"],"backgroundTag":"database-query-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}