{"record":{"id":"8d70f34ae749359c","repo":"apache/seatunnel","slug":"the-offset-to-start-reading-from-has-been-removed","errorCode":null,"errorMessage":"The offset to start reading from has been removed from the database write-ahead log. Create a new snapshot and consider setting of PostgreSQL parameter wal_keep_segments = 0.","messagePattern":"The offset to start reading from has been removed from the database write-ahead log\\. Create a new snapshot and consider setting of PostgreSQL parameter wal_keep_segments = 0\\.","errorType":"exception","errorClass":"ConnectException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-cdc/connector-cdc-opengauss/src/main/java/io/debezium/connector/postgresql/connection/PostgresReplicationConnection.java","lineNumber":559,"sourceCode":"            }\n        } catch (PSQLException e) {\n            if (e.getMessage().matches(\"(?s)ERROR: option .* is unknown.*\")) {\n                // It is possible we are connecting to an old wal2json plug-in\n                LOGGER.warn(\n                        \"Could not register for streaming with metadata in messages, falling back to messages without metadata\");\n\n                // re-init the slot after a failed start of slot, as this\n                // may have closed the slot\n                if (useTemporarySlot()) {\n                    initReplicationSlot();\n                }\n\n                s = startPgReplicationStream(startLsn, messageDecoder::optionsWithoutMetadata);\n                messageDecoder.setContainsMetadata(false);\n            } else if (e.getMessage()\n                    .matches(\"(?s)ERROR: requested WAL segment .* has already been removed.*\")) {\n                LOGGER.error(\"Cannot rewind to last processed WAL position\", e);\n                throw new ConnectException(\n                        \"The offset to start reading from has been removed from the database write-ahead log. Create a new snapshot and consider setting of PostgreSQL parameter wal_keep_segments = 0.\");\n            } else {\n                throw e;\n            }\n        }\n\n        final PGReplicationStream stream = s;\n\n        return new ReplicationStream() {\n\n            private static final int CHECK_WARNINGS_AFTER_COUNT = 100;\n            private int warningCheckCounter = CHECK_WARNINGS_AFTER_COUNT;\n            private ExecutorService keepAliveExecutor = null;\n            private AtomicBoolean keepAliveRunning;\n            private final Metronome metronome =\n                    Metronome.sleeper(statusUpdateInterval, Clock.SYSTEM);\n\n            // make sure this is volatile since multiple threads may be interested in this value","sourceCodeStart":541,"sourceCodeEnd":577,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-cdc/connector-cdc-opengauss/src/main/java/io/debezium/connector/postgresql/connection/PostgresReplicationConnection.java#L541-L577","documentation":"PostgreSQL has already recycled/removed the WAL segment containing the slot's confirmed flush LSN, so the replication stream cannot be rewound to the stored offset. Debezium (bundled in connector-cdc-opengauss) detects the server's 'requested WAL segment ... has already been removed' reply during startPgReplicationStream and converts it into a ConnectException, because continuing would silently skip data. The only recovery is a new snapshot; wal_keep_segments/wal_keep_size only delays, not prevents, this.","triggerScenarios":"createReplicationStream -> startStreaming calls startPgReplicationStream(startLsn,...) and the server responds 'ERROR: requested WAL segment ... has already been removed' because the connector was down (or the slot lagged) long enough for PostgreSQL to recycle the needed WAL segment.","commonSituations":"Connector stopped for hours/days while the database kept writing; too-small wal_keep_size/replication slot lag; WAL disk pressure forcing recycling; restart after a long paused pipeline; slot unused while archives were purged.","solutions":["Take a new snapshot (reset offsets/delete the connector state) so streaming restarts from a current LSN","Increase PostgreSQL wal_keep_size (or legacy wal_keep_segments) and/or max_slot_wal_keep_size so required segments survive outages","Reduce connector downtime: monitor replication slot lag (pg_replication_slots) and alert before WAL is recycled","If archive_mode is on, restore/purge policies can be tuned; ensure segments referenced by the slot are not manually deleted"],"exampleFix":"-- before (postgresql.conf)\nwal_keep_size = 0\n-- after\nwal_keep_size = 512MB\nmax_slot_wal_keep_size = 10GB","handlingStrategy":"validation","validationCode":"SELECT slot_name, active, pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS retained_bytes, safe_wal_size FROM pg_replication_slots WHERE slot_name = 'debezium';\n-- also check: SHOW wal_keep_size; and that required segments exist:\nSELECT pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) < 2147483648 AS wal_within_keep;","typeGuard":null,"tryCatchPattern":"try {\n    stream = connection.createReplicationStream();\n} catch (ConnectException e) {\n    if (e.getMessage().contains(\"removed from the database write-ahead log\")) {\n        // WAL segment gone: cannot resume — trigger full re-snapshot / reset offsets\n        planner.recoverWithSnapshot();\n    } else {\n        throw e; // transient: allow retry\n    }\n}","preventionTips":["Monitor pg_replication_slots restart_lsn lag and alert before WAL recycling","Set generous wal_keep_size / max_slot_wal_keep_size for expected outage windows","Avoid leaving the connector stopped for long; resume pipelines quickly","Schedule snapshots after maintenance that pauses CDC for extended periods"],"tags":["cdc","postgres","opengauss","wal","replication-slot","snapshot-required"],"backgroundTag":"wal-segment-removed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}