apache/flink · critical · IllegalStateException

The "before" field of %s message is null, if you are using D

Error message

The "before" field of %s message is null, if you are using Debezium Postgres Connector, please check the Postgres table has been set REPLICA IDENTITY to FULL level.

What it means

Thrown by DebeziumJsonDeserializationSchema when an UPDATE operation arrives with a null "before" field. The format treats this as a misconfigured source: Debezium's Postgres connector only emits full before-images when the table uses REPLICA IDENTITY FULL; otherwise UPDATE before can be null, and Flink cannot build the UPDATE_BEFORE row. The template names the operation ('UPDATE') and points at the replica identity fix.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/debezium/DebeziumJsonDeserializationSchema.java:164

        genericRowDataList.clear();
        try {
            GenericRowData row = (GenericRowData) jsonDeserializer.deserialize(message);
            GenericRowData payload;
            if (schemaInclude) {
                payload = (GenericRowData) row.getField(0);
            } else {
                payload = row;
            }

            GenericRowData before = (GenericRowData) payload.getField(0);
            GenericRowData after = (GenericRowData) payload.getField(1);
            String op = payload.getField(2).toString();
            if (OP_CREATE.equals(op) || OP_READ.equals(op)) {
                after.setRowKind(RowKind.INSERT);
                genericRowDataList.add(handleRow(row, after));
            } else if (OP_UPDATE.equals(op)) {
                if (before == null) {
                    throw new IllegalStateException(
                            String.format(REPLICA_IDENTITY_EXCEPTION, "UPDATE"));
                }
                before.setRowKind(RowKind.UPDATE_BEFORE);
                after.setRowKind(RowKind.UPDATE_AFTER);
                genericRowDataList.add(handleRow(row, before));
                genericRowDataList.add(handleRow(row, after));
            } else if (OP_DELETE.equals(op)) {
                if (before == null) {
                    throw new IllegalStateException(
                            String.format(REPLICA_IDENTITY_EXCEPTION, "DELETE"));
                }
                before.setRowKind(RowKind.DELETE);
                genericRowDataList.add(handleRow(row, before));
            } else {
                if (!ignoreParseErrors) {
                    throw new IOException(
                            format(
                                    "Unknown \"op\" value \"%s\". The Debezium JSON message is '%s'",

View on GitHub (pinned to 2f3c205e92)

Solutions

On the Postgres source table run: ALTER TABLE <table> REPLICA IDENTITY FULL; then let Debezium capture subsequent updates
If the sink only needs the after-image, use an append-only style consumption or filter so UPDATE handling does not require before
As a stopgap, 'debezium-json' with ignore-parse-errors skips the failing message — but data will be silently lost, so prefer the replica identity fix
Verify with a sample message (jq '.payload.before') that before is populated after the change

Example fix

-- before
CREATE TABLE orders (...);  -- REPLICA IDENTITY DEFAULT

-- after
ALTER TABLE orders REPLICA IDENTITY FULL;

Defensive patterns

Strategy: validation

Validate before calling

// In Postgres, verify before relying on before-images for updates:
// SELECT relreplident FROM pg_class WHERE relname = '<table>';  -- want 'f' (FULL)
// If not 'f': ALTER TABLE <table> REPLICA IDENTITY FULL;

Try / catch

catch (IllegalStateException e) on REPLICA_IDENTITY_EXCEPTION — stop, fix the source table's replica identity, and restart from the checkpoint; the record is not retryable as-is.

Prevention

Set REPLICA IDENTITY FULL on every CDC-captured Postgres table at provisioning time
Check relreplident in infrastructure checks
Never paper over this with ignore-parse-errors (silent data loss)

When it happens

Trigger: Debezium Postgres connector capturing a table whose REPLICA IDENTITY is DEFAULT (or NOTHING), producing an UPDATE event with before = null, consumed via format 'debezium-json'. Fails the job at runtime on the first such record.

Common situations: Newly captured Postgres tables not altered to REPLICA IDENTITY FULL; defaults on managed Postgres (RDS/CloudSQL) where identity is DEFAULT; switching a Debezium topic from append-only tooling to Flink changelog consumption.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/5cb7ac7d315abcf2. Report an issue: GitHub.