apache/flink · error · IOException

Unknown "op" value "%s". The Debezium JSON message is '%s'

Error message

Unknown "op" value "%s". The Debezium JSON message is '%s'

What it means

Thrown by DebeziumJsonDeserializationSchema when a Debezium message's "op" field is not one of 'c' (create), 'r' (read/snapshot), 'u' (update), 'd' (delete) — e.g. truncate ('t') or no-op events. With 'json.ignore-parse-errors' disabled (default) this IOException fails the job; with it enabled the message is logged at DEBUG and skipped.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/debezium/DebeziumJsonDeserializationSchema.java:180

            } else if (OP_UPDATE.equals(op)) {
                if (before == null) {
                    throw new IllegalStateException(
                            String.format(REPLICA_IDENTITY_EXCEPTION, "UPDATE"));
                }
                before.setRowKind(RowKind.UPDATE_BEFORE);
                after.setRowKind(RowKind.UPDATE_AFTER);
                genericRowDataList.add(handleRow(row, before));
                genericRowDataList.add(handleRow(row, after));
            } else if (OP_DELETE.equals(op)) {
                if (before == null) {
                    throw new IllegalStateException(
                            String.format(REPLICA_IDENTITY_EXCEPTION, "DELETE"));
                }
                before.setRowKind(RowKind.DELETE);
                genericRowDataList.add(handleRow(row, before));
            } else {
                if (!ignoreParseErrors) {
                    throw new IOException(
                            format(
                                    "Unknown \"op\" value \"%s\". The Debezium JSON message is '%s'",
                                    op, new String(message)));
                }
                if (LOG.isDebugEnabled()) {
                    LOG.debug(
                            "Unknown \"op\" value '{}'. The Debezium JSON message is '{}'.",
                            op,
                            new String(message));
                }
            }
        } catch (Throwable t) {
            // a big try catch to protect the processing.
            if (!ignoreParseErrors) {
                throw new IOException(
                        format("Corrupt Debezium JSON message '%s'.", new String(message)), t);
            }
            if (LOG.isDebugEnabled()) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set 'json.ignore-parse-errors' = true to skip unsupported op types when losing truncate/heartbeat events is acceptable
2. Disable TRUNCATE emission at the Debezium connector if those events are not needed (e.g. timestamper/skipped ddl options depending on connector)
3. Route only data-change events to the consumed topic via connector's topic routing/filter (SMT) configuration
4. Verify the payload is genuinely Debezium JSON ('op' at payload level) and not schema-wrapped twice

Example fix

// before
WITH ('connector'='kafka', 'format'='debezium-json')

// after
WITH ('connector'='kafka', 'format'='debezium-json',
  'json.ignore-parse-errors'='true')

String op = payload.get("op").asText();
if (!Set.of("c","r","u","d").contains(op)) { /* skip / route elsewhere */ }

static boolean isHandledDebeziumOp(String op) {
    return op != null && op.length() == 1 && "crud".indexOf(op) >= 0; // c,u,d plus r snapshot
}

catch (IOException e) on 'Unknown "op" value' — enable ignore-parse-errors for benign ops (t/heartbeat) or filter them at the connector; no point retrying.

When it happens

Trigger: A Debezium topic containing 't' (truncate), 'n' (heartbeat/signal op variants), or vendor/connector-specific op values, consumed via format 'debezium-json' with default options.

Common situations: Debezium TRUNCATE events enabled via snapshot.mode/endpoint options (Postgres/MySQL truncate handling); mixed topics with signal or heartbeat events; newer Debezium versions emitting additional op kinds; mislabeled topics carrying non-Debezium JSON.

Related errors

• Unknown "type" value "%s". The Canal JSON message is '%s'
• Corrupt Debezium JSON message '%s'.
• Please invoke DeserializationSchema#deserialize(byte[], Coll
• The "before" field of %s message is null, if you are using D
• Could not find field with name '%s'.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/fce06e9b42c702d7. Report an issue: GitHub.