apache/seatunnel · error · SeaTunnelRuntimeException

COMMON-02

COMMON-02

Error message

Json JSON convert/parse '<payload>' operation failed.

What it means

CanalJsonDeserializationSchema.deserialize wraps any RuntimeException thrown while parsing a Canal JSON message or mapping it to a SeaTunnelRow into jsonOperationError, unless ignoreParseErrors is enabled. The raw JSON payload is embedded in the message. It means the message did not conform to the expected Canal CDC envelope (op/data/old fields) or the row converters rejected the values.

Source

Thrown at seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/canal/CanalJsonDeserializationSchema.java:215

                    for (int i = 0; i < dataNode.size(); i++) {
                        SeaTunnelRow row = convertJsonNode(dataNode.get(i));
                        row.setRowKind(RowKind.DELETE);
                        if (tablePath != null && !tablePath.toString().isEmpty()) {
                            row.setTableId(tablePath.toString());
                        }
                        if (tsNode != null) {
                            MetadataUtil.setEventTime(row, tsNode.asLong());
                        }
                        out.collect(row);
                    }
                    break;
                default:
                    throw new IllegalStateException(
                            String.format("Unknown operation type '%s'.", op));
            }
        } catch (RuntimeException e) {
            if (!ignoreParseErrors) {
                throw CommonError.jsonOperationError(FORMAT, jsonNode.toString(), e);
            }
        }
    }

    private ObjectNode convertBytes(byte[] message) throws SeaTunnelRuntimeException {
        if (message == null || message.length == 0) {
            return null;
        }

        try {
            return (ObjectNode) jsonDeserializer.deserializeToJsonNode(message);
        } catch (Throwable t) {
            if (!ignoreParseErrors) {
                throw CommonError.jsonOperationError(FORMAT, new String(message), t);
            }
            return null;
        }
    }

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Log and inspect the payload in the error message and compare it against the expected Canal JSON envelope (data, old, type, ts)
  2. Validate the canal instance configuration so only DML events reach the connector (filter DDL/heartbeat events)
  3. Set ignore-parse-errors=true if dirty/skippable messages should not fail the job
  4. Refresh the connector's table schema (or restart with updated schema) if upstream DDL changed column types

Example fix

// before: job fails on any dirty message
CanalJsonDeserializationSchema.builder(...).build();
// after: tolerate malformed messages
CanalJsonDeserializationSchema.builder(...).setIgnoreParseErrors(true).build();
Defensive patterns

Strategy: try-catch

Validate before calling

// validate Canal envelope before deserialize
JsonNode n = JsonUtils.stringToJsonNode(new String(message));
if (!n.has("type") || !n.has("data")) throw new SkipRecordException();

Type guard

boolean isCanalEnvelope(JsonNode n) {
    return n != null && n.isObject() && n.has("type") && n.has("data");
}

Try / catch

try {
    schema.deserialize(message, out);
} catch (SeaTunnelRuntimeException e) {
    log.warn("Skipping malformed canal message: {}", new String(message), e);
}

Prevention

When it happens

Trigger: deserialize(byte[] message, Collector<SeaTunnelRow>) receives a Canal message whose JSON structure, 'op' value, or 'data' fields do not match the expected schema; converters throw RuntimeException during row conversion.

Common situations: Canal server emitting messages with format version mismatches or non-standard fields; topic polluted with non-CDC messages (DDL events, heartbeats); table schema changed upstream so row conversion fails; 'op' values outside the supported set hitting the IllegalStateException branch.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/791e10853e6c3931. Report an issue: GitHub.