apache/seatunnel · critical · MongodbConnectorException

Non-heartbeat record has no documentKey field, this is unexp

Error message

Non-heartbeat record has no documentKey field, this is unexpected. Record: {}

What it means

MongodbFetchTaskContext.isRecordBetween performs resume-token-based range checks on change stream documents and requires every non-heartbeat event to carry a documentKey field. When a record lacks documentKey but is also not recognized as a heartbeat event, the connector cannot classify it, logs the warning, and throws MongodbConnectorException(ILLEGAL_ARGUMENT) with this message. This is an internal invariant: known Debezium/MongoDB event types (insert/update/delete/replace, DDL, heartbeat) always either have documentKey or are heartbeats.

Source

Thrown at seatunnel-connectors-v2/connector-cdc/connector-cdc-mongodb/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mongodb/source/fetch/MongodbFetchTaskContext.java:178

    }

    @Override
    public boolean isDataChangeRecord(SourceRecord record) {
        return MongodbRecordUtils.isDataChangeRecord(record);
    }

    @Override
    public boolean isRecordBetween(
            SourceRecord record, @Nonnull Object[] splitStart, @Nonnull Object[] splitEnd) {
        BsonDocument documentKey = getDocumentKey(record);
        if (documentKey == null) {
            if (isHeartbeatEvent(record)) {
                log.debug(
                        "Heartbeat record has no documentKey field, skipping range check. Record: {}",
                        record);
                return false;
            }
            log.warn(
                    "Non-heartbeat record has no documentKey field, this is unexpected. Record: {}",
                    record);
            throw new MongodbConnectorException(
                    ILLEGAL_ARGUMENT,
                    "Record has no documentKey field but is not a heartbeat event. "
                            + "This indicates an unexpected record type: "
                            + record);
        }
        BsonDocument splitKeys = (BsonDocument) splitStart[0];
        String firstKey = splitKeys.getFirstKey();
        BsonValue keyValue = documentKey.get(firstKey);
        BsonValue lowerBound = ((BsonDocument) splitStart[1]).get(firstKey);
        BsonValue upperBound = ((BsonDocument) splitEnd[1]).get(firstKey);

        if (isFullRange(lowerBound, upperBound)) {
            return true;
        }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Upgrade the SeaTunnel MongoDB CDC connector to a version whose event handling matches your MongoDB server/driver version
2. Remove or fix any custom change stream pipeline (e.g. $project/$match stages) that could strip the documentKey field from emitted events
3. Inspect the logged Record value to identify the unexpected op/event type and confirm it is legitimate; if it is a known new event type, patch isHeartbeatEvent/record classification to handle it
4. Verify no external writers are injecting malformed documents into the watched collection/change stream

Example fix

// before (record classification misses the new event type)
if (isHeartbeatEvent(record)) { ... return false; }
throw new MongodbConnectorException(ILLEGAL_ARGUMENT, "Record has no documentKey field...");
// after (handle the unexpected type gracefully)
if (isHeartbeatEvent(record)) { return false; }
if (isKnownNonDocumentEvent(record)) { return false; } // e.g. DDL/invalidate events
log.warn("Skipping record without documentKey: {}", record);
return false;

// Classify the record before calling range checks
boolean isClassifiable(Document record) {
    return record.containsKey("documentKey")
        || record.containsKey("id") /* heartbeat */
        || KNOWN_NON_DOCUMENT_EVENT_TYPES.contains(record.getString("operation"));
}

boolean hasDocumentKey(Document record) {
    return record != null && record.containsKey("documentKey");
}

try {
    boolean inRange = isRecordBetween(record);
} catch (MongodbConnectorException e) {
    if (e.getErrorCode() == ILLEGAL_ARGUMENT && e.getMessage().contains("documentKey")) {
        log.warn("Skipping unclassifiable change stream record: {}", record);
        return; // skip, do not crash the pipeline
    }
    throw e;
}

When it happens

Trigger: isRecordBetween() receives a Document with no 'documentKey' field and isHeartbeatEvent(record) returns false — i.e. an unexpected change stream event type (unrecognized op type, malformed driver-converted document, third-party writer inserting events with missing fields, or connector/driver version mismatch producing a record shape the code doesn't know).

Common situations: Upgrading MongoDB server or driver to a version emitting new event types (e.g. new DDL/change-stream events) while the connector's isHeartbeatEvent/record parsing predates them; custom aggregation pipeline in the change stream that strips documentKey from events; corrupted or hand-crafted change events; mixing connector versions (older fetch task against newer record format).

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• ILLEGAL_ARGUMENT
• ILLEGAL_ARGUMENT
• UNSUPPORTED_OPERATION
• Change stream cursor has expired, trying to recreate cursor
• Resume token has expired, fallback to timestamp restart mode

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/9f9a060470e8ce4c. Report an issue: GitHub.