apache/seatunnel · error · CouchbaseConnectorException

WRITE_RECORDS_FAILED

WRITE_RECORDS_FAILED

Error message

'upsert-enable' is true but 'primary-key' is not configured. Upsert requires a primary key to build a stable document key.

What it means

The Couchbase sink validates at createSink time that upsert mode has a primary key, because upserts require a deterministic document key. If 'upsert-enable' is true and 'primary-key' is absent or empty, configuration is invalid and job startup fails with WRITE_RECORDS_FAILED.

Source

Thrown at seatunnel-connectors-v2/connector-couchbase/src/main/java/org/apache/seatunnel/connectors/seatunnel/couchbase/sink/CouchbaseSinkFactory.java:79

                        CouchbaseSinkOptions.UPSERT_ENABLE,
                        CouchbaseSinkOptions.PRIMARY_KEY)
                .build();
    }

    @Override
    public TableSink createSink(TableSinkFactoryContext context) {
        ReadonlyConfig config = context.getOptions();

        // Issue 2 guard: upsert without a primary key produces random-UUID document keys,
        // which breaks upsert semantics. Fail fast here rather than silently degrading.
        boolean upsertEnabled =
                config.getOptional(CouchbaseSinkOptions.UPSERT_ENABLE).orElse(false);
        boolean hasPrimaryKey =
                config.getOptional(CouchbaseSinkOptions.PRIMARY_KEY)
                        .map(keys -> !keys.isEmpty())
                        .orElse(false);
        if (upsertEnabled && !hasPrimaryKey) {
            throw new CouchbaseConnectorException(
                    CouchbaseConnectorErrorCode.WRITE_RECORDS_FAILED,
                    "'upsert-enable' is true but 'primary-key' is not configured. "
                            + "Upsert requires a primary key to build a stable document key.");
        }

        CouchbaseWriterOptions.Builder builder =
                CouchbaseWriterOptions.builder()
                        .withConnectionString(config.get(CouchbaseSinkOptions.CONNECTION_STRING))
                        .withUsername(config.get(CouchbaseSinkOptions.USERNAME))
                        .withPassword(config.get(CouchbaseSinkOptions.PASSWORD))
                        .withBucket(config.get(CouchbaseSinkOptions.BUCKET))
                        .withScope(config.get(CouchbaseSinkOptions.SCOPE))
                        .withCollection(config.get(CouchbaseSinkOptions.COLLECTION));

        config.getOptional(CouchbaseSinkOptions.BUFFER_FLUSH_MAX_ROWS)
                .ifPresent(builder::withFlushSize);
        config.getOptional(CouchbaseSinkOptions.RETRY_MAX).ifPresent(builder::withRetryMax);
        config.getOptional(CouchbaseSinkOptions.RETRY_INTERVAL)

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Add primary-key = ["field"] to the sink config
  2. Set upsert-enable = false if append semantics are intended
  3. Ensure the configured primary key fields exist in the source schema

Example fix

// before
sink {
  Couchbase {
    upsert-enable = true
  }
}
// after
sink {
  Couchbase {
    upsert-enable = true
    primary-key = ["user_id"]
  }
}
Defensive patterns

Strategy: validation

Validate before calling

java
boolean upsert = config.getOptional(UPSERT_ENABLE).orElse(false);
boolean hasPk = config.getOptional(PRIMARY_KEY).map(k -> !k.isEmpty()).orElse(false);
if (upsert && !hasPk) throw new IllegalArgumentException("upsert-enable=true requires primary-key");

Try / catch

java
try {
    sink = factory.createSink(context);
} catch (CouchbaseConnectorException e) {
    // check upsert-enable / primary-key config before resubmitting
}

Prevention

When it happens

Trigger: Sink config with upsert-enable = true but no primary-key list; primary-key configured as an empty array; programmatic sink creation via CouchbaseSinkFactory.createSink with such options.

Common situations: Copying an upsert example but omitting primary-key; switching a plain append sink config to upsert-enable without adding a key.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/30823817e1417292. Report an issue: GitHub.