apache/pulsar · error · IllegalArgumentException

When effectively once processing guarantee is specified, ret

Error message

When effectively once processing guarantee is specified, retain Key ordering cannot be set

What it means

Retain-key-ordering requires keyed (non-effectively-once) processing semantics; combining EFFECTIVELY_ONCE processing guarantees with retainKeyOrdering=true is rejected as semantically contradictory by the sink validation.

Source

Thrown at pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/SinkConfigUtils.java:562

                    ValidatorUtils.validateSchema(consumerSpec.getSchemaType(), typeArg,
                            inputFunction.getTypePool(), true);
                }
                if (consumerSpec.getCryptoConfig() != null) {
                    ValidatorUtils.validateCryptoKeyReader(consumerSpec.getCryptoConfig(),
                            inputFunction.getTypePool(), false);
                }
                if (consumerSpec.getMessagePayloadProcessorConfig() != null) {
                    ValidatorUtils.validateMessagePayloadProcessor(consumerSpec.getMessagePayloadProcessorConfig(),
                            inputFunction.getTypePool());
                }
            }
        }

        if (sinkConfig.getRetainKeyOrdering() != null
                && sinkConfig.getRetainKeyOrdering()
                && sinkConfig.getProcessingGuarantees() != null
                && sinkConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
            throw new IllegalArgumentException(
                    "When effectively once processing guarantee is specified, retain Key ordering cannot be set");
        }

        if (sinkConfig.getRetainKeyOrdering() != null && sinkConfig.getRetainKeyOrdering()
                && sinkConfig.getRetainOrdering() != null && sinkConfig.getRetainOrdering()) {
            throw new IllegalArgumentException("Only one of retain ordering or retain key ordering can be set");
        }

        // validate user defined config if enabled and classloading is enabled
        if (validateConnectorConfig) {
            if (sinkFunction.isEnableClassloading()) {
                validateSinkConfig(sinkConfig, sinkFunction);
            } else {
                log.warn("Skipping annotation based validation of sink config as classloading is disabled");
            }
        }

        return new ExtractedSinkDetails(sinkClassName, typeArg.asErasure().getTypeName(), functionClassName);

View on GitHub (pinned to 820761864e)

Solutions

1. Set retainKeyOrdering to false/null when using EFFECTIVELY_ONCE
2. Or use ATLEAST_ONCE processing guarantees if key ordering retention is required

Example fix

// before
sinkConfig.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);
sinkConfig.setRetainKeyOrdering(true);
// after
sinkConfig.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);
sinkConfig.setRetainKeyOrdering(false);

if (Boolean.TRUE.equals(config.getRetainKeyOrdering())
    && config.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
  throw new IllegalStateException("retainKeyOrdering incompatible with EFFECTIVELY_ONCE");
}

try {
  admin.sinks().createSink(config, archive);
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("retain Key ordering cannot be set")) {
    config.setRetainKeyOrdering(false);
  } else { throw e; }
}

When it happens

Trigger: sinkConfig.retainKeyOrdering == true AND processingGuarantees == EFFECTIVELY_ONCE at sink create time.

Common situations: Copying a function config tuned for at-least-once ordered delivery into an exactly-once sink; toggling processingGuarantees to EFFECTIVELY_ONCE without clearing retainKeyOrdering.

Related errors

• Only one of serdeClassName or schemaType should be set
• Only one of retain ordering or retain key ordering can be se
• Sink class %s does not implement the correct interface
• Function config is not provided
• Output topic %s is also being used as an input topic (topics

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/e24795496a2ac091. Report an issue: GitHub.