apache/pulsar · error · IllegalArgumentException

When effectively once processing guarantee is specified, ret

Error message

When effectively once processing guarantee is specified, retain Key ordering cannot be set

What it means

Apache Pulsar Functions rejects a function configuration where retainKeyOrdering=true is combined with processingGuarantees=EFFECTIVELY_ONCE. Effectively-once delivery is implemented on top of deduplication semantics that cannot honor per-key ordering, so the combination is considered contradictory and validation fails fast. The check runs in FunctionConfigUtils.doCommonChecks, which is invoked by validateJavaFunction and validateNonJavaFunction before any function is created or updated.

Source

Thrown at pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java:882

                && functionConfig.getProcessingGuarantees() != null
                && functionConfig.getProcessingGuarantees() != FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE) {
            throw new IllegalArgumentException("Message timeout can only be specified with processing guarantee is "
                    + FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE.name());
        }

        if (functionConfig.getMaxMessageRetries() != null && functionConfig.getMaxMessageRetries() >= 0
                && functionConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
            throw new IllegalArgumentException("MaxMessageRetries and Effectively once don't gel well");
        }
        if ((functionConfig.getMaxMessageRetries() == null || functionConfig.getMaxMessageRetries() < 0)
                && !org.apache.commons.lang3.StringUtils.isEmpty(functionConfig.getDeadLetterTopic())) {
            throw new IllegalArgumentException("Dead Letter Topic specified, however max retries is set to infinity");
        }
        if (functionConfig.getRetainKeyOrdering() != null
                && functionConfig.getRetainKeyOrdering()
                && functionConfig.getProcessingGuarantees() != null
                && functionConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
            throw new IllegalArgumentException(
                    "When effectively once processing guarantee is specified, retain Key ordering cannot be set");
        }
        if (functionConfig.getRetainKeyOrdering() != null && functionConfig.getRetainKeyOrdering()
                && functionConfig.getRetainOrdering() != null && functionConfig.getRetainOrdering()) {
            throw new IllegalArgumentException("Only one of retain ordering or retain key ordering can be set");
        }

        if (!isEmpty(functionConfig.getPy()) && !org.apache.pulsar.common.functions.Utils
                .isFunctionPackageUrlSupported(functionConfig.getPy())
                && functionConfig.getPy().startsWith(BUILTIN)) {
            String filename = functionConfig.getPy();
            if (filename.contains("..")) {
                throw new IllegalArgumentException("Invalid filename: " + filename);
            }

            if (!new File(filename).exists()) {
                throw new IllegalArgumentException("The supplied python file does not exist");
            }

View on GitHub (pinned to 820761864e)

Solutions

1. Remove the retainKeyOrdering(true) call (or set it to false/null) from the FunctionConfig
2. Keep retainKeyOrdering and switch processingGuarantees to ATLEAST_ONCE or ATMOST_ONCE if key ordering is the requirement
3. If both ordering properties are desired, check whether retainOrdering (not retainKeyOrdering) is the intended flag for your use case

Example fix

// before
conf.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);
conf.setRetainKeyOrdering(true);
// after
conf.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);
conf.setRetainKeyOrdering(false); // key ordering is incompatible with effectively-once

if (conf.getRetainKeyOrdering() != null && conf.getRetainKeyOrdering()
        && conf.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
    conf.setRetainKeyOrdering(false); // or fail fast in your own pre-submit check
}

boolean isValidGuaranteeOrdering(FunctionConfig c) {
    return !(Boolean.TRUE.equals(c.getRetainKeyOrdering())
        && c.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);
}

try {
    pulsarAdmin.functions().createFunction(functionConfig);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("retain Key ordering")) {
        functionConfig.setRetainKeyOrdering(false);
        pulsarAdmin.functions().createFunction(functionConfig);
    } else throw e;
}

When it happens

Trigger: Calling createFunction/updateFunction (or PUT/POST to the functions admin API) with a FunctionConfig where setRetainKeyOrdering(true) and setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) are both set.

Common situations: Enabling effectively-once processing on an existing config that already had retainKeyOrdering=true; copying settings from docs about keyed ordering without realizing they conflict; a migration from at-least-once where key ordering was tuned and someone later switches guarantees to EFFECTIVELY_ONCE.

Related errors

• Only one of retain ordering or retain key ordering can be se
• Receiver queue size should be >= zero
• Source tenant cannot be null
• Source namespace cannot be null
• Source name cannot be null

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/9828add3c03c1632. Report an issue: GitHub.