apache/pulsar · error · IllegalArgumentException

MaxMessageRetries and Effectively once don't gel well

Error message

MaxMessageRetries and Effectively once don't gel well

What it means

maxMessageRetries with a dead-letter topic implements retry-based redelivery, which is incompatible with EFFECTIVELY_ONCE guarantees (retries would violate exactly-once semantics). doCommonChecks throws this IllegalArgumentException when maxMessageRetries >= 0 is combined with ProcessingGuarantees.EFFECTIVELY_ONCE.

Source

Thrown at pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java:872

        if (functionConfig.getResources() != null) {
            ResourceConfigUtils.validate(functionConfig.getResources());
        }

        if (functionConfig.getTimeoutMs() != null && functionConfig.getTimeoutMs() <= 0) {
            throw new IllegalArgumentException("Function timeout must be a positive number");
        }

        if (functionConfig.getTimeoutMs() != null
                && functionConfig.getProcessingGuarantees() != null
                && functionConfig.getProcessingGuarantees() != FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE) {
            throw new IllegalArgumentException("Message timeout can only be specified with processing guarantee is "
                    + FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE.name());
        }

        if (functionConfig.getMaxMessageRetries() != null && functionConfig.getMaxMessageRetries() >= 0
                && functionConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
            throw new IllegalArgumentException("MaxMessageRetries and Effectively once don't gel well");
        }
        if ((functionConfig.getMaxMessageRetries() == null || functionConfig.getMaxMessageRetries() < 0)
                && !org.apache.commons.lang3.StringUtils.isEmpty(functionConfig.getDeadLetterTopic())) {
            throw new IllegalArgumentException("Dead Letter Topic specified, however max retries is set to infinity");
        }
        if (functionConfig.getRetainKeyOrdering() != null
                && functionConfig.getRetainKeyOrdering()
                && functionConfig.getProcessingGuarantees() != null
                && functionConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
            throw new IllegalArgumentException(
                    "When effectively once processing guarantee is specified, retain Key ordering cannot be set");
        }
        if (functionConfig.getRetainKeyOrdering() != null && functionConfig.getRetainKeyOrdering()
                && functionConfig.getRetainOrdering() != null && functionConfig.getRetainOrdering()) {
            throw new IllegalArgumentException("Only one of retain ordering or retain key ordering can be set");
        }

        if (!isEmpty(functionConfig.getPy()) && !org.apache.pulsar.common.functions.Utils

View on GitHub (pinned to 820761864e)

Solutions

Drop maxMessageRetries (set to null) when using EFFECTIVELY_ONCE
Or switch processingGuarantees to ATLEAST_ONCE if retry semantics are required
Parameterize retry settings per guarantee mode in config generation code

Example fix

// before
config.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);
config.setMaxMessageRetries(5);
// after
config.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE);
config.setMaxMessageRetries(5);

Defensive patterns

Strategy: validation

Validate before calling

if (config.getMaxMessageRetries() != null && config.getMaxMessageRetries() >= 0
        && config.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {
    throw new IllegalArgumentException("retries are incompatible with EFFECTIVELY_ONCE");
}

Type guard

boolean retriesGuaranteeCompatible(FunctionConfig c) {
    return c.getProcessingGuarantees() != FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE
        || c.getMaxMessageRetries() == null || c.getMaxMessageRetries() < 0;
}

Try / catch

try {
    admin.functions().createFunction(functionConfig, sourceConfigLocation);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("MaxMessageRetries")) {
        config.setMaxMessageRetries(null); // or drop EFFECTIVELY_ONCE
    } else { throw e; }
}

Prevention

Keep retry settings only on ATLEAST_ONCE functions
Template separate config profiles per guarantee mode
Validate retry/guarantee pairing in CI before deploying
Avoid global 'enable retries everywhere' flags

When it happens

Trigger: createFunction/updateFunction where functionConfig.getMaxMessageRetries() != null && >= 0 and getProcessingGuarantees() == EFFECTIVELY_ONCE — e.g. adding --max-message-retries 3 to an effectively-once function.

Common situations: Reusing a config template between at-least-once and exactly-once functions; enabling retry features globally in generated manifests without checking the guarantee mode.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/e23afe765031bfd6. Report an issue: GitHub.