{"record":{"id":"9828add3c03c1632","repo":"apache/pulsar","slug":"when-effectively-once-processing-guarantee-is-spec","errorCode":null,"errorMessage":"When effectively once processing guarantee is specified, retain Key ordering cannot be set","messagePattern":"When effectively once processing guarantee is specified, retain Key ordering cannot be set","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java","lineNumber":882,"sourceCode":"                && functionConfig.getProcessingGuarantees() != null\n                && functionConfig.getProcessingGuarantees() != FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE) {\n            throw new IllegalArgumentException(\"Message timeout can only be specified with processing guarantee is \"\n                    + FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE.name());\n        }\n\n        if (functionConfig.getMaxMessageRetries() != null && functionConfig.getMaxMessageRetries() >= 0\n                && functionConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {\n            throw new IllegalArgumentException(\"MaxMessageRetries and Effectively once don't gel well\");\n        }\n        if ((functionConfig.getMaxMessageRetries() == null || functionConfig.getMaxMessageRetries() < 0)\n                && !org.apache.commons.lang3.StringUtils.isEmpty(functionConfig.getDeadLetterTopic())) {\n            throw new IllegalArgumentException(\"Dead Letter Topic specified, however max retries is set to infinity\");\n        }\n        if (functionConfig.getRetainKeyOrdering() != null\n                && functionConfig.getRetainKeyOrdering()\n                && functionConfig.getProcessingGuarantees() != null\n                && functionConfig.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {\n            throw new IllegalArgumentException(\n                    \"When effectively once processing guarantee is specified, retain Key ordering cannot be set\");\n        }\n        if (functionConfig.getRetainKeyOrdering() != null && functionConfig.getRetainKeyOrdering()\n                && functionConfig.getRetainOrdering() != null && functionConfig.getRetainOrdering()) {\n            throw new IllegalArgumentException(\"Only one of retain ordering or retain key ordering can be set\");\n        }\n\n        if (!isEmpty(functionConfig.getPy()) && !org.apache.pulsar.common.functions.Utils\n                .isFunctionPackageUrlSupported(functionConfig.getPy())\n                && functionConfig.getPy().startsWith(BUILTIN)) {\n            String filename = functionConfig.getPy();\n            if (filename.contains(\"..\")) {\n                throw new IllegalArgumentException(\"Invalid filename: \" + filename);\n            }\n\n            if (!new File(filename).exists()) {\n                throw new IllegalArgumentException(\"The supplied python file does not exist\");\n            }","sourceCodeStart":864,"sourceCodeEnd":900,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java#L864-L900","documentation":"Apache Pulsar Functions rejects a function configuration where retainKeyOrdering=true is combined with processingGuarantees=EFFECTIVELY_ONCE. Effectively-once delivery is implemented on top of deduplication semantics that cannot honor per-key ordering, so the combination is considered contradictory and validation fails fast. The check runs in FunctionConfigUtils.doCommonChecks, which is invoked by validateJavaFunction and validateNonJavaFunction before any function is created or updated.","triggerScenarios":"Calling createFunction/updateFunction (or PUT/POST to the functions admin API) with a FunctionConfig where setRetainKeyOrdering(true) and setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) are both set.","commonSituations":"Enabling effectively-once processing on an existing config that already had retainKeyOrdering=true; copying settings from docs about keyed ordering without realizing they conflict; a migration from at-least-once where key ordering was tuned and someone later switches guarantees to EFFECTIVELY_ONCE.","solutions":["Remove the retainKeyOrdering(true) call (or set it to false/null) from the FunctionConfig","Keep retainKeyOrdering and switch processingGuarantees to ATLEAST_ONCE or ATMOST_ONCE if key ordering is the requirement","If both ordering properties are desired, check whether retainOrdering (not retainKeyOrdering) is the intended flag for your use case"],"exampleFix":"// before\nconf.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);\nconf.setRetainKeyOrdering(true);\n// after\nconf.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);\nconf.setRetainKeyOrdering(false); // key ordering is incompatible with effectively-once","handlingStrategy":"validation","validationCode":"if (conf.getRetainKeyOrdering() != null && conf.getRetainKeyOrdering()\n        && conf.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE) {\n    conf.setRetainKeyOrdering(false); // or fail fast in your own pre-submit check\n}","typeGuard":"boolean isValidGuaranteeOrdering(FunctionConfig c) {\n    return !(Boolean.TRUE.equals(c.getRetainKeyOrdering())\n        && c.getProcessingGuarantees() == FunctionConfig.ProcessingGuarantees.EFFECTIVELY_ONCE);\n}","tryCatchPattern":"try {\n    pulsarAdmin.functions().createFunction(functionConfig);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().contains(\"retain Key ordering\")) {\n        functionConfig.setRetainKeyOrdering(false);\n        pulsarAdmin.functions().createFunction(functionConfig);\n    } else throw e;\n}","preventionTips":["Centralize FunctionConfig construction in one factory that enforces guarantee/ordering combinations","Only set retainKeyOrdering for ATLEAST_ONCE/ATMOST_ONCE pipelines","Add a unit test asserting each processingGuarantees value against your standard configs"],"tags":["pulsar-functions","config-validation","illegal-argument"],"backgroundTag":"incompatible-function-config","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}