{"record":{"id":"75ab778ccfbc3e0a","repo":"apache/flink","slug":"cannot-use-custom-partitioner-for-a-non-grouped-gr","errorCode":null,"errorMessage":"Cannot use custom partitioner for a non-grouped GroupReduce (AllGroupReduce)","messagePattern":"Cannot use custom partitioner for a non-grouped GroupReduce \\(AllGroupReduce\\)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/common/operators/base/GroupReduceOperatorBase.java","lineNumber":161,"sourceCode":"            this.combinable = combinable;\n        }\n    }\n\n    /**\n     * Checks whether the operation is combinable.\n     *\n     * @return True, if the UDF is combinable, false if not.\n     * @see #setCombinable(boolean)\n     */\n    public boolean isCombinable() {\n        return this.combinable;\n    }\n\n    public void setCustomPartitioner(Partitioner<?> customPartitioner) {\n        if (customPartitioner != null) {\n            int[] keys = getKeyColumns(0);\n            if (keys == null || keys.length == 0) {\n                throw new IllegalArgumentException(\n                        \"Cannot use custom partitioner for a non-grouped GroupReduce (AllGroupReduce)\");\n            }\n            if (keys.length > 1) {\n                throw new IllegalArgumentException(\n                        \"Cannot use the key partitioner for composite keys (more than one key field)\");\n            }\n        }\n        this.customPartitioner = customPartitioner;\n    }\n\n    public Partitioner<?> getCustomPartitioner() {\n        return customPartitioner;\n    }\n\n    private TypeComparator<IN> getTypeComparator(\n            TypeInformation<IN> typeInfo,\n            int[] sortColumns,\n            boolean[] sortOrderings,","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/operators/base/GroupReduceOperatorBase.java#L143-L179","documentation":"Thrown by GroupReduceOperatorBase.setCustomPartitioner(Partitioner<?>) when a custom partitioner is set on a GroupReduce operation that has no grouping keys (an 'all-group reduce' / AllGroupReduce). A custom partitioner routes records by key to specific partitions; without a group key, there is no key to partition by, so the partitioner is meaningless.","triggerScenarios":"Calling reduceGroup without specifying .groupBy(...) (making it an all-reduce), then calling setCustomPartitioner(partitioner). Attempting to set a partitioner on a non-keyed group reduce.","commonSituations":"Forgetting to call groupBy() before reduceGroup(). Building a reduce operation programmatically and setting a partitioner before configuring keys. Misunderstanding that all-group-reduce sends all data to one reducer regardless of partitioning.","solutions":["Call .groupBy(keyFields) or .groupBy(keySelector) on the DataSet before the reduce operation, so the group reduce is keyed.","If you truly want an all-group-reduce, do not set a custom partitioner.","Ensure keys are configured on the GroupReduceOperatorBase (getKeyColumns(0) returns a non-empty array) before calling setCustomPartitioner."],"exampleFix":"// before\nDataSet<MyType> result = dataSet.reduceGroup(myReducer); // no groupBy -> AllGroupReduce\n((GroupReduceOperatorBase) op).setCustomPartitioner(myPartitioner); // throws\n\n// after\nDataSet<MyType> result = dataSet.groupBy(\"keyField\").reduceGroup(myReducer); // keyed reduce\n((GroupReduceOperatorBase) op).setCustomPartitioner(myPartitioner); // ok","handlingStrategy":"validation","validationCode":"void safeSetCustomPartitioner(GroupReduceOperatorBase<?, ?, ?> op, Partitioner<?> partitioner) {\n    if (partitioner == null) { op.setCustomPartitioner(null); return; }\n    int[] keys = op.getKeyColumns(0);\n    if (keys == null || keys.length == 0) {\n        throw new IllegalArgumentException(\"Cannot use custom partitioner on a non-keyed GroupReduce\");\n    }\n    op.setCustomPartitioner(partitioner);\n}","typeGuard":"boolean isKeyedGroupReduce(GroupReduceOperatorBase<?, ?, ?> op) {\n    int[] keys = op.getKeyColumns(0);\n    return keys != null && keys.length > 0;\n}","tryCatchPattern":"try {\n    reduceOp.setCustomPartitioner(partitioner);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().contains(\"non-grouped\")) {\n        // add a groupBy before the reduce\n        log.error(\"Must call groupBy() before setting custom partitioner\");\n    }\n}","preventionTips":["Always call groupBy() before reduceGroup if you intend to use a custom partitioner.","Check getKeyColumns(0) returns non-empty before setting a partitioner.","Do not use setCustomPartitioner on all-group-reduce (AllGroupReduce) operations."],"tags":["group-reduce","custom-partitioner","keyed-operation","dataset-api","flink-core"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}