apache/flink · error · InvalidProgramException

The partitioner is incompatible with the key type. Partition

Error message

The partitioner is incompatible with the key type. Partitioner type: {typeInfo} , key type: {keyType}

What it means

Thrown by SelectorFunctionKeys.validateCustomPartitioner() when the custom Partitioner's type argument differs from the key type. The partitioner receives the key value, so Partitioner<E> must be parameterised by the same E as the key (e.g. Partitioner<String> for a String key); a mismatch would cause ClassCastException at runtime.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/operators/Keys.java:186

            if (keyFields.size() != 1) {
                throw new InvalidProgramException(
                        "Custom partitioners can only be used with keys that have one key field.");
            }

            if (typeInfo == null) {
                // try to extract key type from partitioner
                try {
                    typeInfo = TypeExtractor.getPartitionerTypes(partitioner);
                } catch (Throwable t) {
                    // best effort check, so we ignore exceptions
                }
            }

            // only check if type is known and not a generic type
            if (typeInfo != null && !(typeInfo instanceof GenericTypeInfo)) {
                // check equality of key and partitioner type
                if (!keyType.equals(typeInfo)) {
                    throw new InvalidProgramException(
                            "The partitioner is incompatible with the key type. "
                                    + "Partitioner type: "
                                    + typeInfo
                                    + " , key type: "
                                    + keyType);
                }
            }
        }

        @Override
        public String toString() {
            return "Key function (Type: " + keyType + ")";
        }
    }

    /** Represents (nested) field access through string and integer-based keys */
    public static class ExpressionKeys<T> extends Keys<T> {

View on GitHub (pinned to 2f3c205e92)

Solutions

Implement Partitioner<E> with E exactly matching the key's TypeInformation.
If the key type changed, write a new partitioner for the new type.
For GenericTypeInfo keys the check is skipped (best-effort), but prefer a concrete key type so the mismatch is caught early.

Example fix

// before (key is String, partitioner is Partitioner<Integer>)
Partitioner<Integer> p = new MyIntPartitioner();
ds.partitionCustom(p, stringSel);
// after
ds.partitionCustom(new MyStringPartitioner(), stringSel);

Defensive patterns

Strategy: validation

Validate before calling

TypeInformation<?> partitionerType = TypeExtractor.getPartitionerTypes(partitioner);
if (partitionerType != null && !partitionerType.equals(keyType)) {
    throw new IllegalArgumentException(
        "Partitioner " + partitionerType + " incompatible with key " + keyType);
}

Type guard

static <E> boolean partitionerMatchesType(Partitioner<E> p, TypeInformation<E> keyType) {
    try {
        TypeInformation<?> pt = TypeExtractor.getPartitionerTypes(p);
        return pt == null || pt.equals(keyType);
    } catch (Throwable t) { return true; }
}

Prevention

Implement Partitioner<E> with E exactly equal to the key TypeInformation.
When the key type changes, write a new partitioner rather than reusing the old one.
Prefer concrete (non-generic) key types so the mismatch is caught at validation, not runtime.

When it happens

Trigger: Calling partitionCustom(new MyIntPartitioner(), stringKeySelector) where the partitioner implements Partitioner<Integer> but the key is a String; attaching a Partitioner<Long> to a key whose TypeInformation is BasicTypeInfo.INT_TYPE_INFO.

Common situations: Reusing a partitioner written for one key type against a different key; type-erasure hiding the mismatch until validation; refactoring a key type (e.g. Long to String) without updating the partitioner.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d5d3b044621f3dd9. Report an issue: GitHub.