apache/flink · error · IllegalArgumentException

The position {} occurs multiple times.

Error message

The position {} occurs multiple times.

What it means

Thrown by checkAndCoSort when the same column position appears more than once in the positions array. The method co-sorts positions into a TreeMap keyed by position, so duplicates are ambiguous and rejected. This is a programming-error guard.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/GenericCsvInputFormat.java:546

    protected static void checkAndCoSort(int[] positions, Class<?>[] types) {
        if (positions.length != types.length) {
            throw new IllegalArgumentException(
                    "The positions and types must be of the same length");
        }

        TreeMap<Integer, Class<?>> map = new TreeMap<Integer, Class<?>>();

        for (int i = 0; i < positions.length; i++) {
            if (positions[i] < 0) {
                throw new IllegalArgumentException(
                        "The field " + " (" + positions[i] + ") is invalid.");
            }
            if (types[i] == null) {
                throw new IllegalArgumentException("The type " + i + " is invalid (null)");
            }

            if (map.containsKey(positions[i])) {
                throw new IllegalArgumentException(
                        "The position " + positions[i] + " occurs multiple times.");
            }

            map.put(positions[i], types[i]);
        }

        int i = 0;
        for (Map.Entry<Integer, Class<?>> entry : map.entrySet()) {
            positions[i] = entry.getKey();
            types[i] = entry.getValue();
            i++;
        }
    }

    protected static void checkForMonotonousOrder(int[] positions, Class<?>[] types) {
        if (positions.length != types.length) {
            throw new IllegalArgumentException(
                    "The positions and types must be of the same length");

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Deduplicate the positions array before calling checkAndCoSort (if reading the same column twice is intended, handle it outside co-sort).
2. Validate positions are unique with a Set before the call.
3. Build positions from a single authoritative list and assert no duplicates.

Example fix

// before
checkAndCoSort(new int[]{0,2,2,4}, types);
// after
checkAndCoSort(new int[]{0,2,4}, types);
// if you really need column 2 twice, read it once and project in a downstream map

if (new HashSet<Integer>(intsToList(positions)).size() != positions.length) {
    throw new IllegalArgumentException("Duplicate positions in " + Arrays.toString(positions));
}
checkAndCoSort(positions, types);

// Deduplicate explicitly when reading the same column twice is not intended
int[] unique = Arrays.stream(positions).distinct().toArray();

try {
    checkAndCoSort(positions, types);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("occurs multiple times")) {
        throw new IllegalArgumentException("Duplicate CSV positions: " + Arrays.toString(positions), e);
    }
    throw e;
}

When it happens

Trigger: A subclass passes positions containing a repeated value, e.g. {0, 2, 2, 4}, meaning the same CSV column is mapped to two different output fields in a co-sort context (which is not meaningful for that operation).

Common situations: Copy-paste duplication when adding a column; index list built from two sources without dedup; refactor that merged projections; off-by-one producing an accidental repeat.

Related errors

• The positions and types must be of the same length
• The field ({} is invalid.
• The type {} is invalid (null)
• The field ({}) is invalid.
• The positions must be strictly increasing (no permutations a

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/3bce9278c17e278b. Report an issue: GitHub.