apache/flink · error · IllegalArgumentException

The collection contains a null element

Error message

The collection contains a null element

What it means

Thrown by the private checkIterable method in IndexLookupGeneratorFunction when any element in the provided collection is null. This check runs in the constructor and also increments numElements per element, so it doubles as the element counter. A null element aborts construction immediately.

Source

Thrown at flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/functions/IndexLookupGeneratorFunction.java:116

    }

    @Override
    public OUT map(Long index) throws Exception {
        return lookupMap.get(index);
    }

    /**
     * Verifies that all elements in the iterable are non-null, and are of the given class, or a
     * subclass thereof.
     *
     * @param elements The iterable to check.
     * @param viewedAs The class to which the elements must be assignable to.
     */
    private void checkIterable(Iterable<OUT> elements, Class<?> viewedAs) {
        for (OUT elem : elements) {
            numElements++;
            if (elem == null) {
                throw new IllegalArgumentException("The collection contains a null element");
            }

            if (!viewedAs.isAssignableFrom(elem.getClass())) {
                throw new IllegalArgumentException(
                        "The elements in the collection are not all subclasses of "
                                + viewedAs.getCanonicalName());
            }
        }
    }

    private void serializeElements(Iterable<OUT> elements) throws IOException {
        Preconditions.checkState(serializer != null, "serializer not set");
        LOG.info("Serializing elements using  {}", serializer);
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        DataOutputViewStreamWrapper wrapper = new DataOutputViewStreamWrapper(baos);

        try {
            for (OUT element : elements) {

View on GitHub (pinned to 2f3c205e92)

Solutions

  1. Filter null elements from the collection before constructing the function.
  2. If nulls represent valid data, wrap them in an Optional or a sentinel object.
  3. Validate the collection upstream before passing it to the constructor.

Example fix

// before
List<String> elements = Arrays.asList("x", null, "y");
new IndexLookupGeneratorFunction<>(Types.STRING, elements);

// after
List<String> elements = Stream.of("x", null, "y")
    .filter(Objects::nonNull).collect(Collectors.toList());
new IndexLookupGeneratorFunction<>(Types.STRING, elements);
Defensive patterns

Strategy: validation

Validate before calling

// Filter nulls before constructing IndexLookupGeneratorFunction
List<OUT> cleanElements = StreamSupport.stream(elements.spliterator(), false)
    .filter(Objects::nonNull)
    .collect(Collectors.toList());
new IndexLookupGeneratorFunction<>(typeInfo, cleanElements);

Try / catch

try {
    new IndexLookupGeneratorFunction<>(typeInfo, elements);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("null element")) {
        elements = StreamSupport.stream(elements.spliterator(), false)
            .filter(Objects::nonNull).collect(Collectors.toList());
        new IndexLookupGeneratorFunction<>(typeInfo, elements);
    } else throw e;
}

Prevention

When it happens

Trigger: Constructing new IndexLookupGeneratorFunction<>(typeInfo, elements) with an Iterable containing one or more null values.

Common situations: Programmatically building an element list where computation produced null; reading elements from an external source that may emit nulls; passing a collection built from nullable data without filtering.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/072a8dc981a6ee87. Report an issue: GitHub.