apache/flink · error · FlinkRuntimeException

A user-provided generator function threw an exception on thi

Error message

A user-provided generator function threw an exception on this input: %s

What it means

Thrown by convert() in GeneratingIteratorSourceReader when generatorFunction.map(value) throws any exception. The message uses String.format to include the toString() of the specific input value that caused the failure, and the original exception is wrapped in FlinkRuntimeException. This is the production-oriented iterator source reader.

Source

Thrown at flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/GeneratingIteratorSourceReader.java:60

    public GeneratingIteratorSourceReader(
            SourceReaderContext context, GeneratorFunction<E, O> generatorFunction) {
        super(context);
        this.generatorFunction = checkNotNull(generatorFunction);
    }

    // ------------------------------------------------------------------------

    @Override
    protected O convert(E value) {
        try {
            return generatorFunction.map(value);
        } catch (Exception e) {
            String message =
                    String.format(
                            "A user-provided generator function threw an exception on this input: %s",
                            value.toString());
            throw new FlinkRuntimeException(message, e);
        }
    }

    @Override
    public void start(SourceReaderContext context) {
        try {
            generatorFunction.open(context);
        } catch (Exception e) {
            throw new FlinkRuntimeException("Failed to open the GeneratorFunction", e);
        }
    }

    @Override
    public void close() throws Exception {
        generatorFunction.close();
        super.close();
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

Inspect the input value shown in the error message to identify the problematic data.
Add input validation and null-checks in the GeneratorFunction.map() method.
Add try-catch in map() for known edge cases, returning a default or using a side output.
Fix the root cause in the mapping logic for the failing input.

Example fix

// before: map() throws on null input
GeneratorFunction<Integer, String> fn = value -> value.toString().substring(2);

// after: validate input before processing
GeneratorFunction<Integer, String> fn = value -> {
    if (value == null || value < 100) return "N/A";
    return value.toString().substring(2);
};

Defensive patterns

Strategy: try-catch

Validate before calling

// Validate map() handles all known iterator inputs
GeneratorFunction<E, O> fn = ...;
for (E testInput : sampleInputs) {
    fn.map(testInput); // should not throw for valid inputs
}

Try / catch

try {
    sourceReader.pollNext(output);
} catch (FlinkRuntimeException e) {
    if (e.getMessage().startsWith("A user-provided generator function threw")) {
        // extract the failing input value from the message
        // add validation in map() for this input type
    }
    throw e;
}

Prevention

Add input validation and null-checks in GeneratorFunction.map().
Unit-test map() with representative and edge-case inputs.
Use try-catch inside map() for non-critical edge cases.

When it happens

Trigger: The user-provided GeneratorFunction's map() method throws on a specific input value from the iterator split. Causes include NPE, type cast failures, business logic exceptions, or unhandled edge cases.

Common situations: Generator function that does not handle null or unexpected fields; lookup function index out of bounds; arithmetic error on specific input; type mismatch between the iterator element type and what map() expects.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/bf89d170c8351b7c. Report an issue: GitHub.