apache/flink · error · FlinkRuntimeException

Failed to open the GeneratorFunction

Error message

Failed to open the GeneratorFunction

What it means

Thrown by start() in DoubleEmittingSourceReaderWithCheckpointsInBetween when the user-provided GeneratorFunction.open(context) throws an exception. The original exception is wrapped in FlinkRuntimeException. This source reader is @Experimental and designed for testing purposes (checkpoint-synchronized double emission).

Source

Thrown at flink-connectors/flink-connector-datagen/src/main/java/org/apache/flink/connector/datagen/source/DoubleEmittingSourceReaderWithCheckpointsInBetween.java:83

        super(context);
        this.generatorFunction = checkNotNull(generatorFunction);
        this.allowedToExit = allowedToExit;
    }

    public DoubleEmittingSourceReaderWithCheckpointsInBetween(
            SourceReaderContext context, GeneratorFunction<E, O> generatorFunction) {
        super(context);
        this.generatorFunction = checkNotNull(generatorFunction);
    }

    // ------------------------------------------------------------------------

    @Override
    public void start(SourceReaderContext context) {
        try {
            generatorFunction.open(context);
        } catch (Exception e) {
            throw new FlinkRuntimeException("Failed to open the GeneratorFunction", e);
        }
    }

    @Override
    public InputStatus pollNext(ReaderOutput<O> output) {
        // This is the termination path after the test data has been emitted twice
        if (done) {
            if (allowedToExit != null) { // Termination is controlled externally
                return allowedToExit.getAsBoolean()
                        ? InputStatus.END_OF_INPUT
                        : InputStatus.NOTHING_AVAILABLE;
            } else {
                return InputStatus.END_OF_INPUT;
            }
        }
        // This is the initial path
        if (currentSplit == null) {
            InputStatus inputStatus = tryMoveToNextSplit();

View on GitHub (pinned to 2f3c205e92)

Solutions

  1. Inspect the wrapped cause exception in the FlinkRuntimeException for the root failure.
  2. Fix the root cause in the GeneratorFunction's open() method (e.g., add connection retry, validate configuration).
  3. Ensure the SourceReaderContext is properly configured and accessible.
  4. If the open() method needs external resources, verify they are available before job submission.
Defensive patterns

Strategy: try-catch

Validate before calling

// Validate open() does not throw by testing the GeneratorFunction separately
GeneratorFunction<E, O> fn = ...;
fn.open(testContext); // mock or real SourceReaderContext
fn.close();

Try / catch

try {
    sourceReader.start(context);
} catch (FlinkRuntimeException e) {
    if (e.getMessage().equals("Failed to open the GeneratorFunction")) {
        // inspect e.getCause() for the root failure in open()
    }
    throw e;
}

Prevention

When it happens

Trigger: A custom GeneratorFunction whose open() method throws — e.g., failing to initialize resources, connecting to an unreachable service, or throwing a runtime exception during setup. The source reader calls generatorFunction.open(context) during its start() lifecycle.

Common situations: Generator function trying to open a database or network connection in open() that fails; misconfigured SourceReaderContext; resource initialization failure (file handle, thread pool).

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/bbfcc668c8c3e99a. Report an issue: GitHub.