apache/flink · error · IllegalArgumentException

TaskNumber: {}, numTasks: {}

Error message

TaskNumber: {}, numTasks: {}

What it means

Thrown by FileOutputFormat.open(InitializationContext) when the context reports an invalid parallel execution layout: taskNumber < 0 or numTasks < 1. These values come from the runtime's InitializationContext and describe this operator's position within the parallel subtask set; they must be internally consistent.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/FileOutputFormat.java:188

        }

        // check if have not been set and use the defaults in that case
        if (this.writeMode == null) {
            this.writeMode = DEFAULT_WRITE_MODE;
        }

        if (this.outputDirectoryMode == null) {
            this.outputDirectoryMode = DEFAULT_OUTPUT_DIRECTORY_MODE;
        }
    }

    @Override
    public void open(InitializationContext context) throws IOException {
        int numTasks = context.getNumTasks();
        int taskNumber = context.getTaskNumber();

        if (taskNumber < 0 || numTasks < 1) {
            throw new IllegalArgumentException(
                    "TaskNumber: " + taskNumber + ", numTasks: " + numTasks);
        }

        if (LOG.isDebugEnabled()) {
            LOG.debug(
                    "Opening stream for output ("
                            + (taskNumber + 1)
                            + "/"
                            + numTasks
                            + "). WriteMode="
                            + writeMode
                            + ", OutputDirectoryMode="
                            + outputDirectoryMode);
        }

        Path p = this.outputFilePath;
        if (p == null) {
            throw new IOException("The file path is null.");

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the sink runs with parallelism >= 1 (env.setParallelism(n>=1)).
2. In tests, build the InitializationContext with valid taskNumber in [0, numTasks) and numTasks >= 1.
3. If using a custom ExecutionEnvironment, verify it emits a correct subtask context.

Example fix

// before (test harness)
InitializationContext ctx = new InitializationContext() {
  public int getTaskNumber() { return 0; }
  public int getNumTasks() { return 0; } // invalid
};
format.open(ctx);

// after
InitializationContext ctx = ...getTaskNumber()==0, getNumTasks()==1;
format.open(ctx);

// Ensure parallelism >= 1 for the sink
int p = env.getParallelism();
if (p < 1) {
    throw new IllegalStateException("Sink parallelism must be >= 1, got " + p);
}

When it happens

Trigger: The InitializationContext supplied to open has a negative taskNumber or a numTasks of 0; a custom ExecutionEnvironment or test harness that builds an InitializationContext with bad values; a framework bug or edge case where parallelism resolves to 0 for the sink.

Common situations: Unit tests that construct an InitializationContext manually and pass 0 for numTasks; a misconfigured local execution with parallelism 0; embedding Flink in a custom runtime that fabricates the context.

Related errors

• Currently, filesystem sink doesn't support setting paralleli
• The file path is null.
• Table options do not contain an option key '%s' for discover
• Invalid time zone for '%s'. The value should be a Time Zone
• Could not find any format factory for identifier '%s' in the

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/e0d8fd1950c48786. Report an issue: GitHub.