apache/flink · error · IllegalArgumentException

Number of input splits has to be at least 1.

Error message

Number of input splits has to be at least 1.

What it means

Thrown by GenericInputFormat.createInputSplits(numSplits) when numSplits < 1. The framework calls createInputSplits with the job's configured parallelism to produce GenericInputSplit partitions; a non-positive value is meaningless and rejected. Note the check runs before the NonParallelInput special-case, so even parallelism-1 sources must pass a positive count.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/GenericInputFormat.java:53

    protected int partitionNumber;

    // --------------------------------------------------------------------------------------------

    @Override
    public void configure(Configuration parameters) {
        //	nothing by default
    }

    @Override
    public BaseStatistics getStatistics(BaseStatistics cachedStatistics) throws IOException {
        // no statistics available, by default.
        return cachedStatistics;
    }

    @Override
    public GenericInputSplit[] createInputSplits(int numSplits) throws IOException {
        if (numSplits < 1) {
            throw new IllegalArgumentException("Number of input splits has to be at least 1.");
        }

        numSplits = (this instanceof NonParallelInput) ? 1 : numSplits;
        GenericInputSplit[] splits = new GenericInputSplit[numSplits];
        for (int i = 0; i < splits.length; i++) {
            splits[i] = new GenericInputSplit(i, numSplits);
        }
        return splits;
    }

    @Override
    public DefaultInputSplitAssigner getInputSplitAssigner(GenericInputSplit[] splits) {
        return new DefaultInputSplitAssigner(splits);
    }

    // --------------------------------------------------------------------------------------------

    @Override

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Set a positive parallelism on the environment or the source operator (>= 1).
2. If deriving parallelism from configuration, validate it is >= 1 before job submission and fall back to a sane default like the available parallelism or 1.
3. For non-parallel sources, ensure the underlying type implements NonParallelInput so the special-case reduces to 1 split — but still pass numSplits >= 1.

Example fix

// before
env.setParallelism(0);
// after
env.setParallelism(1); // or omit to use default parallelism

int parallelism = env.getParallelism();
if (parallelism < 1) {
    throw new IllegalStateException("Parallelism must be >= 1, got " + parallelism);
}
// or when deriving from config
int p = Integer.parseInt(config.get("source.parallelism").orElse("1"));
if (p < 1) p = 1;

static int positiveParallelism(int p) {
    if (p < 1) throw new IllegalArgumentException("parallelism must be >= 1, got " + p);
    return p;
}

try {
    source.createInputSplits(parallelism);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("at least 1")) {
        env.setParallelism(1);
        return source.createInputSplits(1);
    }
    throw e;
}

When it happens

Trigger: The execution environment / source is configured with parallelism <= 0, or a custom source driver invokes createInputSplits(0) / createInputSplits(-1). It can also surface from an ExecutionConfig or env.setDefaultParallelism set to a non-positive value.

Common situations: env.setParallelism(0) by mistake; a source whose parallelism is derived from a config/variable that resolved to 0 or negative (e.g. an unset env var parsed as 0); programmatic submission with an explicit zero split count; tests that mock the input format and call createInputSplits(0).

Related errors

• Task id too large.
• Error: Invalid number of samples: ${numSamples}
• File path was not specified in input format or configuration
• Number of input splits has to be at least 1.
• Delimiter must not be null

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/467b4711319ccf28. Report an issue: GitHub.