apache/flink · error · FlinkRuntimeException

Cannot have more than {streamingJobCountLimit} streaming job

Error message

Cannot have more than {streamingJobCountLimit} streaming jobs in a single environment.

What it means

Thrown by StreamContextEnvironment when the number of streaming jobs submitted in a single environment exceeds the configured streamingJobCountLimit. This limit prevents unbounded job submission in application mode. The limit is passed into setAsContext and enforced per-environment via a counter incremented on each streaming executeAsync.

Source

Thrown at flink-clients/src/main/java/org/apache/flink/client/program/StreamContextEnvironment.java:280

        }
        jobIdManager.updateJobId(streamGraph);
        final JobClient jobClient = super.executeAsync(streamGraph);

        if (!suppressSysout) {
            System.out.println("Job has been submitted with JobID " + jobClient.getJobID());
        }

        return jobClient;
    }

    private void validateAllowedExecution(StreamGraph streamGraph) {
        if (streamGraph.getJobType() == JobType.STREAMING) {
            streamingJobCount++;
        }
        jobCount++;

        if (streamingJobCount > streamingJobCountLimit) {
            throw new FlinkRuntimeException(
                    "Cannot have more than "
                            + streamingJobCountLimit
                            + " streaming jobs in a single environment.");
        }
        if (jobCount > jobCountLimit) {
            throw new FlinkRuntimeException(
                    "Cannot have more than " + jobCountLimit + " jobs in a single environment.");
        }
    }

    // --------------------------------------------------------------------------------------------

    public static void setAsContext(
            final PipelineExecutorServiceLoader executorServiceLoader,
            final Configuration clusterConfiguration,
            final ClassLoader userCodeClassLoader,
            final int jobCountLimit,
            final int streamingJobCountLimit,

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Increase the streaming job count limit via the relevant deployment configuration option.
2. Refactor to submit fewer streaming jobs per environment, or use separate environments.
3. If submitting multiple jobs is intended, use application mode with a configured limit (e.g., execution.attached or job-count settings).
4. Consolidate multiple DataStream pipelines into a single job using unions or side outputs.

Example fix

// before: submitting many streaming jobs in a loop
for (Source source : sources) {
    env.fromSource(source, ...).print();
    env.execute(); // exceeds limit on 2nd iteration
}

// after: single job with unioned sources
DataStream<?> combined = env.union(sources.stream()
    .map(s -> env.fromSource(s, ...))
    .collect(Collectors.toList()));
combined.print();
env.execute();

// before calling executeAsync, track streaming job count
int streamingJobsPlanned = countStreamingPipelines();
if (streamingJobsPlanned > configuredStreamingLimit) {
    throw new IllegalStateException(
        "Planned streaming jobs (" + streamingJobsPlanned
        + ") exceed limit (" + configuredStreamingLimit + ")");
}

try {
    env.executeAsync(streamGraph);
} catch (FlinkRuntimeException e) {
    if (e.getMessage().contains("streaming jobs in a single environment")) {
        // increase limit or reduce job count
    }
    throw e;
}

When it happens

Trigger: Submitting more streaming jobs than the allowed limit within one StreamExecutionEnvironment in application mode. Each call to execute/executeAsync that produces a STREAMING job graph increments the counter; exceeding the limit throws FlinkRuntimeException.

Common situations: Application mode with a loop that submits many streaming jobs, or a program that calls execute() multiple times without using separate environments. The default limit is typically 1 for application mode unless configured higher.

Related errors

• Cannot have more than {jobCountLimit} jobs in a single envir
• Application Mode not supported by standalone deployments.
• Could not create the Dispatcher rpc endpoint.
• Invalid cluster id "%s". The expected format is [0-9a-fA-F]{
• Could not execute application.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/3bd8459366d86cb0. Report an issue: GitHub.