apache/flink · error · UnsupportedOperationException

DynamicFileSplitEnumerator only supports batch execution.

Error message

DynamicFileSplitEnumerator only supports batch execution.

What it means

The DynamicFileSplitEnumerator is a batch-only SplitEnumerator that supports dynamic filtering (DynamicFilteringEvent). Its snapshotState method unconditionally throws UnsupportedOperationException because checkpointing state — which is inherent to streaming execution — is not implemented for this enumerator. The enumerator is designed to enumerate and assign splits once in bounded (batch) mode, filtering splits based on dynamic filtering data received from the planner.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/src/impl/DynamicFileSplitEnumerator.java:198

        splitAssigner = splitAssignerFactory.create(splits);
    }

    @Override
    public void addSplitsBack(List<SplitT> splits, int subtaskId) {
        LOG.debug("Dynamic File Source Enumerator adds splits back: {}", splits);
        if (splitAssigner != null) {
            List<FileSourceSplit> fileSplits = new ArrayList<>(splits);
            // Only add back splits enumerating. A split may be filtered after it is assigned.
            fileSplits.removeIf(s -> !allEnumeratingSplits.contains(s.splitId()));
            // Added splits should be removed from assignedSplits for re-assignment
            fileSplits.forEach(s -> assignedSplits.remove(s.splitId()));
            splitAssigner.addSplits(fileSplits);
        }
    }

    @Override
    public PendingSplitsCheckpoint<SplitT> snapshotState(long checkpointId) {
        throw new UnsupportedOperationException(
                "DynamicFileSplitEnumerator only supports batch execution.");
    }

    @Override
    public void handleSourceEvent(int subtaskId, int attemptNumber, SourceEvent sourceEvent) {
        // Only recognize events that don't care attemptNumber
        handleSourceEvent(subtaskId, sourceEvent);
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the job executes in BATCH mode: set execution.runtime-mode = BATCH in configuration, or call StreamExecutionEnvironment.setRuntimeMode(RuntimeExecutionMode.BATCH).
2. Remove dynamic filtering from the query if streaming execution is required, or restructure the query so the file source does not use DynamicFileSplitEnumerator.
3. Do not set source.monitor-interval on the file source when using dynamic filtering — its absence marks the source as bounded (batch).

Example fix

// before
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setRuntimeMode(RuntimeExecutionMode.STREAMING);
// table uses dynamic filtering -> triggers snapshotState

// after
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setRuntimeMode(RuntimeExecutionMode.BATCH);

// Before using dynamic filtering on a file source, verify batch mode
if (env.getRuntimeMode() != RuntimeExecutionMode.BATCH) {
    throw new IllegalStateException(
        "Dynamic filtering on file sources requires BATCH execution mode");
}

When it happens

Trigger: The framework calls snapshotState(checkpointId) during a checkpoint barrier, which only occurs in streaming (continuous) execution mode. This happens when the FileSource is configured with dynamic filtering AND the job runs in STREAMING mode (e.g. when source.monitor-interval is set or the job is unbounded). The DynamicFileSplitEnumerator is instantiated internally by the table planner when dynamic filtering is enabled on a batch file source; if that job is accidentally or intentionally run as streaming, checkpointing triggers this method.

Common situations: Running a batch SQL/Table query with dynamic filtering (e.g. a dimension join that pushes down a filter) but the execution environment defaults to or is explicitly set to streaming mode. Migrating a batch pipeline to streaming without removing dynamic filtering. Enabling checkpointing on a job that uses dynamic file filtering.

Related errors

• SimpleStreamFormat is not splittable, but found split end (%
• Failed to create enumerator for sourceIndex={currentSourceIn
• Invalid version %d
• Invalid option %s. Must be a positive integer.
• ${message}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d43a037e5fae62d1. Report an issue: GitHub.