apache/seatunnel · error · IllegalArgumentException

Single file mode is not supported when checkpoint is enabled

Error message

Single file mode is not supported when checkpoint is enabled or in streaming mode.

What it means

BaseFileSink.preCheckConfig rejects combining SINGLE_FILE_MODE with checkpoint-enabled (batch-with-checkpoint or streaming) jobs, because single-file output relies on a single writer and cannot be reconciled with checkpointed, possibly rescaled sinks.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/BaseFileSink.java:70

    public BaseFileSink(ReadonlyConfig pluginConfig, CatalogTable catalogTable) {
        this.pluginConfig = pluginConfig;
        this.catalogTable = catalogTable;
        this.fileSinkConfig = new FileSinkConfig(pluginConfig, catalogTable.getSeaTunnelRowType());
        this.hadoopConf = initHadoopConf();
    }

    protected abstract HadoopConf initHadoopConf();

    @Override
    public Optional<CatalogTable> getWriteCatalogTable() {
        return Optional.of(catalogTable);
    }

    public void preCheckConfig() {
        if (pluginConfig.getOptional(FileBaseSinkOptions.SINGLE_FILE_MODE).isPresent()
                && pluginConfig.get(FileBaseSinkOptions.SINGLE_FILE_MODE)
                && jobContext.isEnableCheckpoint()) {
            throw new IllegalArgumentException(
                    "Single file mode is not supported when checkpoint is enabled or in streaming mode.");
        }
        if (pluginConfig.getOptional(FileBaseSinkOptions.CREATE_EMPTY_FILE_WHEN_NO_DATA).isPresent()
                && pluginConfig.get(FileBaseSinkOptions.CREATE_EMPTY_FILE_WHEN_NO_DATA)
                && !fileSinkConfig.getPartitionFieldList().isEmpty()) {
            throw new IllegalArgumentException(
                    "Generate empty file when no data is not supported when partition is enabled.");
        }
    }

    @Override
    public void setJobContext(JobContext jobContext) {
        this.jobContext = jobContext;
        this.jobId = jobContext.getJobId();
        preCheckConfig();
    }

    @Override

View on GitHub (pinned to cf67b549a7)

Solutions

1. Disable single_file_mode and instead set a parallelism of 1 for the sink to get a single output file
2. Run the job in batch mode with checkpoint disabled if single-file output is essential
3. Merge output files in a post-processing step after the job finishes

Example fix

// before
sink {
  LocalFile {
    single_file_mode = true
  }
}
// streaming job
// after
sink {
  LocalFile {
    parallelism = 1
  }
}

if (config.get("single_file_mode") == true && (jobMode == STREAMING || checkpointEnabled)) {
    throw new IllegalArgumentException("single_file_mode requires batch mode with checkpoint disabled");
}

When it happens

Trigger: Configuring single_file_mode = true on any file sink while running in streaming mode or with checkpoint enabled (e.g. checkpoint_interval set, or -e run in streaming), causing preCheckConfig (invoked from setJobContext) to throw.

Common situations: Users want one merged output file in a streaming job; enabling checkpoint for exactly-once while keeping single_file_mode from a batch template; job mode switched from batch to streaming without revisiting sink options.

Related errors

• Single file mode is not supported when checkpoint is enabled
• Failed to flush data in prepareCommit
• Failed to flush data during prepareCommit()
• TABLE_PRE_COMMIT_FAILED
• checkpoint {} do not exist or have already been committed.

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/a38a60a9e9bcc3ce. Report an issue: GitHub.