apache/seatunnel · error · IllegalArgumentException

Generate empty file when no data is not supported when parti

Error message

Generate empty file when no data is not supported when partition is enabled.

What it means

BaseFileSink.preCheckConfig rejects create_empty_file_when_no_data = true when the sink is partitioned, because generating an empty file per nonexistent partition combination is undefined; the connector throws IllegalArgumentException.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/BaseFileSink.java:76

    protected abstract HadoopConf initHadoopConf();

    @Override
    public Optional<CatalogTable> getWriteCatalogTable() {
        return Optional.of(catalogTable);
    }

    public void preCheckConfig() {
        if (pluginConfig.getOptional(FileBaseSinkOptions.SINGLE_FILE_MODE).isPresent()
                && pluginConfig.get(FileBaseSinkOptions.SINGLE_FILE_MODE)
                && jobContext.isEnableCheckpoint()) {
            throw new IllegalArgumentException(
                    "Single file mode is not supported when checkpoint is enabled or in streaming mode.");
        }
        if (pluginConfig.getOptional(FileBaseSinkOptions.CREATE_EMPTY_FILE_WHEN_NO_DATA).isPresent()
                && pluginConfig.get(FileBaseSinkOptions.CREATE_EMPTY_FILE_WHEN_NO_DATA)
                && !fileSinkConfig.getPartitionFieldList().isEmpty()) {
            throw new IllegalArgumentException(
                    "Generate empty file when no data is not supported when partition is enabled.");
        }
    }

    @Override
    public void setJobContext(JobContext jobContext) {
        this.jobContext = jobContext;
        this.jobId = jobContext.getJobId();
        preCheckConfig();
    }

    @Override
    public SinkWriter<SeaTunnelRow, FileCommitInfo, FileSinkState> restoreWriter(
            SinkWriter.Context context, List<FileSinkState> states) {
        return new BaseFileSinkWriter(createWriteStrategy(), hadoopConf, context, jobId, states);
    }

    @Override

View on GitHub (pinned to cf67b549a7)

Solutions

1. Remove create_empty_file_when_no_data = true from the partitioned sink config
2. Remove partition_by if empty-file-on-no-data semantics are required more than partitioning
3. Handle 'no data' detection externally (e.g. downstream job monitoring) instead of relying on empty files

Example fix

// before
sink {
  LocalFile {
    partition_by = ["dt"]
    create_empty_file_when_no_data = true
  }
}
// after
sink {
  LocalFile {
    partition_by = ["dt"]
  }
}

if (config.get("create_empty_file_when_no_data") == true && partitionFields != null && !partitionFields.isEmpty()) {
    throw new IllegalArgumentException("create_empty_file_when_no_data cannot be used with partition_by");
}

When it happens

Trigger: Setting create_empty_file_when_no_data = true while sink.partition_by contains one or more fields, so fileSinkConfig.getPartitionFieldList() is non-empty during preCheckConfig.

Common situations: Users want empty files to signal job completion in a partitioned sink; combining two independently useful sink options without realizing they conflict; templated configs where partition fields were added later.

Related errors

• Option 'field_delimiter' cannot be empty
• Option 'max_in_flight' must be greater than zero
• Option 'operation_timeout_ms' must be greater than zero
• TABLE_QUERY_FAILED
• Options 'credentials_path' and 'emulator_host' cannot be con

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/b90fd63fbec3dba3. Report an issue: GitHub.