apache/seatunnel · warning

enable_file_split=true but compress_codec={} or archive_comp

Error message

enable_file_split=true but compress_codec={} or archive_compress_codec={} is not NONE. Falling back to non-splitting mode.

What it means

WARN logged by FileSplitStrategyFactory.initFileSplitStrategy when enable_file_split=true but the configured compress_codec or archive_compress_codec is not NONE. Splitting is unsafe on compressed/archived data (split boundaries would corrupt decoding), so the factory falls back to DefaultFileSplitStrategy (whole-file reads).

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/FileSplitStrategyFactory.java:55

    public static FileSplitStrategy initFileSplitStrategy(
            ReadonlyConfig readonlyConfig, HadoopConf hadoopConf) {
        if (!readonlyConfig.get(FileBaseSourceOptions.ENABLE_FILE_SPLIT)) {
            return new DefaultFileSplitStrategy();
        }
        FileFormat fileFormat = readonlyConfig.get(FileBaseSourceOptions.FILE_FORMAT_TYPE);
        if (!fileFormat.supportFileSplit()) {
            log.warn(
                    "enable_file_split=true but file_format_type={} does not support file split. "
                            + "Falling back to non-splitting mode.",
                    fileFormat);
            return new DefaultFileSplitStrategy();
        }
        CompressFormat compressCodec = readonlyConfig.get(FileBaseSourceOptions.COMPRESS_CODEC);
        ArchiveCompressFormat archiveCompressCodec =
                readonlyConfig.get(FileBaseSourceOptions.ARCHIVE_COMPRESS_CODEC);
        if (compressCodec != CompressFormat.NONE
                || archiveCompressCodec != ArchiveCompressFormat.NONE) {
            log.warn(
                    "enable_file_split=true but compress_codec={} or archive_compress_codec={} is not NONE. "
                            + "Falling back to non-splitting mode.",
                    compressCodec,
                    archiveCompressCodec);
            return new DefaultFileSplitStrategy();
        }

        Objects.requireNonNull(
                hadoopConf, "hadoopConf must not be null when file split is enabled");

        long fileSplitSize = readonlyConfig.get(FileBaseSourceOptions.FILE_SPLIT_SIZE);
        if (fileSplitSize <= 0) {
            throw new SeaTunnelRuntimeException(
                    FileConnectorErrorCode.FILE_SPLIT_SIZE_ILLEGAL,
                    String.format(
                            "file_split_size must be greater than 0 when enable_file_split=true, but got: %d",
                            fileSplitSize));
        }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Set compress_codec=NONE and archive_compress_codec=NONE if you want file splitting.
2. If compression is required, accept the fallback and remove enable_file_split=true for clarity.
3. Pre-decompress/archive-extract data into splittable uncompressed files before ingestion if parallelism is critical.
4. Use natively splittable compression (e.g. bzip2 with formats that support it, or columnar formats with internal block compression like Parquet+snappy) instead of whole-file codecs.

Example fix

// before
enable_file_split = true
compress_codec = "gzip"
// after
enable_file_split = true
compress_codec = "none"
archive_compress_codec = "none"

// Reject conflicting combination before submission
if (enableFileSplit
    && (compressCodec != CompressFormat.NONE
        || archiveCompressCodec != ArchiveCompressFormat.NONE)) {
    throw new IllegalArgumentException(
        "file split requires compress_codec=NONE and archive_compress_codec=NONE");
}

When it happens

Trigger: Config combining enable_file_split=true with compress_codec values like GZIP/LZO/etc., or archive_compress_codec values like ZIP/TAR/GZIP — any non-NONE combination triggers the fallback.

Common situations: Reading gzip-compressed text files with splitting enabled; reading zipped/tarred archives while expecting split parallelism; leftover compress settings from a previous config.

Related errors

• enable_file_split=true but file_format_type={} does not supp
• Excel file must have a .gz extension. File: %s
• The file does not support the compressed file reading
• CanalJson file does not support this compress type: {}
• The {} file format is incorrect. Please check the format in

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/0ae8b1bee74f5019. Report an issue: GitHub.