apache/seatunnel · error · FileConnectorException

SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED

SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED

Error message

PluginName: %s, PluginType: %s, Message: %s

What it means

HdfsFileHadoopConfig.buildWithConfig validates that the three mandatory HDFS source options are present — FILE_PATH, FILE_FORMAT_TYPE, and DEFAULT_FS — using CheckConfigUtil.checkAllExists. If any is missing it throws FileConnectorException with error code CONFIG_VALIDATION_FAILED and a message naming the HDFS plugin, plugin type SOURCE, and the aggregated validation message.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-hadoop/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/hdfs/config/HdfsFileHadoopConfig.java:43

import org.apache.seatunnel.connectors.seatunnel.file.config.FileBaseSourceOptions;
import org.apache.seatunnel.connectors.seatunnel.file.config.FileSystemType;
import org.apache.seatunnel.connectors.seatunnel.file.config.HadoopConf;
import org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException;

public class HdfsFileHadoopConfig extends HadoopConf {
    public HdfsFileHadoopConfig(String hdfsNameKey) {
        super(hdfsNameKey);
    }

    public static HadoopConf buildWithConfig(ReadonlyConfig readonlyConfig) {
        CheckResult result =
                CheckConfigUtil.checkAllExists(
                        readonlyConfig.toConfig(),
                        FileBaseSourceOptions.FILE_PATH.key(),
                        FileBaseSourceOptions.FILE_FORMAT_TYPE.key(),
                        FileBaseSourceOptions.DEFAULT_FS.key());
        if (!result.isSuccess()) {
            throw new FileConnectorException(
                    SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED,
                    String.format(
                            "PluginName: %s, PluginType: %s, Message: %s",
                            FileSystemType.HDFS.getFileSystemPluginName(),
                            PluginType.SOURCE,
                            result.getMsg()));
        }
        HadoopConf hadoopConf =
                new HdfsFileHadoopConfig(readonlyConfig.get(FileBaseSourceOptions.DEFAULT_FS));

        if (readonlyConfig.getOptional(FileBaseSourceOptions.HDFS_SITE_PATH).isPresent()) {
            hadoopConf.setHdfsSitePath(readonlyConfig.get(FileBaseSourceOptions.HDFS_SITE_PATH));
        }

        if (readonlyConfig.getOptional(FileBaseSourceOptions.REMOTE_USER).isPresent()) {
            hadoopConf.setRemoteUser(readonlyConfig.get(FileBaseSourceOptions.REMOTE_USER));
        }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Add all required options to the source config: file_path, file_format_type, and fs.defaultFS (e.g. "hdfs://namenode:8020")
2. Read result.getMsg() in the error message to see exactly which key(s) are missing
3. Fix any misspelled option keys to match FileBaseSourceOptions definitions
4. Verify the options are inside the correct source/sink plugin block so they appear in readonlyConfig

Example fix

// before
HdfsFile {
    file_path = "/data/input"
}
// after
HdfsFile {
    file_path = "/data/input"
    file_format_type = "json"
    fs.defaultFS = "hdfs://namenode:8020"
}

// Java: validate required HDFS options before building config
Config cfg = readonlyConfig.toConfig();
java.util.List<String> missing = new java.util.ArrayList<>();
for (String key : new String[]{"file_path", "file_format_type", "fs.defaultFS"}) {
    if (!cfg.hasPath(key)) missing.add(key);
}
if (!missing.isEmpty()) {
    throw new IllegalArgumentException("Missing required HDFS options: " + missing);
}

try {
    HdfsFileHadoopConfig.buildWithConfig(readonlyConfig);
} catch (FileConnectorException e) {
    if (e.getCode() == SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED) {
        logger.error("HDFS source config incomplete — see message for missing keys: {}", e.getMessage());
    }
    throw e;
}

When it happens

Trigger: buildWithConfig invoked for an HDFS file source whose readonlyConfig lacks any of: FILE_PATH (file_path), FILE_FORMAT_TYPE (file_format_type), or DEFAULT_FS (fs.defaultFS) keys.

Common situations: User omitted file_format_type or fs.defaultFS in the HDFS source config block; typo in option keys (e.g. 'file_format' instead of 'file_format_type'); copied a local-file example that doesn't require default_fs and reused it for HDFS; config section nested under the wrong plugin so keys aren't found.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Schema config can not be empty
• Unknown format type:
• Option '${option}' cannot be blank
• Option '${valuesAndOptions[index + 1]}' is not valid for the
• Option 'field_delimiter' cannot be empty

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/b88e74acefc4f148. Report an issue: GitHub.