apache/seatunnel · error · FileConnectorException

SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED

SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED

Error message

Failed to determine whether file source path is a file or directory: %s

What it means

BinaryReadStrategy.init calls hadoopFileSystemProxy.isFile(basePath) to decide whether the configured path is a single file or a directory. If that check throws an IOException (filesystem unreachable, permission probe failure, HDFS NameNode error, etc.), it wraps it in a FileConnectorException with CONFIG_VALIDATION_FAILED, including the path that could not be classified.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/BinaryReadStrategy.java:66

                    new SeaTunnelDataType[] {
                        PrimitiveByteArrayType.INSTANCE, BasicType.STRING_TYPE, BasicType.LONG_TYPE
                    });

    private String basePath;
    private transient boolean basePathIsFile;
    private int binaryChunkSize = FileBaseSourceOptions.BINARY_CHUNK_SIZE.defaultValue();
    private boolean completeFileMode =
            FileBaseSourceOptions.BINARY_COMPLETE_FILE_MODE.defaultValue();
    private transient String lastReadFingerprint;

    @Override
    public void init(HadoopConf conf) {
        super.init(conf);
        basePath = pluginConfig.getString(FileBaseSourceOptions.FILE_PATH.key());
        try {
            basePathIsFile = hadoopFileSystemProxy.isFile(basePath);
        } catch (IOException e) {
            throw new FileConnectorException(
                    SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED,
                    "Failed to determine whether file source path is a file or directory: "
                            + basePath,
                    e);
        }

        // Load binary chunk size configuration
        if (pluginConfig.hasPath(FileBaseSourceOptions.BINARY_CHUNK_SIZE.key())) {
            binaryChunkSize = pluginConfig.getInt(FileBaseSourceOptions.BINARY_CHUNK_SIZE.key());
            // Validate chunk size - should be positive and reasonable
            if (binaryChunkSize <= 0) {
                throw new IllegalArgumentException(
                        "Binary chunk size must be positive, got: " + binaryChunkSize);
            }
            if (binaryChunkSize > 100 * 1024 * 1024) { // 100MB limit
                throw new IllegalArgumentException(
                        "Binary chunk size too large (max 100MB), got: " + binaryChunkSize);
            }

View on GitHub (pinned to cf67b549a7)

Solutions

Inspect the wrapped 'Caused by' IOException for the root cause (connection refused, auth, timeout).
Verify storage connectivity from the SeaTunnel worker nodes (e.g. hdfs dfs -ls <path>).
Fix HadoopConf settings (fs.defaultFS, endpoint, credentials) and re-run the job.

Example fix

// before
path = "hdfs://namenode-host:9000/data/binary"
// after (corrected reachable FS + valid path)
path = "hdfs://nn1.cluster:8020/data/binary"

Defensive patterns

Strategy: try-catch

Validate before calling

// Java, connectivity pre-check before submitting the job
org.apache.hadoop.fs.FileSystem fs = FileSystem.get(hadoopConf);
if (!fs.exists(new org.apache.hadoop.fs.Path(basePath))) {
    throw new IllegalStateException("Path not reachable: " + basePath);
}

Try / catch

try {
    readStrategy.init(hadoopConf);
} catch (FileConnectorException e) {
    Throwable root = e.getCause();
    log.error("Cannot classify path {}: {}", basePath, root == null ? e : root.getMessage());
    throw e;
}

Prevention

Verify storage access (hdfs dfs -ls / aws s3 ls) from worker nodes before running.
Check fs.defaultFS, endpoints, and credentials in HadoopConf.
Monitor NameNode/storage endpoint health; add retry/backoff at the infra level.

When it happens

Trigger: Initializing a binary file source whose FILE_PATH points to an HDFS/S3/local path where the isFile() metadata call fails: NameNode down, S3 endpoint unreachable, bad credentials, or network interruption.

Common situations: HDFS cluster unavailable; S3/OSS misconfigured endpoint or expired credentials; Kerberos/authentication issues; firewall blocking the storage service; typo causing a resolver-level failure on some filesystems.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/1c97e74c057dced6. Report an issue: GitHub.