What causes "Get file list from this path [%s] failed" in apache/seatunnel?

Verify the path exists and is readable: run hadoop fs -ls (or ls for local) as the job user

apache/seatunnel · critical · FileConnectorException

FILE_LIST_GET_FAILED

FILE_LIST_GET_FAILED

Error message

Get file list from this path [%s] failed

What it means

BaseFileSource's constructor initializes the source by enumerating files under the configured path via readStrategy.getFileNamesByPath(path). If that throws IOException, the source fails fast with FileConnectorException FILE_LIST_GET_FAILED, 'Get file list from this path [%s] failed'. When document routing is enabled it throws with a copied stack trace; otherwise the original cause is attached. This means the connector could not even list input files, before any read started.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/BaseFileSource.java:93

                fileFormat == FileFormat.MARKDOWN
                        && pluginConfig.get(FileBaseSourceOptions.MARKDOWN_RAG_METADATA_ENABLED);
        this.readStrategy.setPluginConfig(pluginConfig.toConfig());
        this.readStrategy.init(hadoopConf);
        String path = pluginConfig.get(FileBaseSourceOptions.FILE_PATH);
        // Fail fast and retain the sanitized root for diagnostics without replacing the logical
        // identity that MarkdownReadStrategy derives for each discovered file.
        String safeDiscoveryRootContext =
                documentRoutingEnabled
                        ? MarkdownKnowledgeSyncMetadata.canonicalizeSourceUri(path)
                        : path;
        try {
            filePaths = readStrategy.getFileNamesByPath(path);
        } catch (IOException e) {
            String errorMsg =
                    String.format(
                            "Get file list from this path [%s] failed", safeDiscoveryRootContext);
            if (documentRoutingEnabled) {
                throw new FileConnectorException(
                        FileConnectorErrorCode.FILE_LIST_GET_FAILED,
                        errorMsg,
                        MarkdownKnowledgeSyncMetadata.copyStackTraceOnly(e));
            }
            throw new FileConnectorException(
                    FileConnectorErrorCode.FILE_LIST_GET_FAILED, errorMsg, e);
        }

        // support user-defined schema
        CatalogTable userDefinedCatalogTable;
        // only json text csv type support user-defined schema now
        if (pluginConfig.getOptional(ConnectorCommonOptions.SCHEMA).isPresent()) {
            switch (fileFormat) {
                case CSV:
                case TEXT:
                case JSON:
                case EXCEL:
                case XML:

View on GitHub (pinned to cf67b549a7)

Solutions

Verify the path exists and is readable: run hadoop fs -ls <path> (or ls for local) as the job user
Check the wrapped IOException cause for permission vs connectivity vs not-found
Fix filesystem configuration (core-site/hdfs credentials, S3 endpoint/keys) on all nodes
Correct the source path in the job config; ensure the directory is populated before job start

Example fix

# before
source {
  File {
    path = "hdfs://nn/data/input-2024*"  # not found / no permission
  }
}
# after: verify with `hadoop fs -ls /data/` then set
    path = "hdfs://nn/data/input"

Defensive patterns

Strategy: try-catch

Validate before calling

// before submit
hadoop fs -test -d <path> && hadoop fs -test -e <path> || echo "path missing/unreadable"

Try / catch

try { source.open(ctx); } catch (FileConnectorException e) { if (e.getErrorCode() == FILE_LIST_GET_FAILED) { log(e.getCause()); /* fix path/permissions, then retry */ } }

Prevention

Pre-validate source paths exist and are readable as the job user before submission
Keep Kerberos/S3 credentials valid on all nodes
Avoid glob/regex paths that can match nothing; create directories before job start

When it happens

Trigger: Source plugin init where the HDFS/local filesystem list operation on the configured path throws IOException — path does not exist, permission denied, namenode unreachable, or invalid URI.

Common situations: Typo in source path or wrong filesystem scheme; Kerberos/credential problems making listing fail; HDFS in safe mode or S3 endpoint misconfigured; directory deleted between job submit and execution; using a file (not directory) path where listing semantics differ.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/76d5ebf5242d4f8a. Report an issue: GitHub.