apache/flink · error · IOException

The given file system URI (${fsUri}) did not describe the au

Error message

The given file system URI (${fsUri}) did not describe the authority (like for example HDFS NameNode address/port or S3 host). The attempt to use a configured default authority failed: Hadoop configuration for default file system ('fs.default.name' or 'fs.defaultFS') contains no valid authority component (like hdfs namenode, S3 host, etc)

What it means

fs.defaultFS parses as a URI but the resulting URI has no authority component (e.g. 'hdfs:///'), so the default-authority fallback still cannot supply a NameNode address. HadoopFsFactory throws IOException stating the default filesystem configuration contains no valid authority.

Source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java:157

                }

                if (configEntry == null) {
                    throw new IOException(
                            getMissingAuthorityErrorPrefix(fsUri)
                                    + "Hadoop configuration did not contain an entry for the default file system ('fs.defaultFS').");
                } else {
                    try {
                        initUri = URI.create(configEntry);
                    } catch (IllegalArgumentException e) {
                        throw new IOException(
                                getMissingAuthorityErrorPrefix(fsUri)
                                        + "The configuration contains an invalid file system default name "
                                        + "('fs.default.name' or 'fs.defaultFS'): "
                                        + configEntry);
                    }

                    if (initUri.getAuthority() == null) {
                        throw new IOException(
                                getMissingAuthorityErrorPrefix(fsUri)
                                        + "Hadoop configuration for default file system ('fs.default.name' or 'fs.defaultFS') "
                                        + "contains no valid authority component (like hdfs namenode, S3 host, etc)");
                    }
                }
            }

            // -- (5) configure the Hadoop file system

            try {
                hadoopFs.initialize(initUri, hadoopConfig);
            } catch (UnknownHostException e) {
                String message =
                        "The Hadoop file system's authority ("
                                + initUri.getAuthority()
                                + "), specified by either the file URI or the configuration, cannot be resolved.";

                throw new IOException(message, e);

View on GitHub (pinned to 2f3c205e92)

Solutions

Set fs.defaultFS to include the authority: hdfs://namenode-host:8020
Prefer specifying the full authority directly in the Flink path/checkpoint URI: hdfs://namenode:8020/flink/checkpoints
Validate the deployed core-site.xml on every node actually contains the host:port

Example fix

# before
fs.defaultFS: hdfs:///
# after
fs.defaultFS: hdfs://namenode:8020

Defensive patterns

Strategy: validation

Validate before calling

URI def = URI.create(conf.get("fs.defaultFS", ""));
if (def.getAuthority() == null) {
    throw new IllegalStateException("fs.defaultFS lacks an authority: expected hdfs://host:port");
}

Try / catch

try {
    fs = FileSystem.get(uri);
} catch (IOException e) {
    if (e.getMessage().contains("no valid authority component")) {
        // set fs.defaultFS to hdfs://namenode:8020 and retry setup
    }
}

Prevention

Standardize fs.defaultFS on the full hdfs://host:port form
Config-validation step in CI that parses and checks the authority
Avoid scheme-only defaults like hdfs:///

When it happens

Trigger: fs.defaultFS set to a scheme-only URI such as 'hdfs:/// ' or 'file:///' while trying to create an HDFS filesystem from an authority-less URI; initUri.getAuthority() returns null in HadoopFsFactory.

Common situations: Copy-pasted core-site.xml with authority stripped; default config of shaded Hadoop; someone replaced the NameNode address with a placeholder during templating.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/2628002718af3dff. Report an issue: GitHub.