apache/flink · error · IOException

The Hadoop file system's authority (${initUri.getAuthority()

Error message

The Hadoop file system's authority (${initUri.getAuthority()}), specified by either the file URI or the configuration, cannot be resolved.

What it means

After the Hadoop FileSystem instance is created, hadoopFs.initialize(initUri, config) throws UnknownHostException: the authority (host) from the URI or fs.defaultFS cannot be resolved by DNS. The factory wraps it in an IOException naming the unresolvable authority.

Source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java:175

                        throw new IOException(
                                getMissingAuthorityErrorPrefix(fsUri)
                                        + "Hadoop configuration for default file system ('fs.default.name' or 'fs.defaultFS') "
                                        + "contains no valid authority component (like hdfs namenode, S3 host, etc)");
                    }
                }
            }

            // -- (5) configure the Hadoop file system

            try {
                hadoopFs.initialize(initUri, hadoopConfig);
            } catch (UnknownHostException e) {
                String message =
                        "The Hadoop file system's authority ("
                                + initUri.getAuthority()
                                + "), specified by either the file URI or the configuration, cannot be resolved.";

                throw new IOException(message, e);
            }

            HadoopFileSystem fs = new HadoopFileSystem(hadoopFs);

            // create the Flink file system, optionally limiting the open connections
            if (flinkConfig != null) {
                return limitIfConfigured(fs, scheme, flinkConfig);
            } else {
                return fs;
            }
        } catch (ReflectiveOperationException | LinkageError e) {
            throw new UnsupportedFileSystemSchemeException(
                    "Cannot support file system for '"
                            + fsUri.getScheme()
                            + "' via Hadoop, because Hadoop is not in the classpath, or some classes "
                            + "are missing from the classpath.",
                    e);
        } catch (IOException e) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the hostname: run `nslookup <host>` or `ping <host>` from the Flink node and fix typos in the URI/fs.defaultFS
2. Add the correct DNS record or /etc/hosts entry on every TaskManager/JobManager node (or K8s NetworkPolicy/DNS config)
3. If a proxy/gateway is required, configure it (e.g. hadoop proxy settings) instead of the raw NameNode host
4. Confirm /etc/resolv.conf on containers points to a resolver that knows the storage hostname

Example fix

# before
state.checkpoints.dir: hdfs://namenode01:8020/checkpoints  # typo
# after
state.checkpoints.dir: hdfs://namenode-01:8020/checkpoints

// pre-flight DNS check on the authority host
String host = URI.create(fsPath).getHost();
if (host != null && InetAddress.getByName(host) == null) { /* unreachable practically; getByName throws */ }

try {
    fs = FileSystem.get(uri);
} catch (IOException e) {
    if (e.getCause() instanceof java.net.UnknownHostException) {
        // DNS problem: fix hosts/DNS, then retry creation; transient in K8s during startup
    }
}

When it happens

Trigger: Creating a filesystem for hdfs://bad-host:8020 where 'bad-host' fails DNS resolution during initialize(); similarly any S3/other Hadoop-compatible endpoint hostname that does not resolve.

Common situations: Typo in the NameNode hostname; missing /etc/hosts entry or DNS in containers/K8s; network isolation between Flink nodes and the storage system; NameNode host renamed but configs not updated.

Related errors

• fqdn is null
• hostPort should not be null or empty
• The given host:port ('{}') doesn't contain a valid host
• The given host:port ('{}') doesn't contain a valid port
• The configured hostname is not valid

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d9b5c450976d2e94. Report an issue: GitHub.