apache/flink · error · IOException

The given file system URI (${fsUri}) did not describe the au

Error message

The given file system URI (${fsUri}) did not describe the authority (like for example HDFS NameNode address/port or S3 host). The attempt to use a configured default authority failed: Hadoop configuration did not contain an entry for the default file system ('fs.defaultFS').

What it means

The filesystem URI (e.g. hdfs:///path) has no authority component, so HadoopFsFactory falls back to the configured default. Neither fs.defaultFS nor its deprecated alias fs.default.name is present in the effective Hadoop configuration, so there is no way to determine the NameNode/host, and an IOException is thrown with this prefix plus the specific cause.

Source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java:142

                if (LOG.isDebugEnabled()) {
                    LOG.debug(
                            "URI {} does not specify file system authority, trying to load default authority (fs.defaultFS)",
                            fsUri);
                }

                String configEntry = hadoopConfig.get("fs.defaultFS", null);
                if (configEntry == null) {
                    // fs.default.name deprecated as of hadoop 2.2.0 - see
                    // http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
                    configEntry = hadoopConfig.get("fs.default.name", null);
                }

                if (LOG.isDebugEnabled()) {
                    LOG.debug("Hadoop's 'fs.defaultFS' is set to {}", configEntry);
                }

                if (configEntry == null) {
                    throw new IOException(
                            getMissingAuthorityErrorPrefix(fsUri)
                                    + "Hadoop configuration did not contain an entry for the default file system ('fs.defaultFS').");
                } else {
                    try {
                        initUri = URI.create(configEntry);
                    } catch (IllegalArgumentException e) {
                        throw new IOException(
                                getMissingAuthorityErrorPrefix(fsUri)
                                        + "The configuration contains an invalid file system default name "
                                        + "('fs.default.name' or 'fs.defaultFS'): "
                                        + configEntry);
                    }

                    if (initUri.getAuthority() == null) {
                        throw new IOException(
                                getMissingAuthorityErrorPrefix(fsUri)
                                        + "Hadoop configuration for default file system ('fs.default.name' or 'fs.defaultFS') "
                                        + "contains no valid authority component (like hdfs namenode, S3 host, etc)");

View on GitHub (pinned to 2f3c205e92)

Solutions

Use fully-qualified URIs including authority: hdfs://namenode:8020/path for checkpoint/output paths
Set fs.defaultFS in flink-conf.yaml (fs.hdfs.hdfsdefault) or provide a core-site.xml via HADOOP_CONF_DIR
Set fs.defaultFS explicitly as a Hadoop config option: hadoop.fs.defaultFS: hdfs://namenode:8020 in Flink configuration

Example fix

# before
state.checkpoints.dir: hdfs:///flink/checkpoints
# after
state.checkpoints.dir: hdfs://namenode:8020/flink/checkpoints

Defensive patterns

Strategy: validation

Validate before calling

URI u = URI.create(path);
if (u.getAuthority() == null) {
    String def = hadoopConf.get("fs.defaultFS", hadoopConf.get("fs.default.name"));
    if (def == null) {
        throw new IllegalStateException("Set fs.defaultFS or use a fully-qualified URI: " + path);
    }
}

Try / catch

try {
    fs = FileSystem.get(uri);
} catch (IOException e) {
    if (e.getMessage().contains("did not describe the authority")) {
        // config issue: set fs.defaultFS or fully-qualify the URI; do not retry
    }
}

Prevention

Always use fully-qualified storage URIs in Flink configs
Ship a core-site.xml with fs.defaultFS via HADOOP_CONF_DIR
Add deployment checks that reject authority-less hdfs:// paths

When it happens

Trigger: Creating a filesystem for a URI like 'hdfs:///user/x' (no host:port) while the Hadoop configuration has no fs.defaultFS entry; common when neither a valid core-site.xml nor a Flink config entry (fs.hdfs.hdfsdefault) is provided.

Common situations: Running Flink without HADOOP_CONF_DIR/core-site.xml; checkpoints or output paths written scheme-relative without host; using the shaded Hadoop jar which ships an empty default configuration.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/1ab87fb7f0aed214. Report an issue: GitHub.