apache/flink · error · UnsupportedFileSystemSchemeException

Cannot support file system for '${fsUri.getScheme()}' via Ha

Error message

Cannot support file system for '${fsUri.getScheme()}' via Hadoop, because Hadoop is not in the classpath, or some classes are missing from the classpath.

What it means

While loading/instantiating the Hadoop filesystem, a ReflectiveOperationException or LinkageError occurred, meaning Hadoop classes could not be loaded at all or are inconsistent on the classpath. Flink wraps this in UnsupportedFileSystemSchemeException telling you Hadoop is missing (or partially missing) from the classpath.

Source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java:187

            } catch (UnknownHostException e) {
                String message =
                        "The Hadoop file system's authority ("
                                + initUri.getAuthority()
                                + "), specified by either the file URI or the configuration, cannot be resolved.";

                throw new IOException(message, e);
            }

            HadoopFileSystem fs = new HadoopFileSystem(hadoopFs);

            // create the Flink file system, optionally limiting the open connections
            if (flinkConfig != null) {
                return limitIfConfigured(fs, scheme, flinkConfig);
            } else {
                return fs;
            }
        } catch (ReflectiveOperationException | LinkageError e) {
            throw new UnsupportedFileSystemSchemeException(
                    "Cannot support file system for '"
                            + fsUri.getScheme()
                            + "' via Hadoop, because Hadoop is not in the classpath, or some classes "
                            + "are missing from the classpath.",
                    e);
        } catch (IOException e) {
            throw e;
        } catch (Exception e) {
            throw new IOException("Cannot instantiate file system for URI: " + fsUri, e);
        }
    }

    private static String getMissingAuthorityErrorPrefix(URI fsURI) {
        return "The given file system URI ("
                + fsURI.toString()
                + ") did not describe the authority "
                + "(like for example HDFS NameNode address/port or S3 host). "
                + "The attempt to use a configured default authority failed: ";

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Clean the classpath: keep exactly one consistent set of Hadoop jars (or one Flink filesystem plugin) in Flink lib/ and remove Hadoop classes from user fat-jars
2. Use the prebuilt flink-shaded-hadoop or the scheme-specific Flink plugins (flink-s3-fs-hadoop, flink-gs-fs-hadoop, ...) matching your Flink version
3. Inspect the suppressed cause in the exception to identify the exact missing class and add that dependency
4. Verify no duplicate classes with `mvn dependency:tree` or by listing Hadoop jars: ls lib/*hadoop*

Example fix

# before: mixed jars
lib/hadoop-common-2.8.5.jar lib/hadoop-hdfs-3.3.4.jar
# after: one consistent set
lib/flink-shaded-hadoop-2-uber-2.8.3-1.8.3-10.0.jar

// pre-flight: can we load Hadoop's FileSystem at all?
try {
    Class.forName("org.apache.hadoop.fs.FileSystem", false, Thread.currentThread().getContextClassLoader());
} catch (ClassNotFoundException e) {
    throw new IllegalStateException("Hadoop not on classpath", e);
}

try {
    fs = FileSystem.get(uri);
} catch (UnsupportedFileSystemSchemeException e) {
    // inspect cause: LinkageError/ReflectiveOperationException => classpath conflict,
    // fix jars; this is not retryable as-is
}

When it happens

Trigger: Any reflection failure or LinkageError (NoSuchMethodError, NoClassDefFoundError, ClassNotFoundException) while getting/instantiating the FileSystem class in HadoopFsFactory.create; e.g. hadoop-common present but hadoop-hdfs absent, or mixed Hadoop versions.

Common situations: Multiple Hadoop jars of different versions in lib/ and user jars; a user-fat-jar bundling Hadoop with relocation conflicts; missing dependency of the chosen connector (e.g. hadoop-aws without matching hadoop-common); running on a Hadoop-free Flink distribution without adding any Hadoop filesystem jar.

Related errors

• The program's entry point class '{className}' could not be l
• Hadoop File System abstraction does not support scheme '${sc
• Truncation of file failed because of access/linking problems
• Bad syntax for classpath: {}
• Python command line option detected but the flink-python mod

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/8623f36e010e7b4d. Report an issue: GitHub.