apache/flink · error · UnsupportedFileSystemSchemeException

Hadoop File System abstraction does not support scheme

Error message

Hadoop File System abstraction does not support scheme '${scheme}'. Either no file system implementation exists for that scheme, or the relevant classes are missing from the classpath.

What it means

HadoopFsFactory cannot resolve a FileSystem class for the requested scheme: org.apache.hadoop.fs.FileSystem.getFileSystemClass(scheme, config) threw, so Flink raises UnsupportedFileSystemSchemeException. This means either no Hadoop filesystem implementation is registered for that scheme or its classes are not on the classpath.

Solutions

1. Add the Hadoop filesystem connector jar for that scheme to Flink's lib/ directory on ALL nodes (e.g. hadoop-aws for s3a, hadoop-azure for abfs) and restart
2. Prefer Flink's native filesystem plugins (flink-s3-fs-hadoop, flink-oss-fs-hadoop, etc.) placed in lib/ or opt/ instead of raw Hadoop jars
3. Set fs.<scheme>.impl (and fs.<scheme>.impl.kerberos...) in flink-conf.yaml / core-site.xml if the implementation class exists but is not auto-registered
4. Verify the jar is not shaded with unrelocated META-INF/services entries that break the Hadoop FileSystem service loader

Example fix

# before: scheme 's3a' unsupported
# after: copy plugin and restart
 cp opt/flink-s3-fs-hadoop* lib/ && ./bin/start-cluster.sh

// pre-flight: can Hadoop resolve the scheme?
try {
    org.apache.hadoop.fs.FileSystem.getFileSystemClass(
            uri.getScheme(), new org.apache.hadoop.conf.Configuration());
} catch (IOException e) {
    throw new IllegalStateException("Add the connector jar for scheme " + uri.getScheme(), e);
}

try {
    fs = FileSystem.get(uri);
} catch (UnsupportedFileSystemSchemeException e) {
    // missing connector jar: fail fast with setup instructions, not a retry
}

When it happens

Trigger: Calling FileSystem.get(uri) for a scheme (e.g. 'hdfs', 's3a', 'abfs') whose Hadoop connector jar is absent, or whose service-provider entry in core-site.xml (fs.<scheme>.impl) is missing, so getFileSystemClass fails.

Common situations: Using the bare flink-shaded-hadoop jar which bundles only HDFS; putting an S3/OBS/Azure filesystem path in state.checkpoints.dir or output paths without adding the corresponding Hadoop connector jar to Flink's lib/ directory; shaded-class relocation issues where the service loader file was not relocated.

Related errors

• Bad syntax for classpath
• Cannot instantiate file system for URI
• Cannot support file system for
• Could not instantiate class
• Could not load class for serialization config

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/7fc78a8a6312eb89. Report an issue: GitHub.

Appendix: source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java:100

                hadoopConfig = this.hadoopConfig;
            } else if (flinkConfig != null) {
                hadoopConfig = HadoopUtils.getHadoopConfiguration(flinkConfig);
                this.hadoopConfig = hadoopConfig;
            } else {
                LOG.warn(
                        "Hadoop configuration has not been explicitly initialized prior to loading a Hadoop file system."
                                + " Using configuration from the classpath.");

                hadoopConfig = new org.apache.hadoop.conf.Configuration();
            }

            // -- (2) get the Hadoop file system class for that scheme

            final Class<? extends org.apache.hadoop.fs.FileSystem> fsClass;
            try {
                fsClass = org.apache.hadoop.fs.FileSystem.getFileSystemClass(scheme, hadoopConfig);
            } catch (IOException e) {
                throw new UnsupportedFileSystemSchemeException(
                        "Hadoop File System abstraction does not support scheme '"
                                + scheme
                                + "'. "
                                + "Either no file system implementation exists for that scheme, "
                                + "or the relevant classes are missing from the classpath.",
                        e);
            }

            // -- (3) instantiate the Hadoop file system

            LOG.debug(
                    "Instantiating for file system scheme {} Hadoop File System {}",
                    scheme,
                    fsClass.getName());

            final org.apache.hadoop.fs.FileSystem hadoopFs = fsClass.newInstance();

            // -- (4) create the proper URI to initialize the file system

View on GitHub (pinned to 2f3c205e92)