{"record":{"id":"7fc78a8a6312eb89","repo":"apache/flink","slug":"hadoop-file-system-abstraction-does-not-support-sc","errorCode":null,"errorMessage":"Hadoop File System abstraction does not support scheme '${scheme}'. Either no file system implementation exists for that scheme, or the relevant classes are missing from the classpath.","messagePattern":"Hadoop File System abstraction does not support scheme '(.+?)'\\. Either no file system implementation exists for that scheme, or the relevant classes are missing from the classpath\\.","errorType":"exception","errorClass":"UnsupportedFileSystemSchemeException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java","lineNumber":100,"sourceCode":"                hadoopConfig = this.hadoopConfig;\n            } else if (flinkConfig != null) {\n                hadoopConfig = HadoopUtils.getHadoopConfiguration(flinkConfig);\n                this.hadoopConfig = hadoopConfig;\n            } else {\n                LOG.warn(\n                        \"Hadoop configuration has not been explicitly initialized prior to loading a Hadoop file system.\"\n                                + \" Using configuration from the classpath.\");\n\n                hadoopConfig = new org.apache.hadoop.conf.Configuration();\n            }\n\n            // -- (2) get the Hadoop file system class for that scheme\n\n            final Class<? extends org.apache.hadoop.fs.FileSystem> fsClass;\n            try {\n                fsClass = org.apache.hadoop.fs.FileSystem.getFileSystemClass(scheme, hadoopConfig);\n            } catch (IOException e) {\n                throw new UnsupportedFileSystemSchemeException(\n                        \"Hadoop File System abstraction does not support scheme '\"\n                                + scheme\n                                + \"'. \"\n                                + \"Either no file system implementation exists for that scheme, \"\n                                + \"or the relevant classes are missing from the classpath.\",\n                        e);\n            }\n\n            // -- (3) instantiate the Hadoop file system\n\n            LOG.debug(\n                    \"Instantiating for file system scheme {} Hadoop File System {}\",\n                    scheme,\n                    fsClass.getName());\n\n            final org.apache.hadoop.fs.FileSystem hadoopFs = fsClass.newInstance();\n\n            // -- (4) create the proper URI to initialize the file system","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFsFactory.java#L82-L118","documentation":"HadoopFsFactory cannot resolve a FileSystem class for the requested scheme: org.apache.hadoop.fs.FileSystem.getFileSystemClass(scheme, config) threw, so Flink raises UnsupportedFileSystemSchemeException. This means either no Hadoop filesystem implementation is registered for that scheme or its classes are not on the classpath.","triggerScenarios":"Calling FileSystem.get(uri) for a scheme (e.g. 'hdfs', 's3a', 'abfs') whose Hadoop connector jar is absent, or whose service-provider entry in core-site.xml (fs.<scheme>.impl) is missing, so getFileSystemClass fails.","commonSituations":"Using the bare flink-shaded-hadoop jar which bundles only HDFS; putting an S3/OBS/Azure filesystem path in state.checkpoints.dir or output paths without adding the corresponding Hadoop connector jar to Flink's lib/ directory; shaded-class relocation issues where the service loader file was not relocated.","solutions":["Add the Hadoop filesystem connector jar for that scheme to Flink's lib/ directory on ALL nodes (e.g. hadoop-aws for s3a, hadoop-azure for abfs) and restart","Prefer Flink's native filesystem plugins (flink-s3-fs-hadoop, flink-oss-fs-hadoop, etc.) placed in lib/ or opt/ instead of raw Hadoop jars","Set fs.<scheme>.impl (and fs.<scheme>.impl.kerberos...) in flink-conf.yaml / core-site.xml if the implementation class exists but is not auto-registered","Verify the jar is not shaded with unrelocated META-INF/services entries that break the Hadoop FileSystem service loader"],"exampleFix":"# before: scheme 's3a' unsupported\n# after: copy plugin and restart\n cp opt/flink-s3-fs-hadoop* lib/ && ./bin/start-cluster.sh","handlingStrategy":"validation","validationCode":"// pre-flight: can Hadoop resolve the scheme?\ntry {\n    org.apache.hadoop.fs.FileSystem.getFileSystemClass(\n            uri.getScheme(), new org.apache.hadoop.conf.Configuration());\n} catch (IOException e) {\n    throw new IllegalStateException(\"Add the connector jar for scheme \" + uri.getScheme(), e);\n}","typeGuard":null,"tryCatchPattern":"try {\n    fs = FileSystem.get(uri);\n} catch (UnsupportedFileSystemSchemeException e) {\n    // missing connector jar: fail fast with setup instructions, not a retry\n}","preventionTips":["Install the scheme's filesystem jar in lib/ on every node as part of provisioning","Validate checkpoint/output URIs against installed plugins at deployment time","Prefer Flink-native filesystem plugins over ad-hoc Hadoop jars"],"tags":["hdfs","classpath","filesystem-scheme","configuration"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}