{"record":{"id":"d4ee0506abd2b938","repo":"apache/flink","slug":"failed-to-initialize-googlehadoopfilesystem","errorCode":null,"errorMessage":"Failed to initialize GoogleHadoopFileSystem","messagePattern":"Failed to initialize GoogleHadoopFileSystem","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-gs-fs-hadoop/src/main/java/org/apache/flink/fs/gs/GSFileSystemFactory.java","lineNumber":156,"sourceCode":"    }\n\n    @Override\n    public String getScheme() {\n        return SCHEME;\n    }\n\n    @Override\n    public FileSystem create(URI fsUri) throws IOException {\n        LOGGER.info(\"Creating GSFileSystem for uri {} with options {}\", fsUri, fileSystemOptions);\n\n        Preconditions.checkNotNull(fsUri);\n\n        // create the Google Hadoop file system\n        GoogleHadoopFileSystem googleHadoopFileSystem = new GoogleHadoopFileSystem();\n        try {\n            googleHadoopFileSystem.initialize(fsUri, hadoopConfig);\n        } catch (IOException ex) {\n            throw new IOException(\"Failed to initialize GoogleHadoopFileSystem\", ex);\n        }\n\n        // create the file system\n        return new GSFileSystem(googleHadoopFileSystem, storage, fileSystemOptions);\n    }\n\n    @VisibleForTesting\n    Storage getStorage() {\n        return storage;\n    }\n\n    /** Config context implementation used at runtime. */\n    private static class RuntimeConfigContext implements ConfigUtils.ConfigContext {\n\n        @Override\n        public Optional<String> getenv(String name) {\n            return Optional.ofNullable(System.getenv(name));\n        }","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-gs-fs-hadoop/src/main/java/org/apache/flink/fs/gs/GSFileSystemFactory.java#L138-L174","documentation":"Thrown by GSFileSystemFactory.create(URI) when GoogleHadoopFileSystem.initialize(fsUri, hadoopConfig) fails with an IOException. This is the entry point for the gs:// filesystem plugin, so the wrapped failure usually means the Google Cloud Storage credentials or configuration supplied to the plugin are invalid — the real reason is in the cause chain.","triggerScenarios":"First access of a gs:// path triggers plugin filesystem creation; initialize fails on bad service-account credentials (malformed JSON, wrong key), missing GCS project/bucket config, or an inaccessible GCS endpoint.","commonSituations":"Wrong value for gs.credentials.* options in flink-conf.yaml (e.g. fs.gs.auth.service.account.json.keyfile pointing at a missing file); service account without storage permissions; copying configs between environments without moving the key file.","solutions":["Read the cause exception — it carries the actual GoogleHadoopFileSystem error (auth, project, endpoint)","Verify the credential config: fs.gs.auth.service.account.json.keyfile exists, is valid JSON, and the account has roles/storage.objectAdmin on the bucket","Test the same key file with 'gsutil -o Credentials:...' or gcloud storage ls to isolate Flink from GCS","Ensure gs:// URIs include the bucket (gs://bucket/...) and the plugin jar (flink-gs-fs-hadoop) is in Flink's plugins directory"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before first gs:// access, sanity-check the credential config\nString keyfile = hadoopConf.get(\"fs.gs.auth.service.account.json.keyfile\");\nif (keyfile != null && !new java.io.File(keyfile).canRead()) {\n    throw new IllegalStateException(\"GCS key file unreadable: \" + keyfile);\n}","typeGuard":null,"tryCatchPattern":"try {\n    FileSystem.get(new URI(\"gs://bucket/\"), hadoopConf);\n} catch (java.io.IOException e) {\n    // unwrap cause: usually auth/project config failure from GoogleHadoopFileSystem\n    log.error(\"GS filesystem init failed\", e.getCause());\n}","preventionTips":["Validate GCS credentials with gcloud/gsutil before deploying Flink","Put the GCS plugin jar in plugins/ and keep credential paths absolute","Grant the service account the needed storage IAM roles up front"],"tags":["gcs","google-cloud","credentials","filesystem","initialization"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}