{"record":{"id":"b74fb973fce8ff7f","repo":"apache/seatunnel","slug":"error-adding-hadoop-resource-resource-was-not-b74fb9","errorCode":null,"errorMessage":"Error adding Hadoop resource {}, resource was not added","messagePattern":"Error adding Hadoop resource (.+?), resource was not added","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/IcebergCatalogLoader.java","lineNumber":103,"sourceCode":"            Object result = configClass.getDeclaredConstructor().newInstance();\n            DynMethods.BoundMethod addResourceMethod =\n                    DynMethods.builder(\"addResource\").impl(configClass, URL.class).build(result);\n            DynMethods.BoundMethod setMethod =\n                    DynMethods.builder(\"set\")\n                            .impl(configClass, String.class, String.class)\n                            .build(result);\n\n            //  load any config files in the specified config directory\n            String hadoopConfPath = config.getHadoopConfPath();\n            if (hadoopConfPath != null) {\n                HADOOP_CONF_FILES.forEach(\n                        confFile -> {\n                            Path path = Paths.get(hadoopConfPath, confFile);\n                            if (Files.exists(path)) {\n                                try {\n                                    addResourceMethod.invoke(path.toUri().toURL());\n                                } catch (IOException e) {\n                                    log.warn(\n                                            \"Error adding Hadoop resource {}, resource was not added\",\n                                            path,\n                                            e);\n                                }\n                            }\n                        });\n            }\n            config.getHadoopProps().forEach(setMethod::invoke);\n            // kerberos authentication\n            doKerberosLogin((Configuration) result);\n            log.info(\"Hadoop config initialized: {}\", configClass.getName());\n            return result;\n        } catch (InstantiationException\n                | IllegalAccessException\n                | NoSuchMethodException\n                | InvocationTargetException e) {\n            log.warn(\n                    \"Hadoop found on classpath but could not create config, proceeding without config\",","sourceCodeStart":85,"sourceCodeEnd":121,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/IcebergCatalogLoader.java#L85-L121","documentation":"This warning comes from IcebergCatalogLoader.loadHadoopConfig() when adding a Hadoop configuration resource file from the configured hadoop conf directory fails with an IOException via reflection (addResourceMethod.invoke). The catalog is still created, but that particular XML/config resource is not loaded, so any settings it carries (e.g. fs defaults, kerberos settings) are missing from the Hadoop Configuration object.","triggerScenarios":"hadoop-conf-path is set and contains a conf file (e.g. core-site.xml, hdfs-site.xml); Files.exists(path) is true, but invoking Configuration.addResource(URL) on the file throws IOException during reflection.","commonSituations":"Unreadable files due to permissions; malformed or special files that the Hadoop resource loader cannot open via URL; network-mounted conf dirs that briefly fail; wrong file types placed in the conf directory.","solutions":["Check the file's readability/permissions for the process user and that it is a valid Hadoop XML config.","Check the attached exception (logged with this warning) for the root cause; fix the file or path accordingly.","Alternatively inline the needed settings (fs.defaultFS, kerberos keys) into iceberg sink/catalog options or a known-good core-site.xml and remove the problematic file."],"exampleFix":"// before\n# hadoop-conf-path points to /etc/hadoop/conf containing a broken hdfs-site.xml\n// after\nls -l /etc/hadoop/conf/*.xml && xmllint --noout /etc/hadoop/conf/hdfs-site.xml  # fix or remove the broken file","handlingStrategy":"try-catch","validationCode":"for (String f : List.of(\"core-site.xml\", \"hdfs-site.xml\")) {\n    Path p = Paths.get(hadoopConfPath, f);\n    if (Files.exists(p) && !Files.isReadable(p)) throw new IllegalStateException(\"Unreadable conf: \" + p);\n}","typeGuard":null,"tryCatchPattern":"try {\n    addResourceMethod.invoke(path.toUri().toURL());\n} catch (Exception e) {\n    log.error(\"Failed to add Hadoop resource {} - required settings may be missing\", path, e);\n    throw new IllegalStateException(\"Hadoop config resource load failed\", e);\n}","preventionTips":["Ensure conf files are valid XML and readable by the job process user","Keep hadoop conf on local disk, not flaky network mounts","Check the cause exception attached to this warning to find the root failure"],"tags":["hadoop","iceberg","config","io"],"backgroundTag":"config-file-not-found","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}