{"record":{"id":"1aab4fa5d833ac38","repo":"apache/iceberg","slug":"error-adding-hadoop-resource-resource-was-not","errorCode":null,"errorMessage":"Error adding Hadoop resource {}, resource was not added","messagePattern":"Error adding Hadoop resource (.+?), resource was not added","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/CatalogUtils.java","lineNumber":79,"sourceCode":"\n    try {\n      Object result = DynConstructors.builder().hiddenImpl(configClass).build().newInstance();\n      BoundMethod addResourceMethod =\n          DynMethods.builder(\"addResource\").impl(configClass, URL.class).build(result);\n      BoundMethod setMethod =\n          DynMethods.builder(\"set\").impl(configClass, String.class, String.class).build(result);\n\n      //  load any config files in the specified config directory\n      String hadoopConfDir = config.hadoopConfDir();\n      if (hadoopConfDir != null) {\n        HADOOP_CONF_FILES.forEach(\n            confFile -> {\n              Path path = Paths.get(hadoopConfDir, confFile);\n              if (Files.exists(path)) {\n                try {\n                  addResourceMethod.invoke(path.toUri().toURL());\n                } catch (IOException e) {\n                  LOG.warn(\"Error adding Hadoop resource {}, resource was not added\", path, e);\n                }\n              }\n            });\n      }\n\n      // set any Hadoop properties specified in the sink config\n      config.hadoopProps().forEach(setMethod::invoke);\n\n      LOG.info(\"Hadoop config initialized: {}\", configClass.getName());\n      return result;\n    } catch (Exception e) {\n      LOG.warn(\n          \"Hadoop found on classpath but could not create config, proceeding without config\", e);\n    }\n    return null;\n  }\n\n  private CatalogUtils() {}","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/CatalogUtils.java#L61-L97","documentation":"CatalogUtils.loadHadoopConfig reflectively adds files from the configured hadoopConfDir to the Hadoop Configuration via conf.addResource(URL). If adding a specific resource file throws IOException, that file is skipped with this warning; the rest of the config initialization continues.","triggerScenarios":"hadoopConfDir contains a file listed to be added that cannot be read/loaded — unreadable file permissions, malformed URL conversion, file disappearing between Files.exists check and invoke.","commonSituations":"Kubernetes mounts partially populated; wrong hadoop-conf-dir path; filename casing not matching hadoop-core-site.xml/hdfs-site.xml conventions; restrictive container filesystems.","solutions":["Verify the hadoopConfDir path and that referenced config files exist and are readable (chmod/chown)","Ensure the Connect worker process has filesystem read permissions for the config directory","Check the file is a valid Hadoop XML config; corrupt files can fail resource loading","Mount configs correctly in the container and restart the Connect worker"],"exampleFix":"// before\n\"hadoop-conf-dir\": \"/etc/hadoop/confTypo\"\n// after\n\"hadoop-conf-dir\": \"/etc/hadoop/conf\"  // contains core-site.xml, hdfs-site.xml, readable by worker","handlingStrategy":"validation","validationCode":"Path dir = Paths.get(hadoopConfDir);\nif (!Files.isDirectory(dir) || !Files.isReadable(dir)) throw new IllegalStateException(\"hadoop-conf-dir unreadable: \" + dir);\ntry (var s = Files.list(dir)) { s.forEach(p -> { if (!Files.isReadable(p)) LOG.warn(\"unreadable config file: \" + p); }); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate hadoop-conf-dir exists, is a directory, and is readable by the worker user","Use canonical filenames (core-site.xml, hdfs-site.xml)","Verify container volume mounts before starting Connect"],"tags":["hadoop","kafka-connect","configuration","file"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}