What causes "Error adding Hadoop resource {}, resource was not added" in apache/seatunnel?

Check the hadoop_conf_dir path for special characters or broken symlinks and fix them

Error adding Hadoop resource {}, resource was not added

Error message

Error adding Hadoop resource {}, resource was not added

What it means

AbstractStorage.loadHiveBaseHadoopConfig adds each XML file found in the configured hadoop conf directory to the Hadoop Configuration via addResource(URL). If a file exists but its URL cannot be converted/loaded (IOException), the file is skipped with this warning, so missing properties from that file may cause later failures.

Source

Thrown at seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/AbstractStorage.java:82

     *
     * @return
     */
    protected Configuration loadHiveBaseHadoopConfig(ReadonlyConfig readonlyConfig) {
        try {
            Configuration configuration = new Configuration();
            // Try to load from hadoop_conf_path(The Bucket configuration is typically in
            // core-site.xml)
            Optional<String> hadoopConfPath =
                    readonlyConfig.getOptional(HiveBaseOptions.HADOOP_CONF_PATH);
            if (hadoopConfPath.isPresent()) {
                HADOOP_CONF_FILES.forEach(
                        confFile -> {
                            java.nio.file.Path path = Paths.get(hadoopConfPath.get(), confFile);
                            if (Files.exists(path)) {
                                try {
                                    configuration.addResource(path.toUri().toURL());
                                } catch (IOException e) {
                                    log.warn(
                                            "Error adding Hadoop resource {}, resource was not added",
                                            path,
                                            e);
                                }
                            }
                        });
            }
            String hiveSitePath = readonlyConfig.get(HiveBaseOptions.HIVE_SITE_PATH);
            String hdfsSitePath = readonlyConfig.get(HiveBaseOptions.HDFS_SITE_PATH);
            if (StringUtils.isNotBlank(hdfsSitePath)) {
                configuration.addResource(new File(hdfsSitePath).toURI().toURL());
            }

            if (StringUtils.isNotBlank(hiveSitePath)) {
                configuration.addResource(new File(hiveSitePath).toURI().toURL());
            }
            // Try to load from hadoopConf
            Optional<Map<String, String>> hadoopConf =

View on GitHub (pinned to cf67b549a7)

Solutions

Check the hadoop_conf_dir path for special characters or broken symlinks and fix them
Ensure the conf files (core-site.xml, hdfs-site.xml, hive-site.xml) are readable by the job user
Re-deploy/refresh the hadoop conf directory if files were replaced concurrently
Add the required properties explicitly in the connector config if a file cannot be loaded

Example fix

// before
"hadoop_conf_path" = "/etc/hadoop conf/" // space/special chars
// after
"hadoop_conf_path" = "/etc/hadoop/conf"

Defensive patterns

Strategy: validation

Validate before calling

java.nio.file.Path p = Paths.get(hadoopConfPath, "core-site.xml");
if (!java.nio.file.Files.isRegularFile(p) || !p.toUri().isAbsolute() && p.toString().contains(" "))
  throw new IllegalArgumentException("Bad hadoop conf path: " + p);

Try / catch

try {
  configuration.addResource(path.toUri().toURL());
} catch (IOException e) {
  log.warn("Skipping unreadable hadoop conf {}", path, e);
}

Prevention

Use absolute conf paths without spaces or special characters
Check file readability with the job's user before launch
Keep conf files stable during job runs (no in-place replacement)
Explicitly set critical properties if conf-dir loading is unreliable

When it happens

Trigger: loadHiveBaseHadoopConfig() iterates hadoop conf files; Paths.get(hadoopConfPath, confFile) exists but path.toUri().toURL() throws IOException — e.g. malformed path characters, unreadable/symlinked file, or IO error constructing the URL.

Common situations: hadoop_conf_dir pointing to a mounted/flaky filesystem, unusual characters in the conf path, files replaced mid-read during deployment.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/bb490fd8504e4efa. Report an issue: GitHub.