What causes "Error adding Hadoop config {}" in apache/seatunnel?

Inspect the logged exception for the offending path and fix permissions or special characters

Error adding Hadoop config {}

Error message

Error adding Hadoop config {}

What it means

HiveMetaStoreCatalog.buildHiveConf adds each Hadoop conf XML (core-site.xml, hdfs-site.xml, etc.) from the configured hadoop conf directory to the HiveConf. If converting an existing file's URL fails with IOException, that file is skipped with this warning; resulting HiveConf may lack critical properties, breaking metastore connectivity.

Source

Thrown at seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/utils/HiveMetaStoreCatalog.java:382

            if (StringUtils.isNotBlank(normalizedMetastoreUris)) {
                hiveConf.set("hive.metastore.uris", normalizedMetastoreUris);
            }
        }
        hiveConf.setBoolVar(HiveConf.ConfVars.METASTORE_EXECUTE_SET_UGI, false);
        hiveConf.setBoolean("hive.metastore.client.capability.check", false);
        hiveConf.setBoolean("hive.metastore.client.filter.enabled", false);
        hiveConf.setInt("hive.metastore.client.socket.timeout", 600);
        hiveConf.setInt("hive.metastore.client.connect.retry.delay", 5);
        hiveConf.setInt("hive.metastore.failure.retries", 3);

        if (StringUtils.isNotBlank(hadoopConfDir)) {
            for (String fileName : HADOOP_CONF_FILES) {
                Path path = Paths.get(hadoopConfDir, fileName);
                if (Files.exists(path)) {
                    try {
                        hiveConf.addResource(path.toUri().toURL());
                    } catch (IOException e) {
                        log.warn("Error adding Hadoop config {}", path, e);
                    }
                }
            }
        }
        if (StringUtils.isNotBlank(hiveSitePath)) {
            try {
                hiveConf.addResource(new File(hiveSitePath).toURI().toURL());
            } catch (MalformedURLException e) {
                log.warn("Invalid hiveSitePath {}", hiveSitePath, e);
            }
        }
        hadoopProperties.forEach(hiveConf::set);
        log.debug("Hive client configuration initialized");
        return hiveConf;
    }

    private IMetaStoreClient loginWithKerberos(HiveConf hiveConf) throws Exception {
        Configuration authConf = new Configuration();

View on GitHub (pinned to cf67b549a7)

Solutions

Inspect the logged exception for the offending path and fix permissions or special characters
Point 'hadoop_conf_dir' at a clean, readable directory containing valid XML files
Validate each XML file loads (e.g. try reading it with Hadoop's Configuration in a test)
Set required properties explicitly via hadoopProperties map if files can't be loaded

Example fix

// before
Path path = Paths.get("/etc/hadoop conf/", fileName); // space breaks URL
// after
Path path = Paths.get("/etc/hadoop/conf", fileName);

Defensive patterns

Strategy: validation

Validate before calling

for (String f : List.of("core-site.xml","hdfs-site.xml")) {
  java.nio.file.Path p = Paths.get(hadoopConfDir, f);
  if (!java.nio.file.Files.isRegularFile(p)) continue;
  new Configuration().addResource(p.toUri().toURL()); // throws early if URL malformed
}

Try / catch

try {
  hiveConf.addResource(path.toUri().toURL());
} catch (IOException e) {
  log.warn("Cannot add hadoop conf {}", path, e);
}

Prevention

Use absolute hadoop_conf_dir without special characters
Verify all expected conf XMLs exist and load before submitting
Prefer classpath-based conf loading when possible
Set critical keys explicitly via hadoopProperties to survive skipped files

When it happens

Trigger: buildHiveConf() (called from initializeClient) iterates HADOOP_CONF_FILES; the file exists but path.toUri().toURL() throws IOException — malformed path, permission/IO error, or unsupported URI characters.

Common situations: hadoop_conf_dir with spaces/illegal chars, files on a flaky mount, conf directory pointing to the wrong location so only some files resolve.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/a5a8a391eae3941d. Report an issue: GitHub.