What causes "Error adding Hadoop resource {}, resource was not added" in apache/seatunnel?

Verify hadoopConfPath files exist and are readable by the SeaTunnel process user (ls -l, check permissions).

Error adding Hadoop resource {}, resource was not added

Error message

Error adding Hadoop resource {}, resource was not added

What it means

PaimonSecurityContext.loadHadoopConfig scans the configured Hadoop conf directory and calls configuration.addResource for each known conf file. If reading a file's URL throws an IOException, the resource is skipped with this warning instead of failing the job, so Hadoop config may be partially loaded.

Source

Thrown at seatunnel-connectors-v2/connector-paimon/src/main/java/org/apache/seatunnel/connectors/seatunnel/paimon/security/PaimonSecurityContext.java:84

     *
     * @return
     */
    public static PaimonHadoopConfiguration loadHadoopConfig(PaimonConfig paimonConfig) {
        PaimonHadoopConfiguration configuration = new PaimonHadoopConfiguration();
        String hdfsSitePath = paimonConfig.getHdfsSitePath();
        if (StringUtils.isNotBlank(hdfsSitePath)) {
            configuration.addResource(new Path(hdfsSitePath));
        }
        String hadoopConfPath = paimonConfig.getHadoopConfPath();
        if (StringUtils.isNotBlank(hadoopConfPath)) {
            HADOOP_CONF_FILES.forEach(
                    confFile -> {
                        java.nio.file.Path path = Paths.get(hadoopConfPath, confFile);
                        if (Files.exists(path)) {
                            try {
                                configuration.addResource(path.toUri().toURL());
                            } catch (IOException e) {
                                log.warn(
                                        "Error adding Hadoop resource {}, resource was not added",
                                        path,
                                        e);
                            }
                        }
                    });
        }
        paimonConfig.getHadoopConfProps().forEach((k, v) -> configuration.set(k, v));
        // This configuration is enabled to avoid affecting other hadoop filesystem jobs
        // refer:
        // org.apache.seatunnel.connectors.seatunnel.file.hadoop.HadoopFileSystemProxy.createConfiguration
        configuration.setBoolean(FS_DISABLE_CACHE, true);
        log.info("Hadoop config initialized: {}", configuration.getClass().getName());
        return configuration;
    }

    /**
     * Check if we need to verify kerberos authentication

View on GitHub (pinned to cf67b549a7)

Solutions

Verify hadoopConfPath files exist and are readable by the SeaTunnel process user (ls -l, check permissions).
Check that the path is a plain local filesystem path, not an unsupported scheme that produces a bad URL.
Re-check Kerberos/HDFS access if the resource lives on a shared mount; retry the job after fixing the mount.
If config is truly partial, fail fast: set the conf path correctly or embed needed Hadoop properties in job config instead of relying on directory scan.

Example fix

// before
if (Files.exists(path)) { try { configuration.addResource(path.toUri().toURL()); } catch (IOException e) { log.warn(...); } }
// after
if (Files.isReadable(path)) { try { configuration.addResource(path.toUri().toURL()); } else { log.error("Hadoop conf file {} is missing or unreadable", path); throw new IOException(path.toString()); } }

Defensive patterns

Strategy: validation

Validate before calling

java.nio.file.Path p = java.nio.file.Paths.get(hadoopConfPath, confFile);
if (!java.nio.file.Files.isReadable(p)) throw new IllegalStateException("Unreadable Hadoop conf: " + p);

Prevention

Check file readability of all files in hadoopConfPath at job startup
Use plain local paths for hadoop conf directory
Avoid regenerating conf files while jobs start

When it happens

Trigger: addResource(path.toUri().toURL()) throws IOException while adding core-site.xml/hdfs-site.xml etc. from hadoopConfPath — typically because the file was deleted between the Files.exists check and the URL open, the URL is unreadable, or a filesystem/permission error occurs.

Common situations: HADOOP_HOME/conf directory mounted into a container but file unreadable by the job user; race with config file regeneration; network filesystem hiccup; wrong hadoop-conf-dir pointing at a stale mount.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/b383a2f0ac3a45d3. Report an issue: GitHub.