Hadoop found on classpath but could not create config, proce

Error message

Hadoop found on classpath but could not create config, proceeding without config

What it means

This warning is emitted by IcebergCatalogLoader.loadHadoopConfig() when Hadoop classes are present on the classpath but instantiating the Hadoop Configuration object (reflectively, e.g. new Configuration()) fails with InstantiationException, IllegalAccessException, NoSuchMethodException, or InvocationTargetException. The loader proceeds with a null config, so the catalog runs without Hadoop settings — often breaking HDFS-backed catalogs.

Source

Thrown at seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/IcebergCatalogLoader.java:120

                                } catch (IOException e) {
                                    log.warn(
                                            "Error adding Hadoop resource {}, resource was not added",
                                            path,
                                            e);
                                }
                            }
                        });
            }
            config.getHadoopProps().forEach(setMethod::invoke);
            // kerberos authentication
            doKerberosLogin((Configuration) result);
            log.info("Hadoop config initialized: {}", configClass.getName());
            return result;
        } catch (InstantiationException
                | IllegalAccessException
                | NoSuchMethodException
                | InvocationTargetException e) {
            log.warn(
                    "Hadoop found on classpath but could not create config, proceeding without config",
                    e);
        }
        return null;
    }

    /**
     * kerberos authentication
     *
     * @param configuration Configuration
     */
    private Configuration doKerberosLogin(Configuration configuration) {
        String kerberosKrb5ConfPath = config.getKerberosKrb5ConfPath();
        String kerberosKeytabPath = config.getKerberosKeytabPath();
        String kerberosPrincipal = config.getKerberosPrincipal();

        if (StringUtils.isNotEmpty(kerberosPrincipal)
                && StringUtils.isNotEmpty(kerberosKrb5ConfPath)

View on GitHub (pinned to cf67b549a7)

Solutions

Check the attached exception stack trace to identify the reflective failure (usually ClassNotFound/NoSuchMethod from version conflicts).
Align Hadoop versions: use a single consistent hadoop-client set matching your cluster, and avoid multiple hadoop-common jars on the classpath.
As a fallback, pass required Hadoop settings explicitly via connector options (e.g. catalog impl, warehouse, fs settings) that do not depend on the config object.

Example fix

// before
# $SEATUNNEL_HOME/lib contains hadoop-common-2.6.0.jar and hadoop-client-api-3.3.4.jar
// after
# remove the older conflicting jar
rm $SEATUNNEL_HOME/lib/hadoop-common-2.6.0.jar
sh bin/seatunnel.sh --config job.config -e local

Defensive patterns

Strategy: try-catch

Validate before calling

URL[] jars = classpath hadoop-common jars;
if (distinctVersions(jars).count() > 1) throw new IllegalStateException("Conflicting hadoop-common versions on classpath");

Try / catch

try {
    Configuration conf = new Configuration();
} catch (Throwable e) {
    log.error("Hadoop Configuration creation failed - check for hadoop jar conflicts", e);
    throw e;
}

Prevention

Keep exactly one consistent Hadoop version in $SEATUNNEL_HOME/lib and plugin dirs
Avoid mixing hadoop-common and shaded hadoop-client-api jars
Pin Hadoop jars to match your cluster version
Inspect the reflective exception stack trace when this warning appears

When it happens

Trigger: Reflection-based construction of the Hadoop Configuration class throws one of the caught reflective exceptions inside loadHadoopConfig(), called from loadCatalog().

Common situations: Mixed/shaded Hadoop versions on the classpath (hadoop-common vs shaded client conflicts); Configuration constructor signature changes across Hadoop versions; fat jars with conflicting hadoop-common copies; incomplete plugin jars missing hadoop dependencies.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/6d680b1a27365178. Report an issue: GitHub.