juicedata/juicefs · error

Problem loading configuration: %s

Error message

Problem loading configuration: %s

What it means

newHDFS builds an HDFS object-storage client from Hadoop configuration loaded via hadoopconf.LoadFromEnvironment() (core-site.xml/hdfs-site.xml via HADOOP_CONF_DIR or similar). When that loader fails — unreadable files or malformed XML — the client creation is aborted and the underlying loader error is wrapped as "Problem loading configuration: %s". This is a startup-time configuration failure, not a runtime data-path error.

Source

Thrown at pkg/object/hdfs.go:303

func (h *hdfsclient) Chmod(key string, mode os.FileMode) error {
	return h.c.Chmod(h.path(key), mode)
}

func (h *hdfsclient) Chown(key string, owner, group string) error {
	if owner == "root" {
		owner = superuser
	}
	if group == "root" {
		group = supergroup
	}
	return h.c.Chown(h.path(key), owner, group)
}

func newHDFS(addr, username, sk, token string) (ObjectStorage, error) {
	conf, err := hadoopconf.LoadFromEnvironment()
	if err != nil {
		return nil, fmt.Errorf("Problem loading configuration: %s", err)
	}

	rpcAddr, basePath := parseHDFSAddr(addr, conf)
	options := hdfs.ClientOptionsFromConf(conf)
	if addr != "" {
		options.Addresses = rpcAddr
		logger.Infof("HDFS Addresses: %s, basePath: %s", rpcAddr, basePath)
	}

	if options.KerberosClient != nil {
		options.KerberosClient, err = getKerberosClient()
		if err != nil {
			return nil, fmt.Errorf("Problem with kerberos authentication: %s", err)
		}
	} else {
		if username == "" {
			username = os.Getenv("HADOOP_USER_NAME")
		}

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Set HADOOP_CONF_DIR to the directory containing valid core-site.xml and hdfs-site.xml (e.g. export HADOOP_CONF_DIR=/etc/hadoop/conf).
2. Verify the files are readable by the process user: ls -l $HADOOP_CONF_DIR/core-site.xml and check XML well-formedness (xmllint).
3. If configs live elsewhere, copy or symlink them into the configured directory and retry.
4. As a last resort pass the minimum required properties (fs.defaultFS etc.) via a minimal core-site.xml.

Example fix

// before
$ ./juicefs format hdfs://mynamenode:9000/data jfs   # HADOOP_CONF_DIR unset
Problem loading configuration: open /etc/hadoop/conf/core-site.xml: no such file or directory
// after
$ export HADOOP_CONF_DIR=/etc/hadoop/conf
$ ./juicefs format hdfs://mynamenode:9000/data jfs

confDir := os.Getenv("HADOOP_CONF_DIR")
if confDir == "" {
    confDir = "/etc/hadoop/conf"
}
for _, f := range []string{"core-site.xml", "hdfs-site.xml"} {
    if _, err := os.Stat(filepath.Join(confDir, f)); err != nil {
        return fmt.Errorf("Hadoop config %s missing in %s: %w", f, confDir, err)
    }
}

os, err := object.CreateStorage("hdfs", addr, "", "")
if err != nil && strings.Contains(err.Error(), "Problem loading configuration") {
    logger.Fatalf("HDFS setup failed: %v — set HADOOP_CONF_DIR to a dir with valid core-site.xml/hdfs-site.xml", err)
}

When it happens

Trigger: Calling newHDFS (e.g. via juicefs format/mount with an hdfs:// address, or the object-storage tests TestHDFS/TestHDFS2/TestMarsharl) when HADOOP_CONF_DIR is unset, points to a nonexistent directory, or contains core-site.xml/hdfs-site.xml files that are unreadable or invalid XML.

Common situations: Running JuiceFS on a node that is not part of the Hadoop cluster so no Hadoop conf dir exists; HADOOP_CONF_DIR typo pointing at the wrong path; partially written or corrupt XML configs; permission problems reading /etc/hadoop/conf.

Understand the failure class

Background: "Config file not found": what it means and how to fix it in docker-sync, Maven, Vagrant, Turborepo and other tools — this error's family across 60 libraries.

Related errors

• object storage: %s
• object storage: %s
• Invalid endpoint: %v, error: %v
• Invalid endpoint: %v
• Invalid endpoint %s: %s

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/a177d0586652e589. Report an issue: GitHub.