{"record":{"id":"a177d0586652e589","repo":"juicedata/juicefs","slug":"problem-loading-configuration-s","errorCode":null,"errorMessage":"Problem loading configuration: %s","messagePattern":"Problem loading configuration: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/object/hdfs.go","lineNumber":303,"sourceCode":"\nfunc (h *hdfsclient) Chmod(key string, mode os.FileMode) error {\n\treturn h.c.Chmod(h.path(key), mode)\n}\n\nfunc (h *hdfsclient) Chown(key string, owner, group string) error {\n\tif owner == \"root\" {\n\t\towner = superuser\n\t}\n\tif group == \"root\" {\n\t\tgroup = supergroup\n\t}\n\treturn h.c.Chown(h.path(key), owner, group)\n}\n\nfunc newHDFS(addr, username, sk, token string) (ObjectStorage, error) {\n\tconf, err := hadoopconf.LoadFromEnvironment()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"Problem loading configuration: %s\", err)\n\t}\n\n\trpcAddr, basePath := parseHDFSAddr(addr, conf)\n\toptions := hdfs.ClientOptionsFromConf(conf)\n\tif addr != \"\" {\n\t\toptions.Addresses = rpcAddr\n\t\tlogger.Infof(\"HDFS Addresses: %s, basePath: %s\", rpcAddr, basePath)\n\t}\n\n\tif options.KerberosClient != nil {\n\t\toptions.KerberosClient, err = getKerberosClient()\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"Problem with kerberos authentication: %s\", err)\n\t\t}\n\t} else {\n\t\tif username == \"\" {\n\t\t\tusername = os.Getenv(\"HADOOP_USER_NAME\")\n\t\t}","sourceCodeStart":285,"sourceCodeEnd":321,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/object/hdfs.go#L285-L321","documentation":"newHDFS builds an HDFS object-storage client from Hadoop configuration loaded via hadoopconf.LoadFromEnvironment() (core-site.xml/hdfs-site.xml via HADOOP_CONF_DIR or similar). When that loader fails — unreadable files or malformed XML — the client creation is aborted and the underlying loader error is wrapped as \"Problem loading configuration: %s\". This is a startup-time configuration failure, not a runtime data-path error.","triggerScenarios":"Calling newHDFS (e.g. via juicefs format/mount with an hdfs:// address, or the object-storage tests TestHDFS/TestHDFS2/TestMarsharl) when HADOOP_CONF_DIR is unset, points to a nonexistent directory, or contains core-site.xml/hdfs-site.xml files that are unreadable or invalid XML.","commonSituations":"Running JuiceFS on a node that is not part of the Hadoop cluster so no Hadoop conf dir exists; HADOOP_CONF_DIR typo pointing at the wrong path; partially written or corrupt XML configs; permission problems reading /etc/hadoop/conf.","solutions":["Set HADOOP_CONF_DIR to the directory containing valid core-site.xml and hdfs-site.xml (e.g. export HADOOP_CONF_DIR=/etc/hadoop/conf).","Verify the files are readable by the process user: ls -l $HADOOP_CONF_DIR/core-site.xml and check XML well-formedness (xmllint).","If configs live elsewhere, copy or symlink them into the configured directory and retry.","As a last resort pass the minimum required properties (fs.defaultFS etc.) via a minimal core-site.xml."],"exampleFix":"// before\n$ ./juicefs format hdfs://mynamenode:9000/data jfs   # HADOOP_CONF_DIR unset\nProblem loading configuration: open /etc/hadoop/conf/core-site.xml: no such file or directory\n// after\n$ export HADOOP_CONF_DIR=/etc/hadoop/conf\n$ ./juicefs format hdfs://mynamenode:9000/data jfs","handlingStrategy":"validation","validationCode":"confDir := os.Getenv(\"HADOOP_CONF_DIR\")\nif confDir == \"\" {\n    confDir = \"/etc/hadoop/conf\"\n}\nfor _, f := range []string{\"core-site.xml\", \"hdfs-site.xml\"} {\n    if _, err := os.Stat(filepath.Join(confDir, f)); err != nil {\n        return fmt.Errorf(\"Hadoop config %s missing in %s: %w\", f, confDir, err)\n    }\n}","typeGuard":null,"tryCatchPattern":"os, err := object.CreateStorage(\"hdfs\", addr, \"\", \"\")\nif err != nil && strings.Contains(err.Error(), \"Problem loading configuration\") {\n    logger.Fatalf(\"HDFS setup failed: %v — set HADOOP_CONF_DIR to a dir with valid core-site.xml/hdfs-site.xml\", err)\n}","preventionTips":["Set HADOOP_CONF_DIR explicitly in deployment manifests/Dockerfiles.","Validate XML configs with xmllint in CI before deploying.","Run the JuiceFS process as a user with read access to the Hadoop conf directory.","Bake the Hadoop conf files into the container image."],"tags":["hdfs","configuration","object-storage","startup"],"backgroundTag":"config-file-not-found","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}