juicedata/juicefs · error

new HDFS client %s: %s

Error message

new HDFS client %s: %s

What it means

newHDFS finalizes by calling hdfs.NewClient(options) to establish the RPC connection to the NameNode. If the client cannot be constructed — unreachable NameNode, bad addresses, protocol/ negotiation failure — the error is wrapped as "new HDFS client %s: %s" with the resolved RPC addresses. Client object creation succeeded up to this point; this is a connectivity or cluster-level failure.

Source

Thrown at pkg/object/hdfs.go:334

			return nil, fmt.Errorf("Problem with kerberos authentication: %s", err)
		}
	} else {
		if username == "" {
			username = os.Getenv("HADOOP_USER_NAME")
		}
		if username == "" {
			current, err := user.Current()
			if err != nil {
				return nil, fmt.Errorf("get current user: %s", err)
			}
			username = current.Username
		}
		options.User = username
	}

	c, err := hdfs.NewClient(options)
	if err != nil {
		return nil, fmt.Errorf("new HDFS client %s: %s", rpcAddr, err)
	}
	if os.Getenv("HADOOP_SUPER_USER") != "" {
		superuser = os.Getenv("HADOOP_SUPER_USER")
	}
	if os.Getenv("HADOOP_SUPER_GROUP") != "" {
		supergroup = os.Getenv("HADOOP_SUPER_GROUP")
	}

	var replication = 3
	if v, found := conf["dfs.replication"]; found {
		if x, err := strconv.Atoi(v); err == nil {
			replication = x
		}
	}
	var umask uint16 = 022
	if v, found := conf["fs.permissions.umask-mode"]; found {
		if x, err := strconv.ParseUint(v, 8, 16); err == nil {
			umask = uint16(x)

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Check the wrapped inner error and test connectivity to the RPC address: nc -vz namenode 8020 (or the printed rpcAddr).
2. Verify fs.defaultFS in core-site.xml and the hdfs:// address match the actual NameNode and port.
3. Confirm the NameNode is up and out of safe mode (hdfs dfsadmin -safemode get).
4. Fix DNS/hostname resolution (add entries to /etc/hosts) if the error is a lookup failure.

Example fix

// before
$ ./juicefs mount hdfs://nn:9000/data /mnt/jfs
new HDFS client nn:9000: dial tcp: lookup nn: no such host
// after
$ grep fs.defaultFS $HADOOP_CONF_DIR/core-site.xml   # confirm host/port
$ echo "10.0.0.5 nn" | sudo tee -a /etc/hosts
$ ./juicefs mount hdfs://nn:9000/data /mnt/jfs

host, port := "namenode", 8020
conn, err := net.DialTimeout("tcp", net.JoinHostPort(host, port), 3*time.Second)
if err != nil {
    return fmt.Errorf("NameNode %s unreachable: %w", net.JoinHostPort(host, port), err)
}
conn.Close()

os, err := object.CreateStorage("hdfs", addr, "", "")
if err != nil && strings.Contains(err.Error(), "new HDFS client") {
    // transient NameNode unavailability: retry with backoff
    for i := 0; i < 5; i++ {
        time.Sleep(time.Duration(1<<i) * time.Second)
        if os, err = object.CreateStorage("hdfs", addr, "", ""); err == nil {
            break
        }
    }
}

When it happens

Trigger: newHDFS called when the NameNode host:port from the hdfs:// addr / core-site.xml (fs.defaultFS) is unreachable, DNS fails, the cluster is down, or client options (user, kerberos) are rejected during connection setup.

Common situations: NameNode in safe mode or down; firewall blocking RPC port 8020/9000; wrong fs.defaultFS; hostname resolution failure inside containers; version incompatibility between client and cluster.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• Failed to create bucket %s: %s, previous error: %s
• (max tries) upload block %s: %s (after %d tries)
• get %s: %s
• Unable to get endpoint of container %s: %s
• Can't create connection to cluster %s for user %s: %s

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/54fbe6f86d4eb7ef. Report an issue: GitHub.