mount issued for NFS V3 but unable to run rpcbind: Output:

Error message

mount issued for NFS V3 but unable to run rpcbind:
 Output: %s
 Error: %v

What it means

Returned by mountInChroot (mounter.go:83) when the original mount failed with the NFS V3 rpc.statd-not-running message, so the mounter attempted to start rpcbind inside the chroot ('/sbin/rpcbind -w'), and rpcbind itself failed. The output and error from the rpcbind invocation are surfaced.

Source

Thrown at cluster/gce/gci/mounter/mounter.go:83

func mountInChroot(rootfsPath string, args []string) error {
	if _, err := os.Stat(rootfsPath); os.IsNotExist(err) {
		return fmt.Errorf("path <%s> does not exist", rootfsPath)
	}
	args = append([]string{rootfsPath, mountBin}, args...)
	output, err := exec.Command(chrootCmd, args...).CombinedOutput()
	if err == nil {
		return nil
	}

	if !strings.EqualFold(string(output), nfsRPCBindErrMsg) {
		// Mount failed but not because of RPC bind error
		return fmt.Errorf("mount failed: %v\nMounting command: %s\nMounting arguments: %v\nOutput: %s", err, chrootCmd, args, string(output))
	}

	// Mount failed because it is NFS V3 and we need to run rpcBind
	output, err = exec.Command(chrootCmd, rootfsPath, rpcBindCmd, "-w").CombinedOutput()
	if err != nil {
		return fmt.Errorf("mount issued for NFS V3 but unable to run rpcbind:\n Output: %s\n Error: %v", string(output), err)
	}

	// Rpcbind is running, try mounting again
	output, err = exec.Command(chrootCmd, args...).CombinedOutput()

	if err != nil {
		return fmt.Errorf("mount failed for NFS V3 even after running rpcBind %s, %v", string(output), err)
	}

	return nil
}

View on GitHub (pinned to b882c60b40)

Solutions

Ensure /sbin/rpcbind and its shared libs exist in /home/kubernetes/containerized_mounter/rootfs
Read the Error and Output fields for the rpcbind failure (missing lib, permission denied, port in use)
Free port 111 on the host if another portmapper is running, or run rpcbind outside the chroot
Rebuild the containerized mounter rootfs image to include rpcbind

Example fix

# before: rootfs lacks rpcbind
chroot rootfs /sbin/rpcbind -w  # No such file

# after: include rpcbind in the rootfs build
# (in the rootfs build script)
apt-get install -y rpcbind
install -Dm0755 /usr/sbin/rpcbind rootfs/sbin/rpcbind
ldd /usr/sbin/rpcbind | awk '{print $3}' | xargs -I{} cp --parents {} rootfs/

Defensive patterns

Strategy: validation

Validate before calling

// Ensure rpcbind is present in the rootfs before attempting NFS mounts.
func ensureRpcbind(rootfsPath string) error {
    p := filepath.Join(rootfsPath, "sbin", "rpcbind")
    if _, err := os.Stat(p); err != nil {
        return fmt.Errorf("rpcbind missing at %s; rebuild the mounter rootfs to include it", p)
    }
    return nil
}

Try / catch

// Differentiate rpcbind-startup failure from other mount failures.
if err := mountInChroot(rootfs, args); err != nil {
    if strings.Contains(err.Error(), "unable to run rpcbind") {
        // rootfs packaging problem: surface as actionable
        return fmt.Errorf("rpcbind unavailable in mounter rootfs: %w", err)
    }
    return err
}

Prevention

Include rpcbind and its shared libraries when building the containerized mounter rootfs
CI-test NFS V3 mounts on the node image so rpcbind regressions surface before deploy
Document that the mounter rootfs must ship /sbin/rpcbind for NFS V3 support

When it happens

Trigger: rpcbind binary missing or broken in the containerized mounter rootfs; rpcbind needs capabilities/ports unavailable in the chroot; another portmapper already bound port 111; shared library missing for rpcbind.

Common situations: mounter rootfs image built without rpcbind or its dependencies; older GCI image; port 111 already held by host portmap; SELinux denying rpcbind.

Related errors

AI-assisted analysis of kubernetes/kubernetes@b882c60b40 (2026-08-07). Data as JSON: /api/errors/f5e9fe74cef8e16d. Report an issue: GitHub.