{"record":{"id":"f5e9fe74cef8e16d","repo":"kubernetes/kubernetes","slug":"mount-issued-for-nfs-v3-but-unable-to-run-rpcbind","errorCode":null,"errorMessage":"mount issued for NFS V3 but unable to run rpcbind:\n Output: %s\n Error: %v","messagePattern":"mount issued for NFS V3 but unable to run rpcbind:\n Output: (.+?)\n Error: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cluster/gce/gci/mounter/mounter.go","lineNumber":83,"sourceCode":"func mountInChroot(rootfsPath string, args []string) error {\n\tif _, err := os.Stat(rootfsPath); os.IsNotExist(err) {\n\t\treturn fmt.Errorf(\"path <%s> does not exist\", rootfsPath)\n\t}\n\targs = append([]string{rootfsPath, mountBin}, args...)\n\toutput, err := exec.Command(chrootCmd, args...).CombinedOutput()\n\tif err == nil {\n\t\treturn nil\n\t}\n\n\tif !strings.EqualFold(string(output), nfsRPCBindErrMsg) {\n\t\t// Mount failed but not because of RPC bind error\n\t\treturn fmt.Errorf(\"mount failed: %v\\nMounting command: %s\\nMounting arguments: %v\\nOutput: %s\", err, chrootCmd, args, string(output))\n\t}\n\n\t// Mount failed because it is NFS V3 and we need to run rpcBind\n\toutput, err = exec.Command(chrootCmd, rootfsPath, rpcBindCmd, \"-w\").CombinedOutput()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"mount issued for NFS V3 but unable to run rpcbind:\\n Output: %s\\n Error: %v\", string(output), err)\n\t}\n\n\t// Rpcbind is running, try mounting again\n\toutput, err = exec.Command(chrootCmd, args...).CombinedOutput()\n\n\tif err != nil {\n\t\treturn fmt.Errorf(\"mount failed for NFS V3 even after running rpcBind %s, %v\", string(output), err)\n\t}\n\n\treturn nil\n}\n","sourceCodeStart":65,"sourceCodeEnd":95,"githubUrl":"https://github.com/kubernetes/kubernetes/blob/b882c60b4023bdf09264c2d5d30a2cadebc240fb/cluster/gce/gci/mounter/mounter.go#L65-L95","documentation":"Returned by mountInChroot (mounter.go:83) when the original mount failed with the NFS V3 rpc.statd-not-running message, so the mounter attempted to start rpcbind inside the chroot ('/sbin/rpcbind -w'), and rpcbind itself failed. The output and error from the rpcbind invocation are surfaced.","triggerScenarios":"rpcbind binary missing or broken in the containerized mounter rootfs; rpcbind needs capabilities/ports unavailable in the chroot; another portmapper already bound port 111; shared library missing for rpcbind.","commonSituations":"mounter rootfs image built without rpcbind or its dependencies; older GCI image; port 111 already held by host portmap; SELinux denying rpcbind.","solutions":["Ensure /sbin/rpcbind and its shared libs exist in /home/kubernetes/containerized_mounter/rootfs","Read the Error and Output fields for the rpcbind failure (missing lib, permission denied, port in use)","Free port 111 on the host if another portmapper is running, or run rpcbind outside the chroot","Rebuild the containerized mounter rootfs image to include rpcbind"],"exampleFix":"# before: rootfs lacks rpcbind\nchroot rootfs /sbin/rpcbind -w  # No such file\n\n# after: include rpcbind in the rootfs build\n# (in the rootfs build script)\napt-get install -y rpcbind\ninstall -Dm0755 /usr/sbin/rpcbind rootfs/sbin/rpcbind\nldd /usr/sbin/rpcbind | awk '{print $3}' | xargs -I{} cp --parents {} rootfs/","handlingStrategy":"validation","validationCode":"// Ensure rpcbind is present in the rootfs before attempting NFS mounts.\nfunc ensureRpcbind(rootfsPath string) error {\n    p := filepath.Join(rootfsPath, \"sbin\", \"rpcbind\")\n    if _, err := os.Stat(p); err != nil {\n        return fmt.Errorf(\"rpcbind missing at %s; rebuild the mounter rootfs to include it\", p)\n    }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"// Differentiate rpcbind-startup failure from other mount failures.\nif err := mountInChroot(rootfs, args); err != nil {\n    if strings.Contains(err.Error(), \"unable to run rpcbind\") {\n        // rootfs packaging problem: surface as actionable\n        return fmt.Errorf(\"rpcbind unavailable in mounter rootfs: %w\", err)\n    }\n    return err\n}","preventionTips":["Include rpcbind and its shared libraries when building the containerized mounter rootfs","CI-test NFS V3 mounts on the node image so rpcbind regressions surface before deploy","Document that the mounter rootfs must ship /sbin/rpcbind for NFS V3 support"],"tags":["gce","gci","mount","nfs","rpcbind","chroot"],"analyzedSha":"b882c60b4023bdf09264c2d5d30a2cadebc240fb","analyzedAt":"2026-08-07T04:07:48.144Z","schemaVersion":2},"datasetVersion":"2026-08-07T07:17:06.508Z"}