k3s-io/k3s · error · passwordError

password hash not found in node secret

Error message

password hash not found in node secret

What it means

Node-password verification looks up the per-node secret in kube-system and validates the 'hash' entry with the password hasher. If the secret exists but its Data has no 'hash' key, verification cannot proceed and the mismatch is wrapped in a passwordError carrying the node name.

Source

Thrown at pkg/nodepassword/nodepassword.go:60

func errVerifyFailed() error { return &passwordError{} }

func getSecretName(nodeName string) string {
	return strings.ToLower(nodeName + ".node-password." + version.Program)
}

func (npc *nodePasswordController) verifyHash(nodeName, pass string, cached bool) error {
	secret, err := npc.getSecret(nodeName, cached)
	if err != nil {
		return &passwordError{node: nodeName, err: err}
	}
	if hash, ok := secret.Data["hash"]; ok {
		if err := Hasher.VerifyHash(string(hash), pass); err != nil {
			return &passwordError{node: nodeName, err: err}
		}
		return nil
	}
	return &passwordError{node: nodeName, err: errors.New("password hash not found in node secret")}
}

// ensure will verify a node-password secret if it exists, otherwise it will create one
func (npc *nodePasswordController) ensure(nodeName, pass string) error {
	err := npc.verifyHash(nodeName, pass, true)
	if apierrors.IsNotFound(err) {
		var hash string
		hash, err = Hasher.CreateHash(pass)
		if err != nil {
			return &passwordError{node: nodeName, err: err}
		}
		_, err = npc.secrets.Create(&v1.Secret{
			ObjectMeta: metav1.ObjectMeta{
				Name:      getSecretName(nodeName),
				Namespace: metav1.NamespaceSystem,
			},
			Immutable: ptr.To(true),
			Data:      map[string][]byte{"hash": []byte(hash)},

View on GitHub (pinned to 6ba341e396)

Solutions

1. Delete the offending secret: kubectl -n kube-system delete secret node-password-<node> and let the controller recreate it with a proper hash.
2. Restart the agent on the node so ensure() re-registers the password.
3. If managed by manifests/GitOps, remove the node-password secret from source control - the controller owns it.

Example fix

# before: secret applied without hash key (verification always fails)
kubectl -n kube-system apply -f node-password-mynode.yaml
# after: let the controller own it
kubectl -n kube-system delete secret node-password-mynode
systemctl restart k3s-agent  # or k3s on the node

// check the secret shape before verifying
secret, err := npc.getSecret(nodeName, true)
if err == nil && len(secret.Data["hash"]) == 0 {
    // malformed: delete so ensure() recreates it, or repair from the node's password file
}

err := npc.ensure(nodeName, pass)
if err != nil {
    var pe *passwordError
    if errors.As(err, &pe) && strings.Contains(pe.err.Error(), "password hash not found") {
        // fallback: delete malformed secret and retry once so it is recreated
    }
}

When it happens

Trigger: The kube-system secret 'node-password-<nodename>' (or equivalent) exists but was created/modified without a hash key - e.g. applied by a manifest, mutated by an operator, or restored from a backup made by a different version.

Common situations: GitOps applying node-password secrets as plain data; upgrades that changed the secret schema; a node renamed so the lookup hits a stale secret.

Related errors

• header node name does not match auth node name
• node name not set
• node password not set
• failed to update secret: %v
• invalid username/password combination

AI-assisted analysis of k3s-io/k3s@6ba341e396 (2026-08-15). Data as JSON: /api/errors/526f81ba3c0b8d80. Report an issue: GitHub.