rancher/rancher · error

timeout waiting for plan

Error message

timeout waiting for plan

What it means

The final stage of findSA watches Secrets labeled for the discovered plan service account, waiting for the serviceaccounttoken controller to publish a token. Despite the name, there is no explicit timer: this error is returned when the secret watch channel closes without delivering a single Secret event. The node's system-agent treats it as retryable and re-issues the HTTP request on its next polling cycle.

Source

Thrown at pkg/capr/configserver/server.go:472

	})
	if err != nil {
		return "", nil, err
	}
	defer func() {
		respSecret.Stop()
		//nolint:revive
		for range respSecret.ResultChan() {
			// Intentionally drain the channel.
		}
	}()
	for event := range respSecret.ResultChan() {
		if secret, ok := event.Object.(*corev1.Secret); ok {
			logrus.Infof("[rke2configserver] %s/%s machineID: %s delivering planSecret %s with token secret %s/%s to system-agent from secret watch", ref.Namespace, ref.Name, machineID, planSecret, secret.Namespace, secret.Name)
			return planSecret, secret, nil
		}
	}

	return "", nil, fmt.Errorf("timeout waiting for plan")
}

func (r *RKE2ConfigServer) setOrUpdateMachineID(ref *corev1.ObjectReference, machineID string) error {
	if ref == nil {
		return nil
	}

	// Only update CAPI Machines; no-op for other kinds (e.g., management Nodes).
	if ref.APIVersion != capi.GroupVersion.String() || ref.Kind != "Machine" {
		return nil
	}

	machineNamespace, machineName := ref.Namespace, ref.Name

	machine, err := r.machineCache.Get(machineNamespace, machineName)
	if err != nil {
		return err
	}

View on GitHub (pinned to 932558d4e6)

Solutions

1. Retry: system-agent polls again and controllers usually catch up, resolving the condition on a later request
2. Verify token issuance works: kubectl get secrets -n <ns> -l kubernetes.io/service-account.name=<plan-sa-name>
3. Check kube-controller-manager health and logs on the management cluster
4. If behind an ingress/proxy, raise idle timeouts so config-server watches are not cut off early

When it happens

Trigger: kube-controller-manager has not generated a token secret for the plan SA within the watch window; the token secret is created without the expected serviceaccounttoken label (version skew); the watch is closed by the API server before any event arrives.

Common situations: Overloaded management cluster slowing token issuance; Kubernetes version changes to service-account token behavior; admission policies blocking token secret creation; watches timing out behind proxies with short idle limits.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• timeout waiting for %s/%s to be ready
• could not start secret watch for token secret
• machine not found by request
• can't create token: %w
• error getting service account token: %w

AI-assisted analysis of rancher/rancher@932558d4e6 (2026-08-16). Data as JSON: /api/errors/4ddc90e2786ba0dd. Report an issue: GitHub.