kubernetes/kubernetes · error

%v failed with : %v

Error message

%v failed with : %v

What it means

Top-level catch-all logged by processNextWorkItem when dsc.syncHandler (syncDaemonSet) returns a non-nil error for a DaemonSet key. It is the wrapper that means 'reconciling this DaemonSet failed for some reason' -- the underlying cause is the second %v. After logging, the key is re-queued via AddRateLimited, so the controller retries with exponential backoff. Seeing this repeatedly means the underlying error (transient API failure, bad object, etc.) is not resolving.

Source

Thrown at pkg/controller/daemon/daemon_controller.go:411

	for dsc.processNextWorkItem(ctx) {
	}
}

// processNextWorkItem deals with one key off the queue.  It returns false when it's time to quit.
func (dsc *DaemonSetsController) processNextWorkItem(ctx context.Context) bool {
	dsKey, quit := dsc.queue.Get()
	if quit {
		return false
	}
	defer dsc.queue.Done(dsKey)

	err := dsc.syncHandler(ctx, dsKey)
	if err == nil {
		dsc.queue.Forget(dsKey)
		return true
	}

	utilruntime.HandleError(fmt.Errorf("%v failed with : %v", dsKey, err))
	dsc.queue.AddRateLimited(dsKey)

	return true
}

func (dsc *DaemonSetsController) enqueue(ds *apps.DaemonSet) {
	key, err := controller.KeyFunc(ds)
	if err != nil {
		utilruntime.HandleError(fmt.Errorf("Couldn't get key for object %#v: %v", ds, err))
		return
	}

	// TODO: Handle overlapping controllers better. See comment in ReplicationManager.
	dsc.queue.Add(key)
}

func (dsc *DaemonSetsController) enqueueDaemonSetAfter(obj interface{}, after time.Duration) {
	key, err := controller.KeyFunc(obj)

View on GitHub (pinned to b882c60b40)

Solutions

1. Read the second %v in the log line to find the real root cause; this wrapper is never the actionable error itself.
2. If the cause is transient (timeout, connection refused, etcd leader election), verify apiserver health and let the rate-limited retry succeed.
3. If the cause repeats for the same key indefinitely, inspect that DaemonSet object (kubectl get ds -n <ns> <name> -o yaml) for a malformed spec.
4. Check kube-controller-manager logs around the same timestamp for the unwrapped error and any related event recordings.

Example fix

// This is an internal controller log; no caller code change fixes it directly.
// Action: grep the underlying error from the same log stream, e.g.
//   kubectl -n kube-system logs <kube-controller-manager-pod> | grep '<ds-key> failed with'
// then resolve the nested cause (see errors 541-559).

// Internal controller log; verify controller-manager health before relying on it.
// kubectl -n kube-system get pod -l component=kube-controller-manager
// kubectl -n kube-system logs <pod> --previous | grep '<ds-key> failed with'

// The controller already retries via AddRateLimited. To classify the wrapped error:
// if apierrors.IsTimeout(err) || apierrors.IsServerTimeout(err) { /* transient */ }
// if apierrors.IsConflict(err) { /* retry with fresh GET */ }
// if apierrors.IsInvalid(err) { /* fix the object spec */ }

When it happens

Trigger: Any error returned from syncDaemonSet for a given namespace/name key: apiserver flake during DS/pod/node lookup, conflict on status update, failure to construct history, or failure to create/delete daemon pods. Logged once per failed reconciliation attempt.

Common situations: Control-plane under load or apiserver briefly unreachable; a DaemonSet whose spec is malformed (bad selector, unparseable update strategy); etcd latency causing conflicts; informer cache still warming up at controller start.

Related errors

• %v failed with : %w
• error storing status for daemon set %#v: %w
• found a stale collision count (%d, expected %d) of DaemonSet
• error syncing item %s: %v
• failed to orphan dependents of owner %s, got errors: %s

AI-assisted analysis of kubernetes/kubernetes@b882c60b40 (2026-08-07). Data as JSON: /api/errors/aefb9bf38820dfa6. Report an issue: GitHub.