argoproj/argo-workflows · error

failed to get pod from informer store: %w

Error message

failed to get pod from informer store: %w

What it means

podExists() queries the pod informer's index (NodeIDIndex) to find an existing pod for a node ID. If the index lookup itself fails (not 'no results', but an actual error from GetPodsByIndex), the controller wraps it with this message.

Source

Thrown at workflow/controller/workflowpod.go:971

		newActiveDeadlineSeconds := int64(templateDeadline.Sub(pb.in.now).Seconds())
		if newActiveDeadlineSeconds <= 1 {
			return nil, fmt.Errorf("%s exceeded its deadline", nodeName)
		}
		log.WithFields(logging.Fields{"newActiveDeadlineSeconds": newActiveDeadlineSeconds, "podNamespace": pod.Namespace, "podName": pod.Name}).Debug(ctx, "Setting new activeDeadlineSeconds")
		pod.Spec.ActiveDeadlineSeconds = &newActiveDeadlineSeconds
	}

	// build is pure: the pod is fully constructed but not yet created. submitPod
	// performs the impure operations (ConfigMap/pod creation, rate limiting,
	// active-pod accounting, progress write).
	result.Pod = pod
	return result, nil
}

func (woc *wfOperationCtx) podExists(nodeID string) (existing *apiv1.Pod, exists bool, err error) {
	objs, err := woc.controller.PodController.GetPodsByIndex(indexes.NodeIDIndex, woc.wf.Namespace+"/"+nodeID)
	if err != nil {
		return nil, false, fmt.Errorf("failed to get pod from informer store: %w", err)
	}

	objectCount := len(objs)

	if objectCount == 0 {
		return nil, false, nil
	}

	if objectCount > 1 {
		return nil, false, fmt.Errorf("expected 1 pod, got %d. This can happen when multiple workflow-controller "+
			"pods are running and both reconciling this Workflow. Check your Argo Workflows installation for a rogue "+
			"workflow-controller. Otherwise, this is a bug", len(objs))
	}

	if existing, ok := objs[0].(*apiv1.Pod); ok {
		return existing, true, nil
	}

View on GitHub (pinned to 35bff19146)

Solutions

1. Retry the workflow reconciliation — usually transient; check controller logs around startup
2. Ensure only one workflow-controller version is running and informers are healthy
3. Check controller metrics/logs for informer sync failures at startup
4. If persistent, restart the workflow-controller deployment

// check informer health before relying on indexed lookups
// controller log: wait for 'informer synced' at startup

objs, err := woc.controller.PodController.GetPodsByIndex(indexes.NodeIDIndex, key)
if err != nil {
    return nil, false, fmt.Errorf("failed to get pod from informer store: %w", err)
} // caller should treat as transient and requeue

When it happens

Trigger: woc.controller.PodController.GetPodsByIndex returns an error for the wf.Namespace/nodeID key — informer store not yet synced, index misconfigured, or internal cache error — reached via nodePodExist, executeTemplate, or createWorkflowPod.

Common situations: Controller just started and informers are still syncing; memory-pressure or cache invalidation in the informer store; bugs after controller upgrades affecting the NodeIDIndex.

Related errors

• failed to get pod from informer store: %w
• could not save to config map cache: %w
• error creating cache entry: %w. Please check out this page f
• failed to delete ConfigMap %s: %w
• failed to update ConfigMap %s: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/19069f996f2e88d8. Report an issue: GitHub.