argoproj/argo-workflows · error

unable to delete Successful Workflows of CronWorkflow '%s':

Error message

unable to delete Successful Workflows of CronWorkflow '%s': %w

What it means

The controller's history-limit enforcement could not delete surplus successful child workflows of a CronWorkflow. The operator keeps spec.successfulJobsHistoryLimit workflows (default 3) and calls deleteOldestWorkflows; any error from the Kubernetes delete/list path is wrapped with this message and surfaces in the controller's cron reconciliation error.

Source

Thrown at workflow/cron/operator.go:437

		if wf.Labels[common.LabelKeyCronWorkflow] != woc.cronWf.Name {
			continue
		}
		if wf.Status.Fulfilled() {
			if wf.Status.Successful() {
				successfulWorkflows = append(successfulWorkflows, wf)
			} else {
				failedWorkflows = append(failedWorkflows, wf)
			}
		}
	}

	workflowsToKeep := int32(3)
	if woc.cronWf.Spec.SuccessfulJobsHistoryLimit != nil && *woc.cronWf.Spec.SuccessfulJobsHistoryLimit >= 0 {
		workflowsToKeep = *woc.cronWf.Spec.SuccessfulJobsHistoryLimit
	}
	err := woc.deleteOldestWorkflows(ctx, successfulWorkflows, int(workflowsToKeep))
	if err != nil {
		return fmt.Errorf("unable to delete Successful Workflows of CronWorkflow '%s': %w", woc.cronWf.Name, err)
	}

	workflowsToKeep = int32(1)
	if woc.cronWf.Spec.FailedJobsHistoryLimit != nil && *woc.cronWf.Spec.FailedJobsHistoryLimit >= 0 {
		workflowsToKeep = *woc.cronWf.Spec.FailedJobsHistoryLimit
	}
	err = woc.deleteOldestWorkflows(ctx, failedWorkflows, int(workflowsToKeep))
	if err != nil {
		return fmt.Errorf("unable to delete Failed Workflows of CronWorkflow '%s': %w", woc.cronWf.Name, err)
	}
	return nil
}

func (woc *cronWfOperationCtx) deleteOldestWorkflows(ctx context.Context, jobList []v1alpha1.Workflow, workflowsToKeep int) error {
	if workflowsToKeep >= len(jobList) {
		return nil
	}

View on GitHub (pinned to 35bff19146)

Solutions

  1. Check the wrapped error in the controller log for the underlying k8s API failure cause
  2. Verify the workflow-controller service account has delete/update permissions on workflows in the namespace (Role/ClusterRole from the official manifests)
  3. Look for stuck finalizers on the old workflows (kubectl get wf -o yaml) and clear the blocking finalizer
  4. Retry — the controller re-runs enforceHistoryLimit on every cron sync tick

Example fix

// before: custom Role missing workflow deletion
rules:
- apiGroups: [argoproj.io]
  resources: [workflows]
  verbs: [get, list, watch]
// after
rules:
- apiGroups: [argoproj.io]
  resources: [workflows]
  verbs: [get, list, watch, delete]
Defensive patterns

Strategy: retry

Validate before calling

// Verify the controller can delete workflows before relying on history limits
authCanI, _ := exec.Command("kubectl", "auth", "can-i", "delete",
    "workflows.argoproj.io", "-n", ns,
    "--as=system:serviceaccount:argo:workflow-controller").Output()
// expect "yes"

Prevention

When it happens

Trigger: deleteOldestWorkflows returns an error while pruning successfulWorkflows — e.g. a workflow Delete call fails with a non-NotFound API error, or the earlier list of successful workflows failed.

Common situations: RBAC: the controller's service account lacks delete permission on workflows (argoworkflows/finalizers) in the namespace; API server throttling or timeouts; a workflow stuck with finalizers preventing deletion.

Related errors


AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/0f47abc823959127. Report an issue: GitHub.