argoproj/argo-workflows · error

unable to delete Failed Workflows of CronWorkflow '%s': %w

Error message

unable to delete Failed Workflows of CronWorkflow '%s': %w

What it means

Same mechanism as the successful-jobs pruning error but for failed child workflows: the controller keeps spec.failedJobsHistoryLimit failed workflows (default 1) and the deleteOldestWorkflows call failed. The error is wrapped with the CronWorkflow name and returned from enforceHistoryLimit.

Source

Thrown at workflow/cron/operator.go:446

		}
	}

	workflowsToKeep := int32(3)
	if woc.cronWf.Spec.SuccessfulJobsHistoryLimit != nil && *woc.cronWf.Spec.SuccessfulJobsHistoryLimit >= 0 {
		workflowsToKeep = *woc.cronWf.Spec.SuccessfulJobsHistoryLimit
	}
	err := woc.deleteOldestWorkflows(ctx, successfulWorkflows, int(workflowsToKeep))
	if err != nil {
		return fmt.Errorf("unable to delete Successful Workflows of CronWorkflow '%s': %w", woc.cronWf.Name, err)
	}

	workflowsToKeep = int32(1)
	if woc.cronWf.Spec.FailedJobsHistoryLimit != nil && *woc.cronWf.Spec.FailedJobsHistoryLimit >= 0 {
		workflowsToKeep = *woc.cronWf.Spec.FailedJobsHistoryLimit
	}
	err = woc.deleteOldestWorkflows(ctx, failedWorkflows, int(workflowsToKeep))
	if err != nil {
		return fmt.Errorf("unable to delete Failed Workflows of CronWorkflow '%s': %w", woc.cronWf.Name, err)
	}
	return nil
}

func (woc *cronWfOperationCtx) deleteOldestWorkflows(ctx context.Context, jobList []v1alpha1.Workflow, workflowsToKeep int) error {
	if workflowsToKeep >= len(jobList) {
		return nil
	}

	sort.SliceStable(jobList, func(i, j int) bool {
		return jobList[i].Status.FinishedAt.After(jobList[j].Status.FinishedAt.Time)
	})

	for _, wf := range jobList[workflowsToKeep:] {
		err := woc.wfClient.Delete(ctx, wf.Name, v1.DeleteOptions{})
		if err != nil {
			if apierrors.IsNotFound(err) {
				woc.log.WithField("workflow", wf.Name).Info(ctx, "Workflow was already deleted")

View on GitHub (pinned to 35bff19146)

Solutions

1. Read the wrapped cause in the controller log to identify the exact API error
2. Grant the controller RBAC delete rights on argoproj.io/workflows
3. Remove blocking finalizers from stale failed workflows
4. Increase failedJobsHistoryLimit if you simply have too many old failed workflows and want less churn

Example fix

// before
spec:
  failedJobsHistoryLimit: 0   # aggressive pruning, fails loudly on every tick
// after
spec:
  failedJobsHistoryLimit: 3

kubectl auth can-i delete workflows.argoproj.io -n <ns> \
  --as=system:serviceaccount:<controller-ns>:workflow-controller

When it happens

Trigger: A non-NotFound error is returned while deleting the oldest failed workflows of the CronWorkflow during a sync tick — k8s API delete failure, permission denial, or an internal error in the pruning loop.

Common situations: Controller RBAC missing workflow delete verbs, failed workflows carrying finalizers, or transient API server errors during heavy load.

Related errors

• unable to delete Successful Workflows of CronWorkflow '%s':
• failed to list SSO RBAC service accounts: %w
• failed to get workflow template: %w
• failed to check if secret %s exists: %w
• failed to get token volumes: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/8176b1e641da09a0. Report an issue: GitHub.