argoproj/argo-workflows · error

error deleting workflow '%s': %w

Error message

error deleting workflow '%s': %w

What it means

A specific child workflow could not be deleted while enforcing CronWorkflow history limits. NotFound errors are tolerated and logged (the workflow was already gone), but any other Kubernetes API error from wfClient.Delete is wrapped with the workflow name and aborts the pruning loop.

Source

Thrown at workflow/cron/operator.go:467

}

func (woc *cronWfOperationCtx) deleteOldestWorkflows(ctx context.Context, jobList []v1alpha1.Workflow, workflowsToKeep int) error {
	if workflowsToKeep >= len(jobList) {
		return nil
	}

	sort.SliceStable(jobList, func(i, j int) bool {
		return jobList[i].Status.FinishedAt.After(jobList[j].Status.FinishedAt.Time)
	})

	for _, wf := range jobList[workflowsToKeep:] {
		err := woc.wfClient.Delete(ctx, wf.Name, v1.DeleteOptions{})
		if err != nil {
			if apierrors.IsNotFound(err) {
				woc.log.WithField("workflow", wf.Name).Info(ctx, "Workflow was already deleted")
				continue
			}
			return fmt.Errorf("error deleting workflow '%s': %w", wf.Name, err)
		}
		woc.log.WithField("workflow", wf.Name).Info(ctx, "Deleted Workflow due to CronWorkflow history limit")
	}
	return nil
}

func (woc *cronWfOperationCtx) reportCronWorkflowError(ctx context.Context, conditionType v1alpha1.ConditionType, errString string) {
	woc.log.WithField("conditionType", conditionType).Error(ctx, errString)
	woc.cronWf.Status.Conditions.UpsertCondition(v1alpha1.Condition{
		Type:    conditionType,
		Message: errString,
		Status:  v1.ConditionTrue,
	})
	if conditionType == v1alpha1.ConditionTypeSpecError {
		woc.metrics.CronWorkflowSpecError(ctx)
	} else {
		if conditionType == v1alpha1.ConditionTypeSubmissionError {
			woc.cronWf.Status.Failed++

View on GitHub (pinned to 35bff19146)

Solutions

  1. Inspect the wrapped k8s error in the controller log (Forbidden/Conflict/Timeout)
  2. Fix RBAC so the controller can delete workflows in the target namespace
  3. Check kubectl describe wf <name> for stuck finalizers and resolve them
  4. If API throttling, scale down concurrent deletions or wait — the next sync retries

Example fix

# before
kubectl auth can-i delete workflows.argoproj.io -n my-ns --as=system:serviceaccount:argo:workflow-controller
# no
# after: add to controller role
- apiGroups: [argoproj.io]
  resources: [workflows]
  verbs: [delete]
Defensive patterns

Strategy: retry

Validate before calling

// Pre-check deletability
wf, err := wfClient.Get(ctx, wfName, metav1.GetOptions{})
if err == nil && len(wf.Finalizers) > 0 {
    // resolve finalizers before history pruning will succeed
}

Prevention

When it happens

Trigger: wfClient.Delete(ctx, wf.Name, DeleteOptions{}) fails with a non-NotFound error: 403 Forbidden from RBAC, 409 Conflict from finalizers/ownership, 429 from API throttling, or network failure to the API server.

Common situations: Namespace-scoped controller lacking delete rights in another namespace, workflows with blocking finalizers, or API server connectivity issues in the cluster.

Related errors


AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/0f3f7a786e01132d. Report an issue: GitHub.