argoproj/argo-workflows · error

error stopping workflow %s: %w

Error message

error stopping workflow %s: %w

What it means

terminateOutstandingWorkflows (used by the Replace concurrency policy) stops each active Workflow listed in the CronWorkflow's Status.Active via util.TerminateWorkflow. If termination fails for a workflow — and it is neither a NotFound (already gone) nor an AlreadyShutdownError — the error is wrapped with the workflow name and aborts the cron run.

Source

Thrown at workflow/cron/operator.go:306

		}
	}
	return true, nil
}

func (woc *cronWfOperationCtx) terminateOutstandingWorkflows(ctx context.Context) error {
	for _, wfObjectRef := range woc.cronWf.Status.Active {
		woc.log.WithField("name", wfObjectRef.Name).Info(ctx, "stopping")
		err := util.TerminateWorkflow(ctx, woc.wfClient, wfObjectRef.Name)
		if err != nil {
			if apierrors.IsNotFound(err) {
				woc.log.WithField("name", wfObjectRef.Name).Warn(ctx, "workflow not found when trying to terminate outstanding workflows")
				continue
			}
			if alreadyShutdownErr, ok := errors.AsType[util.AlreadyShutdownError](err); ok {
				woc.log.Warn(ctx, alreadyShutdownErr.Error())
				continue
			}
			return fmt.Errorf("error stopping workflow %s: %w", wfObjectRef.Name, err)
		}
	}
	return nil
}

func (woc *cronWfOperationCtx) runOutstandingWorkflows(ctx context.Context) (bool, error) {
	missedExecutionTime, err := woc.shouldOutstandingWorkflowsBeRun(ctx)
	if err != nil {
		return false, err
	}
	if !missedExecutionTime.IsZero() {
		woc.run(ctx, missedExecutionTime)
		return true, nil
	}
	return false, nil
}

func (woc *cronWfOperationCtx) shouldOutstandingWorkflowsBeRun(ctx context.Context) (time.Time, error) {

View on GitHub (pinned to 35bff19146)

Solutions

1. Check the inner wrapped error for the root cause (RBAC vs conflict vs timeout)
2. Ensure the controller's service account can patch/update workflows in the namespace
3. Verify Status.Active is accurate; manually stop orphaned workflows with argo stop <name>
4. Retry — conflicts and transient API errors usually resolve on the next cron tick

kubectl auth can-i patch workflows.argoproj.io --as=system:serviceaccount:<ns>:<controller-sa> -n <ns>

// tolerate benign termination errors, fail on real ones
if apierrors.IsNotFound(err) { continue }
if _, ok := errors.AsType[util.AlreadyShutdownError](err); ok { continue }
return fmt.Errorf("error stopping workflow %s: %w", wfObjectRef.Name, err)

err := util.TerminateWorkflow(ctx, woc.wfClient, name)
if err != nil {
    if apierrors.IsNotFound(err) || errors.AsType[util.AlreadyShutdownError](err) != nil {
        continue // already gone or already shutting down
    }
    return fmt.Errorf("error stopping workflow %s: %w", name, err)
}

When it happens

Trigger: TerminateWorkflow returns an error while patching shutdown (e.g. RBAC denial on workflow update/patch, API server conflict or timeout, workflow already terminating with a conflicting patch) for a workflow referenced in cronWf.Status.Active.

Common situations: CronWorkflow's controller service account lacking update permission on workflows; API server flakiness under load; stale Status.Active entries pointing at workflows in a bad state; concurrent shutdown by another actor.

Related errors

• failed to get existing cluster workflow template %q to updat
• failed to list SSO RBAC service accounts: %w
• failed to get service account secret: %w
• failed to create secret: %w
• failed to read secret: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/e6fb2afa4763176b. Report an issue: GitHub.