node named %q returned by WorkflowArtifactGCTask %q wasn't f

Error message

node named %q returned by WorkflowArtifactGCTask %q wasn't found in Workflow %q Status

What it means

During artifact garbage collection, the controller reconciles a completed WorkflowArtifactGCTask pod by matching each node reported in the task's ArtifactResultsByNode against the Workflow's Status.Nodes. This error is returned when a node name recorded in the GC task's status cannot be found in the Workflow status — meaning the GC pod reported results for a node the Workflow no longer tracks.

Source

Thrown at workflow/controller/artifact_gc.go:687

				woc.log.WithField("name", task.Name).WithError(err).Error(ctx, "error deleting WorkflowArtifactGCTask")
			}
		}
	}
	return nil
}

// process the Status in the WorkflowArtifactGCTask which was completed and reflect it in Workflow Status; then delete the Task CRD Object
// return true if all artifacts succeeded, else false
func (woc *wfOperationCtx) processCompletedWorkflowArtifactGCTask(ctx context.Context, artifactGCTask *wfv1.WorkflowArtifactGCTask, strategy wfv1.ArtifactGCStrategy) (bool, error) {
	woc.log.WithField("name", artifactGCTask.Name).Debug(ctx, "processing WorkflowArtifactGCTask")

	foundGCFailure := false
	for nodeName, nodeResult := range artifactGCTask.Status.ArtifactResultsByNode {
		// find this node result in the Workflow Status
		wfNode, err := woc.wf.Status.Nodes.Get(nodeName)
		if err != nil {
			woc.log.WithField("node", nodeName).WithError(err).Error(ctx, "Was unable to obtain node for")
			return false, fmt.Errorf("node named %q returned by WorkflowArtifactGCTask %q wasn't found in Workflow %q Status", nodeName, artifactGCTask.Name, woc.wf.Name)
		}
		if wfNode.Outputs == nil {
			return false, fmt.Errorf("node named %q returned by WorkflowArtifactGCTask %q doesn't seem to have Outputs in Workflow Status", nodeName, artifactGCTask.Name)
		}
		for i, wfArtifact := range wfNode.Outputs.Artifacts {
			// find artifact in the WorkflowArtifactGCTask Status
			artifactResult, foundArt := nodeResult.ArtifactResults[wfArtifact.Name]
			if !foundArt {
				// could be in a different WorkflowArtifactGCTask
				continue
			}

			wfNode.Outputs.Artifacts[i].Deleted = artifactResult.Success
			woc.wf.Status.Nodes.Set(ctx, nodeName, *wfNode)

			if artifactResult.Error != nil {
				woc.addArtGCCondition(fmt.Sprintf("%s (artifactGCTask: %s)", *artifactResult.Error, artifactGCTask.Name))
				// issue an Event if there was an error - just do this once to prevent flooding the system with Events

View on GitHub (pinned to 35bff19146)

Solutions

Inspect the WorkflowArtifactGCTask pod status (kubectl get wfartgc -o yaml) and compare its ArtifactResultsByNode keys with `argo get <wf>` node names to find the stale node.
Check whether workflow status offloading is active (workflow-controller-configmap nodeStatusOffload) and whether hydration lost nodes; re-run the GC reconciliation after full hydration.
Delete the stale WorkflowArtifactGCTask resource so the controller recreates it with current node references.
If this happened after retry/resubmit, ensure the retried workflow's artifactGCScope/strategy matches and old GC tasks from the original run are deleted.
Report with controller logs (node=<name>) if node status legitimately contains the node — may indicate a hydrator bug.

Example fix

// before: reconciling old GC task against retried workflow
kubectl delete wfartgctask <stale-gc-task-name>
// after: GC task recreated against current nodes

Defensive patterns

Strategy: validation

Validate before calling

const nodes = wf.status?.nodes || {};
const gcTask = await k8s.getNamespacedCustomObject('argoproj.io','v1alpha1',ns,'workflowartifactgctasks',name);
const missing = Object.keys(gcTask.status?.artifactResultsByNode || {}).filter(n => !(n in nodes));
if (missing.length) throw new Error(`GC task references missing nodes: ${missing.join(',')}`);

Type guard

function hasNode(wf, nodeName) {
  return Boolean(wf.status && wf.status.nodes && wf.status.nodes[nodeName]);
}

Prevention

Before deleting/pruning workflow status nodes, check no WorkflowArtifactGCTask references them.
Avoid manual edits to Status.Nodes on workflows with artifactGC enabled.
Keep status offloading/hydration healthy — watch for hydrator errors in controller logs.
After retry/resubmit, delete GC tasks from the original run.

When it happens

Trigger: A WorkflowArtifactGCTask pod completes and its Status.ArtifactResultsByNode contains a nodeName that is absent from wf.Status.Nodes — e.g. the node was pruned, the status was offloaded/truncated and lost that node, or the GC task references a stale nodeID from a retried/deleted workflow version.

Common situations: Running workflows with artifactGC enabled where the workflow status was compressed/offloaded and node entries dropped; resubmitting or retrying workflows so an old artifact-gc pod reconciles against a new workflow; manually editing or trimming Status.Nodes; upgrading across versions with large node status.

Related errors

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/ff0017f35c173dcd. Report an issue: GitHub.