argoproj/argo-workflows · critical

critical error; unable to find %s

Error message

critical error; unable to find %s

What it means

This error is thrown by the DAG reconciler in executeDAG when it builds the child node list of a task-group node (a task with sub-tasks, i.e. withChildren/when-agg results). It looks up each child node ID in wf.Status.Nodes; if the ID is missing from the status map it logs 'was unable to obtain node for nodeID' and returns this critical error. The controller treats a missing child as a corrupted/inconsistent workflow state rather than a transient condition, so reconciliation of the DAG aborts.

Source

Thrown at workflow/controller/dag.go:397

		return node, nil
	}

	// set outputs from tasks in order for DAG templates to support outputs
	scope := createScope(tmpl)
	for _, task := range tmpl.DAG.Tasks {
		taskNode := dagCtx.getTaskNode(ctx, task.Name)
		if taskNode == nil {
			// Can happen when dag.target was specified
			continue
		}

		if taskNode.Type == wfv1.NodeTypeTaskGroup {
			childNodes := make([]wfv1.NodeStatus, len(taskNode.Children))
			for i, childID := range taskNode.Children {
				childNode, childErr := woc.wf.Status.Nodes.Get(childID)
				if childErr != nil {
					woc.log.WithField("nodeID", childID).Error(ctx, "was unable to obtain node for nodeID")
					return nil, fmt.Errorf("critical error; unable to find %s", childID)
				}
				childNodes[i] = *childNode
			}
			aggErr := woc.processAggregateNodeOutputs(scope, varkeys.TasksAggregate, task.Name, childNodes)
			if aggErr != nil {
				woc.log.Error(ctx, "unable to processAggregateNodeOutputs")
				return nil, argoerrors.InternalWrapError(aggErr)
			}
		}
		woc.buildLocalScope(scope, varkeys.TasksNodeRef, task.Name, taskNode)
		// Skipped/omitted tasks produced no Outputs; populate their declared output parameters so that
		// DAG-level output aggregation (parameter refs and ValueFrom.Expression) can resolve them.
		woc.addSkippedNodeOutputsToScope(ctx, dagCtx.tmplCtx, scope, varkeys.TasksNodeRef, task.Name, taskNode, &task, false)
		woc.addOutputsToGlobalScope(ctx, taskNode.Outputs)
	}
	outputs, err := woc.getTemplateOutputsFromScope(ctx, tmpl, scope)
	if err != nil {
		woc.log.Error(ctx, "unable to get outputs")

View on GitHub (pinned to 35bff19146)

Solutions

1. Inspect the Workflow's wf.Status.Nodes (argo get / kubectl get wf <name> -o yaml) and confirm whether the child ID in the task-group's Children list is really missing; look at the controller logs for the nodeID field.
2. If the status was manually modified or corrupted, delete and resubmit the workflow from a known-good spec (argo resubmit / argo submit) rather than patching status.
3. Check controller version for known bugs around node offloading/hydration; upgrade to the latest patch release of your Argo Workflows version.
4. If the workflow pod is still running and only status is inconsistent, retrying the workflow (argo retry) can rebuild node state from existing pods.
5. Reduce node/status size (limit outputs, use S3/artifact logging, configure node status offloading) to avoid future status truncation.

Example fix

// before: relying on possibly-missing node in status
childNode, childErr := woc.wf.Status.Nodes.Get(childID)
if childErr != nil {
	return nil, fmt.Errorf("critical error; unable to find %s", childID)
}
// after: guard at caller / avoid hand-editing status; resubmit instead
// kubectl delete wf <name> (keep pods) then:
// argo resubmit <name> --restart-successful

// Validate before running: ensure every task-group child ID exists in status
// (client-side check when inspecting a workflow)
wf, _ := wfClient.ArgoprojV1alpha1().Workflows(ns).Get(ctx, name, metav1.GetOptions{})
for _, n := range wf.Status.Nodes {
	if n.Type == wfv1.NodeTypeTaskGroup {
		for _, cid := range n.Children {
			if _, ok := wf.Status.Nodes[cid]; !ok {
				return fmt.Errorf("child node %s missing from status; resubmit workflow", cid)
			}
		}
	}
}

When it happens

Trigger: executeDAG iterates taskNode.Children for a NodeTypeTaskGroup node and calls wf.Status.Nodes.Get(childID), which fails because the child node ID referenced in taskNode.Children no longer exists in wf.Status.Nodes (e.g. the status was truncated/offloaded, pruned, or never created).

Common situations: Workflows whose status grew too large and was compressed/offloaded (large DAGs, huge outputs) causing node-map inconsistencies; manual edits to the Workflow object (kubectl edit/delete of nodes in status); controller bugs or upgrades where child node creation failed mid-reconcile; restoring archived workflows without full node status.

Related errors

• no Node found by the name of %s; wf.Status.Nodes=%+v
• no Retry Node found by the name of %s; wf.Status.Nodes=%+v
• cannot fetch workflow spec without workflowTemplateRef
• failed to read container args file %s: %w
• failed to unmarshal container args: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/66f75c0dac9c2704. Report an issue: GitHub.