argoproj/argo-workflows · error

failed to create Agent pod: %w

Error message

failed to create Agent pod: %w

What it means

After building the Agent pod spec, createAgentPod asks the controller to create it (via createPodFromBuild). When pod creation fails and the failure is not a transient requeue case, the error is wrapped as 'failed to create Agent pod: %w' so agent-pod creation failures are distinguishable from workload-pod failures in logs/status.

Source

Thrown at workflow/controller/agent.go:290

			// The rate limiter runs before Create, so on the rate-limited path
			// the AlreadyExists→Get recovery inside createPodFromBuild never
			// runs. A pod created by a prior reconcile that has not yet reached
			// the informer (checked at the top of this function) must be
			// recovered with a direct Get rather than requeued as though no pod
			// existed.
			if errors.Is(err, ErrResourceRateLimitReached) {
				if existing, getErr := woc.getPod(ctx, podName); getErr == nil {
					log.Info(ctx, "Recovered existing Agent pod on rate-limited create")
					return existing, nil
				}
			}
			woc.requeue()
			return nil, nil
		}
		// createPodFromBuild wraps non-transient failures generically; add the
		// agent-pod context so an agent-pod creation failure is distinguishable
		// from a workload-pod one in logs/status.
		return nil, fmt.Errorf("failed to create Agent pod: %w", err)
	}
	log.Info(ctx, "Created Agent pod")
	return created, nil
}

func (woc *wfOperationCtx) getExecutorPlugins(ctx context.Context) ([]apiv1.Container, []apiv1.Volume, error) {
	var sidecars []apiv1.Container
	var volumes []apiv1.Volume
	namespaces := map[string]bool{} // de-dupes executorPlugins when their namespaces are the same
	namespaces[woc.controller.namespace] = true
	namespaces[woc.wf.Namespace] = true
	wFPlugins, err := woc.execWf.Spec.AsExecutorPluginSpec()
	if err != nil {
		return nil, nil, err
	}
	isGetPluginsFromWorkflow := len(wFPlugins) > 0
	if isGetPluginsFromWorkflow && !woc.controller.enableWorkflowLevelExecutorPlugins {
		return nil, nil, fmt.Errorf(

View on GitHub (pinned to 35bff19146)

Solutions

1. Read the wrapped inner error for the exact API reason (quota, webhook denial, invalid spec)
2. Check namespace ResourceQuota/LimitRange: `kubectl describe resourcequota -n <ns>`
3. Inspect admission webhook rejections in the error and adjust webhook policy or plugin spec
4. Retry the workflow if the cause was transient (API blip, race with informer store)

kubectl describe resourcequota -n <workflow-namespace>  # and check admission webhooks

created, err := createAgentPod(ctx)
if err != nil {
    if util.errors.IsTransientErr(ctx, err) { woc.requeue(); return }
    return fmt.Errorf("failed to create Agent pod: %w", err)
}

When it happens

Trigger: The k8s pod Create call fails during reconcileAgentPod: invalid pod spec (e.g. plugin container misconfiguration), quota exceeded, admission webhook rejection, API server error, or duplicate pod name race (already exists but not found in informer store yet).

Common situations: ResourceQuota in the namespace blocks pod creation; a mutating admission webhook (e.g. Istio sidecar injection policy, Kyverno) rejects the agent pod spec; API version incompatibilities from CRDs/validating webhooks; namespace terminating while workflow still running.

Related errors

• failed to create pod: %w
• failed to read container args file %s: %w
• failed to unmarshal container args: %w
• failed to read template: %w
• failed to start command: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/0cfe82a4a2783f49. Report an issue: GitHub.