argoproj/argo-workflows · error

error creating cache entry: %w. Please check out this page f

Error message

error creating cache entry: %w. Please check out this page for help: https://argo-workflows.readthedocs.io/en/latest/memoization/#faqs

What it means

When saving a memoization entry to an existing ConfigMap, the controller calls ConfigMaps.Update. A Kubernetes API error during this update is wrapped with this message plus a link to the memoization FAQ, because the most common cause is RBAC or resource-version conflicts on the memoization ConfigMap.

Source

Thrown at workflow/controller/cache/configmap_cache.go:206

		CreationTimestamp: metav1.Time{Time: creationTime},
		LastHitTimestamp:  metav1.Time{Time: creationTime},
	}

	entryJSON, err := json.Marshal(newEntry)
	if err != nil {
		c.logError(ctx, err, logging.Fields{"key": key, "nodeID": nodeID}, "Unable to marshal cache entry")
		return fmt.Errorf("unable to marshal cache entry: %w", err)
	}

	if cache.Data == nil {
		cache.Data = make(map[string]string)
	}
	cache.Data[key] = string(entryJSON)

	_, err = c.kubeClient.CoreV1().ConfigMaps(c.namespace).Update(ctx, cache, metav1.UpdateOptions{})
	if err != nil {
		c.logError(ctx, err, logging.Fields{"key": key, "nodeID": nodeID}, "Kubernetes error creating new cache entry")
		return fmt.Errorf("error creating cache entry: %w. Please check out this page for help: https://argo-workflows.readthedocs.io/en/latest/memoization/#faqs", err)
	}
	return nil
}

View on GitHub (pinned to 35bff19146)

Solutions

  1. Read the wrapped underlying error in controller logs ('Kubernetes error creating new cache entry') — it names the exact Kubernetes cause (Forbidden/Conflict/TooLarge).
  2. If 409 Conflict: re-run the workflow; the save will retry against the latest resourceVersion.
  3. If Forbidden: add update/get/create verbs on configmaps for the controller SA per the memoization FAQ RBAC example.
  4. If payload too large: split memoization across multiple ConfigMaps (distinct memoize configMap names per template) or prune old entries.
  5. Consult the linked FAQ: https://argo-workflows.readthedocs.io/en/latest/memoization/#faqs.

Example fix

# before: insufficient verbs
verbs: ["get", "create"]
# after
verbs: ["create", "get", "update", "delete"]
Defensive patterns

Strategy: retry

Validate before calling

kubectl auth can-i update configmaps -n <ns> --as=system:serviceaccount:<argo-ns>:workflow-controller
# and monitor ConfigMap size: kubectl get cm <name> -o jsonpath='{.metadata.resourceVersion}' — 409s mean concurrent writers

Try / catch

err := cache.Save(ctx, key, nodeID, entry)
if err != nil {
  if apierrors.IsConflict(err) {
    // stale resourceVersion: re-get ConfigMap and retry save with backoff
  } else if apierrors.IsForbidden(err) {
    // fix RBAC (update verb on configmaps)
  }
  return err
}

Prevention

When it happens

Trigger: CoreV1().ConfigMaps(ns).Update(cache) fails after validation passed: SA lacks update permission on configmaps, conflict (the ConfigMap was modified concurrently — stale resourceVersion), ConfigMap size limit (1 MiB) exceeded by too many entries, or API server errors.

Common situations: High-fanout memoized workflows writing many keys to one ConfigMap (size/conflict pressure); RBAC missing update verb; concurrent workflow completions hitting the same ConfigMap causing 409 conflicts; clusters where Argo was installed without the memoization RBAC rules.

Related errors


AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/d5ad53210974f5f8. Report an issue: GitHub.