{"record":{"id":"f3d4a0fc35f87824","repo":"argoproj/argo-workflows","slug":"could-not-save-to-config-map-cache-w","errorCode":null,"errorMessage":"could not save to config map cache: %w","messagePattern":"could not save to config map cache: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"workflow/controller/cache/configmap_cache.go","lineNumber":173,"sourceCode":"\t\tc.logError(ctx, err, logging.Fields{\"key\": key}, errString)\n\t\treturn err\n\t}\n\n\tc.lock.Lock()\n\tdefer c.lock.Unlock()\n\n\tc.logInfo(ctx, logging.Fields{\"key\": key, \"nodeID\": nodeID}, \"Saving ConfigMap cache entry\")\n\n\tcache, err := c.kubeClient.CoreV1().ConfigMaps(c.namespace).Get(ctx, c.name, metav1.GetOptions{})\n\tif apierr.IsNotFound(err) || cache == nil {\n\t\tcache, err = c.kubeClient.CoreV1().ConfigMaps(c.namespace).Create(ctx, &apiv1.ConfigMap{\n\t\t\tObjectMeta: metav1.ObjectMeta{\n\t\t\t\tName: c.name,\n\t\t\t},\n\t\t}, metav1.CreateOptions{})\n\t\tif err != nil {\n\t\t\tc.logError(ctx, err, logging.Fields{\"key\": key, \"nodeID\": nodeID}, \"Error saving to ConfigMap cache\")\n\t\t\treturn fmt.Errorf(\"could not save to config map cache: %w\", err)\n\t\t}\n\t} else {\n\t\tvalidateErr := c.validateConfigmap(ctx, cache)\n\t\tif validateErr != nil {\n\t\t\treturn validateErr\n\t\t}\n\t}\n\n\tcreationTime := time.Now()\n\tcache.SetLabels(map[string]string{common.LabelKeyConfigMapType: common.LabelValueTypeConfigMapCache})\n\n\tnewEntry := Entry{\n\t\tNodeID:            nodeID,\n\t\tOutputs:           value,\n\t\tCreationTimestamp: metav1.Time{Time: creationTime},\n\t\tLastHitTimestamp:  metav1.Time{Time: creationTime},\n\t}\n","sourceCodeStart":155,"sourceCodeEnd":191,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/controller/cache/configmap_cache.go#L155-L191","documentation":"save() persists a memoization entry into a ConfigMap; if the ConfigMap does not yet exist it is created via the Kubernetes API. When that Create call fails (RBAC denial, namespace issues, quota, API errors), the failure is wrapped as 'could not save to config map cache' and the memoized node is marked failed.","triggerScenarios":"First-ever save for a given memoization ConfigMap where CoreV1().ConfigMaps(ns).Create fails: service account lacks create permission on configmaps, namespace mismatch, resource quota exceeded, API server unreachable, or name/label conflicts.","commonSituations":"Memoization ConfigMap in a namespace where the controller's SA has no create rights (RBAC not synced after installing Argo); Kubernetes 1.22+ with stale manifests; air-gapped clusters with API latency; the ConfigMap was deleted mid-run causing a create/conflict race.","solutions":["Check controller logs for the underlying error (logError 'Error saving to ConfigMap cache') to see the real Kubernetes cause.","Verify RBAC: the workflow-controller service account needs create/update/get on configmaps in the workflow namespace (see the memoization docs RBAC snippet).","Confirm the ConfigMap name/namespace from the memoize config and that no quota (resourcequota on configmap count) blocks creation.","Re-run the workflow after fixing — memoization save failures are not retried automatically for the same node.","If a name conflict race, retry — the next save will take the update path instead of create."],"exampleFix":"# before: RBAC missing\n# after: grant the controller SA access\nkind: Role\nrules:\n  - apiGroups: [\"\"]\n    resources: [\"configmaps\"]\n    verbs: [\"create\", \"get\", \"update\", \"delete\"]\n# bind it to the workflow-controller service account in the workflow namespace","handlingStrategy":"try-catch","validationCode":"// pre-check permissions before submitting memoized workflows\nkubectl auth can-i create configmaps -n <workflow-ns> --as=system:serviceaccount:<argo-ns>:workflow-controller","typeGuard":null,"tryCatchPattern":"err := cache.Save(ctx, key, nodeID, entry)\nif err != nil {\n  var serr *apierrors.StatusError\n  if errors.As(err, &serr) && apierrors.IsForbidden(err) {\n    // fix RBAC then re-run; memoization save is not auto-retried\n  }\n  return fmt.Errorf(\"memoization save failed: %w\", err)\n}","preventionTips":["Install the memoization RBAC rules from the Argo docs for the controller SA in every workflow namespace.","Check resource quotas limiting configmap counts in target namespaces.","Uniquely name memoization ConfigMaps per memoize config to avoid create/update races.","Verify kubectl auth can-i create/update configmaps before enabling memoization in a new namespace."],"tags":["memoization","cache","kubernetes","rbac"],"backgroundTag":"configmap-save-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}