{"record":{"id":"fb4a656ac7de20af","repo":"argoproj/argo-workflows","slug":"failed-to-update-configmap-s-w","errorCode":null,"errorMessage":"failed to update ConfigMap %s: %w","messagePattern":"failed to update ConfigMap (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"workflow/controller/cache_gc.go","lineNumber":65,"sourceCode":"\t\t}\n\t\tif time.Since(entry.LastHitTimestamp.Time) > wfc.gcAfterNotHitDuration {\n\t\t\tlogger.WithFields(logging.Fields{\"key\": key, \"configMap\": cm.Name, \"gcAfterNotHitDuration\": wfc.gcAfterNotHitDuration}).Info(ctx, \"Deleting entry in ConfigMap since it's not been hit\")\n\t\t\tdelete(cm.Data, key)\n\t\t\tmodified = true\n\t\t}\n\t}\n\tif len(cm.Data) == 0 {\n\t\terr := wfc.kubeclientset.CoreV1().ConfigMaps(cm.Namespace).Delete(ctx, cm.Name, metav1.DeleteOptions{})\n\t\tif err != nil {\n\t\t\tif apierr.IsNotFound(err) {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"failed to delete ConfigMap %s: %w\", cm.Name, err)\n\t\t}\n\t} else if modified {\n\t\t_, err := wfc.kubeclientset.CoreV1().ConfigMaps(cm.Namespace).Update(ctx, cm, metav1.UpdateOptions{})\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to update ConfigMap %s: %w\", cm.Name, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n","sourceCodeStart":47,"sourceCodeEnd":71,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/controller/cache_gc.go#L47-L71","documentation":"Returned by cleanupUnusedCache in workflow/controller/cache_gc.go:65 when the periodic memoization-cache GC removes stale entries from a cache ConfigMap and the subsequent ConfigMap Update API call fails. The underlying Kubernetes client error is wrapped with %w, so errors.Is/As on the cause (e.g. Conflict, Forbidden) works. It is logged (not fatal) by syncAllCacheForGC and retried on the next GC tick.","triggerScenarios":"GC deletes expired entries from a cache ConfigMap (workflow-controller-configmap referenced by a memoize template with caches->configMap) and calls CoreV1().ConfigMaps(...).Update(ctx, cm, ...) which fails — most commonly an optimistic-concurrency 409 Conflict because the memoization loop or another controller updated the ConfigMap since it was read from the informer, or RBAC failure, or the ConfigMap was deleted concurrently (404).","commonSituations":"Heavy memoization traffic on the same cache ConfigMap causing constant update conflicts; controller service account lacking update permission on ConfigMaps in the namespace; the cache ConfigMap deleted mid-GC; extremely large ConfigMaps nearing the 1MiB limit so writes are rejected.","solutions":["Retry is built in: the next GC cycle re-reads the ConfigMap from the informer and re-applies deletion — usually no action needed.","If conflicts are frequent, reduce concurrent writers to the cache ConfigMap (fewer memoization hits racing) or accept GC lag.","Check RBAC: the controller service account needs configmaps update/delete on the workflow namespace.","Verify the ConfigMap still exists and its size is under the 1MiB limit; prune huge cache entries or use a shorter gcAfterNotHitDuration."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before relying on GC, confirm RBAC and concurrency\nconst cm = await k8s.readNamespacedConfigMap(name, ns)\n// ensure resourceVersion is fresh before update; expect 409 under contention","typeGuard":null,"tryCatchPattern":"try {\n  await gcCacheConfigMap(cm)\n} catch (err) {\n  if (isConflict(err)) { /* resync from informer; retry next GC tick */ }\n  else if (isForbidden(err)) { fixControllerRBAC() }\n  else { logAndRetryLater(err) }\n}","preventionTips":["Ensure the controller service account can update/delete ConfigMaps in managed namespaces","Minimize concurrent writers to memoization cache ConfigMaps","Keep ConfigMap size well under the 1MiB limit","Remember GC is self-healing: transient failures are retried next cycle"],"tags":["kubernetes","configmap","cache","gc","conflict"],"backgroundTag":"configmap-update-conflict","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}