{"record":{"id":"c6b8d85b8e5b1dec","repo":"argoproj/argo-workflows","slug":"unable-to-compress-or-offload-workflow-nodes-w","errorCode":null,"errorMessage":"unable to compress or offload workflow nodes: %w","messagePattern":"unable to compress or offload workflow nodes: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"workflow/util/util.go","lineNumber":567,"sourceCode":"\t\t\tif node.IsActiveSuspendNode() {\n\t\t\t\tif err := OverrideOutputParametersWithDefault(node.Outputs); err != nil {\n\t\t\t\t\treturn false, err\n\t\t\t\t}\n\t\t\t\tnode.Phase = wfv1.NodeSucceeded\n\t\t\t\tif node.Message != \"\" {\n\t\t\t\t\tuiMsg = node.Message + \"; \" + uiMsg\n\t\t\t\t}\n\t\t\t\tnode.Message = uiMsg\n\t\t\t\tnode.FinishedAt = metav1.Time{Time: time.Now().UTC()}\n\t\t\t\twf.Status.Nodes.Set(ctx, nodeID, node)\n\t\t\t\tworkflowUpdated = true\n\t\t\t}\n\t\t}\n\n\t\tif workflowUpdated {\n\t\t\terr := hydrator.Dehydrate(ctx, wf)\n\t\t\tif err != nil {\n\t\t\t\treturn false, fmt.Errorf(\"unable to compress or offload workflow nodes: %w\", err)\n\t\t\t}\n\t\t\tcreator.LabelActor(ctx, wf, creator.ActionResume)\n\t\t\t_, err = wfIf.Update(ctx, wf, metav1.UpdateOptions{})\n\t\t\tif err != nil {\n\t\t\t\tif apierr.IsConflict(err) {\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t}\n\t\treturn true, nil\n\t})\n\treturn err\n}\n\nfunc SelectorMatchesNode(selector fields.Selector, node wfv1.NodeStatus) bool {\n\tnodeFields := fields.Set{\n\t\t\"displayName\":  node.DisplayName,","sourceCodeStart":549,"sourceCodeEnd":585,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/util/util.go#L549-L585","documentation":"During resume/retry-style reconciliation in workflow/util, when node status was changed the controller dehydrates (compresses/offloads) the workflow node status before updating the object via hydrator.Dehydrate. If compression or SQL offloading fails, the update is aborted with this error. It protects the cluster from writing an over-sized Workflow object that the API server or etcd would reject.","triggerScenarios":"Resuming/operating on a workflow whose Status.Nodes exceeds the in-object size limit and whose offload target (SQL archive database) is unreachable, misconfigured, or fails; or compression of the node status fails.","commonSituations":"Misconfigured persistence (persistence section in workflow-controller-configmap pointing at a down Postgres/MySQL), expired DB credentials, or very large node histories from deeply-fanned-out DAGs.","solutions":["Verify the workflow-controller-configmap persistence settings (archive, postgresql/mysql host, credentials) and that the DB is reachable.","Check controller logs for the wrapped hydrator/persist error for the exact cause.","Reduce node status size (avoid huge captured outputs/parameters in nodes) or archive/clean old workflows.","Retry the operation once the database is healthy."],"exampleFix":"// before (workflow-controller-configmap)\npersistence: {}\n// after\npersistence:\n  archive: true\n  postgresql:\n    host: postgres.postgres\n    database: argo\n    userNameSecret: {name: argo-postgres-config}\n    passwordSecret: {name: argo-postgres-config}","handlingStrategy":"retry","validationCode":"cfg, _ := config.ControllerConfig()\nif cfg.Persistence == nil || !cfg.Persistence.Archive {\n\t// offloading disabled; large workflows will fail dehydrate\n}","typeGuard":null,"tryCatchPattern":"updated, err := util.OperateWorkflow(ctx, ...)\nif err != nil && strings.Contains(err.Error(), \"unable to compress or offload workflow nodes\") {\n\t// transient: requeue and retry after checking DB health\n\treturn requeueAfter(time.Minute)\n}","preventionTips":["Keep the archive database (Postgres/MySQL) reachable and credentials current.","Monitor controller logs for persist/sqldb errors.","Keep node outputs small (avoid dumping huge payloads into parameters).","Rotate old workflows out of the archive."],"tags":["hydration","persistence","controller"],"backgroundTag":"workflow-node-offload-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}