{"record":{"id":"48f18b1e94383c72","repo":"vxcontrol/pentagi","slug":"failed-to-update-container-status-to-deleted-w","errorCode":null,"errorMessage":"failed to update container status to deleted: %w","messagePattern":"failed to update container status to deleted: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/pkg/docker/client.go","lineNumber":777,"sourceCode":"\toptions := client.ContainerRemoveOptions{\n\t\tRemoveVolumes: true,\n\t\tForce:         true,\n\t}\n\tif _, err := dc.client.ContainerRemove(ctx, containerID, options); err != nil {\n\t\tif !cerrdefs.IsNotFound(err) {\n\t\t\treturn fmt.Errorf(\"failed to remove container: %w\", err)\n\t\t}\n\t\t// already gone (removed manually, or a prior call already succeeded);\n\t\t// still mark it deleted below so the database row does not go stale.\n\t\tlogger.WithError(err).Warn(\"container not found\")\n\t}\n\n\t_, err := dc.db.UpdateContainerStatus(ctx, database.UpdateContainerStatusParams{\n\t\tStatus: database.ContainerStatusDeleted,\n\t\tID:     dbID,\n\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to update container status to deleted: %w\", err)\n\t}\n\n\tlogger.Info(\"container removed\")\n\n\treturn nil\n}\n\nfunc (dc *dockerClient) Cleanup(ctx context.Context) error {\n\tlogger := dc.logger.WithContext(ctx).WithField(\"docker\", \"cleanup\")\n\tlogger.Info(\"cleaning up containers and making all flows finished...\")\n\n\tflows, err := dc.db.GetFlows(ctx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get all flows: %w\", err)\n\t}\n\n\tcontainers, err := dc.db.GetContainers(ctx)\n\tif err != nil {","sourceCodeStart":759,"sourceCodeEnd":795,"githubUrl":"https://github.com/vxcontrol/pentagi/blob/ea665308baaff015b226f308438a68d929d0f29b/backend/pkg/docker/client.go#L759-L795","documentation":"The Docker container was removed successfully, but marking the DB row as ContainerStatusDeleted failed. The row would go stale (still showing the container as present) while the real container is gone.","triggerScenarios":"dc.db.UpdateContainerStatus with Status=ContainerStatusDeleted errors: lost DB connection, deadlock/lock timeout on the containers row, concurrent delete of the row, or constraint violation.","commonSituations":"Postgres failover or pool exhaustion during cleanup; two teardown paths racing to update the same dbID; long transaction holding a row lock; backend reconnected to a read-only replica.","solutions":["Check database connectivity and whether the backend is pointed at a read-only replica","Inspect row locks (`pg_locks`, `pg_stat_activity`) for blocking transactions and resolve them","Retry the update; since the container is already removed, this is a bookkeeping-only operation","Add an idempotency guard: if the row is already deleted or missing, treat the update as done"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := dc.RemoveContainer(ctx, id, dbID); err != nil {\n    if strings.Contains(err.Error(), \"status to deleted\") {\n        // container already gone; reconcile row later\n        reconcileQueue.Add(dbID)\n    }\n}","preventionTips":["Serialize status updates per container row (advisory lock or unique worker)","Use short backoff retries for bookkeeping updates","Ensure the backend never writes through a read-only replica","Run a periodic reconciler that marks rows deleted for missing containers"],"tags":["database","postgresql","stale-state"],"backgroundTag":"database-update-failed","analyzedSha":"ea665308baaff015b226f308438a68d929d0f29b","analyzedAt":"2026-09-01T14:16:31.421Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}