{"record":{"id":"a75151d71577e4ea","repo":"containerd/containerd","slug":"push-is-on-going-w","errorCode":null,"errorMessage":"push is on-going: %w","messagePattern":"push is on-going: %w","errorType":"error_code","errorClass":"errdefs.ErrUnavailable","httpStatus":503,"severity":"warning","filePath":"core/remotes/docker/pusher.go","lineNumber":100,"sourceCode":"\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif p.dockerBase.warningHandler != nil {\n\t\tctx = context.WithValue(ctx, warningSourceKey{}, WarningSource{\n\t\t\tDesc:   &desc,\n\t\t\tDigest: &desc.Digest,\n\t\t})\n\t}\n\tstatus, err := p.tracker.GetStatus(ref)\n\tif err == nil {\n\t\tif status.Committed && status.Offset == status.Total {\n\t\t\treturn nil, fmt.Errorf(\"ref %v: %w\", ref, errdefs.ErrAlreadyExists)\n\t\t}\n\t\tif unavailableOnFail && status.ErrClosed == nil {\n\t\t\t// Another push of this ref is happening elsewhere. The rest of function\n\t\t\t// will continue only when `errdefs.IsNotFound(err) == true` (i.e. there\n\t\t\t// is no actively-tracked ref already).\n\t\t\treturn nil, fmt.Errorf(\"push is on-going: %w\", errdefs.ErrUnavailable)\n\t\t}\n\t\t// TODO: Handle incomplete status\n\t} else if !errdefs.IsNotFound(err) {\n\t\treturn nil, fmt.Errorf(\"failed to get status: %w\", err)\n\t}\n\n\thosts := p.filterHosts(HostCapabilityPush)\n\tif len(hosts) == 0 {\n\t\treturn nil, fmt.Errorf(\"no push hosts: %w\", errdefs.ErrNotFound)\n\t}\n\n\tvar (\n\t\tisManifest bool\n\t\texistCheck []string\n\t\thost       = hosts[0]\n\t)\n\n\tif images.IsManifestType(desc.MediaType) || images.IsIndexType(desc.MediaType) {","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/containerd/containerd/blob/4246446a2bf7d03837b0244118d858799393bd80/core/remotes/docker/pusher.go#L82-L118","documentation":"dockerPusher.push checks the StatusTracker; if another push of the same ref is currently active (a status exists that is not committed and not closed, and unavailableOnFail is set for the Writer path), it returns errdefs.ErrUnavailable with \"push is on-going\". This lets concurrent clients know an upload is already in progress elsewhere.","triggerScenarios":"Calling Pusher.Writer(ctx, ...) (unavailableOnFail=true) while another goroutine/process holds an active, unclosed status for the same ref in the tracker; concurrent pushes of the same layer from parallel jobs in one process using a shared InFlightTracker.","commonSituations":"Parallel layer uploads in one containerd client; retry storms re-entering push while the first attempt is still live; multiple controllers sharing an in-memory status tracker pushing the same manifest concurrently.","solutions":["Serialize pushes per ref (single-flight/mutex) so only one push runs at a time","Wait for the other push to finish, then retry — treat ErrUnavailable as transient","Check whether concurrent workers are deduplicating refs properly (InFlightTracker usage)","If the other push crashed without closing status, restart the client/process to clear tracker state"],"exampleFix":"// before\ngo pushLayer(ctx, desc); go pushLayer(ctx, desc)\n// after\nvar mu sync.Mutex\ngo func(){ mu.Lock(); defer mu.Unlock(); pushLayer(ctx, desc) }()","handlingStrategy":"retry","validationCode":"// check for an active push before starting another\nif st, err := tracker.GetStatus(ref); err == nil && !st.Committed && st.ErrClosed == nil {\n    return fmt.Errorf(\"push active for %s, wait\", ref)\n}","typeGuard":"null","tryCatchPattern":"err := pusher.Push(ctx, desc)\nif errdefs.IsUnavailable(err) {\n    time.Sleep(backoff)\n    return pusher.Push(ctx, desc) // previous push likely finished\n}\nreturn err","preventionTips":["Single-flight pushes per ref across goroutines","Use InFlightTracker-based deduplication for parallel layer uploads","Bound retries with backoff so concurrent pushes resolve instead of racing"],"tags":["go","push","concurrency","unavailable"],"backgroundTag":"push-already-in-progress","analyzedSha":"4246446a2bf7d03837b0244118d858799393bd80","analyzedAt":"2026-09-02T00:14:43.053Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}