containerd/containerd · error

failed to send write: %w

Error message

failed to send write: %w

What it means

remoteWriter.Write sends WriteAction_WRITE requests carrying data chunks through the containerd content proxy. If the RPC fails, the native (errgrpc.ToNative) error is wrapped with 'failed to send write' and Write returns 0 bytes written.

Source

Thrown at core/content/proxy/content_writer.go:91

	}, nil
}

func (rw *remoteWriter) Digest() digest.Digest {
	return rw.digest
}

func (rw *remoteWriter) Write(p []byte) (n int, err error) {
	const maxBufferSize = defaults.DefaultMaxSendMsgSize >> 1
	for data := range slices.Chunk(p, maxBufferSize) {
		offset := rw.offset

		resp, err := rw.send(&contentapi.WriteContentRequest{
			Action: contentapi.WriteAction_WRITE,
			Offset: offset,
			Data:   data,
		})
		if err != nil {
			return 0, fmt.Errorf("failed to send write: %w", errgrpc.ToNative(err))
		}

		written := int(resp.Offset - offset)
		rw.offset += int64(written)
		if resp.Digest != "" {
			rw.digest = digest.Digest(resp.Digest)
		}
		n += written

		if written < len(data) {
			return n, io.ErrShortWrite
		}
	}
	return n, nil
}

func (rw *remoteWriter) Commit(ctx context.Context, size int64, expected digest.Digest, opts ...content.Opt) (err error) {
	defer func() {

View on GitHub (pinned to 4246446a2b)

Solutions

1. Check the wrapped gRPC code: ResourceExhausted means reduce chunk size (default write buffer) or raise the message size limit
2. Retry the transfer from the last known offset using Status(); the proxy writer supports resuming via the offset field
3. Ensure the context is not cancelled/timed out prematurely during large writes
4. Verify the containerd daemon and ttrpc socket stayed connected for the duration of the write

Example fix

// before: one huge chunk
n, err := rw.Write(bigBuffer) // ResourceExhausted
// after: bounded chunks
const chunk = 1 << 20
for len(bigBuffer) > 0 {
    c := bigBuffer
    if len(c) > chunk { c = c[:chunk] }
    if _, err := rw.Write(c); err != nil { return err }
    bigBuffer = bigBuffer[len(c):]
}

// check remote offset before writing next chunk
st, err := rw.Status()
if err != nil { return err }
if st.Offset != expectedOffset {
    return fmt.Errorf("offset drift: remote %d != local %d", st.Offset, expectedOffset)
}

n, err := rw.Write(chunk)
if err != nil {
    switch {
    case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
        return err
    case status.Code(err) == codes.ResourceExhausted:
        return retryWithSmallerChunk(chunk[:len(chunk)/2])
    default:
        return resumeFromStatus(ctx) // re-sync via Status() and continue
    }
}

When it happens

Trigger: Calling Write on a proxied content writer when the underlying write RPC fails — connection reset by the daemon, context cancellation, socket buffer overflow with oversized data chunks, or remote ingest closed.

Common situations: Pushing large layers to a daemon that restarts mid-transfer; writing chunks larger than the ttrpc/gRPC message size limit; cancelled context during blob upload; network interruption between proxy and remote worker.

Related errors

• error getting writer status: %w
• commit failed: %w
• NotImplemented
• unsupported content client %T: %w
• unable to make connection: %w

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/b7d5a47563d5f510. Report an issue: GitHub.