containerd/containerd · error

truncate failed: %w

Error message

truncate failed: %w

What it means

The content service's Write RPC failed to truncate the underlying writer back to the requested offset. This happens only when a client restarts a write at offset 0 after having already written data, and the writer's Truncate call fails (I/O or state error).

Source

Thrown at plugins/services/content/contentserver/contentserver.go:391

		}

		switch req.Action {
		case api.WriteAction_STAT:
			msg.Digest = wr.Digest().String()
			msg.StartedAt = protobuf.ToTimestamp(ws.StartedAt)
			msg.UpdatedAt = protobuf.ToTimestamp(ws.UpdatedAt)
			msg.Total = total
		case api.WriteAction_WRITE, api.WriteAction_COMMIT:
			if req.Offset > 0 {
				// validate the offset if provided
				if req.Offset != ws.Offset {
					return status.Errorf(codes.OutOfRange, "write @%v must occur at current offset %v", req.Offset, ws.Offset)
				}
			}

			if req.Offset == 0 && ws.Offset > 0 {
				if err := wr.Truncate(req.Offset); err != nil {
					return fmt.Errorf("truncate failed: %w", err)
				}
				msg.Offset = req.Offset
			}

			// issue the write if we actually have data.
			if len(req.Data) > 0 {
				// While this looks like we could use io.WriterAt here, because we
				// maintain the offset as append only, we just issue the write.
				n, err := wr.Write(req.Data)
				if err != nil {
					return errgrpc.ToGRPC(err)
				}

				if n != len(req.Data) {
					// TODO(stevvooe): Perhaps, we can recover this by including it
					// in the offset on the write return.
					return status.Errorf(codes.DataLoss, "wrote %v of %v bytes", n, len(req.Data))
				}

View on GitHub (pinned to 4246446a2b)

Solutions

1. Retry the write after the underlying I/O condition (disk space, read-only fs) is fixed
2. Abort the existing write session (content Abort) and start a fresh Write
3. Ensure only one writer is active per ref to avoid conflicting offsets
4. Update the client to resume from the server-reported offset instead of restarting at 0

Example fix

// before: client restarts upload at 0 after partial write
req := &contentapi.WriteRequest{Ref: ref, Offset: 0, Data: data}
// after: resume from server status or abort first
// st, _ := client.Status(ctx, ref); if st.Offset > 0 { client.Abort(ctx, ref) }

// client: check server-side offset before restarting at 0
st, err := client.Status(ctx, ref)
if err == nil && st.Offset > 0 { /* resume instead of Offset:0 */ }

err := stream.Send(req); ...
if err != nil && strings.Contains(err.Error(), "truncate failed") {
    client.Abort(ctx, ref)
    return startFreshWrite(ctx, ref, data)
}

When it happens

Trigger: Calling the content Write stream with Offset==0 while the server-side writer status shows Offset>0, and wr.Truncate(0) fails — e.g. the writer is no longer truncatable or the backing file I/O errors.

Common situations: A client restarting an interrupted blob upload from scratch while a previous partial write exists; disk full or read-only filesystem during truncate; concurrent writers on the same blob ref.

Related errors

• unable to discard to offset
• failed to get reader: %w
• failed to perform sync: %w
• writer has been reset
• bufpipe: read/write on closed pipe

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/d93f4bee940677cf. Report an issue: GitHub.