hashicorp/terraform · error

error reading source block %d: %w

Error message

error reading source block %d: %w

What it means

Raised inside uploadPartsWorker when reading a source block from its in-memory io.SectionReader fails. Each block is an io.SectionReader over a bytes.Reader built from the in-memory state payload (multipart_upload.go:182), so a read failure here is unexpected: it indicates the section was misconfigured or the underlying buffer changed/shrank under the worker. The error names the offending block number.

Source

Thrown at internal/backend/remote-state/oci/multipart_upload.go:212

func SplitSizeToOffsetsAndLimits(size int64) ([]int64, int64, error) {
	partSize := DefaultFilePartSize
	totalParts := (size + partSize - 1) / partSize
	if totalParts > MaxCount {
		return nil, 0, fmt.Errorf("file exceeds maximum part count")
	}
	offsets := make([]int64, totalParts)
	for i := range offsets {
		offsets[i] = int64(i) * partSize
	}
	return offsets, partSize, nil
}

func (ctx *objectStorageMultiPartUploadContext) uploadPartsWorker() {
	for block := range ctx.sourceBlocks {
		buffer := make([]byte, block.section.Size())
		_, err := block.section.Read(buffer)
		if err != nil {
			ctx.errChan <- fmt.Errorf("error reading source block %d: %w", block.blockNumber, err)
			return
		}
		tmpLength := int64(len(buffer))
		sum := md5.Sum(buffer)
		uploadPartRequest := &objectstorage.UploadPartRequest{
			UploadId:       ctx.multipartUploadResponse.UploadId,
			ObjectName:     ctx.multipartUploadResponse.Object,
			NamespaceName:  ctx.multipartUploadResponse.Namespace,
			BucketName:     ctx.multipartUploadResponse.Bucket,
			ContentLength:  &tmpLength,
			UploadPartBody: io.NopCloser(bytes.NewReader(buffer)),
			UploadPartNum:  block.blockNumber,
			ContentMD5:     common.String(base64.StdEncoding.EncodeToString(sum[:])),
			RequestMetadata: common.RequestMetadata{
				RetryPolicy: getDefaultRetryPolicy(),
			},
		}

View on GitHub (pinned to c9def3e214)

Solutions

1. Treat MultipartUploadData.Data as immutable for the whole upload; never write to it after multiPartUploadImpl starts.
2. If you construct MultipartUploadData in custom code, pass a defensive copy so the caller cannot mutate it mid-upload.
3. Verify SplitSizeToOffsetsAndLimits offsets stay within len(Data); under a stock Terraform build this is guaranteed, so a read error suggests external mutation or memory corruption.
4. Run with the race detector (-race) in tests to catch concurrent buffer access.

Example fix

// before: shared buffer mutated while workers read it
data := getStateBytes()
mu := MultipartUploadData{client: c, Data: data}
go mutate(data)          // data race -> error reading source block
mu.multiPartUploadImpl(ctx)
// after: pass an immutable copy
original := getStateBytes()
data := make([]byte, len(original))
copy(data, original)
mu := MultipartUploadData{client: c, Data: data}
mu.multiPartUploadImpl(ctx)

// Ensure Data is immutable for the upload duration; pass a copy when constructing:
buf := make([]byte, len(original))
copy(buf, original)
mu := MultipartUploadData{client: c, Data: buf}
// offsets guaranteed within len(buf) by SplitSizeToOffsetsAndLimits.

// A read error here is almost always a data race; classify:
if errors.Is(err, io.EOF) { /* section shorter than expected -> miscomputed offset */ }

if _, err := block.section.Read(buffer); err != nil {
    ctx.errChan <- fmt.Errorf("error reading source block %d: %w", block.blockNumber, err)
    return
}

When it happens

Trigger: block.section.Read(buffer) at multipart_upload.go:210 returns a non-nil, non-io.EOF error: the SectionReader is asked for more bytes than remain, or the underlying slice was mutated/replaced concurrently with the read (data race on MultipartUploadData.Data).

Common situations: Concurrent mutation of the Data slice while multipart upload workers are reading it (sharing the buffer across goroutines unsafely); a miscomputed offset/size in SplitSizeToOffsetsAndLimits producing a SectionReader beyond the buffer length; a custom integration reusing the Data buffer before upload completes.

Related errors

• not all parts uploaded successfully, multipart upload aborte
• unable to read 'content' from response: %w
• failed to read existing lock file content: %w
• error splitting source data: %s
• error creating multipart upload: %s

AI-assisted analysis of hashicorp/terraform@c9def3e214 (2026-08-07). Data as JSON: /api/errors/9ad55099b6905813. Report an issue: GitHub.