What causes "error splitting data into parts: %s" in hashicorp/terraform?

Investigate why the state file is so large — check for accidental binary blobs, base64-encoded secrets, or corrupted state data.

hashicorp/terraform · error

error splitting data into parts

Error message

error splitting data into parts: %s

What it means

Thrown by objectMultiPartSplit() in the OCI remote-state backend when SplitSizeToOffsetsAndLimits() fails to divide the state file payload into upload parts. The underlying cause is surfaced via %s, which is typically the 'file exceeds maximum part count' error. This indicates the state data is too large to be chunked within OCI's multipart upload constraints (DefaultFilePartSize=128MB, MaxCount=10000).

Solutions

Investigate why the state file is so large — check for accidental binary blobs, base64-encoded secrets, or corrupted state data.
If running in a test, ensure DefaultFilePartSize is reset to its original 128MB value after the test mutates it, or use smaller test fixtures.
If the state genuinely needs to exceed MaxCount*DefaultFilePartSize, you cannot use this multipart path — consider splitting state across multiple workspaces or reducing state size.
Verify the fallback logic in client.go:147 works — when multipart fails and dataSize <= MaxFilePartSize, it falls back to single-part upload.

Example fix

// In test code, always defer restoration of DefaultFilePartSize:
DefaultFilePartSize = 100
defer func() { DefaultFilePartSize = 128 * 1024 * 1024 }()

Defensive patterns

Strategy: validation

Validate before calling

// Before uploading state, validate size against multipart constraints
const (
    DefaultFilePartSize int64 = 128 * 1024 * 1024 // 128MB
    MaxCount            int64 = 10000
)

func validateStateSizeForMultipart(data []byte) error {
    dataSize := int64(len(data))
    if dataSize <= DefaultFilePartSize {
        return nil // single-part upload, no limit concern
    }
    totalParts := (dataSize + DefaultFilePartSize - 1) / DefaultFilePartSize
    if totalParts > MaxCount {
        return fmt.Errorf("state file size %d bytes requires %d parts, exceeding max %d; reduce state size", dataSize, totalParts, MaxCount)
    }
    return nil
}

Prevention

Keep Terraform/OpenTofu state files small by splitting large resources into separate workspaces.
Avoid storing binary blobs or base64-encoded large secrets in state — use external secret backends.
In tests, always restore DefaultFilePartSize to its original value after mutation.

When it happens

Trigger: Called from client.go:Put() when dataSize > 128MB (DefaultFilePartSize). SplitSizeToOffsetsAndLimits(dataSize) computes totalParts = ceil(dataSize/128MB) and returns an error when totalParts > 10000, i.e. the state file exceeds ~1.28TB. The wrapped error propagates up through multiPartUploadImpl.

Common situations: State files are virtually never this large in practice — hitting this means either a corrupted/garbage state blob or a test harness that set DefaultFilePartSize to a very small value (tests set it to 100 bytes) while feeding large data. In production, this error path is nearly unreachable with well-formed state.

Related errors

AI-assisted analysis of hashicorp/terraform@d32a084675 (2026-08-11). Data as JSON: /api/errors/5d20b7694917d883. Report an issue: GitHub.

Appendix: source

Thrown at internal/backend/remote-state/oci/multipart_upload.go:172

		ObjectName:         multipartUploadResponse.Object,
		OpcClientRequestId: multipartUploadResponse.OpcClientRequestId,
		RequestMetadata:    multipartUploadRequest.RequestMetadata,
		CommitMultipartUploadDetails: objectstorage.CommitMultipartUploadDetails{
			PartsToCommit: commitMultipartUploadPartDetails,
		},
	}
	_, err = multipartUploadData.client.objectStorageClient.CommitMultipartUpload(context.Background(), commitMultipartUploadRequest)
	if err != nil {
		return fmt.Errorf("failed to commit multipart upload: %s", err)
	}

	return nil
}
func (m MultipartUploadData) objectMultiPartSplit() ([]objectStorageSourceBlock, error) {
	dataSize := int64(len(m.Data))
	offsets, partSize, err := SplitSizeToOffsetsAndLimits(dataSize)
	if err != nil {
		return nil, fmt.Errorf("error splitting data into parts: %s", err)
	}
	sourceBlocks := make([]objectStorageSourceBlock, len(offsets))
	for i := range offsets {
		start := offsets[i]
		end := start + partSize
		if end > dataSize {
			end = dataSize
		}
		sourceBlocks[i] = objectStorageSourceBlock{
			section:     io.NewSectionReader(bytes.NewReader(m.Data), start, end-start),
			blockNumber: common.Int(i + 1),
		}
	}
	return sourceBlocks, nil
}

/*
SplitSizeToOffsetsAndLimits splits a file size into chunks based on DefaultFilePartSize.

View on GitHub (pinned to d32a084675)