hashicorp/terraform · error

error splitting source data: %s

Error message

error splitting source data: %s

What it means

Raised at the top of multiPartUploadImpl when objectMultiPartSplit() fails. objectMultiPartSplit delegates to SplitSizeToOffsetsAndLimits, which errors when the data size would require more than MaxCount (10000) parts at DefaultFilePartSize (128MiB). So this is the multipart path's size-guard: the state payload is too large to upload in 10000 parts.

Source

Thrown at internal/backend/remote-state/oci/multipart_upload.go:58

	sourceBlocks            chan objectStorageSourceBlock
	osUploadPartResponses   chan objectStorageUploadPartResponse
	wg                      *sync.WaitGroup
	errChan                 chan error
	multipartUploadResponse objectstorage.CreateMultipartUploadResponse
	multipartUploadRequest  objectstorage.CreateMultipartUploadRequest
	logger                  hclog.Logger
}

type objectStorageSourceBlock struct {
	section     *io.SectionReader
	blockNumber *int
}

func (multipartUploadData MultipartUploadData) multiPartUploadImpl(ctx context.Context) error {
	logger := ctx.Value("logger").(hclog.Logger).Named("multiPartUpload")
	sourceBlocks, err := multipartUploadData.objectMultiPartSplit()
	if err != nil {
		return fmt.Errorf("error splitting source data: %s", err)
	}

	multipartUploadRequest := &objectstorage.CreateMultipartUploadRequest{
		NamespaceName:   common.String(multipartUploadData.client.namespace),
		BucketName:      common.String(multipartUploadData.client.bucketName),
		RequestMetadata: multipartUploadData.RequestMetadata,
		CreateMultipartUploadDetails: objectstorage.CreateMultipartUploadDetails{
			Object: common.String(multipartUploadData.client.path),
		},
	}
	if multipartUploadData.client.kmsKeyID != "" {
		multipartUploadRequest.OpcSseKmsKeyId = common.String(multipartUploadData.client.kmsKeyID)
	} else if multipartUploadData.client.SSECustomerKey != "" && multipartUploadData.client.SSECustomerKeySHA256 != "" {
		multipartUploadRequest.OpcSseCustomerKey = common.String(multipartUploadData.client.SSECustomerKey)
		multipartUploadRequest.OpcSseCustomerKeySha256 = common.String(multipartUploadData.client.SSECustomerKeySHA256)
		multipartUploadRequest.OpcSseCustomerAlgorithm = common.String(multipartUploadData.client.SSECustomerAlgorithm)
	}

View on GitHub (pinned to c9def3e214)

Solutions

1. Reduce state size: remove large sensitive values, split the configuration into multiple workspaces/states, or prune deleted resources from state.
2. Raise DefaultFilePartSize (var, multipart_upload.go:21) so fewer parts are needed, keeping under MaxCount — only if the larger part size stays within OCI's per-part limits.
3. Confirm this is not a one-off huge object being written to the state key by mistake.
4. If legitimately large, store the bulk data in object storage outside Terraform and reference it, keeping tfstate small.

Example fix

// before (default 128MiB parts -> >10000 parts for >1.28TiB)
var DefaultFilePartSize int64 = 128 * 1024 * 1024
// after (larger parts reduce part count)
var DefaultFilePartSize int64 = 512 * 1024 * 1024 // keep under OCI per-part max & MaxCount

// Validate size before entering multipart:
dataSize := int64(len(data))
if totalParts := (dataSize + DefaultFilePartSize - 1) / DefaultFilePartSize; totalParts > MaxCount {
    return fmt.Errorf("state of %d bytes would need %d parts (>MaxCount %d); reduce size or raise DefaultFilePartSize", dataSize, totalParts, MaxCount)
}

sourceBlocks, err := multipartUploadData.objectMultiPartSplit()
if err != nil {
    return fmt.Errorf("error splitting source data: %s", err)
}

When it happens

Trigger: multiPartUploadImpl is entered (data > DefaultFilePartSize per client.go:137) and objectMultiPartSplit returns an error — concretely when dataSize > 10000 * 128MiB (~1.28 TiB), making totalParts exceed MaxCount at multipart_upload.go:197.

Common situations: An abnormally huge terraform.tfstate (1+ TiB) due to runaway resource count, large embedded binary blobs in state, or state bloat from mismanaged resources; testing with an oversized synthetic state.

Related errors

• error splitting data into parts: %s
• file exceeds maximum part count
• error creating multipart upload: %s
• failed to upload part: %s
• not all parts uploaded successfully, multipart upload aborte

AI-assisted analysis of hashicorp/terraform@c9def3e214 (2026-08-07). Data as JSON: /api/errors/916e8db0ddf16916. Report an issue: GitHub.