What causes "error splitting source data: %s" in hashicorp/terraform?

Reduce state size: split the root module into workspaces/modules so each state file is smaller.

error splitting source data

Error message

error splitting source data: %s

What it means

objectMultiPartSplit failed, almost always because SplitSizeToOffsetsAndLimits returned 'file exceeds maximum part count' — the state byte size implies more than MaxCount (10000) parts at DefaultFilePartSize (128 MiB), i.e. a state file larger than ~1.28 TiB. This is reached only when dataSize > DefaultFilePartSize so multipart is chosen.

Solutions

Reduce state size: split the root module into workspaces/modules so each state file is smaller.
Raise DefaultFilePartSize so fewer parts are needed (must stay within OCI's per-part max and ≤ MaxFilePartSize 50 GiB; keep partSize*MaxCount above your state size).
Prune stale/deleted resources from state with `terraform state rm` / `state pull` + manual cleanup.
If hit during a one-off import, split the import into smaller batches.

Example fix

// before: state of 1.4 TiB with DefaultFilePartSize = 128 MiB
//   totalParts = ceil(1.4 TiB / 128 MiB) = 11200 > MaxCount(10000) -> 315
// after: raise part size so parts fit under MaxCount
var DefaultFilePartSize int64 = 256 * 1024 * 1024 // 256 MiB -> 5600 parts
// and split the root module across workspaces to keep state bounded

Defensive patterns

Strategy: validation

Validate before calling

// Validate state size against the multipart limits before uploading
func validateStateForMultipart(size int64) error {
    parts := (size + DefaultFilePartSize - 1) / DefaultFilePartSize
    if parts > MaxCount {
        return fmt.Errorf("state (%d bytes) needs %d parts > MaxCount %d; split the workspace or raise DefaultFilePartSize",
            size, parts, MaxCount)
    }
    return nil
}

Prevention

Keep state files small by splitting large root modules into workspaces/modules.
When tuning DefaultFilePartSize down, recompute partSize*MaxCount against your largest state.
Prune deleted resources from state periodically.
Alert on state size growth so you act before hitting the limit.

When it happens

Trigger: State byte size exceeds MaxCount * DefaultFilePartSize; DefaultFilePartSize was lowered globally without recomputing the limit; an enormous state from a monolithic root module.

Common situations: A single root module tracking tens of thousands of resources; tuning DefaultFilePartSize down for memory reasons; importing a huge dataset into state.

Related errors

AI-assisted analysis of hashicorp/terraform@d32a084675 (2026-08-11). Data as JSON: /api/errors/916e8db0ddf16916. Report an issue: GitHub.

Appendix: source

Thrown at internal/backend/remote-state/oci/multipart_upload.go:58

	sourceBlocks            chan objectStorageSourceBlock
	osUploadPartResponses   chan objectStorageUploadPartResponse
	wg                      *sync.WaitGroup
	errChan                 chan error
	multipartUploadResponse objectstorage.CreateMultipartUploadResponse
	multipartUploadRequest  objectstorage.CreateMultipartUploadRequest
	logger                  hclog.Logger
}

type objectStorageSourceBlock struct {
	section     *io.SectionReader
	blockNumber *int
}

func (multipartUploadData MultipartUploadData) multiPartUploadImpl(ctx context.Context) error {
	logger := ctx.Value("logger").(hclog.Logger).Named("multiPartUpload")
	sourceBlocks, err := multipartUploadData.objectMultiPartSplit()
	if err != nil {
		return fmt.Errorf("error splitting source data: %s", err)
	}

	multipartUploadRequest := &objectstorage.CreateMultipartUploadRequest{
		NamespaceName:   common.String(multipartUploadData.client.namespace),
		BucketName:      common.String(multipartUploadData.client.bucketName),
		RequestMetadata: multipartUploadData.RequestMetadata,
		CreateMultipartUploadDetails: objectstorage.CreateMultipartUploadDetails{
			Object: common.String(multipartUploadData.client.path),
		},
	}
	if multipartUploadData.client.kmsKeyID != "" {
		multipartUploadRequest.OpcSseKmsKeyId = common.String(multipartUploadData.client.kmsKeyID)
	} else if multipartUploadData.client.SSECustomerKey != "" && multipartUploadData.client.SSECustomerKeySHA256 != "" {
		multipartUploadRequest.OpcSseCustomerKey = common.String(multipartUploadData.client.SSECustomerKey)
		multipartUploadRequest.OpcSseCustomerKeySha256 = common.String(multipartUploadData.client.SSECustomerKeySHA256)
		multipartUploadRequest.OpcSseCustomerAlgorithm = common.String(multipartUploadData.client.SSECustomerAlgorithm)
	}

View on GitHub (pinned to d32a084675)