benbjohnson/litestream · error

s3: upload to %s: part size must be at least %d bytes

Error message

s3: upload to %s: part size must be at least %d bytes

What it means

WriteLTXFile refuses to upload when the configured multipart part size is below the AWS SDK's manager.MinUploadPartSize (5 MiB). The check mirrors the uploader's own validation so the misconfiguration fails identically on the single-put path instead of silently succeeding for small objects. It is a deterministic configuration error, not a transient fault.

Source

Thrown at s3/replica_client.go:735

// to avoid the multipart upload manager's 5 MiB part buffers (issue #1327).
func (c *ReplicaClient) WriteLTXFile(ctx context.Context, level int, minTXID, maxTXID ltx.TXID, r io.Reader) (*ltx.FileInfo, error) {
	if err := c.Init(ctx); err != nil {
		return nil, err
	}

	filename := ltx.FormatFilename(minTXID, maxTXID)
	key := c.Path + "/" + fmt.Sprintf("%04x/%s", level, filename)

	partSize := int64(manager.DefaultUploadPartSize)
	if c.PartSize > 0 {
		partSize = c.PartSize
	}

	// The uploader rejects part sizes below the SDK minimum on every upload.
	// Fail identically here so the single-put path cannot mask the
	// misconfiguration for small objects.
	if partSize < manager.MinUploadPartSize {
		return nil, fmt.Errorf("s3: upload to %s: part size must be at least %d bytes", key, manager.MinUploadPartSize)
	}

	size, timestamp, etag, err := c.uploadLTX(ctx, key, r, partSize)
	if err != nil {
		return nil, err
	}

	internal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, "PUT").Inc()
	internal.OperationBytesCounterVec.WithLabelValues(ReplicaClientType, "PUT").Add(float64(size))

	// ETag indicates successful upload
	if etag == nil {
		return nil, fmt.Errorf("s3: upload failed: no ETag returned")
	}

	return &ltx.FileInfo{
		Level:     level,
		MinTXID:   minTXID,

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Set partSize to at least manager.MinUploadPartSize (5*1024*1024 bytes) in the config or call site
2. Clamp the value programmatically: if partSize < s3manager.MinUploadPartSize { partSize = s3manager.MinUploadPartSize } before calling WriteLTXFile
3. If the value comes from config, validate it at config-load time so the process fails fast at startup

Example fix

// before
partSize := 1024 * 1024 // 1 MiB
client.WriteLTXFile(ctx, key, r, level, minTXID, maxTXID, timestamp, partSize)

// after
const minPartSize = 5 * 1024 * 1024
if partSize < minPartSize {
    partSize = minPartSize
}
client.WriteLTXFile(ctx, key, r, level, minTXID, maxTXID, timestamp, partSize)

if partSize < s3manager.MinUploadPartSize {
    return fmt.Errorf("partSize %d below SDK minimum %d", partSize, s3manager.MinUploadPartSize)
}

var cfgErr *fmt.Errorf // match message
if err != nil && strings.Contains(err.Error(), "part size must be at least") {
    // fix config and restart; retrying will not help
}

When it happens

Trigger: Calling WriteLTXFile with a partSize argument smaller than manager.MinUploadPartSize (e.g. a config file sets a custom part size like 1MB, or code passes a hand-computed value below 5*1024*1024).

Common situations: Users tune upload part sizes in litestream YAML to reduce memory or costs and pick a value under the S3 5 MiB minimum; copying part-size defaults from older SDK versions where the minimum differed.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• heartbeat URL must be a valid HTTP or HTTPS URL
• heartbeat interval must be at least 1 minute
• database config #%d: duplicate path %q (already used by data
• database config #%d: 'pattern' is required when using 'dir'
• database config #%d: 'watch' can only be enabled with a dire

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/b8e6e6f820382fbf. Report an issue: GitHub.