{"record":{"id":"631120bb7e6285c0","repo":"vitessio/vitess","slug":"errpartsize","errorCode":"ErrPartSize","errorMessage":"minimum S3 part size must be between 5MiB and 5GiB","messagePattern":"minimum S3 part size must be between 5MiB and 5GiB","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/mysqlctl/s3backupstorage/s3.go","lineNumber":101,"sourceCode":"\n\t// forcePath is used to ensure that the certificate and path used match the endpoint + region\n\tforcePath bool\n\n\ttlsSkipVerifyCert bool\n\n\t// verboseLogging provides more verbose logging of AWS actions\n\trequiredLogLevel string\n\n\t// sse is the server-side encryption algorithm used when storing this object in S3\n\tsse string\n\n\t// path component delimiter\n\tdelimiter = \"/\"\n\n\t// minimum part size\n\tminPartSize int64 = 5 * 1024 * 1024 // 5MiB - AWS requirement\n\n\tErrPartSize = errors.New(\"minimum S3 part size must be between 5MiB and 5GiB\")\n)\n\nfunc registerFlags(fs *pflag.FlagSet) {\n\tutils.SetFlagStringVar(fs, &region, \"s3-backup-aws-region\", \"us-east-1\", \"AWS region to use.\")\n\tutils.SetFlagIntVar(fs, &retryCount, \"s3-backup-aws-retries\", -1, \"AWS request retries.\")\n\tutils.SetFlagStringVar(fs, &endpoint, \"s3-backup-aws-endpoint\", \"\", \"endpoint of the S3 backend (region must be provided).\")\n\tutils.SetFlagStringVar(fs, &bucket, \"s3-backup-storage-bucket\", \"\", \"S3 bucket to use for backups.\")\n\tutils.SetFlagStringVar(fs, &root, \"s3-backup-storage-root\", \"\", \"root prefix for all backup-related object names.\")\n\tutils.SetFlagBoolVar(fs, &forcePath, \"s3-backup-force-path-style\", false, \"force the s3 path style.\")\n\tutils.SetFlagBoolVar(fs, &tlsSkipVerifyCert, \"s3-backup-tls-skip-verify-cert\", false, \"skip the 'certificate is valid' check for SSL connections.\")\n\tutils.SetFlagStringVar(fs, &requiredLogLevel, \"s3-backup-log-level\", \"LogOff\", \"determine the S3 loglevel to use from LogOff, LogDebug, LogDebugWithSigning, LogDebugWithHTTPBody, LogDebugWithRequestRetries, LogDebugWithRequestErrors.\")\n\tutils.SetFlagStringVar(fs, &sse, \"s3-backup-server-side-encryption\", \"\", \"server-side encryption algorithm (e.g., AES256, aws:kms, sse_c:/path/to/key/file).\")\n\tutils.SetFlagInt64Var(fs, &minPartSize, \"s3-backup-aws-min-partsize\", minPartSize, \"Minimum part size to use, defaults to 5MiB but can be increased due to the dataset size.\")\n}\n\nfunc init() {\n\tservenv.OnParseFor(\"vtbackup\", registerFlags)\n\tservenv.OnParseFor(\"vtctl\", registerFlags)","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/mysqlctl/s3backupstorage/s3.go#L83-L119","documentation":"ErrPartSize is thrown by calculateUploadPartSize when the derived S3 multipart upload part size falls outside the AWS-mandated bounds: each part must be at least 5MiB and at most 5GiB. S3 multipart uploads reject parts outside this range, so the backup handle refuses to start with an invalid part size rather than failing mid-upload.","triggerScenarios":"Calling S3BackupHandle.AddFile (which calls calculateUploadPartSize) with a filesize whose computed part size is below 5MiB or above 5GiB — e.g. an extremely small backup where the intended part size would undercut the AWS 5MiB minimum, or a part count/part size division yielding out-of-range values.","commonSituations":"Backing up tiny databases where a computed part size dips under the AWS minimum; custom part-size math against huge backups exceeding limits; misconfigured backup flags that influence part sizing.","solutions":["Clamp the computed part size to at least 5MiB (5*1024*1024) before starting the multipart upload.","Verify the backup file size passed to AddFile is sensible; tiny test backups may not support multipart semantics.","If using custom part-size configuration, ensure the resulting size stays within [5MiB, 5GiB]."],"exampleFix":"// before\npartSize := fileSize / maxParts\nbh.AddFile(ctx, filename, fileSize)\n\n// after\nconst minPartSize = 5 * 1024 * 1024\nconst maxPartSize = 5 * 1024 * 1024 * 1024\nif partSize < minPartSize {\n    partSize = minPartSize\n}\nif partSize > maxPartSize {\n    partSize = maxPartSize\n}","handlingStrategy":"validation","validationCode":"const minPart = int64(5 * 1024 * 1024)     // 5MiB\nconst maxPart = int64(5 * 1024 * 1024 * 1024) // 5GiB\nif partSize < minPart || partSize > maxPart {\n    return fmt.Errorf(\"part size %d outside [%d, %d]\", partSize, minPart, maxPart)\n}\n","typeGuard":null,"tryCatchPattern":"wc, err := bh.AddFile(ctx, filename, fileSize)\nif err != nil {\n    if strings.Contains(err.Error(), \"part size\") {\n        return vterrors.Errorf(vtrpcpb.Code_INVALID_ARGUMENT, \"backup size yields invalid S3 part size: %v\", err)\n    }\n    return err\n}","preventionTips":["Clamp computed part sizes to [5MiB, 5GiB] before initiating multipart uploads.","Test the backup pipeline with both tiny and multi-TB backup sizes to exercise both bounds.","Rely on calculateUploadPartSize instead of custom part-size math."],"tags":["s3","backup","multipart-upload","validation"],"backgroundTag":"s3-part-size-invalid","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}