{"record":{"id":"008f0930ba09b129","repo":"vitessio/vitess","slug":"filesize-v-is-too-large-to-upload-to-az-blob-m","errorCode":null,"errorMessage":"filesize (%v) is too large to upload to az blob (max size %v)","messagePattern":"filesize \\((.+?)\\) is too large to upload to az blob \\(max size (.+?)\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/mysqlctl/azblobbackupstorage/azblob.go","lineNumber":239,"sourceCode":"// Directory implements BackupHandle.\nfunc (bh *AZBlobBackupHandle) Directory() string {\n\treturn bh.dir\n}\n\n// Name implements BackupHandle.\nfunc (bh *AZBlobBackupHandle) Name() string {\n\treturn bh.name\n}\n\n// AddFile implements BackupHandle.\nfunc (bh *AZBlobBackupHandle) AddFile(ctx context.Context, filename string, filesize int64) (io.WriteCloser, error) {\n\tif bh.readOnly {\n\t\treturn nil, errors.New(\"AddFile cannot be called on read-only backup\")\n\t}\n\t// Error out if the file size it too large ( ~4.75 TB)\n\tmaxSize := int64(azblob.BlockBlobMaxStageBlockBytes * azblob.BlockBlobMaxBlocks)\n\tif filesize > maxSize {\n\t\treturn nil, fmt.Errorf(\"filesize (%v) is too large to upload to az blob (max size %v)\", filesize, maxSize)\n\t}\n\n\tobj := objName(bh.dir, bh.name, filename)\n\tcontainerURL, err := bh.bs.containerURL()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tblockBlobURL := containerURL.NewBlockBlobURL(obj)\n\n\treader, writer := io.Pipe()\n\n\tbh.waitGroup.Go(func() {\n\t\t// The upload must honor both the per-file context and the handle-level\n\t\t// abort context: callers cancel the per-file ctx to stop a failing\n\t\t// backup promptly, while AbortBackup cancels bh.ctx. Waiting on this\n\t\t// upload (via Wait) without honoring the per-file ctx would otherwise\n\t\t// let a stalled upload block the caller from ever reaching AbortBackup.","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/mysqlctl/azblobbackupstorage/azblob.go#L221-L257","documentation":"azblobbackupstorage.AddFile uploads a backup file to Azure Blob storage in blocks. Azure caps a single block blob at BlockBlobMaxStageBlockBytes * BlockBlobMaxBlocks bytes (~4.75 TB). The library proactively rejects larger files with this error instead of failing mid-upload.","triggerScenarios":"Taking a backup whose file (e.g. a large tablet data file or the combined backup artifact) exceeds ~4.75 TB, then calling Backup with the azblob backup storage backend.","commonSituations":"Very large single-shard datasets, insufficient sharding so one tablet's backup balloons in size, backing up uncompressed data that exceeds the blob cap, misconfigured backup that includes unintended files.","solutions":["Reduce file size: enable compression and exclude unneeded files from the backup.","Shard/resplit the keyspace so no single tablet's backup exceeds the Azure blob limit.","Switch to a backend without this cap (e.g. filesystem-backed backups or a different storage plugin) for very large shards.","Free disk/split the data and retry with a smaller backup."],"exampleFix":"// before: one huge shard > 4.75 TB\n// after: reshard so each shard backup fits\n// vtctldclient Reshard --shards='-80,80-' keyspace '-40,40-80,80-c0,c0-'","handlingStrategy":"validation","validationCode":"fi, err := os.Stat(path)\nif err != nil {\n\treturn err\n}\nconst maxBlob = int64(azblob.BlockBlobMaxStageBlockBytes * azblob.BlockBlobMaxBlocks)\nif fi.Size() > maxBlob {\n\treturn fmt.Errorf(\"file %s is %d bytes; az blob max is %d\", path, fi.Size(), maxBlob)\n}","typeGuard":null,"tryCatchPattern":"err := bh.AddFile(ctx, filename, filesize) // within Backup\nif err != nil {\n\treturn fmt.Errorf(\"az backup upload failed: %w\", err)\n}","preventionTips":["Enable compression and trim unnecessary files before backup","Reshard keyspaces so single-shard backups stay under ~4.75 TB","Estimate backup size during planning when using the azblob backend"],"tags":["go","backup","azure-blob","limit-exceeded"],"backgroundTag":"backup-file-too-large","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}