vitessio/vitess · error

filesize (%v) is too large to upload to az blob (max size %v

Error message

filesize (%v) is too large to upload to az blob (max size %v)

What it means

azblobbackupstorage.AddFile uploads a backup file to Azure Blob storage in blocks. Azure caps a single block blob at BlockBlobMaxStageBlockBytes * BlockBlobMaxBlocks bytes (~4.75 TB). The library proactively rejects larger files with this error instead of failing mid-upload.

Source

Thrown at go/vt/mysqlctl/azblobbackupstorage/azblob.go:239

// Directory implements BackupHandle.
func (bh *AZBlobBackupHandle) Directory() string {
	return bh.dir
}

// Name implements BackupHandle.
func (bh *AZBlobBackupHandle) Name() string {
	return bh.name
}

// AddFile implements BackupHandle.
func (bh *AZBlobBackupHandle) AddFile(ctx context.Context, filename string, filesize int64) (io.WriteCloser, error) {
	if bh.readOnly {
		return nil, errors.New("AddFile cannot be called on read-only backup")
	}
	// Error out if the file size it too large ( ~4.75 TB)
	maxSize := int64(azblob.BlockBlobMaxStageBlockBytes * azblob.BlockBlobMaxBlocks)
	if filesize > maxSize {
		return nil, fmt.Errorf("filesize (%v) is too large to upload to az blob (max size %v)", filesize, maxSize)
	}

	obj := objName(bh.dir, bh.name, filename)
	containerURL, err := bh.bs.containerURL()
	if err != nil {
		return nil, err
	}

	blockBlobURL := containerURL.NewBlockBlobURL(obj)

	reader, writer := io.Pipe()

	bh.waitGroup.Go(func() {
		// The upload must honor both the per-file context and the handle-level
		// abort context: callers cancel the per-file ctx to stop a failing
		// backup promptly, while AbortBackup cancels bh.ctx. Waiting on this
		// upload (via Wait) without honoring the per-file ctx would otherwise
		// let a stalled upload block the caller from ever reaching AbortBackup.

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Reduce file size: enable compression and exclude unneeded files from the backup.
2. Shard/resplit the keyspace so no single tablet's backup exceeds the Azure blob limit.
3. Switch to a backend without this cap (e.g. filesystem-backed backups or a different storage plugin) for very large shards.
4. Free disk/split the data and retry with a smaller backup.

Example fix

// before: one huge shard > 4.75 TB
// after: reshard so each shard backup fits
// vtctldclient Reshard --shards='-80,80-' keyspace '-40,40-80,80-c0,c0-'

fi, err := os.Stat(path)
if err != nil {
	return err
}
const maxBlob = int64(azblob.BlockBlobMaxStageBlockBytes * azblob.BlockBlobMaxBlocks)
if fi.Size() > maxBlob {
	return fmt.Errorf("file %s is %d bytes; az blob max is %d", path, fi.Size(), maxBlob)
}

err := bh.AddFile(ctx, filename, filesize) // within Backup
if err != nil {
	return fmt.Errorf("az backup upload failed: %w", err)
}

When it happens

Trigger: Taking a backup whose file (e.g. a large tablet data file or the combined backup artifact) exceeds ~4.75 TB, then calling Backup with the azblob backup storage backend.

Common situations: Very large single-shard datasets, insufficient sharding so one tablet's backup balloons in size, backing up uncompressed data that exceeds the blob cap, misconfigured backup that includes unintended files.

Related errors

• AbortBackup cannot be called on read-only backup
• ReadFile cannot be called on read-write backup
• cannot use backup between different flavors: %q vs. %q
• ReadFile cannot be called on read-write backup
• AddFile cannot be called on read-only backup

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/008f0930ba09b129. Report an issue: GitHub.