What causes "wrong data size uploaded to %q at %s; got %d bytes; want %d bytes" in VictoriaMetrics/VictoriaMetrics?

Ensure the origin data is not modified during the backup (stop writes or use fsfreeze/snapshot).

wrong data size uploaded to %q at %s; got %d bytes; want %d

Error message

wrong data size uploaded to %q at %s; got %d bytes; want %d bytes

What it means

UploadPart verifies that the number of bytes read from r (and handed to the GCS writer) equals the expected part size p.Size. A mismatch means the uploaded object will not match the backup manifest, so the upload is treated as failed. This protects against truncated or over-long source data corrupting the backup.

Source

Thrown at lib/backup/gcsremote/gcs.go:215

	return nil
}

// UploadPart uploads part p from r to fs.
func (fs *FS) UploadPart(p common.Part, r io.Reader) error {
	o := fs.object(p)
	w := o.NewWriter(fs.ctx)
	if len(fs.Metadata) > 0 {
		w.Metadata = fs.Metadata
	}
	n, err := io.Copy(w, r)
	if err1 := w.Close(); err1 != nil && err == nil {
		err = err1
	}
	if err != nil {
		return fmt.Errorf("cannot upload data to %q at %s (remote path %q): %w", p.Path, fs, o.ObjectName(), err)
	}
	if uint64(n) != p.Size {
		return fmt.Errorf("wrong data size uploaded to %q at %s; got %d bytes; want %d bytes", p.Path, fs, n, p.Size)
	}
	return nil
}

func (fs *FS) object(p common.Part) *storage.ObjectHandle {
	path := p.RemotePath(fs.Dir)
	return fs.bkt.Object(path)
}

// DeleteFile deletes filePath at fs if it exists.
//
// The function does nothing if the filePath doesn't exists.
func (fs *FS) DeleteFile(filePath string) error {
	path := path.Join(fs.Dir, filePath)
	return fs.delete(path)
}

func (fs *FS) delete(path string) error {

View on GitHub (pinned to 5079fb58f1)

Solutions

Ensure the origin data is not modified during the backup (stop writes or use fsfreeze/snapshot).
Re-run the backup so p.Size is recomputed against current data.
Check the source part file integrity (size, filesystem errors) and re-create it if truncated.
Delete the bad remote object and re-upload the part.

Defensive patterns

Strategy: validation

Validate before calling

fi, err := os.Open(partPath)
if err != nil {
    return err
}
info, _ := fi.Stat()
if uint64(info.Size()) != p.Size {
    return fmt.Errorf("local part size %d != expected %d; re-run backup", info.Size(), p.Size)
}

Try / catch

if err := fs.UploadPart(p, r); err != nil {
    if strings.Contains(err.Error(), "wrong data size uploaded") {
        // source data changed mid-backup; quiesce writes and re-run
    }
}

Prevention

Quiesce or snapshot origin data before backing up
Verify local part file sizes against metadata before upload
Recompute part metadata on every backup run instead of caching

When it happens

Trigger: io.Copy returned n != p.Size — the reader r produced fewer bytes than declared (e.g. the local part file was truncated/modified after metadata was computed) or more bytes than expected (source data changed mid-upload).

Common situations: Local data files being modified while the backup runs (no snapshot/quiesce); corrupted local part files; stale part metadata from a previous run; disk errors while reading the source file.

Related errors

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/f6dfc7ebfde8552d. Report an issue: GitHub.