What causes "wrong data size uploaded to %q; got %d bytes; want %d bytes" in VictoriaMetrics/VictoriaMetrics?

Re-snapshot the source data so file sizes are stable, recompute parts, then re-upload

wrong data size uploaded to %q; got %d bytes; want %d bytes

Error message

wrong data size uploaded to %q; got %d bytes; want %d bytes

What it means

After a successful copy/sync/close, UploadPart compares the number of bytes written (io.Copy's return) with p.Size from the part metadata. A mismatch means the reader produced fewer (or more) bytes than the part metadata promised. The partially-uploaded file is removed and this error is returned so a corrupt part never remains on the remote.

Source

Thrown at lib/backup/fsremote/fsremote.go:183

	}
	w, err := os.Create(path)
	if err != nil {
		return fmt.Errorf("cannot create file %q: %w", path, err)
	}
	n, err := io.Copy(w, r)
	if err := w.Sync(); err != nil {
		return fmt.Errorf("cannot fsync file: %q: %w", w.Name(), err)
	}
	if err1 := w.Close(); err1 != nil && err == nil {
		err = err1
	}
	if err != nil {
		_ = os.RemoveAll(path)
		return fmt.Errorf("cannot upload data to %q: %w", path, err)
	}
	if uint64(n) != p.Size {
		_ = os.RemoveAll(path)
		return fmt.Errorf("wrong data size uploaded to %q; got %d bytes; want %d bytes", path, n, p.Size)
	}
	return nil
}

func (fs *FS) mkdirAll(filePath string) error {
	dir := filepath.Dir(filePath)
	if err := os.MkdirAll(dir, 0700); err != nil {
		return fmt.Errorf("cannot create directory %q: %w", dir, err)
	}
	return nil
}

func (fs *FS) path(p common.Part) string {
	return filepath.Join(p.LocalPath(fs.Dir), fmt.Sprintf("%016X_%016X_%016X", p.FileSize, p.Offset, p.Size))
}

// DeleteFile deletes filePath at fs.
//

View on GitHub (pinned to 5079fb58f1)

Solutions

Re-snapshot the source data so file sizes are stable, recompute parts, then re-upload
Verify the reader passed to UploadPart covers exactly p.Size bytes at p.Offset (use io.LimitReader or pread with the correct offsets)
Re-run the upload with fresh part metadata — the mismatched file was already removed
Never mutate source files during backup; use the snapshot mechanism the tool provides

Example fix

// before: reading the whole file instead of the exact part window
f, _ := os.Open(srcPath)
err := fsRemote.UploadPart(part, f) // size mismatch: whole file != part.Size
// after: bound the reader to the part's size at its offset
f, _ := os.Open(srcPath)
if _, err := f.Seek(int64(part.Offset), io.SeekStart); err != nil { return err }
err = fsRemote.UploadPart(part, io.LimitReader(f, int64(part.Size)))

Defensive patterns

Strategy: validation

Validate before calling

import "os"
func partReader(r io.Reader, p Part) (io.Reader, error) {
	f, ok := r.(*os.File)
	if !ok { return nil, fmt.Errorf("need *os.File to seek to part offset") }
	if _, err := f.Seek(int64(p.Offset), io.SeekStart); err != nil { return nil, err }
	return io.LimitReader(f, int64(p.Size)), nil
}

Try / catch

err := fsRemote.UploadPart(part, r)
if err != nil && strings.Contains(err.Error(), "wrong data size uploaded") {
	// reader produced != part.Size bytes: recompute parts from a fresh snapshot and retry
}

Prevention

Always bound the reader to exactly part.Size bytes (io.LimitReader after seeking to Offset)
Compute part metadata and readers from the same immutable snapshot
Never reuse part lists from a previous backup run against changed data
Cross-check source file sizes against part.FileSize before uploading

When it happens

Trigger: Calling UploadPart with a reader whose data length doesn't match p.Size — the part filename encodes Size as %016X, so this fires when the source snapshot changed size after the part list was computed, the wrong reader/offset was passed, or a compressing/limiting reader returned short reads.

Common situations: Database files modified during backup (snapshot taken without proper locking / hardlink snapshot skipped); passing a size-limited reader with the wrong limit; metadata computed from an earlier backup run reused with new data; reading through a flaky pipe that returns EOF early.

Related errors

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/623d177d7966938c. Report an issue: GitHub.