weaviate/weaviate · error

failed to preallocate file: %w

Error message

failed to preallocate file: %w

What it means

Fired during replication stream copy when f.Truncate(meta.Size) cannot preallocate the local .tmp file to the expected shard segment size reported by the source node. Indicates local disk failure (out of space, quota, I/O error) or an invalid size, aborting the shard copy before data transfer.

Source

Thrown at cluster/replication/copier/copier.go:349

		tmpPath := localFilePath + ".tmp"

		if err := func() error {
			f, err := os.Create(tmpPath)
			if err != nil {
				return fmt.Errorf("open file %q for writing: %w", tmpPath, err)
			}
			eg := enterrors.NewErrorGroupWrapper(c.logger)
			eg.SetLimit(_NUMCPU)
			// Drain writers before closing the FD: a late WriteAt against a
			// recycled descriptor would otherwise land on the wrong file.
			defer func() {
				_ = eg.Wait()
				if f != nil {
					_ = f.Close()
				}
			}()
			if err := f.Truncate(meta.Size); err != nil {
				return fmt.Errorf("failed to preallocate file: %w", err)
			}

			for {
				chunk, err := stream.Recv()
				if err != nil {
					return fmt.Errorf("failed to receive file chunk for %s: %w", meta.FileName, err)
				}
				if len(chunk.Data) > 0 {
					eg.Go(func() error {
						// Test-only: forces a deterministic WriteAt-after-Close window.
						if sleep := os.Getenv("WEAVIATE_TEST_DOWNLOAD_WRITE_SLEEP"); sleep != "" {
							if d, err := time.ParseDuration(sleep); err == nil {
								time.Sleep(d)
							}
						}
						if _, err := f.WriteAt(chunk.Data, chunk.Offset); err != nil {
							return fmt.Errorf("writing chunk to file %q: %w", tmpPath, err)
						}

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Free disk space — the volume must have at least meta.Size bytes free for the .tmp file.
2. Avoid network filesystems (NFS/FUSE) for PERSISTENCE_DATA_PATH; use local SSD storage.
3. If meta.Size is implausible, verify source and target run compatible Weaviate versions and re-initiate replication.
4. Retry after resolving; the .tmp file is removed on failure so no cleanup is needed.

Example fix

// before: volume too small for the shard file
$ df -h /var/lib/weaviate  # 2GB free, file is 5GB
// after: expand the volume or move PERSISTENCE_DATA_PATH to larger storage
$ kubectl patch pvc weaviate-data -p '{"spec":{"resources":{"requests":{"storage":"200Gi"}}}}'

var st syscall.Statfs_t
if err := syscall.Statfs(shardDir, &st); err == nil {
    free := st.Bavail * uint64(st.Bsize)
    if free < uint64(meta.Size)+margin {
        return fmt.Errorf("only %d bytes free, need %d", free, meta.Size)
    }
}

When it happens

Trigger: Truncate fails on a full filesystem (cannot grow the file to meta.Size), on filesystems that disallow growing via truncate beyond limits (e.g. some FUSE/network mounts), when meta.Size is negative/corrupt, or when the fd became invalid.

Common situations: Very large shard files on a volume that is nearly full; downloading to NFS/CIFS/glusterfs mounts with size limits; corrupted metadata returned by a buggy/mismatched source node version.

Related errors

• create parent folder for %s: %w
• open file %q for writing: %w
• writing chunk to file %q: %w
• writing chunks to file %q: %w
• fsyncing file %q for writing: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/dcb3b83abd3aec9a. Report an issue: GitHub.