weaviate/weaviate · error

write : %w

Error message

write : %w

What it means

Read() streams the backup file into the io.WriteCloser w via io.Copy. This error (note the cosmetic double space in 'write :') wraps any failure while writing to the consumer side — i.e. the restore pipeline's writer (the node receiving the object data), not the local file. The reader side was fine; the destination rejected or failed the data.

Source

Thrown at modules/backup-filesystem/backup.go:200

	if err != nil {
		return -1, err
	}
	sourcePath, err := m.getObjectPath(ctx, basePath, backupID, key)
	if err != nil {
		return 0, fmt.Errorf("source path %s/%s: %w", backupID, key, err)
	}

	// open file
	f, err := os.Open(sourcePath)
	if err != nil {
		return 0, fmt.Errorf("open file %q: %w", sourcePath, err)
	}
	defer f.Close()

	// copy file
	read, err := io.Copy(w, f)
	if err != nil {
		return 0, fmt.Errorf("write : %w", err)
	}

	if metric, err := monitoring.GetMetrics().BackupRestoreDataTransferred.
		GetMetricWithLabelValues(m.Name(), "class"); err == nil {
		metric.Add(float64(read))
	}
	return read, err
}

func (m *Module) SourceDataPath() string {
	return m.dataPath
}

func (m *Module) initBackupBackend(ctx context.Context, backupsPath string) error {
	if backupsPath == "" {
		return fmt.Errorf("empty backup path provided")
	}
	backupsPath = filepath.Clean(backupsPath)

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Inspect the wrapped cause: 'broken pipe'/'connection reset' points to the consumer node or network — check the restoring node's logs and health.
2. Check disk space on the restoring node (destination, not the backup source).
3. Increase operation timeouts if the wrapped error is a deadline/context cancellation.
4. Retry the restore after stabilizing the destination node; the backup source file is unaffected.

Example fix

// before
# destination node runs out of disk during restore
// after
$ df -h /var/lib/weaviate   # on the RESTORING node
# free space / expand PVC, then retry restore

// check destination node health and free space before restoring on the target
resp, err := http.Get(destNode + "/v1/nodes")
if err != nil || resp.StatusCode != 200 {
    return fmt.Errorf("destination node unhealthy before restore")
}

if err != nil {
    if strings.Contains(err.Error(), "broken pipe") || strings.Contains(err.Error(), "connection reset") {
        // consumer dropped: wait for node recovery, then retry the restore
        time.Sleep(30 * time.Second)
        return retryRestore(backupID)
    }
    return err
}

When it happens

Trigger: Read() where the receiving write end of the restore stream returns an error: destination shard write failure, broken pipe because the consuming node/connection died, or context cancellation causing the writer to fail mid-copy.

Common situations: Restore target node crashed or restarted mid-transfer; network interruption between nodes during distributed restore; destination disk full on the restoring node; gRPC stream deadline exceeded.

Related errors

• io.copy for read %q: %w
• write file %q: %w
• find backup %s: %w
• close stream for object %s
• read stream for object %s

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/7ccdf9373b56fb35. Report an issue: GitHub.