weaviate/weaviate · error

snapshot phase failed: %w

Error message

snapshot phase failed: %w

What it means

Wraps the failure of the shard-snapshot phase that Commit waits on (pending.done channel). The snapshot goroutine started during Prepare finished with a non-nil error (pending.err), and Commit wraps it with this prefix before cleaning up snapshots and releasing the reservation. The underlying error is in the wrapped %w chain.

Source

Thrown at usecases/export/participant.go:275

	backendStore, backendErr := p.backends.BackupBackend(req.Backend, modulecapabilities.BackendUseCaseExport)
	if backendErr == nil {
		if backendErr = backendStore.Initialize(ctx, req.ID, req.Bucket, req.Path); backendErr != nil {
			backendStore = nil
		}
	}

	// Wait for the snapshot goroutine started during Prepare. The mutex
	// is NOT held here so that Abort can cancel the snapshot if needed.
	snapshotWaitStart := time.Now()
	var snapshots []shardSnapshot
	var skipped []skippedShard
	var snapshotErr error
	select {
	case <-pending.done:
		snapshots = pending.snapshots
		skipped = pending.skipped
		if pending.err != nil {
			snapshotErr = fmt.Errorf("snapshot phase failed: %w", pending.err)
		}
	case <-ctx.Done():
		snapshotErr = ctx.Err()
	}
	snapshotDuration := time.Since(snapshotWaitStart)
	if snapshotErr != nil {
		p.logger.WithField("action", "export_participant").
			WithField("export_id", exportID).
			WithField("duration_ms", snapshotDuration.Milliseconds()).
			Errorf("snapshot phase failed: %v", snapshotErr)
		p.cleanupSnapshots(snapshots)
		func() {
			p.mu.Lock()
			defer p.mu.Unlock()
			p.clearAndRelease()
		}()
		return snapshotErr
	}

View on GitHub (pinned to 75aa4b6d11)

Solutions

  1. Inspect the wrapped cause (errors.Unwrap / %v) — fix the underlying snapshot failure (disk space, permissions, shard health) first.
  2. Check node disk health and free space; snapshots write shard data to temporary directories.
  3. Avoid cancelling the request context mid-export; use a generous timeout for large collections.
  4. Retry the whole export (new Prepare + Commit) once the node is healthy; the slot is fully cleaned up after this error.

Example fix

// before
if err := participant.Commit(ctx, id); err != nil {
    log.Print(err) // "snapshot phase failed: ..." with unknown cause
}
// after
if err := participant.Commit(ctx, id); err != nil {
    var snapErr error
    if errors.As(err, &snapErr) || strings.Contains(err.Error(), "snapshot phase failed") {
        log.Printf("snapshot cause: %v", errors.Unwrap(err))
    }
}
Defensive patterns

Strategy: try-catch

Validate before calling

if err := ctx.Err(); err != nil {
    return fmt.Errorf("context already cancelled before commit: %w", err)
}
// pre-check node disk headroom if the backend supports it

Try / catch

if err := participant.Commit(ctx, id); err != nil {
    var cause error
    if errors.Unwrap(err) != nil { cause = errors.Unwrap(err) }
    logger.Errorf("export commit failed (snapshot phase): %v (cause: %v)", err, cause)
    // inspect cause: disk-full -> free space; context.Canceled -> widen timeout
    return err
}

Prevention

When it happens

Trigger: The snapshot goroutine fails to pause/flush shards or snapshot directories on this node (disk I/O error, permission problem, shard already closing); also raised when the caller's ctx is cancelled while waiting on pending.done (then snapshotErr is the ctx error, wrapped the same way).

Common situations: Disk full or failing volume on the export node; another operation (shard drop, tenant offloading) racing the snapshot; operator cancels the request context because the export appears hung on a large dataset.

Related errors


AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/3ca736b85b2f34c7. Report an issue: GitHub.