{"record":{"id":"3ca736b85b2f34c7","repo":"weaviate/weaviate","slug":"snapshot-phase-failed-w","errorCode":null,"errorMessage":"snapshot phase failed: %w","messagePattern":"snapshot phase failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"usecases/export/participant.go","lineNumber":275,"sourceCode":"\tbackendStore, backendErr := p.backends.BackupBackend(req.Backend, modulecapabilities.BackendUseCaseExport)\n\tif backendErr == nil {\n\t\tif backendErr = backendStore.Initialize(ctx, req.ID, req.Bucket, req.Path); backendErr != nil {\n\t\t\tbackendStore = nil\n\t\t}\n\t}\n\n\t// Wait for the snapshot goroutine started during Prepare. The mutex\n\t// is NOT held here so that Abort can cancel the snapshot if needed.\n\tsnapshotWaitStart := time.Now()\n\tvar snapshots []shardSnapshot\n\tvar skipped []skippedShard\n\tvar snapshotErr error\n\tselect {\n\tcase <-pending.done:\n\t\tsnapshots = pending.snapshots\n\t\tskipped = pending.skipped\n\t\tif pending.err != nil {\n\t\t\tsnapshotErr = fmt.Errorf(\"snapshot phase failed: %w\", pending.err)\n\t\t}\n\tcase <-ctx.Done():\n\t\tsnapshotErr = ctx.Err()\n\t}\n\tsnapshotDuration := time.Since(snapshotWaitStart)\n\tif snapshotErr != nil {\n\t\tp.logger.WithField(\"action\", \"export_participant\").\n\t\t\tWithField(\"export_id\", exportID).\n\t\t\tWithField(\"duration_ms\", snapshotDuration.Milliseconds()).\n\t\t\tErrorf(\"snapshot phase failed: %v\", snapshotErr)\n\t\tp.cleanupSnapshots(snapshots)\n\t\tfunc() {\n\t\t\tp.mu.Lock()\n\t\t\tdefer p.mu.Unlock()\n\t\t\tp.clearAndRelease()\n\t\t}()\n\t\treturn snapshotErr\n\t}","sourceCodeStart":257,"sourceCodeEnd":293,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/usecases/export/participant.go#L257-L293","documentation":"Wraps the failure of the shard-snapshot phase that Commit waits on (pending.done channel). The snapshot goroutine started during Prepare finished with a non-nil error (pending.err), and Commit wraps it with this prefix before cleaning up snapshots and releasing the reservation. The underlying error is in the wrapped %w chain.","triggerScenarios":"The snapshot goroutine fails to pause/flush shards or snapshot directories on this node (disk I/O error, permission problem, shard already closing); also raised when the caller's ctx is cancelled while waiting on pending.done (then snapshotErr is the ctx error, wrapped the same way).","commonSituations":"Disk full or failing volume on the export node; another operation (shard drop, tenant offloading) racing the snapshot; operator cancels the request context because the export appears hung on a large dataset.","solutions":["Inspect the wrapped cause (errors.Unwrap / %v) — fix the underlying snapshot failure (disk space, permissions, shard health) first.","Check node disk health and free space; snapshots write shard data to temporary directories.","Avoid cancelling the request context mid-export; use a generous timeout for large collections.","Retry the whole export (new Prepare + Commit) once the node is healthy; the slot is fully cleaned up after this error."],"exampleFix":"// before\nif err := participant.Commit(ctx, id); err != nil {\n    log.Print(err) // \"snapshot phase failed: ...\" with unknown cause\n}\n// after\nif err := participant.Commit(ctx, id); err != nil {\n    var snapErr error\n    if errors.As(err, &snapErr) || strings.Contains(err.Error(), \"snapshot phase failed\") {\n        log.Printf(\"snapshot cause: %v\", errors.Unwrap(err))\n    }\n}","handlingStrategy":"try-catch","validationCode":"if err := ctx.Err(); err != nil {\n    return fmt.Errorf(\"context already cancelled before commit: %w\", err)\n}\n// pre-check node disk headroom if the backend supports it","typeGuard":null,"tryCatchPattern":"if err := participant.Commit(ctx, id); err != nil {\n    var cause error\n    if errors.Unwrap(err) != nil { cause = errors.Unwrap(err) }\n    logger.Errorf(\"export commit failed (snapshot phase): %v (cause: %v)\", err, cause)\n    // inspect cause: disk-full -> free space; context.Canceled -> widen timeout\n    return err\n}","preventionTips":["Monitor disk space on export nodes; snapshots need temp space for shard data.","Use generous context timeouts sized to your largest collection.","Avoid dropping/offloading shards while an export is prepared.","Always unwrap and log the cause — the prefix alone does not tell you what failed."],"tags":["snapshot","export","io","context-canceled"],"backgroundTag":"snapshot-phase-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}