{"record":{"id":"16736dbd83c66a50","repo":"weaviate/weaviate","slug":"lazyloadshard-preventshutdown-w","errorCode":null,"errorMessage":"LazyLoadShard::preventShutdown: %w","messagePattern":"LazyLoadShard::preventShutdown: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/shard_lazyloader.go","lineNumber":854,"sourceCode":"\tl.mutex.Lock()\n\tdefer l.mutex.Unlock()\n\n\tif !l.loaded {\n\t\treturn nil\n\t}\n\n\tif err := l.shard.Shutdown(ctx); err != nil {\n\t\treturn err\n\t}\n\n\t// Mark as unloaded so drop() knows the correct state\n\tl.loaded = false\n\treturn nil\n}\n\nfunc (l *LazyLoadShard) preventShutdown() (release func(), err error) {\n\tif err := l.Load(context.Background()); err != nil {\n\t\treturn func() {}, fmt.Errorf(\"LazyLoadShard::preventShutdown: %w\", err)\n\t}\n\treturn l.shard.preventShutdown()\n}\n\n// HashTreeLevel maps unloaded to ErrAsyncReplicationNotActive; an empty level would read as convergence.\nfunc (l *LazyLoadShard) HashTreeLevel(ctx context.Context, level int, discriminant *hashtree.Bitset) (digests []hashtree.Digest, err error) {\n\tif !l.isLoaded() {\n\t\treturn nil, errAsyncReplicationNotActive\n\t}\n\treturn l.shard.HashTreeLevel(ctx, level, discriminant)\n}\n\n// CreateAsyncCheckpoint maps unloaded to ErrAsyncReplicationNotActive so transports\n// return REST 412 / gRPC FailedPrecondition (matching the loaded path).\nfunc (l *LazyLoadShard) CreateAsyncCheckpoint(ctx context.Context, cutoffMs int64, createdAt time.Time) error {\n\tif !l.isLoaded() {\n\t\treturn errAsyncReplicationNotActive\n\t}","sourceCodeStart":836,"sourceCodeEnd":872,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/shard_lazyloader.go#L836-L872","documentation":"preventShutdown ensures a shard is loaded and protected from being shut down (e.g. during replication or maintenance), returning a release func to undo the protection. When the LazyLoadShard is cold, it first calls Load; this error wraps any Load failure (memory pressure rejection, load-limiter permit failure, reindex errors), so the shutdown protection cannot be established.","triggerScenarios":"Calling preventShutdown on an unloaded shard while Load fails — most commonly memory-pressure reservation failure (CheckMappingAndReserve) or inability to acquire a shard-load permit (context cancelled/timeout).","commonSituations":"Nodes under memory pressure refusing shard loads; saturated shard-load limiter during bulk operations; operations like async replication or offloading trying to protect a cold tenant's shard.","solutions":["Free memory or wait for load pressure to drop, then retry the operation.","Increase the shard-load limiter concurrency or reduce competing shard loads.","Retry the parent operation (e.g. replication step) once the shard can load.","Investigate the inner Load error from the message — it names the exact cause (memory pressure vs permit vs reindex)."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// before operations that protect shards from shutdown, check node memory headroom\n// if memoryPressure() { return errors.New(\"defer: node under memory pressure\") }","typeGuard":null,"tryCatchPattern":"release, err := lazyShard.preventShutdown()\nif err != nil {\n\tif strings.Contains(err.Error(), \"preventShutdown\") && strings.Contains(err.Error(), \"memory pressure\") {\n\t\ttime.Sleep(backoff); release, err = lazyShard.preventShutdown() // retry after pressure drops\n\t}\n\tif err != nil { return err }\n}\ndefer release()","preventionTips":["Keep memory headroom above the shard-load reservation thresholds.","Avoid running replication/offload maintenance during peak memory usage.","Tune the shard-load limiter so protection loads aren't starved.","Read the wrapped inner Load error to target the exact cause (pressure vs permit vs reindex)."],"tags":["shard","lazy-loading","memory-pressure","shutdown"],"backgroundTag":"shard-load-refused","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}