{"record":{"id":"8c11e7f7b6459807","repo":"weaviate/weaviate","slug":"local-shard-q-w","errorCode":null,"errorMessage":"local shard %q: %w","messagePattern":"local shard %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/index.go","lineNumber":3575,"sourceCode":") {\n\treturn i.getOptInitLocalShard(ctx, shardName, true, callerMovement)\n}\n\n// getLoadedShard returns the shard only if it is already loaded, never\n// materializing one, plus a release that drops the shutdown refcount. A nil\n// shard and a nil error mean the shard is not loaded here.\n//\n// The refcount rather than shardCreateLocks keeps the shard alive, because\n// shardCreateLocks is on the shard's own read and write path and callers can\n// run long. Same reasoning as backupShardWithHardlinks.\nfunc (i *Index) getLoadedShard(shardName string) (shard ShardLike, release func(), err error) {\n\t// Every shard teardown holds one of these two: Index.Shutdown takes closeLock\n\t// for write, the unload paths take shardCreateLocks for write.\n\ti.closeLock.RLock()\n\tdefer i.closeLock.RUnlock()\n\n\tif i.closed {\n\t\treturn nil, func() {}, fmt.Errorf(\"local shard %q: %w\", shardName, errAlreadyShutdown)\n\t}\n\n\ti.shardCreateLocks.RLock(shardName)\n\tdefer i.shardCreateLocks.RUnlock(shardName)\n\n\tshard = i.shards.Loaded(shardName)\n\tif shard == nil {\n\t\treturn nil, func() {}, nil\n\t}\n\t// Both locks span the map read and the refcount: a lazy shard's\n\t// preventShutdown loads, so a teardown landing in between would have it\n\t// rebuild the shard outside i.shards, where nothing can ever shut it down.\n\trelease, err = shard.preventShutdown()\n\tif err != nil {\n\t\treturn nil, func() {}, fmt.Errorf(\"shard %q, no shutdown: %w\", shardName, err)\n\t}\n\treturn shard, release, nil\n}","sourceCodeStart":3557,"sourceCodeEnd":3593,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/index.go#L3557-L3593","documentation":"getOrInitShard (or similar shard lookup) was called after the Index was shut down; it wraps errAlreadyShutdown. The index-level closed flag under closeLock indicates the whole collection index has been closed, so no shard can be served.","triggerScenarios":"Any object read/write/lookup arriving on an index whose Shutdown() already completed — e.g. requests in flight during graceful node shutdown, or use of a DB/Index handle after Close.","commonSituations":"Client requests hitting a node mid-rolling-restart; load balancer not yet draining the node; application code holding a client connection to a node that has been stopped.","solutions":["Retry against another node — this node is shutting down; load balancers should mark it unhealthy.","Check errors.Is(err, errAlreadyShutdown) to distinguish from other local-shard failures.","Fix client routing so drained nodes stop receiving traffic during rolling upgrades.","If this happens outside a shutdown, investigate what called Index.Shutdown (schema deletion of the class, node drop)."],"exampleFix":"shard, release, err := i.getOrInitShard(ctx, shardName)\nif err != nil {\n    if errors.Is(err, errAlreadyShutdown) {\n        return ErrNodeDraining // surface as retryable/redirect\n    }\n    return err\n}","handlingStrategy":"try-catch","validationCode":"// health-check the node before sending traffic\nhealth, _ := client.Nodes().Getter().Do(ctx) // node status should be HEALTHY","typeGuard":null,"tryCatchPattern":"if errors.Is(err, errAlreadyShutdown) {\n    return redirectOrRetry(otherNode) // node is draining\n}","preventionTips":["Drain nodes from the load balancer before shutdown","Use rolling restarts with readiness probes","Treat already-shutdown as redirectable, not retryable locally"],"tags":["shutdown","lifecycle","shard","availability"],"backgroundTag":"already-shutdown","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}