{"record":{"id":"1c9174456a5a398d","repo":"weaviate/weaviate","slug":"get-neighbor-vec","errorCode":null,"errorMessage":"get neighbor vec","messagePattern":"get neighbor vec","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"adapters/repos/db/vector/hnsw/delete.go","lineNumber":656,"sourceCode":"\t}\n\tneighborNode.Unlock()\n\n\tvar neighborVec []float32\n\tvar compressorDistancer compressionhelpers.CompressorDistancer\n\tif h.compressed.Load() {\n\t\tcompressorDistancer, err = h.compressor.NewDistancerFromID(neighbor)\n\t} else {\n\t\tneighborVec, err = h.cache.Get(context.Background(), neighbor)\n\t}\n\n\tif err != nil {\n\t\tvar e storobj.ErrNotFound\n\t\tif errors.As(err, &e) {\n\t\t\th.handleDeletedNode(e.DocID, \"reassignNeighbor\")\n\t\t\treturn true, nil\n\t\t} else {\n\t\t\t// not a typed error, we can recover from, return with err\n\t\t\treturn false, errors.Wrap(err, \"get neighbor vec\")\n\t\t}\n\t}\n\n\t// the new recursive implementation no longer needs an entrypoint, so we can\n\t// just pass this dummy value to make the neighborFinderConnector happy\n\tneighborNode.markAsMaintenance()\n\tdefer neighborNode.unmarkAsMaintenance()\n\n\tdummyEntrypoint := uint64(0)\n\tif err := h.reconnectNeighboursOf(ctx, neighborNode, dummyEntrypoint, neighborVec, compressorDistancer,\n\t\tneighborLevel, currentMaximumLayer, deleteList, processedIDs); err != nil {\n\t\treturn false, errors.Wrap(err, \"find and connect neighbors\")\n\t}\n\n\th.metrics.CleanedUp()\n\treturn true, nil\n}\n","sourceCodeStart":638,"sourceCodeEnd":674,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/repos/db/vector/hnsw/delete.go#L638-L674","documentation":"Wrapped failure in reassignNeighbor when fetching the vector of a neighbor node fails with an error that is not a recoverable storobj.ErrNotFound. The cleanup worker needs the neighbor's vector to reconnect it after its tombstoned neighbor is removed; if the vector cannot be read for any reason other than 'object was deleted', the error is wrapped as 'get neighbor vec' and aborts the reassignment so the node stays in maintenance state.","triggerScenarios":"During tombstone cleanup, reassignNeighbor reads the neighbor's vector via the store; the read returns a non-ErrNotFound error (I/O failure, store closed/corrupted segment, deserialization failure). ErrNotFound is handled separately (the neighbor is treated as deleted).","commonSituations":"Disk I/O errors or corrupted LSM segments on the shard; a race where the shard is being dropped/closed while cleanup runs; store-level failures after an unclean shutdown.","solutions":["Unwrap the error to see the underlying store failure and address it (disk, segment corruption, closed store)","Retry cleanup — tombstoned nodes are re-attempted on later cleanup passes","Check shard/segment integrity; restore from backup if segments are corrupted","Ensure the shard is not being dropped or the store closed concurrently with cleanup","If vectors are persistently unreadable for specific IDs, delete and re-insert those objects"],"exampleFix":null,"handlingStrategy":"type-guard","validationCode":"// Verify neighbor object exists and is readable before reassignment\nvec, err := h.store.GetObject(ctx, neighborID)\nif err != nil { return fmt.Errorf(\"neighbor %d unreadable: %w\", neighborID, err) }","typeGuard":"var e storobj.ErrNotFound\nif errors.As(err, &e) {\n    h.handleDeletedNode(e.DocID, \"reassignNeighbor\")\n    return true, nil // recoverable\n}","tryCatchPattern":"vec, err := h.vectorByIndexNode(ctx, node)\nif err != nil {\n    var e storobj.ErrNotFound\n    if errors.As(err, &e) { h.handleDeletedNode(e.DocID, \"reassignNeighbor\"); return true, nil }\n    return false, fmt.Errorf(\"get neighbor vec: %w\", err)\n}","preventionTips":["Treat storobj.ErrNotFound as success (node already deleted) — use errors.As, not string matching","Monitor store I/O errors; run filesystem checks on the shard's storage volume","Never drop a shard or close its store while cleanup goroutines are active","Restore corrupted segments from backup instead of letting cleanup retry indefinitely"],"tags":["hnsw","vector-index","tombstone-cleanup","go"],"backgroundTag":"hnsw-neighbor-reassignment-failed","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}