weaviate/weaviate · error

intermediate rescore: fde for vector %d

Error message

intermediate rescore: fde for vector %d

What it means

Wraps an error returned by bucket.Get while reading the raw FDE (fixed dimensional encoding) bytes of a candidate vector during the intermediate rescore stage of an HFresh FDE search. The candidate id came from the posting scan, but the underlying LSMKV bucket read failed (I/O, corruption, bucket closed). Unlike the empty-read case (deleted vector, skipped gracefully), this is an actual storage-level failure and aborts the search.

Source

Thrown at adapters/repos/db/vector/hfresh/search.go:687

// bucket plus the fold); the pipeline shape stays as is.
func (h *HFresh) rescoreFDECandidates(queryFDE []float32, candidates *ResultSet, rerankBudget int) ([]uint64, error) {
	bucket := h.store.Bucket(h.id + "_muvera_vectors")
	if bucket == nil {
		return nil, errors.New("intermediate rescore: muvera vectors bucket not found")
	}

	// empty struct, costs nothing to create per call
	dotProvider := distancer.NewDotProductProvider()
	cosine := h.needsNormalization
	rescored := NewResultSet(rerankBudget)
	keyBuf := make([]byte, 8)
	var fdeBuf []float32

	for id := range candidates.Iter() {
		binary.BigEndian.PutUint64(keyBuf, id)
		raw, err := bucket.Get(keyBuf)
		if err != nil {
			return nil, errors.Wrapf(err, "intermediate rescore: fde for vector %d", id)
		}
		if len(raw) == 0 {
			// deleted between the posting scan and this stage; skip stale
			// entries gracefully, like the single-vector rescore does
			continue
		}
		fdeBuf = float32SliceInto(fdeBuf, raw)

		var dist float32
		if cosine {
			negDot, err := dotProvider.SingleDist(queryFDE, fdeBuf)
			if err != nil {
				return nil, errors.Wrapf(err, "intermediate rescore: distance for vector %d", id)
			}
			negNormSq, err := dotProvider.SingleDist(fdeBuf, fdeBuf)
			if err != nil {
				return nil, errors.Wrapf(err, "intermediate rescore: norm for vector %d", id)
			}

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Inspect the wrapped inner error (the cause is appended by errors.Wrapf) to find the storage-level root cause (I/O, corruption, closed bucket).
2. Check disk health and free space on the node (dmesg, df) and repair or restore the affected shard from backup if corruption is reported.
3. Retry the query — transient errors during compaction/shutdown often resolve once the node is stable.
4. If it recurs on the same shard, drop and re-import the collection or restore from a backup.

// before querying, verify the collection/shard is readable
schema := client.Schema.GetExtractorClass(class)
if schema == nil { return fmt.Errorf("class %s not found", class) }

result, err := client.GraphQL().Get().WithClassName(class).Run(ctx)
if err != nil {
    if isStorageError(err) { // transient I/O
        // retry with backoff, then alert if persistent
    }
    return fmt.Errorf("fde rescore read failed: %w", err)
}

When it happens

Trigger: Calling a search that uses FDE rescoring (searchByFDE) when bucket.Get(keyBuf) returns a non-nil error for a candidate id — e.g. disk I/O error, corrupted segment, or a closed/compacting bucket.

Common situations: Disk failures or full disks on a node; LSMKV segment corruption; querying a shard whose store is shutting down or was dropped mid-query; races with shard offloading/tenant moves.

Related errors

• failed to get posting %d for reassign after split
• failed to get version for %d
• could not copy all data into buffer
• object bucket does not exist
• hfresh distancer is not initialized

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/0ad579ada6429dfc. Report an issue: GitHub.