weaviate/weaviate · error

%w: version got=%d want=%d

Error message

%w: version got=%d  want=%d

What it means

Returned by Store.WaitToRestoreDB when the context is cancelled before the raft last-applied index reaches the requested schema version. It wraps types.ErrDeadlineExceeded with got/want index values, meaning the node has not yet replayed the raft log up to the version required to serve requests.

Source

Thrown at cluster/store.go:811

		}
	}, st.log)
	return func() { close(done) }
}

// WaitForAppliedIndex waits until the update with the given version is propagated to this follower node
func (st *Store) WaitForAppliedIndex(ctx context.Context, period time.Duration, version uint64) error {
	if idx := st.lastAppliedIndex.Load(); idx >= version {
		return nil
	}
	ctx, cancel := context.WithTimeout(ctx, st.cfg.ConsistencyWaitTimeout)
	defer cancel()
	ticker := time.NewTicker(period)
	defer ticker.Stop()
	var idx uint64
	for {
		select {
		case <-ctx.Done():
			return fmt.Errorf("%w: version got=%d  want=%d", types.ErrDeadlineExceeded, idx, version)
		case <-ticker.C:
			if idx = st.lastAppliedIndex.Load(); idx >= version {
				return nil
			} else {
				st.log.WithFields(logrus.Fields{
					"got":  idx,
					"want": version,
				}).Debug("wait for update version")
			}
		}
	}
}

// IsLeader returns whether this node is the leader of the cluster
func (st *Store) IsLeader() bool {
	return st.raft != nil && st.raft.State() == raft.Leader
}

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Increase the caller's context timeout so the node has time to replay the raft log
2. Wait and retry — the node continues applying entries in the background
3. If replay is persistently too slow, snapshot the node: remove it from the cluster and re-add with a fresh data dir (or take a raft snapshot)
4. Check disk I/O throughput on the node and the 'got' index progress rate in logs to size the timeout

Example fix

// before
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
err := store.WaitToRestoreDB(ctx, time.Second, closeCh)
// after
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
err := store.WaitToRestoreDB(ctx, time.Second, closeCh)

if ctx, err := context.WithTimeout(parent, replayBudget); err != nil || replayBudget < expectedLogReplayTime {
    return errors.New("timeout too short for raft log replay")
}

if err := store.WaitToRestoreDB(ctx, period, closeCh); err != nil {
    if errors.Is(err, types.ErrDeadlineExceeded) {
        // inspect got/want in message and retry with a longer ctx
        return retryWithLongerTimeout(err)
    }
    return err
}

When it happens

Trigger: WaitToRestoreDB(ctx, period, close) polls lastAppliedIndex every tick; ctx.Done() fires (caller timeout or shutdown) while idx < version, so it returns deadline-exceeded with the current and target index.

Common situations: Node rejoining after a long offline period with a huge raft log to replay; slow disk making log replay slower than the caller's timeout; the requested version far ahead because the leader committed many schema updates while this node was down.

Related errors

• ErrBadRequest
• store not open
• unknown command
• deadline exceeded for waiting for update
• gRPC PutObjects: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/b34a00707dc18453. Report an issue: GitHub.