{"record":{"id":"b34a00707dc18453","repo":"weaviate/weaviate","slug":"w-version-got-d-want-d","errorCode":null,"errorMessage":"%w: version got=%d  want=%d","messagePattern":"%w: version got=(.+?)  want=(.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cluster/store.go","lineNumber":811,"sourceCode":"\t\t}\n\t}, st.log)\n\treturn func() { close(done) }\n}\n\n// WaitForAppliedIndex waits until the update with the given version is propagated to this follower node\nfunc (st *Store) WaitForAppliedIndex(ctx context.Context, period time.Duration, version uint64) error {\n\tif idx := st.lastAppliedIndex.Load(); idx >= version {\n\t\treturn nil\n\t}\n\tctx, cancel := context.WithTimeout(ctx, st.cfg.ConsistencyWaitTimeout)\n\tdefer cancel()\n\tticker := time.NewTicker(period)\n\tdefer ticker.Stop()\n\tvar idx uint64\n\tfor {\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn fmt.Errorf(\"%w: version got=%d  want=%d\", types.ErrDeadlineExceeded, idx, version)\n\t\tcase <-ticker.C:\n\t\t\tif idx = st.lastAppliedIndex.Load(); idx >= version {\n\t\t\t\treturn nil\n\t\t\t} else {\n\t\t\t\tst.log.WithFields(logrus.Fields{\n\t\t\t\t\t\"got\":  idx,\n\t\t\t\t\t\"want\": version,\n\t\t\t\t}).Debug(\"wait for update version\")\n\t\t\t}\n\t\t}\n\t}\n}\n\n// IsLeader returns whether this node is the leader of the cluster\nfunc (st *Store) IsLeader() bool {\n\treturn st.raft != nil && st.raft.State() == raft.Leader\n}\n","sourceCodeStart":793,"sourceCodeEnd":829,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/cluster/store.go#L793-L829","documentation":"Returned by Store.WaitToRestoreDB when the context is cancelled before the raft last-applied index reaches the requested schema version. It wraps types.ErrDeadlineExceeded with got/want index values, meaning the node has not yet replayed the raft log up to the version required to serve requests.","triggerScenarios":"WaitToRestoreDB(ctx, period, close) polls lastAppliedIndex every tick; ctx.Done() fires (caller timeout or shutdown) while idx < version, so it returns deadline-exceeded with the current and target index.","commonSituations":"Node rejoining after a long offline period with a huge raft log to replay; slow disk making log replay slower than the caller's timeout; the requested version far ahead because the leader committed many schema updates while this node was down.","solutions":["Increase the caller's context timeout so the node has time to replay the raft log","Wait and retry — the node continues applying entries in the background","If replay is persistently too slow, snapshot the node: remove it from the cluster and re-add with a fresh data dir (or take a raft snapshot)","Check disk I/O throughput on the node and the 'got' index progress rate in logs to size the timeout"],"exampleFix":"// before\nctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\nerr := store.WaitToRestoreDB(ctx, time.Second, closeCh)\n// after\nctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)\nerr := store.WaitToRestoreDB(ctx, time.Second, closeCh)\n","handlingStrategy":"retry","validationCode":"if ctx, err := context.WithTimeout(parent, replayBudget); err != nil || replayBudget < expectedLogReplayTime {\n    return errors.New(\"timeout too short for raft log replay\")\n}","typeGuard":null,"tryCatchPattern":"if err := store.WaitToRestoreDB(ctx, period, closeCh); err != nil {\n    if errors.Is(err, types.ErrDeadlineExceeded) {\n        // inspect got/want in message and retry with a longer ctx\n        return retryWithLongerTimeout(err)\n    }\n    return err\n}","preventionTips":["Size the caller timeout against the expected raft log backlog when rejoining nodes","Monitor lastAppliedIndex progress rate to detect slow replay","Snapshot nodes that fall far behind so replay stays short","Ensure adequate disk IOPS for raft log replay"],"tags":["raft","timeout","replication","startup"],"backgroundTag":"raft-log-replay-timeout","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}