weaviate/weaviate · critical

open raft store: %w

Error message

open raft store: %w

What it means

Weaviate's cluster service startup wraps any failure to open the RAFT log/snapshot store (cluster/service.go Open). RAFT keeps persistent state in the configured RAFT data dir; if opening that store fails the whole node refuses to start so it never serves with inconsistent cluster state.

Source

Thrown at cluster/service.go:231

						}
					}
				}, c.logger)
				return
			}
		}
	}
}

// Open internal RPC service to handle node communication,
// bootstrap the Raft node, and restore the database state
func (c *Service) Open(ctx context.Context, db schema.Indexer) error {
	c.logger.WithField("servers", c.config.NodeNameToPortMap).Info("open cluster service")
	if err := c.rpcServer.Open(); err != nil {
		return fmt.Errorf("start rpc service: %w", err)
	}

	if err := c.Raft.Open(ctx, db); err != nil {
		return fmt.Errorf("open raft store: %w", err)
	}

	hasState, err := raft.HasExistingState(c.Raft.store.logCache, c.Raft.store.logStore, c.Raft.store.snapshotStore)
	if err != nil {
		return err
	}
	c.log.WithField("hasState", hasState).Info("raft init")

	// If we have a state in raft, we only want to re-join the nodes in raft_join list to ensure that we update the
	// configuration with our current ip.
	// If we have no state, we want to do the bootstrap procedure where we will try to join a cluster or notify other
	// peers that we are ready to form a new cluster.
	bootstrapCtx, bCancel := context.WithTimeout(ctx, c.config.BootstrapTimeout)
	defer bCancel()
	if hasState {
		joiner := bootstrap.NewJoiner(c.rpcClient, c.config.NodeID, c.raftAddr, c.config.Voter)
		err = backoff.Retry(func() error {
			joinNodes := bootstrap.ResolveRemoteNodes(c.config.NodeSelector, c.config.NodeNameToPortMap)

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Check RAFT_HOME / work dir exists, is writable, and only one weaviate process uses it
2. Fix disk space / permissions on the data volume
3. If state is disposable or cluster is being rebuilt, stop all nodes, clear the RAFT dir, and re-bootstrap with RAFT_JOIN=true
4. Enable one-node recovery (RAFT_FORCE_ONE_NODE_RECOVERY) for single-node clusters with stale state
5. Inspect inner wrapped message for the precise failing store component (bolt db, snapshot store, etc.)

Example fix

// before
services := cluster.New(...)
if err := services.Open(ctx, db); err != nil { log.Fatal(err) }
// after
// ensure RAFT_HOME dir exists and is writable before starting:
if err := os.MkdirAll(raftHome, 0o755); err != nil { log.Fatal(err) }
if err := services.Open(ctx, db); err != nil { log.Fatalf("open cluster: %v", err) }

// pre-start check (shell)
test -w "$RAFT_HOME" && ! pgrep -f 'weaviate.*' >/dev/null || echo 'dir unwritable or another weaviate running'

if err := clusterService.Open(ctx, db); err != nil {
    var perr *fs.PathError
    if errors.As(err, &perr) { /* fix dir perms/path per perr */ }
    log.Fatalf("cluster open failed: %v", err)
}

When it happens

Trigger: c.Raft.Open(ctx, db) fails: RAFT work dir missing/unwritable, corrupted bolt store or snapshot files, stale lock from another process, or inner 'initialize raft store' errors (mkdir/bolt/log-cache/snapshot/transport).

Common situations: Running two weaviate processes against the same RAFT_HOME dir; disk full or permissions changed on the data volume; corrupted raft.db after unclean shutdown; Kubernetes pod mounting a read-only volume.

Related errors

• initialize raft store: %w
• store not open
• expect integer as raft port: got %s:: %w
• init index %q: %w
• init shard %s of index %s: %w

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/cfb4425f2806b1bb. Report an issue: GitHub.