hashicorp/nomad · warning

cluster ID not ready yet

Error message

cluster ID not ready yet

What it means

ClusterMetadata (cluster ID) generation only happens on the leader. If no persisted cluster ID exists yet and the server handling the request is not the leader, there is nothing it can do, so it returns "cluster ID not ready yet". Callers must retry against the leader once the ID is generated.

Source

Thrown at nomad/server.go:2304

	s.clusterIDLock.Lock()
	defer s.clusterIDLock.Unlock()

	// try to load the cluster ID from state store
	fsmState := s.fsm.State()
	existingMeta, err := fsmState.ClusterMetadata(nil)
	if err != nil {
		s.logger.Named("core").Error("failed to get cluster ID", "error", err)
		return structs.ClusterMetadata{}, err
	}

	// got the cluster ID from state store, cache that and return it
	if existingMeta != nil && existingMeta.ClusterID != "" {
		return *existingMeta, nil
	}

	// if we are not the leader, nothing more we can do
	if !s.IsLeader() {
		return structs.ClusterMetadata{}, errors.New("cluster ID not ready yet")
	}

	// we are the leader, try to generate the ID now
	generatedMD, err := s.generateClusterMetadata()
	if err != nil {
		return structs.ClusterMetadata{}, err
	}

	return generatedMD, nil
}

func (s *Server) isSingleServerCluster() bool {
	return s.config.BootstrapExpect == 1
}

// peersInfoContent is used to help operators understand what happened to the
// peers.json file. This is written to a file called peers.info in the same
// location.

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Retry the request after a short backoff; it succeeds once the leader generates the cluster ID
2. Route the request to the leader (use nomad operator api /v1/status/leader to find it, or ensure the client targets the leader address)
3. Verify cluster health and that a leader exists (`nomad server members`) before depending on cluster ID

Example fix

// before
meta := server.ClusterMetadata()
useClusterID(meta.ID)
// after (retry until ready)
var md structs.ClusterMetadata
var err error
for i := 0; i < 30; i++ {
    md, err = server.ClusterMetadata()
    if err == nil { break }
    if !strings.Contains(err.Error(), "not ready yet") { return err }
    time.Sleep(time.Second)
}

// Confirm a leader exists before depending on cluster ID
for _, m := range client.Agent().Members() {
    if m.Leader { break }
}
// else wait for leader election first

meta, err := getClusterMetadata()
if err != nil && strings.Contains(err.Error(), "cluster ID not ready yet") {
    time.Sleep(2 * time.Second)
    meta, err = getClusterMetadata() // or target the leader
}

When it happens

Trigger: Querying cluster metadata (e.g. via state store access used by RPCs like ClusterMetadata or replication paths) on a follower server before the leader has generated and persisted the cluster ID, such as immediately after forming a new cluster or restoring from a state where cluster ID is absent.

Common situations: New cluster bootstrapping; right after leader election where the new leader has not yet finished generateClusterMetadata; clients hitting a non-leader node in multi-server setups.

Related errors

• failed to reset heartbeat since server is not leader
• unsupported minimum common raft protocol version
• must provide peer id or address
• No cluster leader
• Not ready to serve consistent reads

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/74f81b5ec5470225. Report an issue: GitHub.