hashicorp/nomad · critical

server setup failed: %v

Error message

server setup failed: %v

What it means

setupServer constructs the actual Nomad server via nomad.NewServer; any error from server construction (Raft setup, Serf start, state store open, TLS, etc.) is wrapped as 'server setup failed' and NewAgent aborts.

Source

Thrown at command/agent/agent.go:1204

	// Generate a node ID and persist it if it is the first instance, otherwise
	// read the persisted node ID.
	if err := a.setupNodeID(conf); err != nil {
		return fmt.Errorf("setting up server node ID failed: %s", err)
	}

	// Sets up the keyring for gossip encryption
	if err := a.setupKeyrings(conf); err != nil {
		return fmt.Errorf("failed to configure keyring: %v", err)
	}

	// Create the server
	server, err := nomad.NewServer(conf,
		a.consulCatalog,           // self service discovery
		a.consulConfigEntriesFunc, // writing config entries for gateways
	)
	if err != nil {
		return fmt.Errorf("server setup failed: %v", err)
	}
	a.server = server

	// Consul check addresses default to bind but can be toggled to use advertise
	rpcCheckAddr := a.config.normalizedAddrs.RPC
	serfCheckAddr := a.config.normalizedAddrs.Serf

	defaultConsul := conf.ConsulConfigs[structs.ConsulDefaultCluster]

	if *defaultConsul.ChecksUseAdvertise {
		rpcCheckAddr = a.config.AdvertiseAddrs.RPC
		serfCheckAddr = a.config.AdvertiseAddrs.Serf
	}

	// Create the Nomad Server services for Consul
	if *defaultConsul.AutoAdvertise {
		httpServ := &structs.Service{
			Name:      defaultConsul.ServerServiceName,

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped %v detail for the underlying cause
2. Check for port conflicts (lsof/ss on 4647, 4648) and change ports or stop the conflicting process
3. Back up then inspect/repair data_dir/server state; never run a newer data_dir with an older Nomad binary
4. Verify TLS certificate/key/CA validity and hostname match if TLS is enabled
5. Ensure node ID matches prior state, or restore a consistent data_dir backup

Example fix

// before (shell)
nomad agent -server ...  # Error: server setup failed: ...bind: address already in use
// after (shell)
ss -lntp | grep 4647  # stop the conflicting process or set ports { http=4646 rpc=4647 serf=4648 }
rm -f stale raft state only with a verified backup

// Pre-flight: check Raft/Serf ports are free
for _, p := range []string{":4647", ":4648"} {
    if ln, err := net.Listen("tcp", p); err != nil {
        log.Fatalf("port %s in use: %v", p, err)
    } else {
        ln.Close()
    }
}

server, err := nomad.NewServer(conf, consulCatalog, consulConfigEntriesFunc)
if err != nil {
    if strings.Contains(err.Error(), "address already in use") {
        return fmt.Errorf("server setup failed (port conflict): %w", err)
    }
    return fmt.Errorf("server setup failed: %w", err)
}

When it happens

Trigger: nomad.NewServer returns an error during agent startup: cannot bind Raft/Serf ports, corrupted raft/raft.peers or state store in data_dir, invalid TLS material at runtime, or invalid runtime-tunable config values.

Common situations: Port conflicts on 4647/serf ports, corrupted data_dir after unclean shutdown or version downgrade, mismatched TLS certs/keys, stale raft state after changing node ID, or invalid autopilot/raft protocol settings.

Related errors

• server config setup failed: %s
• Failed to start Raft: %v
• Failed to start serf: %v
• failed to reset heartbeat since server is not leader
• unsupported minimum common raft protocol version

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/7bf2cb452ec41de7. Report an issue: GitHub.