hashicorp/nomad · error

No cluster leader

Error message

No cluster leader

What it means

ErrNoLeader is a sentinel error (nomad/structs/errors.go:53) returned by Nomad servers when an RPC requiring Raft consensus cannot find or reach the cluster leader. Without a leader the server cannot forward requests or serve writes/consistent reads, so it reports this error to clients and internal retry logic, which treats it as transient and retryable.

Source

Thrown at nomad/structs/errors.go:53

	ErrUnknownJobPrefix        = "Unknown job"
	ErrUnknownEvaluationPrefix = "Unknown evaluation"
	ErrUnknownDeploymentPrefix = "Unknown deployment"

	errRPCCodedErrorPrefix = "RPC Error:: "

	errDeploymentTerminalNoCancel    = "can't cancel terminal deployment"
	errDeploymentTerminalNoFail      = "can't fail terminal deployment"
	errDeploymentTerminalNoPause     = "can't pause terminal deployment"
	errDeploymentTerminalNoPromote   = "can't promote terminal deployment"
	errDeploymentTerminalNoResume    = "can't resume terminal deployment"
	errDeploymentTerminalNoUnblock   = "can't unblock terminal deployment"
	errDeploymentTerminalNoRun       = "can't run terminal deployment"
	errDeploymentTerminalNoSetHealth = "can't set health of allocations for a terminal deployment"
	errDeploymentRunningNoUnblock    = "can't unblock running deployment"
)

var (
	ErrNoLeader                   = errors.New(errNoLeader)
	ErrNotReadyForConsistentReads = errors.New(errNotReadyForConsistentReads)
	ErrNoRegionPath               = errors.New(errNoRegionPath)
	ErrTokenNotFound              = errors.New(errTokenNotFound)
	ErrTokenExpired               = errors.New(errTokenExpired)
	ErrTokenInvalid               = errors.New(errTokenInvalid)
	ErrPermissionDenied           = errors.New(errPermissionDenied)
	ErrJobRegistrationDisabled    = errors.New(errJobRegistrationDisabled)
	ErrNoNodeConn                 = errors.New(errNoNodeConn)
	ErrUnknownMethod              = errors.New(errUnknownMethod)
	ErrUnknownNomadVersion        = errors.New(errUnknownNomadVersion)
	ErrNodeLacksRpc               = errors.New(errNodeLacksRpc)
	ErrMissingAllocID             = errors.New(errMissingAllocID)
	ErrIncompatibleFiltering      = errors.New(errIncompatibleFiltering)
	ErrMalformedChooseParameter   = errors.New(errMalformedChooseParameter)

	// ErrResultPaginatorCreation is returned by list RPC handlers when the
	// result paginator cannot be built, for example when the server cannot
	// evaluate a requested filter expression. api.ResultPaginatorErrorContent

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check `nomad server members` and `nomad operator raft list-peers`; ensure a majority of voting servers are running and connected
2. Verify Raft ports (4647) and network connectivity between all servers; inspect server logs for leader election messages
3. If quorum is permanently lost, recover the cluster with `nomad operator raft peer remove` or restore from backup/snapshot, then restart clients so they re-register (clients retry automatically)
4. Simply wait/retry — clients already retry registration and CSI hooks treat this error as retryable

Example fix

// handling in client code: treat as transient and retry
if strings.Contains(err.Error(), structs.ErrNoLeader.Error()) {
    return true // retryable: wait for leader election
}

func isNoLeaderErr(err error) bool {
    return strings.Contains(err.Error(), structs.ErrNoLeader.Error())
}

for attempt := 0; attempt < 5; attempt++ {
    if err := rpcCall(); err == nil {
        break
    } else if isNoLeaderErr(err) {
        time.Sleep(backoff(attempt)) // transient: wait for leader election
        continue
    } else {
        return err
    }
}

When it happens

Trigger: Any RPC handled while no Raft leader is elected: leader election in progress after server start/restart, quorum loss (too few voting servers up), network partitions between servers, or during leader failover. Returned by RPC forwarding helpers (forwardFor, canRetry) and heartbeat/registration paths (client.go:1943).

Common situations: Rolling Nomad server restarts; a server cluster down to one of three nodes; firewall/network issues breaking Raft RPC between servers; freshly bootstrapped cluster not yet elected a leader; clients registering during a failover window.

Related errors

• unsupported minimum common raft protocol version
• failed to reset heartbeat since server is not leader
• must provide peer id or address
• No servers found
• Failed to start Raft: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/6603a813613bba5b. Report an issue: GitHub.