hashicorp/nomad · critical

failed to create server Consul syncer: %v

Error message

failed to create server Consul syncer: %v

What it means

NewServer wraps any error returned by setupConsulSyncer, which builds the Consul-backed syncer used to advertise and discover Nomad servers and services. The wrapped error indicates the Consul syncer infrastructure could not be initialized, so the server cannot start. It is a startup-only failure: NewServer returns nil and the process should not continue.

Source

Thrown at nomad/server.go:512

	// Initialize the wan Serf
	s.serf, err = s.setupSerf(config.SerfConfig, s.eventCh, serfSnapshot)
	if err != nil {
		s.Shutdown()
		s.logger.Error("failed to start serf WAN", "error", err)
		return nil, fmt.Errorf("Failed to start serf: %v", err)
	}

	// Initialize the scheduling workers
	if err := s.setupWorkers(s.shutdownCtx); err != nil {
		s.Shutdown()
		s.logger.Error("failed to start workers", "error", err)
		return nil, fmt.Errorf("Failed to start workers: %v", err)
	}

	// Setup the Consul syncer
	if err := s.setupConsulSyncer(); err != nil {
		s.logger.Error("failed to create server consul syncer", "error", err)
		return nil, fmt.Errorf("failed to create server Consul syncer: %v", err)
	}

	// Setup the deployment watcher.
	if err := s.setupDeploymentWatcher(); err != nil {
		s.logger.Error("failed to create deployment watcher", "error", err)
		return nil, fmt.Errorf("failed to create deployment watcher: %v", err)
	}

	// Setup the volume watcher
	if err := s.setupVolumeWatcher(); err != nil {
		s.logger.Error("failed to create volume watcher", "error", err)
		return nil, fmt.Errorf("failed to create volume watcher: %v", err)
	}
	s.volumeControllerFutures = map[string]context.Context{}

	// Start the eval broker notification system so any subscribers can get
	// updates when the processes SetEnabled is triggered.
	go s.evalBroker.enabledNotifier.Run()

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check the wrapped inner error in the log line just above ('failed to create server consul syncer') to see the root cause.
2. Verify the Consul agent address is reachable (e.g. `consul members` / curl http://127.0.0.1:8500/v1/agent/self).
3. Fix consul block in server config: address, ACL token, CA cert, cert/key paths.
4. Ensure Consul starts before Nomad (systemd dependency or container ordering).

Example fix

// before
consul {
  address = "consul.internal:8500"
  ca_file = "/etc/consul/wrong-ca.pem"
}
// after
consul {
  address = "127.0.0.1:8500"
  ca_file = "/etc/consul/ca.pem"
  token   = "<valid-acl-token>"
}

// before NewServer
dial, err := net.DialTimeout("tcp", cfg.ConsulConfig.ServerAutoJoin != "" ? consulAddr : consulAddr, 2*time.Second)
if err != nil { return fmt.Errorf("consul unreachable: %w", err) }

srv, err := nomad.NewServer(cfg, logger)
if err != nil {
    if strings.Contains(err.Error(), "failed to create server Consul syncer") {
        // surface consul connectivity/config remediation
        return fmt.Errorf("server startup blocked: consul syncer init failed: %w", err)
    }
    return err
}

When it happens

Trigger: NewServer called with a config whose Consul block points to an unreachable/misconfigured Consul agent; setupConsulSyncer fails to create the Consul client (bad address, invalid ACL token, bad TLS/CA settings) or its API client cannot be constructed.

Common situations: Bad consul.address in server config, Consul not running in the dev/VM, invalid consul CA cert or ACL token after a credential rotation, malformed auto-join/advertise-retry settings, or server.startup_timeout issues in containerized environments where Consul starts after Nomad.

Related errors

• consul address must be set on nomad client
• failed to initialize Consul client config: %v
• failed to initialize Consul client: %v
• Failed to initialize Consul client: %v
• client setup failed: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/cce54d996e809e05. Report an issue: GitHub.