hashicorp/nomad · critical

node setup failed

Error message

node setup failed: %v

What it means

NewClient returns this when c.setupNode() fails — the step that loads or generates the node ID/secret, builds the node registration object, and persists initial node state. setupNode errors (e.g. from state store writes or ID resolution) are wrapped here.

Solutions

  1. Inspect the wrapped inner error for the actual cause (state store write vs ID parse).
  2. Ensure state_dir is writable and not corrupt; move/delete state if corruption is confirmed (allocs reschedule).
  3. If a custom node_id is set, verify its format and uniqueness.
  4. Check for another client process using the same data directory.

Example fix

// before
client { node_id = "not a valid id!" }
// after
client { node_id = "9f1e0d2c-3b4a-5f6e-8d7c-1a2b3c4d5e6f" }
Defensive patterns

Strategy: validation

Validate before calling

// verify state dir is writable and node_id (if set) looks like a UUID before startup
if cfg.NodeID != "" {
    if _, err := uuid.Parse(cfg.NodeID); err != nil {
        return fmt.Errorf("invalid node_id %q: %w", cfg.NodeID, err)
    }
}
if err := checkWritable(cfg.StateDir); err != nil {
    return fmt.Errorf("state dir unwritable, node setup would fail: %w", err)
}

Prevention

When it happens

Trigger: Node ID/secret cannot be generated or loaded from the state store; writing the initial node snapshot to the boltdb state DB fails; an explicitly configured node_id is malformed.

Common situations: Unwritable or corrupt state_dir so the node secret cannot be persisted; invalid node_id in the client config; running two clients sharing the same data dir causing ID conflicts.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/73957142263bb16d. Report an issue: GitHub.

Appendix: source

Thrown at client/client.go:455

	c.dynamicRegistry =
		dynamicplugins.NewRegistry(c.stateDB, map[string]dynamicplugins.PluginDispenser{
			dynamicplugins.PluginTypeCSIController: func(info *dynamicplugins.PluginInfo) (any, error) {
				return csi.NewClient(info.ConnectionInfo.SocketPath, logger.Named("csi_client").With("plugin.name", info.Name, "plugin.type", "controller")), nil
			},
			dynamicplugins.PluginTypeCSINode: func(info *dynamicplugins.PluginInfo) (any, error) {
				return csi.NewClient(info.ConnectionInfo.SocketPath, logger.Named("csi_client").With("plugin.name", info.Name, "plugin.type", "client")), nil
			},
		})

	// Setup the clients RPC server
	c.setupClientRpc(rpcs)

	// Initialize the ACL state
	c.clientACLResolver.init()

	// Setup the node
	if err := c.setupNode(); err != nil {
		return nil, fmt.Errorf("node setup failed: %v", err)
	}

	// Add workload identity signer after node secret has been generated/loaded
	c.widsigner = widmgr.NewSigner(widmgr.SignerConfig{
		NodeSecret: c.secretNodeID(),
		Region:     cfg.Region,
		RPC:        c,
	})

	c.fingerprintManager = NewFingerprintManager(
		cfg.PluginSingletonLoader,
		c.GetConfig,
		cfg.Node,
		c.shutdownCh,
		c.updateNodeFromFingerprint,
		c.logger,
	)
	c.pluginManagers = pluginmanager.New(c.logger)

View on GitHub (pinned to 482b49bf1a)