hashicorp/nomad · critical

node setup failed: %v

Error message

node setup failed: %v

What it means

NewClient returns this when c.setupNode() fails — the step that loads or generates the node ID/secret, builds the node registration object, and persists initial node state. setupNode errors (e.g. from state store writes or ID resolution) are wrapped here.

Source

Thrown at client/client.go:455

	c.dynamicRegistry =
		dynamicplugins.NewRegistry(c.stateDB, map[string]dynamicplugins.PluginDispenser{
			dynamicplugins.PluginTypeCSIController: func(info *dynamicplugins.PluginInfo) (any, error) {
				return csi.NewClient(info.ConnectionInfo.SocketPath, logger.Named("csi_client").With("plugin.name", info.Name, "plugin.type", "controller")), nil
			},
			dynamicplugins.PluginTypeCSINode: func(info *dynamicplugins.PluginInfo) (any, error) {
				return csi.NewClient(info.ConnectionInfo.SocketPath, logger.Named("csi_client").With("plugin.name", info.Name, "plugin.type", "client")), nil
			},
		})

	// Setup the clients RPC server
	c.setupClientRpc(rpcs)

	// Initialize the ACL state
	c.clientACLResolver.init()

	// Setup the node
	if err := c.setupNode(); err != nil {
		return nil, fmt.Errorf("node setup failed: %v", err)
	}

	// Add workload identity signer after node secret has been generated/loaded
	c.widsigner = widmgr.NewSigner(widmgr.SignerConfig{
		NodeSecret: c.secretNodeID(),
		Region:     cfg.Region,
		RPC:        c,
	})

	c.fingerprintManager = NewFingerprintManager(
		cfg.PluginSingletonLoader,
		c.GetConfig,
		cfg.Node,
		c.shutdownCh,
		c.updateNodeFromFingerprint,
		c.logger,
	)
	c.pluginManagers = pluginmanager.New(c.logger)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Inspect the wrapped inner error for the actual cause (state store write vs ID parse).
2. Ensure state_dir is writable and not corrupt; move/delete state if corruption is confirmed (allocs reschedule).
3. If a custom node_id is set, verify its format and uniqueness.
4. Check for another client process using the same data directory.

Example fix

// before
client { node_id = "not a valid id!" }
// after
client { node_id = "9f1e0d2c-3b4a-5f6e-8d7c-1a2b3c4d5e6f" }

// verify state dir is writable and node_id (if set) looks like a UUID before startup
if cfg.NodeID != "" {
    if _, err := uuid.Parse(cfg.NodeID); err != nil {
        return fmt.Errorf("invalid node_id %q: %w", cfg.NodeID, err)
    }
}
if err := checkWritable(cfg.StateDir); err != nil {
    return fmt.Errorf("state dir unwritable, node setup would fail: %w", err)
}

When it happens

Trigger: Node ID/secret cannot be generated or loaded from the state store; writing the initial node snapshot to the boltdb state DB fails; an explicitly configured node_id is malformed.

Common situations: Unwritable or corrupt state_dir so the node secret cannot be persisted; invalid node_id in the client config; running two clients sharing the same data dir causing ID conflicts.

Related errors

• failed to restore state
• volume lookup failed: %s %v
• error parsing: root should be an object
• cannot specify Accessor ID
• network already configured but not found in state

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/73957142263bb16d. Report an issue: GitHub.