hashicorp/nomad · error

failed creating state dir: %s

Error message

failed creating state dir: %s

What it means

Client.init() ensures the configured state directory exists using os.MkdirAll with mode 0700. If MkdirAll fails (a parent component of the path is not a directory, permissions, read-only fs), init returns this error, which NewClient surfaces wrapped as "failed to initialize client".

Source

Thrown at client/client.go:687

	c.shutdownGroup.Go(c.emitStats)

	c.logger.Info("started client", "node_id", c.NodeID())
	return c, nil
}

// Ready returns a chan that is closed when the client is fully initialized
func (c *Client) Ready() <-chan struct{} {
	return c.serversContactedCh
}

// init is used to initialize the client and perform any setup
// needed before we begin starting its various components.
func (c *Client) init() error {
	// Ensure the state dir exists if we have one
	conf := c.GetConfig()
	if conf.StateDir != "" {
		if err := os.MkdirAll(conf.StateDir, 0700); err != nil {
			return fmt.Errorf("failed creating state dir: %s", err)
		}

	} else {
		// Otherwise make a temp directory to use.
		p, err := os.MkdirTemp("", "NomadClient")
		if err != nil {
			return fmt.Errorf("failed creating temporary directory for the StateDir: %v", err)
		}

		p, err = filepath.EvalSymlinks(p)
		if err != nil {
			return fmt.Errorf("failed to find temporary directory for the StateDir: %v", err)
		}

		conf = c.UpdateConfig(func(c *config.Config) {
			c.StateDir = p
		})
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check the wrapped inner OS error: "not a directory" means a path component is a file; "permission denied" means fix ownership.
2. Ensure the nomad user owns or can write to state_dir: mkdir -p && chown nomad:nomad.
3. Verify the volume backing state_dir is mounted read-write.
4. If state_dir was intentionally unset, confirm config parsing actually left it empty (then Nomad creates a temp dir instead).

Example fix

// before
client { state_dir = "/etc/nomad.d/state" }  # /etc/nomad.d is root-only
// after
sudo mkdir -p /opt/nomad/state && sudo chown nomad:nomad /opt/nomad/state
client { state_dir = "/opt/nomad/state" }

// precheck the configured state dir before launching the agent
func ensureStateDir(path string) error {
    if fi, err := os.Stat(path); err == nil && !fi.IsDir() {
        return fmt.Errorf("%s is a file, not a directory", path)
    }
    if err := os.MkdirAll(path, 0o700); err != nil {
        return fmt.Errorf("cannot create state dir %s: %w", path, err)
    }
    probe := filepath.Join(path, ".probe")
    if err := os.WriteFile(probe, nil, 0o600); err != nil { return err }
    return os.Remove(probe)
}

When it happens

Trigger: client { state_dir = "..." } set to a path whose creation fails: a path component is an existing regular file, the parent dir is not writable by the nomad user, or the mount is read-only.

Common situations: Typo'd state_dir pointing into a file (e.g. state_dir = "/etc/nomad.conf"); data volume mounted read-only after maintenance; running the agent as a non-root user without write access to the configured path.

Related errors

• failed to initialize client: %v
• failed creating temporary directory for the StateDir: %v
• unable to read rooted allocation directory
• plugin not found
• plugin not executable

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/1c2c2fb2171ddba4. Report an issue: GitHub.