hashicorp/nomad · error

failed creating temporary directory for the StateDir

Error message

failed creating temporary directory for the StateDir: %v

What it means

When no state_dir is configured, Client.init() creates a temporary directory via os.MkdirTemp("", "NomadClient") for the state. If the OS temp-dir creation fails (usually TMPDIR problems or no writable temp location), this error is returned and blocks client startup.

Solutions

1. Fix or unset TMPDIR in the Nomad agent's environment so /tmp (or the target) is writable.
2. Check /tmp (or $TMPDIR) exists, is writable by the nomad user, and has free space (df -h /tmp).
3. Explicitly set client { state_dir } to a writable path to bypass temp-dir creation entirely.
4. Check SELinux/seccomp policies that may deny mkdir for the nomad process in the temp location.

Example fix

// before
Environment=TMPDIR=/mnt/deleted-tmp  (systemd unit)
# after
Environment=TMPDIR=/var/tmp
# or bypass entirely:
client { state_dir = "/var/lib/nomad/state" }

// verify a usable temp location before starting a client without state_dir
if os.Getenv("TMPDIR") != "" {
    if fi, err := os.Stat(os.Getenv("TMPDIR")); err != nil || !fi.IsDir() {
        return fmt.Errorf("TMPDIR %q invalid", os.Getenv("TMPDIR"))
    }
} else if err := checkWritable("/tmp"); err != nil {
    return fmt.Errorf("/tmp not writable: %w", err)
}

When it happens

Trigger: os.MkdirTemp fails because $TMPDIR points to a non-existent or non-writable path, the temp filesystem is full, or hard sandboxing (seccomp/SELinux) blocks mkdir in /tmp.

Common situations: Systemd unit with PrivateTmp plus a misconfigured tmpfs; TMPDIR set to a deleted or read-only path in the agent's environment; /tmp filled to capacity on small instances.

Related errors

• failed creating state dir
• failed to initialize client
• alloc dir must be absolute
• allocation directory is inaccessible
• allocation not found

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/e5417a252d91aa70. Report an issue: GitHub.

Appendix: source

Thrown at client/client.go:694

func (c *Client) Ready() <-chan struct{} {
	return c.serversContactedCh
}

// init is used to initialize the client and perform any setup
// needed before we begin starting its various components.
func (c *Client) init() error {
	// Ensure the state dir exists if we have one
	conf := c.GetConfig()
	if conf.StateDir != "" {
		if err := os.MkdirAll(conf.StateDir, 0700); err != nil {
			return fmt.Errorf("failed creating state dir: %s", err)
		}

	} else {
		// Otherwise make a temp directory to use.
		p, err := os.MkdirTemp("", "NomadClient")
		if err != nil {
			return fmt.Errorf("failed creating temporary directory for the StateDir: %v", err)
		}

		p, err = filepath.EvalSymlinks(p)
		if err != nil {
			return fmt.Errorf("failed to find temporary directory for the StateDir: %v", err)
		}

		conf = c.UpdateConfig(func(c *config.Config) {
			c.StateDir = p
		})
	}
	c.logger.Info("using state directory", "state_dir", conf.StateDir)

	// Open the state database
	db, err := conf.StateDBFactory(c.logger, conf.StateDir)
	if err != nil {
		return fmt.Errorf("failed to open state database: %v", err)
	}

View on GitHub (pinned to 482b49bf1a)