hashicorp/nomad · error

failed creating temporary directory for the StateDir: %v

Error message

failed creating temporary directory for the StateDir: %v

What it means

When no state_dir is configured, Client.init() creates a temporary directory via os.MkdirTemp("", "NomadClient") for the state. If the OS temp-dir creation fails (usually TMPDIR problems or no writable temp location), this error is returned and blocks client startup.

Source

Thrown at client/client.go:694

func (c *Client) Ready() <-chan struct{} {
	return c.serversContactedCh
}

// init is used to initialize the client and perform any setup
// needed before we begin starting its various components.
func (c *Client) init() error {
	// Ensure the state dir exists if we have one
	conf := c.GetConfig()
	if conf.StateDir != "" {
		if err := os.MkdirAll(conf.StateDir, 0700); err != nil {
			return fmt.Errorf("failed creating state dir: %s", err)
		}

	} else {
		// Otherwise make a temp directory to use.
		p, err := os.MkdirTemp("", "NomadClient")
		if err != nil {
			return fmt.Errorf("failed creating temporary directory for the StateDir: %v", err)
		}

		p, err = filepath.EvalSymlinks(p)
		if err != nil {
			return fmt.Errorf("failed to find temporary directory for the StateDir: %v", err)
		}

		conf = c.UpdateConfig(func(c *config.Config) {
			c.StateDir = p
		})
	}
	c.logger.Info("using state directory", "state_dir", conf.StateDir)

	// Open the state database
	db, err := conf.StateDBFactory(c.logger, conf.StateDir)
	if err != nil {
		return fmt.Errorf("failed to open state database: %v", err)
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Fix or unset TMPDIR in the Nomad agent's environment so /tmp (or the target) is writable.
2. Check /tmp (or $TMPDIR) exists, is writable by the nomad user, and has free space (df -h /tmp).
3. Explicitly set client { state_dir } to a writable path to bypass temp-dir creation entirely.
4. Check SELinux/seccomp policies that may deny mkdir for the nomad process in the temp location.

Example fix

// before
Environment=TMPDIR=/mnt/deleted-tmp  (systemd unit)
# after
Environment=TMPDIR=/var/tmp
# or bypass entirely:
client { state_dir = "/var/lib/nomad/state" }

// verify a usable temp location before starting a client without state_dir
if os.Getenv("TMPDIR") != "" {
    if fi, err := os.Stat(os.Getenv("TMPDIR")); err != nil || !fi.IsDir() {
        return fmt.Errorf("TMPDIR %q invalid", os.Getenv("TMPDIR"))
    }
} else if err := checkWritable("/tmp"); err != nil {
    return fmt.Errorf("/tmp not writable: %w", err)
}

When it happens

Trigger: os.MkdirTemp fails because $TMPDIR points to a non-existent or non-writable path, the temp filesystem is full, or hard sandboxing (seccomp/SELinux) blocks mkdir in /tmp.

Common situations: Systemd unit with PrivateTmp plus a misconfigured tmpfs; TMPDIR set to a deleted or read-only path in the agent's environment; /tmp filled to capacity on small instances.

Related errors

• failed to initialize client: %v
• failed creating state dir: %s
• unable to read rooted allocation directory
• plugin not found
• plugin not executable

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/e5417a252d91aa70. Report an issue: GitHub.