hashicorp/nomad · critical

failed to find temporary directory for the StateDir: %v

Error message

failed to find temporary directory for the StateDir: %v

What it means

During Nomad client initialization, when no explicit StateDir is configured, the client creates a temporary directory for state and then calls filepath.EvalSymlinks on it. If EvalSymlinks fails, the resolved path cannot be determined and init aborts with this error wrapping the OS error. It indicates the freshly created temp directory is not accessible via path resolution.

Source

Thrown at client/client.go:699

// needed before we begin starting its various components.
func (c *Client) init() error {
	// Ensure the state dir exists if we have one
	conf := c.GetConfig()
	if conf.StateDir != "" {
		if err := os.MkdirAll(conf.StateDir, 0700); err != nil {
			return fmt.Errorf("failed creating state dir: %s", err)
		}

	} else {
		// Otherwise make a temp directory to use.
		p, err := os.MkdirTemp("", "NomadClient")
		if err != nil {
			return fmt.Errorf("failed creating temporary directory for the StateDir: %v", err)
		}

		p, err = filepath.EvalSymlinks(p)
		if err != nil {
			return fmt.Errorf("failed to find temporary directory for the StateDir: %v", err)
		}

		conf = c.UpdateConfig(func(c *config.Config) {
			c.StateDir = p
		})
	}
	c.logger.Info("using state directory", "state_dir", conf.StateDir)

	// Open the state database
	db, err := conf.StateDBFactory(c.logger, conf.StateDir)
	if err != nil {
		return fmt.Errorf("failed to open state database: %v", err)
	}

	// Upgrade the state database
	if err := db.Upgrade(); err != nil {
		// Upgrade only returns an error on critical persistence
		// failures in which an operator should intervene before the

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check that the effective temp directory (TMPDIR or /tmp) exists, is writable, and has no broken symlink components.
2. Set an explicit state_dir in the client config so the temp-dir creation path is skipped entirely.
3. Inspect the wrapped %v OS error for the exact syscall failure (e.g. ENOENT, EACCES) and fix the filesystem/permissions accordingly.

Example fix

// before (broken TMPDIR)
env TMPDIR=/nonexistent nomad agent -client
// after
mkdir -p /var/lib/nomad/state
# config.hcl
client {
  state_dir = "/var/lib/nomad/state"
}

tmp := os.TempDir()
if fi, err := os.Stat(tmp); err != nil || !fi.IsDir() {
    return fmt.Errorf("temp dir %q unusable: %v", tmp, err)
}

if err := clientInit(); err != nil {
    if strings.Contains(err.Error(), "failed to find temporary directory for the StateDir") {
        // inspect wrapped OS error, fix TMPDIR, retry with explicit state_dir
    }
}

When it happens

Trigger: NewClient/TestClient_Init/TestRPCOnlyClient init path with conf.StateDir empty: os.MkdirTemp succeeds but filepath.EvalSymlinks(p) returns an error immediately after creation.

Common situations: Hosts with broken TMPDIR settings pointing to non-existent or inaccessible directories, container images with missing/incorrect temp volume mounts, or heavily sandboxed environments where symlink resolution fails due to permissions or race conditions.

Related errors

• failed creating temporary directory for the AllocDir: %v
• failed to open state database: %v
• failed creating alloc mounts dir: %w
• failed creating alloc dir: %w
• failed to find temporary directory for the AllocDir: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/82700dd9c673aa4a. Report an issue: GitHub.