What causes "allocation directory is inaccessible: %w" in hashicorp/nomad?

Confirm the alloc dir exists under the client data_dir and is readable by the Nomad client user

allocation directory is inaccessible: %w

Error message

allocation directory is inaccessible: %w

What it means

During Restore (agent restart recovery), the alloc runner checks that the allocation directory still exists before restoring task state. This error is returned when os.Stat on the alloc dir fails — the storage holding the alloc dir is gone or unreadable, so restoring would operate on nothing.

Source

Thrown at client/allocrunner/alloc_runner.go:479

func (ar *allocRunner) setAlloc(updated *structs.Allocation) {
	ar.allocLock.Lock()
	ar.alloc = updated
	ar.allocLock.Unlock()
}

// GetAllocDir returns the alloc dir which is safe for concurrent use.
func (ar *allocRunner) GetAllocDir() allocdir.Interface {
	return ar.allocDir
}

// Restore state from database. Must be called after NewAllocRunner but before
// Run.
func (ar *allocRunner) Restore() error {
	// We should not carry on to restoring an allocation whose directory is
	// inaccessible. This can happen if allocation storage is ephemeral, e.g.
	// a tmpfs or cloud local SSDs.
	if _, err := os.Stat(ar.allocDir.AllocDirPath()); err != nil {
		return fmt.Errorf("allocation directory is inaccessible: %w", err)
	}

	// Retrieve deployment status to avoid reseting it across agent
	// restarts. Once a deployment status is set Nomad no longer monitors
	// alloc health, so we must persist deployment state across restarts.
	ds, err := ar.stateDB.GetDeploymentStatus(ar.id)
	if err != nil {
		return err
	}

	ns, err := ar.stateDB.GetNetworkStatus(ar.id)
	if err != nil {
		return err
	}

	ar.stateLock.Lock()
	ar.state.DeploymentStatus = ds
	ar.state.NetworkStatus = ns

View on GitHub (pinned to 482b49bf1a)

Solutions

Confirm the alloc dir exists under the client data_dir and is readable by the Nomad client user
If storage is genuinely ephemeral, let the scheduler reschedule: the client marks the alloc as lost/restoration-failed and a replacement runs — nothing to fix locally
Point data_dir at persistent storage or enable host_volume/alloc persistence if allocs must survive reboots
Fix host permissions/disk faults preventing os.Stat from reaching the path

Defensive patterns

Strategy: validation

Validate before calling

// Before agent restarts (or in tooling), verify each alloc dir exists
for _, alloc := range allocs {
  p := filepath.Join(clientDataDir, "alloc", alloc.ID)
  if _, err := os.Stat(p); err != nil {
    // storage lost: expect rescheduling instead of restore
  }
}

Try / catch

if err := ar.Restore(); err != nil {
  if strings.Contains(err.Error(), "allocation directory is inaccessible") {
    // treat as lost alloc: rely on scheduler rescheduling; skip restore
  }
  return err
}

Prevention

Avoid ephemeral storage (tmpfs, local SSD) for the Nomad client data_dir if allocs must survive reboots
Do not run host GC or cleanup cron jobs that delete alloc dirs while the agent is down
Keep data_dir ownership consistent across upgrades
Monitor node disks for failures that remove alloc storage

When it happens

Trigger: Agent restart/restore path: os.Stat(ar.allocDir.AllocDirPath()) fails with ENOENT (dir deleted — ephemeral disk, GC while agent down, tmpfs/local SSD wiped) or EACCES/EIO (permissions, disk fault).

Common situations: Clients configured with ephemeral disks (cloud local SSD, tmpfs data dir) that are wiped on reboot; host GC removed the alloc dir while the agent was down; data-dir moved/permissions changed after an upgrade or user switch.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c8454dbc3e18fbec. Report an issue: GitHub.