{"record":{"id":"c8454dbc3e18fbec","repo":"hashicorp/nomad","slug":"allocation-directory-is-inaccessible-w","errorCode":null,"errorMessage":"allocation directory is inaccessible: %w","messagePattern":"allocation directory is inaccessible: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/alloc_runner.go","lineNumber":479,"sourceCode":"func (ar *allocRunner) setAlloc(updated *structs.Allocation) {\n\tar.allocLock.Lock()\n\tar.alloc = updated\n\tar.allocLock.Unlock()\n}\n\n// GetAllocDir returns the alloc dir which is safe for concurrent use.\nfunc (ar *allocRunner) GetAllocDir() allocdir.Interface {\n\treturn ar.allocDir\n}\n\n// Restore state from database. Must be called after NewAllocRunner but before\n// Run.\nfunc (ar *allocRunner) Restore() error {\n\t// We should not carry on to restoring an allocation whose directory is\n\t// inaccessible. This can happen if allocation storage is ephemeral, e.g.\n\t// a tmpfs or cloud local SSDs.\n\tif _, err := os.Stat(ar.allocDir.AllocDirPath()); err != nil {\n\t\treturn fmt.Errorf(\"allocation directory is inaccessible: %w\", err)\n\t}\n\n\t// Retrieve deployment status to avoid reseting it across agent\n\t// restarts. Once a deployment status is set Nomad no longer monitors\n\t// alloc health, so we must persist deployment state across restarts.\n\tds, err := ar.stateDB.GetDeploymentStatus(ar.id)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tns, err := ar.stateDB.GetNetworkStatus(ar.id)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tar.stateLock.Lock()\n\tar.state.DeploymentStatus = ds\n\tar.state.NetworkStatus = ns","sourceCodeStart":461,"sourceCodeEnd":497,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/alloc_runner.go#L461-L497","documentation":"During Restore (agent restart recovery), the alloc runner checks that the allocation directory still exists before restoring task state. This error is returned when os.Stat on the alloc dir fails — the storage holding the alloc dir is gone or unreadable, so restoring would operate on nothing.","triggerScenarios":"Agent restart/restore path: os.Stat(ar.allocDir.AllocDirPath()) fails with ENOENT (dir deleted — ephemeral disk, GC while agent down, tmpfs/local SSD wiped) or EACCES/EIO (permissions, disk fault).","commonSituations":"Clients configured with ephemeral disks (cloud local SSD, tmpfs data dir) that are wiped on reboot; host GC removed the alloc dir while the agent was down; data-dir moved/permissions changed after an upgrade or user switch.","solutions":["Confirm the alloc dir exists under the client data_dir and is readable by the Nomad client user","If storage is genuinely ephemeral, let the scheduler reschedule: the client marks the alloc as lost/restoration-failed and a replacement runs — nothing to fix locally","Point data_dir at persistent storage or enable host_volume/alloc persistence if allocs must survive reboots","Fix host permissions/disk faults preventing os.Stat from reaching the path"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Before agent restarts (or in tooling), verify each alloc dir exists\nfor _, alloc := range allocs {\n  p := filepath.Join(clientDataDir, \"alloc\", alloc.ID)\n  if _, err := os.Stat(p); err != nil {\n    // storage lost: expect rescheduling instead of restore\n  }\n}","typeGuard":null,"tryCatchPattern":"if err := ar.Restore(); err != nil {\n  if strings.Contains(err.Error(), \"allocation directory is inaccessible\") {\n    // treat as lost alloc: rely on scheduler rescheduling; skip restore\n  }\n  return err\n}","preventionTips":["Avoid ephemeral storage (tmpfs, local SSD) for the Nomad client data_dir if allocs must survive reboots","Do not run host GC or cleanup cron jobs that delete alloc dirs while the agent is down","Keep data_dir ownership consistent across upgrades","Monitor node disks for failures that remove alloc storage"],"tags":["restore","filesystem","agent-restart","alloc-dir"],"backgroundTag":"alloc-dir-inaccessible","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}