{"record":{"id":"1645649b13199966","repo":"hashicorp/nomad","slug":"failed-to-create-wal-directory-v","errorCode":null,"errorMessage":"failed to create WAL directory: %v","messagePattern":"failed to create WAL directory: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/server.go","lineNumber":1454,"sourceCode":"\t\t\tbackend = s.config.RaftLogStoreConfig.Backend\n\t\t}\n\n\t\tvar store raftBackend\n\t\tswitch backend {\n\t\tcase LogStoreBackendWAL:\n\t\t\t// Check for an existing BoltDB store that needs migration.\n\t\t\tboltPath := filepath.Join(path, \"raft.db\")\n\t\t\tif _, statErr := os.Stat(boltPath); statErr == nil {\n\t\t\t\treturn fmt.Errorf(\n\t\t\t\t\t\"existing BoltDB raft store found at %s; \"+\n\t\t\t\t\t\t\"run 'nomad operator raft migrate-backend %s' while the server \"+\n\t\t\t\t\t\t\"is stopped to migrate to the WAL backend, then start the server again\",\n\t\t\t\t\tboltPath, s.config.DataDir)\n\t\t\t}\n\n\t\t\twalDir := filepath.Join(path, \"wal\")\n\t\t\tif err := ensurePath(walDir, true); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create WAL directory: %v\", err)\n\t\t\t}\n\n\t\t\twalStore, walErr := s.openRaftWAL(walDir)\n\t\t\tif walErr != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to open WAL log store: %v\", walErr)\n\t\t\t}\n\t\t\tstore = walStore\n\n\t\tcase LogStoreBackendBoltDB:\n\t\t\t// Create the BoltDB backend, with NoFreelistSync option\n\t\t\tnoFreelistSync := false\n\t\t\tif s.config.RaftLogStoreConfig != nil {\n\t\t\t\tnoFreelistSync = s.config.RaftLogStoreConfig.BoltDBNoFreelistSync\n\t\t\t}\n\n\t\t\tboltStore, boltErr := raftboltdb.New(raftboltdb.Options{\n\t\t\t\tPath:   filepath.Join(path, \"raft.db\"),\n\t\t\t\tNoSync: false, // fsync each log write","sourceCodeStart":1436,"sourceCodeEnd":1472,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/server.go#L1436-L1472","documentation":"When the WAL raft log store backend is selected, Nomad creates <raft path>/wal via ensurePath before opening the WAL store. Failure to create this directory (permissions, read-only FS, path conflicts) aborts startup with this error.","triggerScenarios":"setupRaft(): ensurePath(filepath.Join(path, \"wal\"), true) returns an error because the wal directory cannot be created under the raft data path.","commonSituations":"data_dir not writable by the nomad user; read-only container filesystem; a file named 'wal' already exists in the raft directory blocking mkdir; disk full or ENOSPC.","solutions":["Fix data_dir permissions/ownership so the nomad process can create directories","Check for a regular file named 'wal' in <data_dir>/raft and remove/rename it","Verify the filesystem is writable and has free space"],"exampleFix":"# before: cannot create /var/lib/nomad/raft/wal\n$ ls -ld /var/lib/nomad  # owned by root\n# after\n$ sudo chown -R nomad:nomad /var/lib/nomad\n$ sudo -u nomad nomad agent -server","handlingStrategy":"validation","validationCode":"wal := filepath.Join(dataDir, \"raft\", \"wal\")\nif fi, err := os.Stat(wal); err == nil && !fi.IsDir() {\n    return fmt.Errorf(\"%s exists and is not a directory\", wal)\n}\nif err := os.MkdirAll(wal, 0700); err != nil {\n    return fmt.Errorf(\"cannot create wal dir: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"if err := server.Start(); err != nil {\n    if strings.Contains(err.Error(), \"failed to create WAL directory\") {\n        logger.Error(\"fix data_dir writability / stray 'wal' file\")\n    }\n    return err\n}","preventionTips":["Ensure data_dir is owned by the nomad service user","Check the raft directory for stray files named 'wal' before startup","Keep data_dir on writable persistent storage, not tmpfs/ro mounts"],"tags":["filesystem","storage","raft","nomad"],"backgroundTag":"mkdir-permission-denied","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}