{"record":{"id":"cb0586063ec56d69","repo":"hashicorp/nomad","slug":"unable-to-open-raft-logs-that-are-in-use","errorCode":null,"errorMessage":"unable to open raft logs that are in use","messagePattern":"unable to open raft logs that are in use","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"helper/raftutil/state.go","lineNumber":24,"sourceCode":"import (\n\t\"bytes\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/hashicorp/go-msgpack/v2/codec\"\n\t\"github.com/hashicorp/nomad/nomad/structs\"\n\t\"github.com/hashicorp/raft\"\n\traftboltdb \"github.com/hashicorp/raft-boltdb/v2\"\n\traftwal \"github.com/hashicorp/raft-wal\"\n\t\"go.etcd.io/bbolt\"\n)\n\nvar (\n\terrAlreadyOpen = errors.New(\"unable to open raft logs that are in use\")\n)\n\n// RaftStore is the interface returned by RaftStateInfo, satisfied by both\n// *raftboltdb.BoltStore and *raftwal.WAL.\ntype RaftStore interface {\n\traft.LogStore\n\traft.StableStore\n\tClose() error\n}\n\n// RaftStateInfo returns info about the raft state found at path p. The path\n// may point to a BoltDB file (raft.db) or a WAL directory. The returned\n// RaftStore must be closed by the caller.\nfunc RaftStateInfo(p string) (store RaftStore, firstIdx uint64, lastIdx uint64, err error) {\n\tinfo, statErr := os.Stat(p)\n\tif statErr != nil {\n\t\treturn nil, 0, 0, fmt.Errorf(\"failed to stat %s: %v\", p, statErr)\n\t}","sourceCodeStart":6,"sourceCodeEnd":42,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/helper/raftutil/state.go#L6-L42","documentation":"errAlreadyOpen is returned by RaftStateInfo (raftStateInfoBoltDB) when opening the raft BoltDB fails with a lock \"timeout\" — meaning another process (e.g. a running Nomad agent) holds the boltdb file lock. It tells the caller the logs exist but are actively in use.","triggerScenarios":"Running RaftStateInfo or LogEntries against a data dir whose raft.db is locked by a live Nomad server; bbolt Open times out acquiring the flock and strings.HasSuffix(err.Error(), \"timeout\") maps it to this sentinel.","commonSituations":"Running inspection/ops tooling against a production data dir while the server is up; two agents pointed at the same data dir; leftover lock from a crashed process on filesystems without proper flock cleanup.","solutions":["Stop the Nomad server process holding the raft.db lock before inspecting the data directory.","Point the tool at a copy/snapshot of the data dir instead of the live one.","Check for a second agent using the same data_dir (duplicate processes) and terminate it.","Compare with errors.Is(err, raftutil.ErrNoKeyID-style sentinel) — use errors.Is against errAlreadyOpen-equivalent to branch on 'in use' vs other failures."],"exampleFix":"// before\nstore, _, _, err := raftutil.RaftStateInfo(dir)\nif err != nil { return err }\n// after\nstore, _, _, err := raftutil.RaftStateInfo(dir)\nif err != nil {\n    if strings.HasSuffix(err.Error(), \"in use\") {\n        return fmt.Errorf(\"stop the nomad agent using %s before inspecting\", dir)\n    }\n    return err\n}","handlingStrategy":"retry","validationCode":"// verify no live agent holds the data dir before inspection\nif pid := runningNomadPidFor(dataDir); pid != 0 {\n    return fmt.Errorf(\"nomad agent pid %d is using %s; stop it first\", pid, dataDir)\n}","typeGuard":"func isInUseError(err error) bool {\n    return err != nil && strings.HasSuffix(err.Error(), \"in use\")\n}","tryCatchPattern":"store, _, _, err := raftutil.RaftStateInfo(dir)\nif err != nil {\n    if strings.HasSuffix(err.Error(), \"in use\") {\n        return fmt.Errorf(\"raft logs locked; stop the nomad agent or inspect a copy of %s\", dir)\n    }\n    return fmt.Errorf(\"failed to open raft logs: %w\", err)\n}","preventionTips":["Never run raft inspection tools against a live server's data_dir.","Inspect a filesystem snapshot or backup copy instead.","Ensure only one agent process is configured with a given data_dir."],"tags":["raft","boltdb","file-lock"],"backgroundTag":"database-locked","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}