What causes "Error acquiring lock on file %s: %s" in hyperledger/fabric?

Check filesystem permissions and ownership on the lock file's directory and ensure the process user can read/write it

Error acquiring lock on file

Error message

Error acquiring lock on file %s: %s

What it means

This panic occurs in the file-lock wrapper around LevelDB when opening the lock file fails with an error other than syscall.EAGAIN. The library only treats EAGAIN as a normal 'lock already held' condition; any other open failure is considered unrecoverable and panics. It signals that the lock database could not be opened at all, so locking state is unknown.

Solutions

Check filesystem permissions and ownership on the lock file's directory and ensure the process user can read/write it
Inspect/clear corrupted LevelDB lock artifacts (back up first), or point the lock path at a clean directory
Check disk space (df -h) and I/O health in logs (dmesg) for underlying OpenFile failures
Ensure only one process uses the path and the filesystem supports proper locking (avoid NFS for lock/ledger dirs)

Example fix

// before (panics on unexpected open error)
if err != nil {
	panic(fmt.Sprintf("Error acquiring lock on file %s: %s", f.filePath, err))
}
// after (ensure a clean, writable directory before Lock)
if err := os.MkdirAll(filepath.Dir(lockPath), 0755); err != nil { return err }
if err := os.Chmod(filepath.Dir(lockPath), 0755); err != nil { return err }
lock.Lock()

Defensive patterns

Strategy: try-catch

Validate before calling

func canAcquireLock(dir string) error {
	fi, err := os.Stat(dir)
	if err != nil { return err }
	if !fi.IsDir() { return fmt.Errorf("%s is not a directory", dir) }
	tmp := filepath.Join(dir, ".locktest")
	f, err := os.Create(tmp); if err != nil { return err }
	f.Close(); os.Remove(tmp)
	return nil
}

Type guard

func isOpenError(err error) bool { return err != nil && errors.Is(err, syscall.EAGAIN) }

Try / catch

func() {
	defer func() {
		if r := recover(); r != nil {
			log.Errorf("lock acquisition panicked: %v", r)
			// check permissions/disk/corruption, then retry on a clean path
		}
	}()
	lock.Lock()
}()

Prevention

Ensure the lock directory exists and is writable by the process user before locking
Never place ledger/lock directories on NFS; use local filesystems
Check disk space and clean unclean-shutdown artifacts before startup
Run only one process per lock path

When it happens

Trigger: Calling FileLock.Lock() on a path whose underlying leveldb.OpenFile fails with something other than EAGAIN — e.g. corrupted lock DB files, permission denied on the directory, disk I/O error, or the directory being unusable. As seen in TestFileLock/TestFileLockLockUnlockLock, normal usage opens the file in an existing empty dir; failure of OpenFile with an unexpected error triggers the panic.

Common situations: Running a peer/node whose data directory has wrong permissions or is read-only; leftover/corrupted LOCK files from an unclean shutdown; disk full; the lock path pointing at a file where a directory is expected; running two processes on NFS where flock semantics differ and OpenFile fails unexpectedly.

Related errors

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/227b45d4b0cecd91. Report an issue: GitHub.

Appendix: source

Thrown at common/ledger/util/leveldbhelper/leveldb_helper.go:219

// another process, error would be returned. When the db is closed
// or the owner process dies, the lock would be released and hence
// the other process can open the db. We exploit this leveldb
// functionality to acquire and release file lock as the leveldb
// supports this for Windows, Solaris, and Unix.
func (f *FileLock) Lock() error {
	dbOpts := &opt.Options{}
	var err error
	var dirEmpty bool
	if dirEmpty, err = fileutil.CreateDirIfMissing(f.filePath); err != nil {
		panic(fmt.Sprintf("Error creating dir if missing: %s", err))
	}
	dbOpts.ErrorIfMissing = !dirEmpty
	db, err := leveldb.OpenFile(f.filePath, dbOpts)
	if err != nil && err == syscall.EAGAIN {
		return errors.Errorf("lock is already acquired on file %s", f.filePath)
	}
	if err != nil {
		panic(fmt.Sprintf("Error acquiring lock on file %s: %s", f.filePath, err))
	}

	// only mutate the lock db reference AFTER validating that the lock was held.
	f.db = db

	return nil
}

// Determine if the lock is currently held open.
func (f *FileLock) IsLocked() bool {
	return f.db != nil
}

// Unlock releases a previously acquired lock. We achieve this by closing
// the previously opened db. FileUnlock can be called multiple times.
func (f *FileLock) Unlock() {
	if f.db == nil {
		return

View on GitHub (pinned to 2736b63f8f)