What causes "Error acquiring lock on file %s: %s" in hyperledger/fabric?

Check filesystem permissions and ownership on the lock file's directory and ensure the process user can read/write it

Error acquiring lock on file %s: %s

Error message

Error acquiring lock on file %s: %s

What it means

This panic occurs in the file-lock wrapper around LevelDB when opening the lock file fails with an error other than syscall.EAGAIN. The library only treats EAGAIN as a normal 'lock already held' condition; any other open failure is considered unrecoverable and panics. It signals that the lock database could not be opened at all, so locking state is unknown.

Source

Thrown at common/ledger/util/leveldbhelper/leveldb_helper.go:219

// another process, error would be returned. When the db is closed
// or the owner process dies, the lock would be released and hence
// the other process can open the db. We exploit this leveldb
// functionality to acquire and release file lock as the leveldb
// supports this for Windows, Solaris, and Unix.
func (f *FileLock) Lock() error {
	dbOpts := &opt.Options{}
	var err error
	var dirEmpty bool
	if dirEmpty, err = fileutil.CreateDirIfMissing(f.filePath); err != nil {
		panic(fmt.Sprintf("Error creating dir if missing: %s", err))
	}
	dbOpts.ErrorIfMissing = !dirEmpty
	db, err := leveldb.OpenFile(f.filePath, dbOpts)
	if err != nil && err == syscall.EAGAIN {
		return errors.Errorf("lock is already acquired on file %s", f.filePath)
	}
	if err != nil {
		panic(fmt.Sprintf("Error acquiring lock on file %s: %s", f.filePath, err))
	}

	// only mutate the lock db reference AFTER validating that the lock was held.
	f.db = db

	return nil
}

// Determine if the lock is currently held open.
func (f *FileLock) IsLocked() bool {
	return f.db != nil
}

// Unlock releases a previously acquired lock. We achieve this by closing
// the previously opened db. FileUnlock can be called multiple times.
func (f *FileLock) Unlock() {
	if f.db == nil {
		return

View on GitHub (pinned to 2736b63f8f)

Solutions

Check filesystem permissions and ownership on the lock file's directory and ensure the process user can read/write it
Inspect/clear corrupted LevelDB lock artifacts (back up first), or point the lock path at a clean directory
Check disk space (df -h) and I/O health in logs (dmesg) for underlying OpenFile failures
Ensure only one process uses the path and the filesystem supports proper locking (avoid NFS for lock/ledger dirs)

Example fix

// before (panics on unexpected open error)
if err != nil {
	panic(fmt.Sprintf("Error acquiring lock on file %s: %s", f.filePath, err))
}
// after (ensure a clean, writable directory before Lock)
if err := os.MkdirAll(filepath.Dir(lockPath), 0755); err != nil { return err }
if err := os.Chmod(filepath.Dir(lockPath), 0755); err != nil { return err }
lock.Lock()

Defensive patterns

Strategy: try-catch

Validate before calling

func canAcquireLock(dir string) error {
	fi, err := os.Stat(dir)
	if err != nil { return err }
	if !fi.IsDir() { return fmt.Errorf("%s is not a directory", dir) }
	tmp := filepath.Join(dir, ".locktest")
	f, err := os.Create(tmp); if err != nil { return err }
	f.Close(); os.Remove(tmp)
	return nil
}

Type guard

func isOpenError(err error) bool { return err != nil && errors.Is(err, syscall.EAGAIN) }

Try / catch

func() {
	defer func() {
		if r := recover(); r != nil {
			log.Errorf("lock acquisition panicked: %v", r)
			// check permissions/disk/corruption, then retry on a clean path
		}
	}()
	lock.Lock()
}()

Prevention

Ensure the lock directory exists and is writable by the process user before locking
Never place ledger/lock directories on NFS; use local filesystems
Check disk space and clean unclean-shutdown artifacts before startup
Run only one process per lock path

When it happens

Trigger: Calling FileLock.Lock() on a path whose underlying leveldb.OpenFile fails with something other than EAGAIN — e.g. corrupted lock DB files, permission denied on the directory, disk I/O error, or the directory being unusable. As seen in TestFileLock/TestFileLockLockUnlockLock, normal usage opens the file in an existing empty dir; failure of OpenFile with an unexpected error triggers the panic.

Common situations: Running a peer/node whose data directory has wrong permissions or is read-only; leftover/corrupted LOCK files from an unclean shutdown; disk full; the lock path pointing at a file where a directory is expected; running two processes on NFS where flock semantics differ and OpenFile fails unexpectedly.

Related errors

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/227b45d4b0cecd91. Report an issue: GitHub.