gastownhall/beads · error

remove dead spawn marker

Error message

remove dead spawn marker: %w

What it means

This error wraps a failure to remove a spawn marker file that belongs to a process already confirmed dead. The library throws it because a dead marker left on disk would keep triggering spurious spawn-owner checks; if removal fails for a reason other than the file not existing, the failure is surfaced instead of silently ignored.

Solutions

1. Fix permissions on the database root directory so the current user can delete files in it (chown/chmod)
2. Remove the marker file manually: rm <dbdir>/.bd-spawn-marker (actual spawnMarkerFileName)
3. Check the wrapped cause (%w); if the marker path is a directory, remove the directory instead
4. If the marker is already gone, the matching errors.Is(err, fs.ErrNotExist) branch means no action is needed — rerun the operation

Example fix

// before
if err := os.Remove(filepath.Join(rootDir, spawnMarkerFileName)); err != nil && !errors.Is(err, fs.ErrNotExist) {
    return false, fmt.Errorf("remove dead spawn marker: %w", err)
}
// after
markerPath := filepath.Join(rootDir, spawnMarkerFileName)
if err := os.Remove(markerPath); err != nil && !errors.Is(err, fs.ErrNotExist) {
    _ = os.Chmod(rootDir, 0o755)
    if err2 := os.Remove(markerPath); err2 != nil && !errors.Is(err2, fs.ErrNotExist) {
        return false, fmt.Errorf("remove dead spawn marker: %w", err2)
    }
}

func canRemoveMarker(rootDir string) error {
    info, err := os.Stat(rootDir)
    if err != nil {
        return err
    }
    if !info.IsDir() {
        return fmt.Errorf("%s is not a directory", rootDir)
    }
    f, err := os.CreateTemp(rootDir, ".wtest")
    if err != nil {
        return err
    }
    _ = f.Close()
    return os.Remove(f.Name())
}

if err := verifySpawnOwner(rootDir); err != nil {
    if strings.Contains(err.Error(), "remove dead spawn marker") {
        fmt.Fprintf(os.Stderr, "fix permissions on %s, then retry: %v\n", rootDir, err)
    }
}

When it happens

Trigger: After procid.Verify reports the recorded PID is not running, calling os.Remove on <rootDir>/<spawnMarkerFileName> fails with an error that is not fs.ErrNotExist — e.g. EACCES/EPERM on the directory or file, or the path is a directory rather than a file.

Common situations: Read-only or root-owned database directory after restoring a backup with wrong ownership, stale markers under a directory owned by a different user/container UID mismatch, or a directory named like the marker file blocking removal.

Related errors

• artifact(s) could not be removed
• failed to remove
• failed to walk directory tree
• remove expired quarantine
• atomicfile: chmod

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/1dc6ac3272226035. Report an issue: GitHub.

Appendix: source

Thrown at internal/storage/dbproxy/proxy/endpoint.go:1044

}

func inspectSpawnMarkerLocked(rootDir string) (bool, error) {
	marker, err := readSpawnMarker(rootDir)
	if err != nil {
		return false, fmt.Errorf("inspect proxy spawn marker: %w", err)
	}
	if marker == nil {
		return false, nil
	}
	matched, err := procid.Verify(marker.PID, procid.Token(marker.Birth))
	if err != nil {
		return false, fmt.Errorf("verify proxy spawn owner pid %d: %w", marker.PID, err)
	}
	if matched {
		return true, nil
	}
	if err := os.Remove(filepath.Join(rootDir, spawnMarkerFileName)); err != nil && !errors.Is(err, fs.ErrNotExist) {
		return false, fmt.Errorf("remove dead spawn marker: %w", err)
	}
	return false, nil
}

func clearSpawnMarkerAfterLock(rootDir string) error {
	err := os.Remove(filepath.Join(rootDir, spawnMarkerFileName))
	if err != nil && !errors.Is(err, fs.ErrNotExist) {
		return err
	}
	return nil
}

func clearOwnSpawnMarker(rootDir string, own spawnMarker) error {
	deadline := time.Now().Add(shutdownConfirmDeadline)
	for {
		marker, err := readSpawnMarker(rootDir)
		if err != nil || marker == nil {
			return err

View on GitHub (pinned to 71377f2769)