What causes "stat state marker: %w" in multica-ai/multica?

Verify the overlay dir is traversable/readable by the daemon user: `stat ` and `ls -la `.

stat state marker: %w

Error message

stat state marker: %w

What it means

prepareHermesTaskLocalState could not os.Lstat the task-local-state marker, and the error was not 'does not exist'. Unlike a missing marker (which triggers the legacy migration), an inconclusive stat means the code cannot tell whether state.db is safe to reuse, so it fails closed.

Source

Thrown at server/internal/daemon/execenv/hermes_home.go:582

	}
	return nil
}

// prepareHermesTaskLocalState migrates an overlay built by an older daemon away
// from the shared Hermes SQLite session store. Without the marker, state.db and
// its sidecars may be symlinks or independently copied files; neither is safe to
// reuse as task-local state. Remove only those entries inside the generated
// overlay, then record the migration atomically. Hermes lazily creates a fresh
// database, and later prepares preserve it because the marker is present.
func prepareHermesTaskLocalState(hermesHome string) error {
	marker := filepath.Join(hermesHome, hermesTaskLocalStateMarker)
	if fi, err := os.Lstat(marker); err == nil {
		if !fi.Mode().IsRegular() {
			return fmt.Errorf("state marker is not a regular file: %s", marker)
		}
		return nil
	} else if !os.IsNotExist(err) {
		return fmt.Errorf("stat state marker: %w", err)
	}

	entries, err := os.ReadDir(hermesHome)
	if err != nil {
		return fmt.Errorf("read overlay home: %w", err)
	}
	for _, entry := range entries {
		if !isHermesTaskLocalStateEntry(entry.Name()) {
			continue
		}
		path := filepath.Join(hermesHome, entry.Name())
		if err := os.RemoveAll(path); err != nil {
			return fmt.Errorf("remove legacy task state %s: %w", path, err)
		}
	}
	return writeFileAtomic(marker, []byte("task-local Hermes state\n"), 0o600)
}

View on GitHub (pinned to 2c0912b6ec)

Solutions

Verify the overlay dir is traversable/readable by the daemon user: `stat <hermesHome>` and `ls -la <hermesHome>`.
Fix permissions or ownership of the whole overlay dir, then retry.
If the volume is erroring, address the storage problem (remount, replace disk) — this error will recur on the same mount otherwise.
As a reset, delete the regenerable overlay dir.

Defensive patterns

Strategy: try-catch

Validate before calling

if fi, err := os.Stat(filepath.Dir(marker)); err != nil {
	return fmt.Errorf("overlay dir unavailable: %w", err)
} else if !fi.IsDir() {
	return fmt.Errorf("overlay path is not a directory: %s", filepath.Dir(marker))
}

Try / catch

if err := prepareHermesTaskLocalState(hermesHome); err != nil {
	var pe *os.PathError
	if errors.As(err, &pe) && errors.Is(pe.Err, syscall.EACCES) {
		log.Printf("fix permissions on %s and retry", filepath.Dir(pe.Path))
	}
	return err
}

Prevention

Keep overlay dir ownership/permissions stable across daemon restarts.
Watch for storage-level errors on the env volume; stat failures are often the first symptom.

When it happens

Trigger: Permission denied on the overlay dir (cannot stat entries inside it), I/O error from the underlying volume, or path-component issues (a non-directory component in the marker path).

Common situations: Overlay dir with restrictive ACLs after a UID change; failing disk or network volume hosting the env RootDir; security software intercepting stat calls.

Related errors

AI-assisted analysis of multica-ai/multica@2c0912b6ec (2026-08-15). Data as JSON: /api/errors/6f96009de97fb581. Report an issue: GitHub.