multica-ai/multica · error

clear empty hermes store %s before publishing: %w

Error message

clear empty hermes store %s before publishing: %w

What it means

Returned by promoteHermesStoreStaging when os.Remove of the (expected-empty) store dir fails with a non-NotExist error AND hermesStorePopulated cannot confirm another task already published. The empty-dir-only remove is the guard that makes publishing safe: only after it succeeds can staging be renamed into place, and a remove failure that is not explained by a concurrent winner is fatal.

Source

Thrown at server/internal/daemon/execenv/hermes_memory.go:299

// failure. The caller deletes the source directory whenever this returns
// (false, nil), so a permission error, a read-only filesystem or a Windows
// sharing violation must fail closed instead of passing for "someone else
// published".
//
// published answers "did a competitor already publish real state here?" and is
// the caller's to define, because the two stores disagree about what an
// occupied directory looks like: any entry at all means memory, while a session
// store can hold a zero-length database or orphan journal sidecars that carry no
// transcript. Sharing one definition let a session migration read the store as
// empty, copy into staging, then read it as occupied at publish time and report
// a lost race — after which the caller deleted a source database that had never
// been carried over.
func promoteHermesStoreStaging(staging, storeDir string, published func(string) bool) (bool, error) {
	if err := os.Remove(storeDir); err != nil && !os.IsNotExist(err) {
		if published(storeDir) {
			return false, nil // another task published first
		}
		return false, fmt.Errorf("clear empty hermes store %s before publishing: %w", storeDir, err)
	}
	if err := os.Rename(staging, storeDir); err != nil {
		if published(storeDir) {
			return false, nil // lost a narrow race between the remove and the rename
		}
		return false, fmt.Errorf("publish hermes store %s: %w", storeDir, err)
	}
	return true, nil
}

// hermesStorePopulated reports whether storeDir is *confirmed* to hold
// entries. Anything it cannot confirm — an unreadable directory, a path that is
// not a directory — is false, so a caller asking "did another task publish?"
// treats uncertainty as no.
func hermesStorePopulated(storeDir string) bool {
	entries, err := os.ReadDir(storeDir)
	return err == nil && len(entries) > 0
}

View on GitHub (pinned to 2c0912b6ec)

Solutions

  1. Stop concurrent tasks/daemons for the agent and retry — the lost-race path already returns cleanly, so an error means a genuine remove failure.
  2. Fix permissions on the store dir so both the remove and the populated-check can run (0700, daemon-owned).
  3. On Windows, ensure no process (antivirus, another daemon) holds a handle inside the store during upgrade.
Defensive patterns

Strategy: retry

Validate before calling

// Pre-publish check used by the guard itself:
func hermesStorePopulated(dir string) bool {
    ents, err := os.ReadDir(dir)
    return err == nil && len(ents) > 0 // unreadable == NOT confirmed, fails closed
}

Type guard

func isLostRace(err error) bool {
    // promoteHermesStoreStaging already swallows true lost races;
    // reaching the caller means the remove failed for a real reason.
    return false
}

Try / catch

promoted, err := promoteHermesStoreStaging(staging, store, hermesStorePopulated)
if err != nil && strings.Contains(err.Error(), "clear empty hermes store") {
    // stop sibling tasks, then a single retry is safe: staging still exists
    promoted, err = promoteHermesStoreStaging(staging, store, hermesStorePopulated)
}

Prevention

When it happens

Trigger: os.Remove(storeDir) fails because the dir is non-empty and unreadable (so the populated-check cannot confirm entries), the dir is busy (EBUSY, Windows handle open in it), or permissions deny removal — and no concurrent winner explains it.

Common situations: Two concurrent tasks for one agent racing the migration (usually resolved as a lost race, not an error); a daemon restart leaving an open handle inside the store on Windows; a store dir whose mode bits block ReadDir so 'populated' cannot be confirmed.

Related errors

AI-assisted analysis of multica-ai/multica@2c0912b6ec (2026-08-15). Data as JSON: /api/errors/0681fb513be7ec34. Report an issue: GitHub.