What causes "PgVectorBackend has been closed" in MemPalace/mempalace?

Order your shutdown: cancel/finish outstanding work before calling backend.close().

MemPalace/mempalace · error · BackendClosedError

PgVectorBackend has been closed

Error message

PgVectorBackend has been closed

What it means

PgVectorBackend caches one _PgVectorClient per config in a registry guarded by a lock. close() sets _closed and clears the registry; _client() re-checks _closed under the same lock and raises BackendClosedError on any subsequent use. This prevents use-after-close races where a client is created concurrently with shutdown (the same pattern as SQLiteExactBackend._connect).

Source

Thrown at mempalace/backends/pgvector.py:1490

    def _embedder_sidecar_path(palace: PalaceRef) -> Optional[str]:
        if not palace.local_path:
            return None
        return os.path.join(palace.local_path, EMBEDDER_SIDECAR_FILENAME)

    def _get_embedder_identity(self, palace: PalaceRef, collection_name: str):
        return read_embedder_sidecar(self._embedder_sidecar_path(palace), collection_name)

    def _set_embedder_identity(self, palace: PalaceRef, collection_name: str, identity) -> None:
        write_embedder_sidecar(self._embedder_sidecar_path(palace), collection_name, identity)

    # ------------------------------------------------------------------
    def _client(self, config: _PgVectorConfig) -> _PgVectorClient:
        with self._lock:
            # Checked under the lock so a client cannot be created and stored
            # concurrently with close() clearing the registry (mirrors
            # SQLiteExactBackend._connect).
            if self._closed:
                raise BackendClosedError("PgVectorBackend has been closed")
            client = self._clients.get(config)
            if client is None:
                client = _PgVectorClient(config)
                self._clients[config] = client
            return client

    def get_collection(self, *args, **kwargs) -> PgVectorCollection:
        palace, collection_name, create, options = self._normalize_args(args, kwargs)
        config = _PgVectorConfig.from_options(options)
        if palace.namespace and palace.namespace != config.namespace:
            config = _PgVectorConfig(dsn=config.dsn, namespace=palace.namespace)
        client = self._client(config)
        if palace.local_path:
            marker_path = self._marker_path(palace.local_path)
            if os.path.isfile(marker_path):
                self._validate_marker_target(palace, config)
            elif not create:
                raise PalaceNotFoundError(marker_path)

View on GitHub (pinned to 06cb6987f0)

Solutions

Order your shutdown: cancel/finish outstanding work before calling backend.close().
If the backend must keep serving, create a fresh PgVectorBackend instance instead of reusing the closed one.
In tests, use a fresh backend per test (fixture) rather than a shared closed instance.

Example fix

# before
backend.close()
col = backend.get_collection(palace, "notes")  # BackendClosedError

# after
# finish work first, then close
col.query(query_embeddings=[vec])
backend.close()

Defensive patterns

Strategy: try-catch

Validate before calling

if backend._closed:  # internal flag; prefer tracking closes yourself
    raise RuntimeError("backend already closed")
col = backend.get_collection(palace, "notes")

Type guard

def is_open(backend) -> bool:
    return not getattr(backend, "_closed", False)

Try / catch

from mempalace.backends.base import BackendClosedError
try:
    col = backend.get_collection(palace, "notes")
except BackendClosedError:
    backend = PgVectorBackend()
    col = backend.get_collection(palace, "notes")

Prevention

Own an explicit shutdown order: stop workers, await pending tasks, then close().
Register close() in atexit/atypical shutdown only after background work is joined.
Use one backend instance per process lifetime and never reuse after close.

When it happens

Trigger: Calling backend.get_collection(...) (or any path reaching _client()) after backend.close(); a background thread opening a collection while the main thread shuts the backend down at interpreter exit.

Common situations: Module-teardown hooks closing the backend before pending async tasks run; a long-lived palace object used across test cases where a fixture closes the backend; concurrent request handlers racing shutdown.

Related errors

AI-assisted analysis of MemPalace/mempalace@06cb6987f0 (2026-08-15). Data as JSON: /api/errors/97455e73b870d31d. Report an issue: GitHub.