{"record":{"id":"d66f1fb424331d0a","repo":"MemPalace/mempalace","slug":"sqlite-exact-read-only-open-found-an-incomplete-wa","errorCode":null,"errorMessage":"sqlite_exact read-only open found an incomplete WAL sidecar set; open the palace after its writer exits cleanly or restore both the -wal and -shm files","messagePattern":"sqlite_exact read-only open found an incomplete WAL sidecar set; open the palace after its writer exits cleanly or restore both the -wal and -shm files","errorType":"exception","errorClass":"BackendError","httpStatus":null,"severity":"error","filePath":"mempalace/backends/sqlite_exact.py","lineNumber":909,"sourceCode":"        return os.path.join(palace_path, _DB_FILENAME)\n\n    @staticmethod\n    def _wal_sidecar_state(db_path: str) -> tuple[bool, bool]:\n        return (\n            os.path.isfile(f\"{db_path}-wal\"),\n            os.path.isfile(f\"{db_path}-shm\"),\n        )\n\n    @staticmethod\n    def _connect_read_only(db_path: str) -> tuple[sqlite3.Connection, bool]:\n        \"\"\"Open without creating WAL files while preserving an active WAL.\n\n        Returns ``(connection, immutable)``. ``immutable`` is True when the\n        database was clean (no WAL) and was opened with ``immutable=1``.\n        \"\"\"\n        wal_exists, shm_exists = SQLiteExactBackend._wal_sidecar_state(db_path)\n        if wal_exists != shm_exists:\n            raise BackendError(\n                \"sqlite_exact read-only open found an incomplete WAL sidecar set; \"\n                \"open the palace after its writer exits cleanly or restore both \"\n                \"the -wal and -shm files\"\n            )\n\n        db_uri = Path(db_path).resolve().as_uri()\n        if wal_exists:\n            # An active writer's uncheckpointed rows live in the WAL. With both\n            # sidecars already present, mode=ro can read them without creating\n            # filesystem state, including on a read-only mount.\n            db_uri = f\"{db_uri}?mode=ro\"\n            immutable = False\n        else:\n            # A clean WAL-mode database would otherwise make SQLite create new\n            # -wal/-shm files while connecting. Immutable mode is safe here\n            # only until a writer creates sidecars this connection would miss.\n            db_uri = f\"{db_uri}?mode=ro&immutable=1\"\n            immutable = True","sourceCodeStart":891,"sourceCodeEnd":927,"githubUrl":"https://github.com/MemPalace/mempalace/blob/06cb6987f02610784fefbad4b2bd5d026d164ba6/mempalace/backends/sqlite_exact.py#L891-L927","documentation":"Raised by `SQLiteExactBackend._connect_read_only`: exactly one of the two SQLite WAL sidecar files (`-wal`, `-shm`) exists. A live WAL pair is fine (mode=ro reads it), and no WAL at all is fine (opened immutable=1), but a lone `-wal` or lone `-shm` means the database was left in an inconsistent state — a writer crashed or was killed mid-checkpoint — and a read-only open cannot safely recover it because recovery would require writing.","triggerScenarios":"Opening a palace read-only after the writing process was SIGKILLed/power-lossed between creating `-wal` and `-shm`; someone deleted or half-restored one sidecar in a backup/sync copy; copying a palace directory while a writer was active so only one sidecar got copied.","commonSituations":"Restoring a palace from a backup tool or cloud sync that skipped one sidecar; a crashed mining run followed by a read-only search; NFS/sync mounts that lose one file; snapshotting a live palace.","solutions":["Restore BOTH `-wal` and `-shm` files from the backup/copy so the set is complete, then reopen read-only.","Or restore the palace with NO sidecars at all (only the main db, after the writer exited cleanly with a checkpoint).","Or open the palace once in read-write mode from the same machine — SQLite will recover/complete the WAL set — then close cleanly; read-only opens will work afterwards.","Prevent recurrence: stop the writer cleanly (let hooks finish) before copying or backing up the palace directory."],"exampleFix":"# before: backup captured db + -wal but not -shm\n# ls palace/*.db* -> palace.db, palace.db-wal   (missing palace.db-shm)\nbackend.get_collection(palace, \"drawers\", read_only=True)  # BackendError\n\n# after: complete the set (or drop both sidecars after clean close)\ncp backup/palace.db-shm palace/palace.db-shm\nbackend.get_collection(palace, \"drawers\", read_only=True)","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef wal_sidecars_complete(db_path) -> bool:\n    wal, shm = Path(f\"{db_path}-wal\"), Path(f\"{db_path}-shm\")\n    return wal.is_file() == shm.is_file()  # both or neither","typeGuard":null,"tryCatchPattern":"try:\n    backend.get_collection(palace, \"drawers\", read_only=True)\nexcept BackendError as e:\n    if \"incomplete WAL sidecar\" in str(e):\n        # open once read-write to let SQLite recover, then retry read-only\n        backend_rw = SQLiteExactBackend()\n        backend_rw.get_collection(palace, \"drawers\", create=True)\n        backend_rw.close()\n        backend.get_collection(palace, \"drawers\", read_only=True)\n    else:\n        raise","preventionTips":["Always stop the writer cleanly (finish hooks, graceful shutdown) before copying a palace.","Backup tools must copy db, -wal, AND -shm together, or none of the sidecars.","Never snapshot a palace directory while a mining run is active."],"tags":["sqlite-exact","sqlite","wal","read-only","backup","corruption"],"backgroundTag":null,"analyzedSha":"06cb6987f02610784fefbad4b2bd5d026d164ba6","analyzedAt":"2026-08-15T03:03:36.213Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}