linera-io/linera-protocol · critical

Failed to open store

Error message

Failed to open store

What it means

After connecting, `database.open_shared(&[])` opens a shared store handle (column families plus the StorageCacheConfig caches) on the just-opened RocksDB instance. `.expect("Failed to open store")` panics when that open fails: typically the on-disk state does not match what was requested (column families or options differ from how the DB was created) or the database files are corrupted.

Source

Thrown at linera-storage-service/src/server.rs:694

            };
            let storage_cache_config = StorageCacheConfig {
                max_cache_size,
                max_value_entry_size,
                max_find_keys_entry_size,
                max_find_key_values_entry_size,
                max_cache_entries,
                max_cache_value_size,
                max_cache_find_keys_size,
                max_cache_find_key_values_size,
            };
            let config = RocksDbStoreConfig {
                inner_config,
                storage_cache_config,
            };
            let database = RocksDbDatabase::maybe_create_and_connect(&config, &namespace)
                .await
                .expect("store");
            let store = database.open_shared(&[]).expect("Failed to open store");
            let store = LocalStore::RocksDb(store);
            (store, endpoint)
        }
    };
    let pending_big_puts = Arc::new(RwLock::new(BTreeMap::default()));
    let pending_big_reads = Arc::new(RwLock::new(PendingBigReads::default()));
    let store = StorageServer {
        store,
        pending_big_puts,
        pending_big_reads,
    };
    let endpoint = endpoint.parse().unwrap();
    info!("Starting linera_storage_service on endpoint={}", endpoint);
    Server::builder()
        .add_service(StorageServiceServer::new(store))
        .serve(endpoint)
        .await
        .expect("a successful running of the server");

View on GitHub (pinned to 6c226ddcb3)

Solutions

1. Re-run with the exact same --path/namespace and options (same binary version) that created the database
2. Read the RocksDB error printed just before the panic: it names the real cause (corruption vs. column family vs. I/O)
3. Restore the directory from backup, or recreate the namespace and re-import the data if corruption is confirmed
4. Shut down with SIGTERM instead of kill -9 so RocksDB keeps MANIFEST files consistent

Example fix

# before: kill -9 mid-compaction -> next startup panics "Failed to open store"
kill -9 $STORAGE_PID

# after: graceful shutdown, then restart
kill -TERM $STORAGE_PID && wait $STORAGE_PID

# sanity-check the DB directory before start
[ -f /data/linera/CURRENT ] && ls /data/linera/MANIFEST-* >/dev/null 2>&1 \
  || { echo "not a healthy RocksDB dir - do not start"; exit 1; }

When it happens

Trigger: Reopening a RocksDB namespace with different options or column families than it was created with; MANIFEST/SST corruption after a crash or unclean kill during compaction; a cache configuration value rejected when applied to an existing namespace.

Common situations: Upgrading linera-storage-service across versions that changed storage options; kill -9 of the server mid-write; partially copied or moved DB directories; mixing namespaces created by a different linera version.

Related errors

• store
• Cannot apply default storage because the feature 'rocksdb' w
• Not possible to work with RocksDB
• Failed to parse {spawn_mode_name} as a spawn_mode
• Invalid chain description data for chain {chain_id}

AI-assisted analysis of linera-io/linera-protocol@6c226ddcb3 (2026-08-22). Data as JSON: /api/errors/62ad98e693b5a348. Report an issue: GitHub.