CacheStore cannot be used on the worker node! eviction was…

Error message

CacheStore cannot be used on the worker node! eviction was used.

What it means

CacheStore::eviction triggers cache eviction policy execution. On worker nodes it is a deliberate panic! stub (cache_rocksstore.rs:2084): eviction is a master-side responsibility and worker cache stores never run it. The panic indicates eviction was requested on the wrong node.

Solutions

Run eviction only on the master node's store
Point management commands to the master endpoint
Add an is-master assertion before invoking eviction in shared code

Example fix

// before
store.eviction().await?; // runs everywhere
// after
if is_master { master_store.eviction().await?; }

Defensive patterns

Strategy: validation

Validate before calling

if !is_master { return Err(CubeError::internal("eviction is master-only")); }
master_client.eviction().await?;

Type guard

fn supports_eviction(store: &dyn CacheStore) -> bool { node_role == NodeRole::Master }

Try / catch

std::panic::catch_unwind(AssertUnwindSafe(|| master_client.eviction())).map_err(|_| CubeError::internal("eviction called on worker node"))?;

Prevention

Run eviction loops only on the master
Gate eviction triggers with role assertions
Review topology code after failover/promotion
Monitor for panic-rate spikes on worker nodes

When it happens

Trigger: Calling CacheStore::eviction() on a worker-node CacheStore (cache_rocksstore.rs:2084), e.g. from an eviction loop or admin command routed to a worker.

Common situations: Eviction/management loops misconfigured to run cluster-wide; topology changes where a worker inherited master-era scheduler code.

Related errors

AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/0e008437478a990c. Report an issue: GitHub.

Appendix: source

Thrown at rust/cubestore/cubestore/src/cachestore/cache_rocksstore.rs:2084

        _timeout: u64,
    ) -> Result<Option<QueueResultResponse>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_result_blocking was used.")
    }

    async fn queue_merge_extra(&self, _key: QueueKey, _payload: String) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_merge_extra was used.")
    }

    async fn compaction(&self) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! compaction was used.")
    }

    async fn info(&self) -> Result<CachestoreInfo, CubeError> {
        panic!("CacheStore cannot be used on the worker node! info was used.")
    }

    async fn eviction(&self) -> Result<EvictionResult, CubeError> {
        panic!("CacheStore cannot be used on the worker node! eviction was used.")
    }

    async fn persist(&self) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! persist was used.")
    }

    async fn healthcheck(&self) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! healthcheck was used.")
    }

    async fn rocksdb_properties(&self) -> Result<Vec<RocksPropertyRow>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! rocksdb_properties was used.")
    }

    async fn wipe(&self) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! wipe was used.")
    }
}

View on GitHub (pinned to 7d981676b3)