CacheStore cannot be used on the worker node! queue_clear wa

Error message

CacheStore cannot be used on the worker node! queue_clear was used.

What it means

A hard guard panic inside the RocksDB-based CacheStore on a Cube Store worker node: the queue_* methods of the QueueStore trait are intentionally unimplemented there because queue state must live on the router node. Seeing this panic means queue_clear was invoked against a store instance that is only meant to serve cached query results locally on a worker.

Source

Thrown at rust/cubestore/cubestore/src/cachestore/cache_rocksstore.rs:2007

    }

    async fn queue_results_multi_delete(&self, _ids: Vec<u64>) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_results_multi_delete was used.")
    }

    async fn queue_add(&self, _payload: QueueAddPayload) -> Result<QueueAddResponse, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_add was used.")
    }

    async fn queue_add_and_retrieve(
        &self,
        _payload: QueueAddAndRetrievePayload,
    ) -> Result<QueueAddAndRetrieveResponse, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_add_and_retrieve was used.")
    }

    async fn queue_clear(&self) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_clear was used.")
    }

    async fn queue_to_cancel(
        &self,
        _prefix: String,
        _orphaned_timeout: Option<u32>,
        _heartbeat_timeout: Option<u32>,
    ) -> Result<Vec<IdRow<QueueItem>>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_to_cancel was used.")
    }

    async fn queue_list(
        &self,
        _prefix: String,
        _status_filter: Option<QueueItemStatus>,
        _priority_sort: bool,
        _with_payload: bool,
        _caller_process_id: Option<String>,

View on GitHub (pinned to 7d981676b3)

Solutions

Issue queue_clear only against the metastore node.
Verify which node currently holds metastore role before running destructive queue maintenance.
Update ops scripts/dashboards with the metastore address.
Restart the cluster with the correct role assignment if the metastore moved.

Example fix

// before: admin tool pointed at worker
adminUrl = 'http://worker-1:3030/queue/clear';
// after
adminUrl = 'http://metastore-1:3030/queue/clear';

Defensive patterns

Strategy: validation

Validate before calling

if !is_metastore(node) {
    return Err(CubeError::internal("queue_clear requires the metastore node"));
}
node.queueClear().await?;

Type guard

fn is_metastore(node: &CubeStoreNode) -> bool {
    matches!(node.role(), NodeRole::Metastore)
}

Prevention

Double-check destructive queue maintenance targets before executing.
Update ops runbooks with the current metastore address after failover.
Require explicit confirmation of target node role in admin tools.

When it happens

Trigger: Invoking queue_clear on a node whose backing store is CacheStore, e.g. an admin/maintenance endpoint bound to worker replicas.

Common situations: Ops scripts clearing stuck queues that hit the wrong node; admin dashboards connecting to a worker address; stale config after reassigning which node is the metastore.

Related errors

AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/73a2339f280c4e4c. Report an issue: GitHub.