CacheStore cannot be used on the worker node! queue_cancel w

Error message

CacheStore cannot be used on the worker node! queue_cancel was used.

What it means

A hard guard panic inside the RocksDB-based CacheStore on a Cube Store worker node: queue_cancel was invoked on a store whose queue methods are intentionally unimplemented, as queue lifecycle is owned by the router node. Encountering it means a worker node is trying to cancel a queue item, which the Cube Store topology does not support.

Source

Thrown at rust/cubestore/cubestore/src/cachestore/cache_rocksstore.rs:2035

    }

    async fn queue_list(
        &self,
        _prefix: String,
        _status_filter: Option<QueueItemStatus>,
        _priority_sort: bool,
        _with_payload: bool,
        _caller_process_id: Option<String>,
    ) -> Result<Vec<QueueListItem>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_list was used.")
    }

    async fn queue_get(&self, _key: QueueKey) -> Result<Option<QueueGetResponse>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_get was used.")
    }

    async fn queue_cancel(&self, _key: QueueKey) -> Result<Option<QueueCancelResponse>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_cancel was used.")
    }

    async fn queue_heartbeat(&self, _key: QueueKey) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_heartbeat_by_id was used.")
    }

    async fn queue_retrieve_by_path(
        &self,
        _path: String,
        _allow_concurrency: u32,
        _caller_process_id: Option<String>,
    ) -> Result<QueueRetrieveResponse, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_retrieve_by_path was used.")
    }

    async fn queue_ack(&self, _key: QueueKey, _result: Option<String>) -> Result<bool, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_ack was used.")
    }

View on GitHub (pinned to 7d981676b3)

Solutions

Route cancellation requests to the metastore node.
Gate admin cancel actions in tooling to use the metastore endpoint only.
Fix any proxy that forwards mutating queue APIs to workers.
Verify the metastore address in your Cube driver / cubestore connection config.

Example fix

// before
await workerConn.queueCancel(key);
// after
await metastoreConn.queueCancel(key);

Defensive patterns

Strategy: validation

Validate before calling

if !is_metastore(node) {
    return Err(CubeError::internal("queue_cancel requires the metastore node"));
}
node.queueCancel(key).await?;

Type guard

fn is_metastore(node: &CubeStoreNode) -> bool {
    matches!(node.role(), NodeRole::Metastore)
}

Prevention

Restrict mutating admin APIs to the metastore endpoint.
Fix reverse proxies forwarding POST queue routes to workers.
Validate target node role in automation before cancelling jobs.

When it happens

Trigger: Requesting cancellation of a queue item (queue_cancel with QueueKey) while connected to a worker node.

Common situations: Users cancelling stuck pre-aggregation builds via a UI whose backend talks to a worker; automation scripts iterating all nodes to cancel jobs.

Related errors

AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/b26a67a65c43e380. Report an issue: GitHub.