CacheStore cannot be used on the worker node! queue_add was

Error message

CacheStore cannot be used on the worker node! queue_add was used.

What it means

queue_add enqueues new work items and must execute on the metastore node where the authoritative queue tables live. On a worker node CacheStore stubs this method with a panic, so any enqueue attempt routed to a worker aborts the task with this message.

Source

Thrown at rust/cubestore/cubestore/src/cachestore/cache_rocksstore.rs:1996

    }

    async fn queue_all(&self, _limit: Option<usize>) -> Result<Vec<QueueAllItem>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_all was used.")
    }

    async fn queue_results_all(
        &self,
        _limit: Option<usize>,
    ) -> Result<Vec<IdRow<QueueResult>>, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_results_all was used.")
    }

    async fn queue_results_multi_delete(&self, _ids: Vec<u64>) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_results_multi_delete was used.")
    }

    async fn queue_add(&self, _payload: QueueAddPayload) -> Result<QueueAddResponse, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_add was used.")
    }

    async fn queue_add_and_retrieve(
        &self,
        _payload: QueueAddAndRetrievePayload,
    ) -> Result<QueueAddAndRetrieveResponse, CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_add_and_retrieve was used.")
    }

    async fn queue_clear(&self) -> Result<(), CubeError> {
        panic!("CacheStore cannot be used on the worker node! queue_clear was used.")
    }

    async fn queue_to_cancel(
        &self,
        _prefix: String,
        _orphaned_timeout: Option<u32>,
        _heartbeat_timeout: Option<u32>,

View on GitHub (pinned to 7d981676b3)

Solutions

Point queue producers at the metastore node's address.
Fix service discovery/selector so only the metastore pod receives queue API traffic.
If running workers+metastore separately, ensure the client uses the metastore port.
Check cube store logs to confirm which node role received the request.

Example fix

// before: K8s service selects all cubestore pods
queueTarget = 'cubestore-service:3030';
// after: dedicated service for metastore only
queueTarget = 'cubestore-metastore:3030';

Defensive patterns

Strategy: validation

Validate before calling

if !is_metastore(store.node()) {
    return Err(CubeError::internal("queue_add requires the metastore node"));
}
let resp = store.queueAdd(payload).await?;

Type guard

fn is_metastore(node: &CubeStoreNode) -> bool {
    matches!(node.role(), NodeRole::Metastore)
}

Prevention

Configure producers with the metastore address explicitly.
In Kubernetes, use a service selecting only the metastore pod.
Log the resolved queue endpoint at client startup.

When it happens

Trigger: Submitting a queue item (QueueAddPayload) — e.g. scheduling a pre-aggregation partition build — to a CubeStore worker instead of the metastore node.

Common situations: App servers configured with the address of a CubeStore worker rather than the metastore; Kubernetes service selecting all CubeStore pods instead of only the metastore pod.

Related errors

AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/5819af227d35da9f. Report an issue: GitHub.