{"record":{"id":"a8f57ecbcdb1f5c4","repo":"cube-js/cube","slug":"cachestore-cannot-be-used-on-the-worker-node-queu-a8f57e","errorCode":null,"errorMessage":"CacheStore cannot be used on the worker node! queue_results_multi_delete was used.","messagePattern":"CacheStore cannot be used on the worker node! queue_results_multi_delete was used\\.","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"rust/cubestore/cubestore/src/cachestore/cache_rocksstore.rs","lineNumber":1992,"sourceCode":"    }\n\n    async fn cache_incr(&self, _: String) -> Result<IdRow<CacheItem>, CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! cache_incr was used.\")\n    }\n\n    async fn queue_all(&self, _limit: Option<usize>) -> Result<Vec<QueueAllItem>, CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! queue_all was used.\")\n    }\n\n    async fn queue_results_all(\n        &self,\n        _limit: Option<usize>,\n    ) -> Result<Vec<IdRow<QueueResult>>, CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! queue_results_all was used.\")\n    }\n\n    async fn queue_results_multi_delete(&self, _ids: Vec<u64>) -> Result<(), CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! queue_results_multi_delete was used.\")\n    }\n\n    async fn queue_add(&self, _payload: QueueAddPayload) -> Result<QueueAddResponse, CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! queue_add was used.\")\n    }\n\n    async fn queue_add_and_retrieve(\n        &self,\n        _payload: QueueAddAndRetrievePayload,\n    ) -> Result<QueueAddAndRetrieveResponse, CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! queue_add_and_retrieve was used.\")\n    }\n\n    async fn queue_clear(&self) -> Result<(), CubeError> {\n        panic!(\"CacheStore cannot be used on the worker node! queue_clear was used.\")\n    }\n\n    async fn queue_to_cancel(","sourceCodeStart":1974,"sourceCodeEnd":2010,"githubUrl":"https://github.com/cube-js/cube/blob/7d981676b36392fec34088b9afab6bdcad40207c/rust/cubestore/cubestore/src/cachestore/cache_rocksstore.rs#L1974-L2010","documentation":"CacheStore on a CubeStore worker node cannot perform queue maintenance; queue_results_multi_delete (bulk deletion of finished queue results) is implemented only by the real metastore store and panics on the worker. This is a deliberate guard so queue state is never mutated outside the metastore.","triggerScenarios":"Calling queue_results_multi_delete (bulk cleanup of queue results by ids) against a node whose store is CacheStore, i.e. a worker node.","commonSituations":"Cleanup/retention jobs or queue-trimming code that iterates cluster nodes and hits workers; load balancers distributing metastore API requests to worker replicas.","solutions":["Send queue result deletion to the metastore node only.","Exclude worker nodes from any load balancer / client pool used for queue management APIs.","Run cleanup logic on or via the metastore leader process.","Verify node role at startup and skip queue cleanup on workers."],"exampleFix":"// before: cleanup job hits every node incl. workers\nfor node in nodes { node.queueResultsMultiDelete(ids); }\n// after: only metastore handles queue writes\nmetastoreNode.queueResultsMultiDelete(ids);","handlingStrategy":"validation","validationCode":"if !is_metastore(node) {\n    return Err(CubeError::internal(\"queue_results_multi_delete requires the metastore node\"));\n}\nnode.queueResultsMultiDelete(ids).await?;","typeGuard":"fn is_metastore(node: &CubeStoreNode) -> bool {\n    matches!(node.role(), NodeRole::Metastore)\n}","tryCatchPattern":null,"preventionTips":["Run result cleanup jobs on the metastore node only.","Exclude workers from pools used for mutating queue APIs.","Audit cron/systemd units so cleanup targets the metastore address."],"tags":["cubestore","rust","topology","queue"],"backgroundTag":"wrong-node-role","analyzedSha":"7d981676b36392fec34088b9afab6bdcad40207c","analyzedAt":"2026-09-02T03:45:10.400Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}