{"record":{"id":"edd081ca1f275a45","repo":"risingwavelabs/risingwave","slug":"task-got-killed-because-compute-node-running-out-o","errorCode":null,"errorMessage":"Task got killed because compute node running out of memory","messagePattern":"Task got killed because compute node running out of memory","errorType":"exception","errorClass":"SchedulerError","httpStatus":null,"severity":"error","filePath":"src/frontend/src/scheduler/error.rs","lineNumber":39,"sourceCode":"use crate::error::{ErrorCode, RwError};\nuse crate::scheduler::plan_fragmenter::QueryId;\n\n#[derive(Error, Debug)]\npub enum SchedulerError {\n    #[error(\"Pin snapshot error: {0} fails to get epoch {1}\")]\n    PinSnapshot(QueryId, u64),\n\n    #[error(transparent)]\n    RpcError(\n        #[from]\n        #[backtrace]\n        RpcError,\n    ),\n\n    #[error(\"{0}\")]\n    TaskExecutionError(String),\n\n    #[error(\"Task got killed because compute node running out of memory\")]\n    TaskRunningOutOfMemory,\n\n    /// Used when receive cancel request for some reason, such as user cancel or timeout.\n    #[error(\"Query cancelled: {0}\")]\n    QueryCancelled(String),\n\n    #[error(\n        \"Reject query: the {0} query number reaches the limit: {1}. Use `SHOW PROCESSLIST` to check for hanging queries and cancel them if needed.\"\n    )]\n    QueryReachLimit(QueryMode, u64),\n\n    #[error(transparent)]\n    BatchError(\n        #[from]\n        #[backtrace]\n        BatchError,\n    ),\n","sourceCodeStart":21,"sourceCodeEnd":57,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/frontend/src/scheduler/error.rs#L21-L57","documentation":"SchedulerError::TaskRunningOutOfMemory indicates the batch task was killed because the compute node ran out of memory while executing it. The frontend aborts the query with this fixed message. It reflects the batch engine's memory manager deciding to kill a task under memory pressure.","triggerScenarios":"Executing a batch query whose stage memory footprint exceeds the compute node's task memory limit — large scans, huge sorts/hash aggregations without spill, very wide rows, or many concurrent queries sharing one node.","commonSituations":"Ad-hoc analytical SELECT over large materialized views; `SELECT *` of huge tables in one query; too many concurrent queries on an under-provisioned cluster; misconfigured batch memory limits.","solutions":["Add LIMIT or more selective WHERE predicates to reduce the working set of the query.","Run the heavy query when the cluster is less loaded, or increase compute node memory / scale out nodes.","Raise batch execution memory-related config (e.g. `developer.batch_*` memory limits) if limits are set too low.","Split the work into smaller queries or use a streaming (materialized view) computation instead of ad-hoc batch."],"exampleFix":"// before\nSELECT * FROM large_mv;\n// after\nSELECT * FROM large_mv WHERE event_time > now() - interval '1 day' LIMIT 1000;","handlingStrategy":"validation","validationCode":"-- bound the working set before running\nSELECT count(*) FROM large_mv WHERE <selective predicate>;","typeGuard":null,"tryCatchPattern":"// catch OOM and fall back to a narrower query\ncatch (e) { if (e.message.includes('running out of memory')) return narrowQuery(); throw e; }","preventionTips":["Always add LIMIT/predicates to ad-hoc analytical queries","Provision enough compute memory for peak concurrent load","Tune batch memory limits via session config before running heavy queries","Move heavy transformations into materialized views instead of ad-hoc batch"],"tags":["frontend","scheduler","memory","oom","batch"],"backgroundTag":"payload-too-large","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}