{"record":{"id":"aa052de0d287a69e","repo":"pingcap/tidb","slug":"out-of-global-memory-limit","errorCode":null,"errorMessage":"Out Of Global Memory Limit!","messagePattern":"Out Of Global Memory Limit!","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"pkg/executor/select.go","lineNumber":182,"sourceCode":"}\n\n// Action panics when storage usage exceeds storage quota.\nfunc (a *globalPanicOnExceed) Action(t *memory.Tracker) {\n\ta.mutex.Lock()\n\tdefer a.mutex.Unlock()\n\tmsg := \"\"\n\tswitch t.Label() {\n\tcase memory.LabelForGlobalStorage:\n\t\tmsg = globalPanicStorageExceed\n\tcase memory.LabelForGlobalMemory:\n\t\tmsg = globalPanicMemoryExceed\n\tcase memory.LabelForGlobalAnalyzeMemory:\n\t\tmsg = globalPanicAnalyzeMemoryExceed\n\tdefault:\n\t\tmsg = \"Out of Unknown Resource Quota!\"\n\t}\n\t// TODO(hawkingrei): should return error instead.\n\tpanic(msg)\n}\n\n// GetPriority get the priority of the Action\nfunc (*globalPanicOnExceed) GetPriority() int64 {\n\treturn memory.DefPanicPriority\n}\n\n// SelectLockExec represents a select lock executor.\n// It is built from the \"SELECT .. FOR UPDATE\" or the \"SELECT .. LOCK IN SHARE MODE\" statement.\n// For \"SELECT .. FOR UPDATE\" statement, it locks every row key from source Executor.\n// After the execution, the keys are buffered in transaction, and will be sent to KV\n// when doing commit. If there is any key already locked by another transaction,\n// the transaction will rollback and retry.\ntype SelectLockExec struct {\n\texec.BaseExecutor\n\n\tLock *ast.SelectLockInfo\n\tkeys []kv.Key","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/pingcap/tidb/blob/d01f9615c13e02dfa75922745640dec9d81b802e/pkg/executor/select.go#L164-L200","documentation":"Fired by the globalPanicOnExceed memory action in TiDB's executor (select.go). When the tracker labeled LabelForGlobalMemory (the server-wide memory tracker) exceeds its quota, TiDB deliberately panics the owning session/goroutine with 'Out Of Global Memory Limit!' to keep the process from being OOM-killed by the OS. The panic unwinds the query and the error surfaces to the client.","triggerScenarios":"Any query or set of concurrent queries whose combined tracked memory exceeds the global memory limit (memory_usage_limit / tidb_server_memory_limit and friends): huge joins/sorts/aggregations with no spill, wide table scans, many concurrent analytical queries, or ANALYZE under the global tracker.","commonSituations":"Server sized with memory_usage_limit close to physical RAM while workloads spike; spill-to-disk disabled for the operators that would otherwise spill (tidb_enable_spilled_result / spill settings for join/agg/sort); a migration from an older TiDB where only per-query mem-quota-query applied; slow concurrent stats or large DML ballooning global tracked usage.","solutions":["Raise the global memory limit (or add machine memory) via memory_usage_limit / tidb_server_memory_limit so the workload fits.","Reduce peak per-query memory: add LIMIT, filter earlier, split the query, or create suitable indexes so large hash joins/sorts are avoided.","Enable/verify spill for the offending operators (e.g. tidb_enable_pseudo_for_outbound_stats off; tidb_mem_quota_query plus spill for sort/join/agg: tidb_enable_spilled_result_protection, tidb_prefer_flashback... - specifically enable disk spill for Sort/Join/Agg via their tidb_* spill variables) so they go to temp storage instead of RAM.","Kill or throttle the memory-heavy sessions (max_connections, resource control with RUNAWAY queries) so global usage stays under quota.","If usage is dominated by a single system operation (e.g. ANALYZE or GC), schedule it off-peak."],"exampleFix":"-- before: single giant aggregation blows the global limit\nSELECT c_custkey, SUM(o_totalprice) FROM orders GROUP BY c_custkey; -- panics\n\n-- after: bound the query and let TiDB spill\nSET GLOBAL tidb_mem_quota_query = 4294967296; -- per-query bound\nSET SESSION tidb_enable_spilled_result = ON;  -- allow spill for sort/agg\nSELECT c_custkey, SUM(o_totalprice) FROM orders WHERE o_orderdate >= '2024-01-01' GROUP BY c_custkey;","handlingStrategy":"validation","validationCode":"-- before running big queries, check current global memory headroom\nSELECT MEMORY_OFFSET, MEMORY_USAGE, MEMORY_MAX,\n       ROUND((MEMORY_USAGE / MEMORY_MAX) * 100, 1) AS used_pct\nFROM information_schema.INSTANCE_MEMORY_USAGE\nORDER BY MEMORY_USAGE DESC;","typeGuard":null,"tryCatchPattern":"-- catch at the client: the panic surfaces as a query error, so a retry wrapper\ngives the query a chance once memory frees up (do NOT blind-retry hot loops)\nEXECUTE stmt; -- on error 1105 'Out Of Global Memory Limit!': shed load, then retry once","preventionTips":["Set memory_usage_limit / tidb_server_memory_limit with real headroom for concurrency spikes.","Keep spill variables enabled for sort/join/agg so queries fall back to disk instead of RAM.","Use resource control (RUNAWAY watch) to auto-kill sessions that balloon.","Monitor MEMORY_USAGE / INSTANCE_MEMORY_USAGE and alert before the quota, not at it."],"tags":["tidb","memory","quota","oom","panic"],"backgroundTag":null,"analyzedSha":"d01f9615c13e02dfa75922745640dec9d81b802e","analyzedAt":"2026-08-15T12:19:33.324Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}