{"record":{"id":"d356f47c04da8ffa","repo":"apache/druid","slug":"query-needs-s-merge-buffers-but-only-s-merge-bu","errorCode":null,"errorMessage":"Query needs %s merge buffers, but only %s merge buffers were configured","messagePattern":"Query needs (.+?) merge buffers, but only (.+?) merge buffers were configured","errorType":"exception","errorClass":"ResourceLimitExceededException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/query/groupby/GroupingEngine.java","lineNumber":174,"sourceCode":"      BlockingPool<ByteBuffer> mergeBufferPool,\n      boolean usesGroupByMergingQueryRunner,\n      GroupByQueryConfig groupByQueryConfig\n  )\n  {\n\n    final int requiredMergeBufferNumForToolchestMerge =\n        GroupByQueryResources.countRequiredMergeBufferNumForToolchestMerge(query);\n\n    final int requiredMergeBufferNumForMergingQueryRunner =\n        usesGroupByMergingQueryRunner\n        ? GroupByQueryResources.countRequiredMergeBufferNumForMergingQueryRunner(groupByQueryConfig, query)\n        : 0;\n\n    final int requiredMergeBufferNum =\n        requiredMergeBufferNumForToolchestMerge + requiredMergeBufferNumForMergingQueryRunner;\n\n    if (requiredMergeBufferNum > mergeBufferPool.maxSize()) {\n      throw new ResourceLimitExceededException(\n          \"Query needs \" + requiredMergeBufferNum + \" merge buffers, but only \"\n          + mergeBufferPool.maxSize() + \" merge buffers were configured\"\n      );\n    } else if (requiredMergeBufferNum == 0) {\n      return new GroupByQueryResources(null, null);\n    } else {\n      final List<ReferenceCountingResourceHolder<ByteBuffer>> mergeBufferHolders;\n      final QueryContext context = query.context();\n      if (context.hasTimeout()) {\n        mergeBufferHolders = mergeBufferPool.takeBatch(requiredMergeBufferNum, context.getTimeout());\n      } else {\n        mergeBufferHolders = mergeBufferPool.takeBatch(requiredMergeBufferNum);\n      }\n      if (mergeBufferHolders.isEmpty()) {\n        throw QueryCapacityExceededException.withErrorMessageAndResolvedHost(\n            StringUtils.format(\n                \"Cannot acquire %s merge buffers. Try again after current running queries are finished.\",\n                requiredMergeBufferNum","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/query/groupby/GroupingEngine.java#L156-L192","documentation":"Before running a group-by query, GroupingEngine.prepareResource computes the number of merge buffers needed for toolchest merge plus the merging query runner, and throws ResourceLimitExceededException when this exceeds the configured merge buffer pool size (druid.processing.numMergeBuffers).","triggerScenarios":"Submitting a group-by query requiring more merge buffers than available: multi-level groupBy with multiple nested subqueries, high druid.query.groupBy.maxMergingDictionarySize configurations aside — specifically nested subquery depth or broker merge requirements pushing requiredMergeBufferNum above pool maxSize.","commonSituations":"Cluster configured with default numMergeBuffers while running deeply nested groupBy subqueries; many concurrent groupBy queries exhausting (or exceeding sizing for) merge buffers; v2 engine setups where subquery nesting multiplies buffer requirements.","solutions":["Increase druid.processing.numMergeBuffers on historical/broker nodes","Reduce subquery nesting depth or rewrite the query to fewer levels","Lower concurrency (druid.processing.numThreads) so fewer queries contend, or route heavy nested queries to dedicated tier"],"exampleFix":"// before (runtime.properties)\ndruid.processing.numMergeBuffers=2\n// after\ndruid.processing.numMergeBuffers=4","handlingStrategy":"try-catch","validationCode":"if (requiredMergeBuffers(query) > mergeBufferPool.maxSize()) { throw new ResourceLimitExceededException(\"Reduce nesting or increase druid.processing.numMergeBuffers\"); }","typeGuard":"null","tryCatchPattern":"try { groupingEngine.prepareResource(query, ...); } catch (ResourceLimitExceededException e) { /* surface a 429-style error or retry on a larger cluster */ }","preventionTips":["Size druid.processing.numMergeBuffers >= subquery nesting depth + 1 on historicals and brokers","Keep groupBy subquery nesting shallow","Monitor merge-buffer pool usage and alert before exhaustion"],"tags":["groupby","merge-buffers","resource-limit"],"backgroundTag":"resource-limit-exceeded","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}