{"record":{"id":"d3ba39843b127f0d","repo":"apache/druid","slug":"not-enough-space-for-aggregators-needed-d-byt","errorCode":null,"errorMessage":"Not enough space for aggregators, needed [%,d] bytes but have only [%,d].","messagePattern":"Not enough space for aggregators, needed \\[%,d\\] bytes but have only \\[%,d\\]\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/query/timeseries/TimeseriesQueryEngine.java","lineNumber":172,"sourceCode":"          gran,\n          queryInterval\n      );\n\n      if (granularizer == null) {\n        return Sequences.withBaggage(Sequences.empty(), closer);\n      }\n\n      final VectorColumnSelectorFactory columnSelectorFactory = cursor.getColumnSelectorFactory();\n      final AggregatorAdapters aggregators =\n          AggregatorAdapters.factorizeVector(columnSelectorFactory, query.getAggregatorSpecs());\n      closer.register(aggregators::reset);\n\n      final ResourceHolder<ByteBuffer> bufferHolder = closer.register(bufferPool.take());\n\n      final ByteBuffer buffer = bufferHolder.get();\n\n      if (aggregators.spaceNeeded() > buffer.remaining()) {\n        throw new ISE(\n            \"Not enough space for aggregators, needed [%,d] bytes but have only [%,d].\",\n            aggregators.spaceNeeded(),\n            buffer.remaining()\n        );\n      }\n\n      return Sequences.withBaggage(\n          Sequences\n              .simple(granularizer.getBucketIterable())\n              .map(\n                  bucketInterval -> {\n                    // Whether or not the current bucket is empty\n                    boolean emptyBucket = true;\n\n                    while (!cursor.isDone()) {\n                      granularizer.setCurrentOffsets(bucketInterval);\n\n                      if (granularizer.getEndOffset() > granularizer.getStartOffset()) {","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/query/timeseries/TimeseriesQueryEngine.java#L154-L190","documentation":"In the vectorized timeseries path, aggregator intermediates are stored in a single pooled ByteBuffer. Before processing, Druid checks the aggregators' spaceNeeded() against the buffer's remaining bytes and aborts with ISE if the buffer from the pool is too small to hold the aggregator state.","triggerScenarios":"Running a vectorized timeseries query whose aggregators require more intermediate buffer space than the configured aggregation buffer pool provides (small druid.processing.buffer.sizeBytes plus many/large aggregators).","commonSituations":"Queries with many aggregators or large sketch/HLL aggregators on nodes with small processing buffers, misconfigured druid.processing.buffer.sizeBytes, buffer pool contention after config change without restart.","solutions":["Increase druid.processing.buffer.sizeBytes so it exceeds the aggregators' spaceNeeded()","Reduce the number/size of aggregators in the query or split the query","Disable vectorization for the affected query (druid.query.vectorize=false) to fall back to the non-vectorized engine","Verify the processing buffer pool is actually sized after restarts (check startup logs for buffer pool warnings)"],"exampleFix":"// before (runtime.properties)\ndruid.processing.buffer.sizeBytes=50000000\n// after\ndruid.processing.buffer.sizeBytes=1073741824","handlingStrategy":"validation","validationCode":"// before running vectorized: check config in runtime.properties\n// assert druid.processing.buffer.sizeBytes > sum of aggregator spaceNeeded();","typeGuard":"static boolean bufferSufficient(BufferAggregator[] aggs, ByteBuffer buf) {\n  long need = 0; /* sum agg space needs */ return need <= buf.remaining();\n}","tryCatchPattern":"try { return vectorizedProcess(...); }\ncatch (IllegalStateException e) {\n  if (e.getMessage().contains(\"Not enough space for aggregators\")) { return nonVectorizedProcess(...); }\n  throw e;\n}","preventionTips":["Size druid.processing.buffer.sizeBytes for worst-case aggregator counts","Avoid huge numbers of aggregators in single timeseries queries","Set druid.query.vectorize=false for sketch-heavy queries on small nodes"],"tags":["druid","vectorized-query","memory","timeseries"],"backgroundTag":"insufficient-buffer-size","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}