apache/druid · error · TooManyBucketsException

Too many buckets; maximum is [%s]

Error message

Too many buckets; maximum is [%s]

What it means

TombstoneHelper.validateAndIncrementBuckets enforces a cap on the number of tombstone buckets generated when condensing replace intervals; exceeding it throws TooManyBucketsException. The error's message 'Too many buckets; maximum is [%s]' fires when the bucket count reaches the configured maxBuckets. This guards against generating an unbounded number of tombstone segments for very fragmented timelines.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/TombstoneHelper.java:333

              SegmentDetail.none()
          ));
      for (DataSegment usedSegment : usedSegmentsInInputInterval) {
        for (Interval condensedInputInterval : condensedInputIntervals) {
          if (condensedInputInterval.overlaps(usedSegment.getInterval())) {
            retVal.add(usedSegment.getInterval());
            break;
          }
        }
      }
    }

    return JodaUtils.condenseIntervals(retVal);
  }

  private int validateAndIncrementBuckets(final int buckets, final int maxBuckets)
  {
    if (buckets >= maxBuckets) {
      throw new TooManyBucketsException(maxBuckets);
    }
    return buckets + 1;
  }

}

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Increase the bucket limit in the task's tuning/partitioning config if the fragmentation is expected
2. Pre-compact the datasource into coarser granularity segments (e.g., day) to reduce fragmentation before running the replace task
3. Narrow the compaction interval so fewer time chunks are processed per run
4. Enable auto-compaction with smaller skipOffsetFromLatest and regular runs to prevent fragmentation buildup

Example fix

// before: default maxBuckets with heavily fragmented timeline
int buckets = validateAndIncrementBuckets(buckets, maxBuckets);
// after: raise the limit in the compaction config
"tuningConfig": { "type": "index_parallel", "maxNumSegmentsToMerge": 100 } // and/or raise bucket limit

// estimate bucket count before submitting the replace task
int distinctChunks = condenseIntervals(existingIntervals).size();
if (distinctChunks >= maxBuckets) {
  throw new IAE("timeline too fragmented (%d chunks); pre-compact or raise maxBuckets", distinctChunks);
}

catch (TooManyBucketsException e) {
  log.warn("tombstone bucket cap hit (%s); narrowing interval", e.getMaxBuckets());
  return compactInSmallerIntervals(dataSource, narrowerInterval);
}

When it happens

Trigger: computeTombstoneIntervalsForReplace is called on a highly fragmented set of existing segment intervals (many small, non-contiguous time chunks), causing the bucket count to reach the configured max (e.g., maxNumPartitions/bucket limit) during tombstone interval computation.

Common situations: Compacting a datasource that accumulated many small segments across many time chunks over time; auto-compaction on a datasource with per-hour or finer segment granularity and long histories; replace tasks spanning wide intervals with sparse data.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Unable to fetch the version of the segments in use. The lock
• ColumnCapacityExceededException
• Bloom filter aggregators are query-time only
• Unsupported keyFormat. KafkaInputformat only supports input
• ORC flattener does not support JQ

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/467dd34c4f2bbcd9. Report an issue: GitHub.