{"record":{"id":"467dd34c4f2bbcd9","repo":"apache/druid","slug":"too-many-buckets-maximum-is-s","errorCode":null,"errorMessage":"Too many buckets; maximum is [%s]","messagePattern":"Too many buckets; maximum is \\[(.+?)\\]","errorType":"exception","errorClass":"TooManyBucketsException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/TombstoneHelper.java","lineNumber":333,"sourceCode":"              SegmentDetail.none()\n          ));\n      for (DataSegment usedSegment : usedSegmentsInInputInterval) {\n        for (Interval condensedInputInterval : condensedInputIntervals) {\n          if (condensedInputInterval.overlaps(usedSegment.getInterval())) {\n            retVal.add(usedSegment.getInterval());\n            break;\n          }\n        }\n      }\n    }\n\n    return JodaUtils.condenseIntervals(retVal);\n  }\n\n  private int validateAndIncrementBuckets(final int buckets, final int maxBuckets)\n  {\n    if (buckets >= maxBuckets) {\n      throw new TooManyBucketsException(maxBuckets);\n    }\n    return buckets + 1;\n  }\n\n}\n","sourceCodeStart":315,"sourceCodeEnd":339,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/TombstoneHelper.java#L315-L339","documentation":"TombstoneHelper.validateAndIncrementBuckets enforces a cap on the number of tombstone buckets generated when condensing replace intervals; exceeding it throws TooManyBucketsException. The error's message 'Too many buckets; maximum is [%s]' fires when the bucket count reaches the configured maxBuckets. This guards against generating an unbounded number of tombstone segments for very fragmented timelines.","triggerScenarios":"computeTombstoneIntervalsForReplace is called on a highly fragmented set of existing segment intervals (many small, non-contiguous time chunks), causing the bucket count to reach the configured max (e.g., maxNumPartitions/bucket limit) during tombstone interval computation.","commonSituations":"Compacting a datasource that accumulated many small segments across many time chunks over time; auto-compaction on a datasource with per-hour or finer segment granularity and long histories; replace tasks spanning wide intervals with sparse data.","solutions":["Increase the bucket limit in the task's tuning/partitioning config if the fragmentation is expected","Pre-compact the datasource into coarser granularity segments (e.g., day) to reduce fragmentation before running the replace task","Narrow the compaction interval so fewer time chunks are processed per run","Enable auto-compaction with smaller skipOffsetFromLatest and regular runs to prevent fragmentation buildup"],"exampleFix":"// before: default maxBuckets with heavily fragmented timeline\nint buckets = validateAndIncrementBuckets(buckets, maxBuckets);\n// after: raise the limit in the compaction config\n\"tuningConfig\": { \"type\": \"index_parallel\", \"maxNumSegmentsToMerge\": 100 } // and/or raise bucket limit\n","handlingStrategy":"validation","validationCode":"// estimate bucket count before submitting the replace task\nint distinctChunks = condenseIntervals(existingIntervals).size();\nif (distinctChunks >= maxBuckets) {\n  throw new IAE(\"timeline too fragmented (%d chunks); pre-compact or raise maxBuckets\", distinctChunks);\n}","typeGuard":null,"tryCatchPattern":"catch (TooManyBucketsException e) {\n  log.warn(\"tombstone bucket cap hit (%s); narrowing interval\", e.getMaxBuckets());\n  return compactInSmallerIntervals(dataSource, narrowerInterval);\n}","preventionTips":["Run auto-compaction regularly so timelines don't fragment into many small chunks","Use coarse segment granularity for long-retention datasources","Limit each replace/compaction task's interval scope","Raise bucket limits only when fragmentation is genuinely expected"],"tags":["ingestion","tombstone","compaction","limits"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}