apache/druid · error · IllegalArgumentException

Trying to persist an empty index!

Error message

Trying to persist an empty index!

What it means

IndexMergerBase.persist refuses to write an index that contains no rows. Persisting an empty incremental index would produce a meaningless segment (no timestamps, no ranges), so Druid treats it as a caller bug and throws IAE rather than emitting a broken segment.

Solutions

1. Guard the persist call with index.isEmpty() and skip or log instead of persisting an empty index
2. Fix the ingestion spec/filters so events actually reach the index
3. Return a null/absent segment rather than persisting when the input window is empty

Example fix

// before
merger.persist(index, dataInterval, outDir, indexSpec, progress, null);
// after
if (!index.isEmpty()) {
  merger.persist(index, dataInterval, outDir, indexSpec, progress, null);
} else {
  log.warn("Skipping persist: index is empty for interval %s", dataInterval);
}

if (index.isEmpty()) {
  log.warn("Nothing to persist; skipping segment for interval %s", dataInterval);
  return null;
}

try {
  merger.persist(index, dataInterval, outDir, indexSpec, progress, null);
} catch (IAE e) {
  if (e.getMessage().contains("empty index")) {
    // skip: no data in this window; emit no segment
  }
}

When it happens

Trigger: Calling IndexMerger.persist (or the merge/persist helpers built on it) with a deactivated incremental index that has had all rows dropped, or an index where nothing was ever added (e.g. all rows filtered out at firehose level).

Common situations: Ingestion tasks with filters matching zero events; stream ingestion where the window closed before any event arrived; custom code that clears an index (rollup/overwrite paths) and then persists it unconditionally.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Attempt to add row to swapped-out sink for segment
• Bloom filter aggregators are query-time only
• Can't find pushedSegments for segment
• Can't find segmentsForSequence for sequence
• Cannot simultaneously replace and append to existing…

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/6b0c6668292a390e. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/IndexMergerBase.java:135

   */
  protected abstract void makeColumn(
      SegmentFileBuilder segmentFileBuilder,
      String columnName,
      ColumnDescriptor serdeficator
  ) throws IOException;

  @Override
  public File persist(
      final IncrementalIndex index,
      final Interval dataInterval,
      File outDir,
      IndexSpec indexSpec,
      ProgressIndicator progress,
      @Nullable SegmentWriteOutMediumFactory segmentWriteOutMediumFactory
  ) throws IOException
  {
    if (index.isEmpty()) {
      throw new IAE("Trying to persist an empty index!");
    }

    indexSpec = indexSpec.getEffectiveSpec();

    final DateTime firstTimestamp = index.getMinTime();
    final DateTime lastTimestamp = index.getMaxTime();
    if (!(dataInterval.contains(firstTimestamp) && dataInterval.contains(lastTimestamp))) {
      throw new IAE(
          "interval[%s] does not encapsulate the full range of timestamps[%s, %s]",
          dataInterval,
          firstTimestamp,
          lastTimestamp
      );
    }

    FileUtils.mkdirp(outDir);

    log.debug("Starting persist for interval[%s], rows[%,d]", dataInterval, index.numRows());

View on GitHub (pinned to 9b90983fd2)