What causes "Index[%d] >= size[%d]" in apache/druid?

Re-fetch the Indexed instance from the current column holder and use its size() as the loop bound.

apache/druid · error · IllegalArgumentException

Index[%d] >= size[%d]

Error message

Index[%d] >= size[%d]

What it means

SpectatorHistogramIndexed is a dense indexed collection over SpectatorHistogram column values; checkIndex validates that a requested ordinal lies within [0, size). When get() is called with an index at or beyond the number of stored values, the IAE 'Index[%d] >= size[%d]' is thrown to prevent reads of uninitialized offsets into the underlying buffer.

Source

Thrown at extensions-contrib/spectator-histogram/src/main/java/org/apache/druid/spectator/histogram/SpectatorHistogramIndexed.java:105

    // The rest of the buffer is the values
    valueBuffer = buffer.slice();
  }

  /**
   * Checks  if {@code index} a valid `element index` in SpectatorHistogramIndexed.
   * Similar to Preconditions.checkElementIndex() except this method throws {@link IAE} with custom error message.
   * <p>
   * Used here to get existing behavior(same error message and exception) of V1 GenericIndexed.
   *
   * @param index index identifying an element of an SpectatorHistogramIndexed.
   */
  private void checkIndex(int index)
  {
    if (index < 0) {
      throw new IAE("Index[%s] < 0", index);
    }
    if (index >= size) {
      throw new IAE("Index[%d] >= size[%d]", index, size);
    }
  }

  public Class<? extends SpectatorHistogram> getClazz()
  {
    return strategy.getClazz();
  }

  @Override
  public int size()
  {
    return size;
  }

  @Nullable
  @Override
  public SpectatorHistogram get(int index)
  {

View on GitHub (pinned to 9b90983fd2)

Solutions

Re-fetch the Indexed instance from the current column holder and use its size() as the loop bound.
Verify the index source: ensure the caller uses size() from the same Indexed object, not a cached or computed count.
Check for concurrent segment swapping; re-run the query if a segment was replaced mid-scan.
If debugging, log the failing index and size and compare against the column's numRows metadata.

Example fix

// before
for (int i = 0; i < cachedSize; i++) { histogram = indexed.get(i); }
// after
for (int i = 0; i < indexed.size(); i++) { histogram = indexed.get(i); }

Defensive patterns

Strategy: validation

Validate before calling

if (index < 0 || index >= indexed.size()) { throw new IllegalArgumentException("index " + index + " out of range 0.." + (indexed.size() - 1)); }
return indexed.get(index);

Try / catch

try { return indexed.get(i); } catch (IAE e) { log.warn("stale index %d: %s", i, e.getMessage()); return null; }

Prevention

Always bound loops with the same Indexed object's size(), never a cached count.
Re-acquire Indexed references after segment swaps instead of reusing old ones.
Write a unit test that iterates every index 0..size()-1 before deploying index math changes.

When it happens

Trigger: Calling SpectatorHistogramIndexed.get(index) with a negative-derived or stale index, e.g. iterating with a row count from a different segment version, or a caller that computed an index from an out-of-date dictionary size.

Common situations: Segment replacement during query execution making a cached Indexed reference stale; hand-written code scanning rows with a hardcoded or mis-computed bound; dictionary lookups against a column whose size shrunk after compaction.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/9386361031a4378e. Report an issue: GitHub.