apache/druid · error · MSQException

TooManyRowsWithSameKey

TooManyRowsWithSameKey

Error message

TooManyRowsWithSameKeyFault: too many rows with same key %s (bytes %s > max %s)

What it means

The sort-merge join buffers all rows sharing the current join key from both inputs. When every tracker needing more data has hit maxBufferedBytes, nextAwait throws TooManyRowsWithSameKeyFault, since it cannot buffer the key's full row group within the byte limit and still produce a correct merge. This protects workers from unbounded memory use on skewed keys.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/querykit/common/SortMergeJoinFrameProcessor.java:421

    if (awaitSet.isEmpty()) {
      // No tracker reported that it "needsMoreData" to read the current cursor. However, we may still need to read
      // more data to have a complete set for the current mark.
      for (int i = 0; i < inputChannels.size(); i++) {
        final Tracker tracker = trackers.get(i);
        if (!tracker.hasCompleteSetForMark()) {
          if (tracker.canBufferMoreFrames()) {
            awaitSet.add(i);
          } else if (trackerAtLimit < 0) {
            trackerAtLimit = i;
          }
        }
      }
    }

    if (awaitSet.isEmpty() && trackerAtLimit >= 0) {
      // All trackers that need more data are at their max buffered bytes limit. Generate a nice exception.
      final Tracker tracker = trackers.get(trackerAtLimit);
      throw new MSQException(
          new TooManyRowsWithSameKeyFault(
              tracker.readMarkKey(),
              tracker.totalBytesBuffered(),
              maxBufferedBytes
          )
      );
    }

    return ReturnOrAwait.awaitAll(awaitSet);
  }

  /**
   * Whether all trackers return true from {@link Tracker#isAtEnd()}.
   */
  private boolean allTrackersAreAtEnd()
  {
    for (Tracker tracker : trackers) {
      if (!tracker.isAtEnd()) {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Increase the context parameter maxRowsWithSameKey / maxBufferedBytes for the query (memory permitting)
2. Choose a higher-cardinality join key or add columns to the join condition to reduce per-key row counts
3. Pre-aggregate one or both sides before the join to shrink per-key groups
4. Filter rows before joining so fewer duplicates share each key

Example fix

// before
"context": {"maxRowsWithSameKey": 100000}
// after
"context": {"maxRowsWithSameKey": 1000000}

// Estimate max rows per join-key group vs maxRowsWithSameKey budget
long maxGroupRows = estimateMaxRowsPerKeyGroup(leftTable, rightTable, joinKey);
long maxRowsWithSameKey = context.get("maxRowsWithSameKey", 100000);
if (maxGroupRows > maxRowsWithSameKey) {
  throw new IllegalStateException("Join key skew: " + maxGroupRows + " rows per key > limit " + maxRowsWithSameKey);
}

try {
  runMsqQuery(query);
} catch (MSQException e) {
  if (e.getFault() instanceof TooManyRowsWithSameKey) {
    TooManyRowsWithSameKeyFault f = (TooManyRowsWithSameKeyFault) e.getFault();
    // raise maxBufferedBytes/maxRowsWithSameKey above f.getBytes() or de-skew the key
  }
}

When it happens

Trigger: A join key with more matching rows than fit in maxBufferedBytes (context setting, default ~100MiB total across trackers); detected in nextAwait when awaitSet is empty and trackerAtLimit >= 0 during runIncrementally.

Common situations: Joining on a low-cardinality key (e.g. country='US' matching millions of rows); data skew where one key dominates; queries copied with a small maxRowsWithSameKey/maxBufferedBytes context value.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• BroadcastTablesTooLarge
• FrameTooLarge
• Row [%d] out of bounds
• Row compares higher than mark; out-of-order input?
• Cannot handle constant condition: %s

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/47dadc3d7811aed5. Report an issue: GitHub.