apache/druid · error · FrameRowTooLargeException

FrameTooLarge

FrameTooLarge

Error message

FrameRowTooLargeException: row too large for frame allocator capacity %s

What it means

In the offset/limit frame processor, after skipping to startRow the processor writes rows up to endRow into a fresh frame writer whose allocator is unlimited; if addSelection() still fails the code intentionally throws FrameRowTooLargeException rather than retrying, because with an unlimited allocator failure indicates a logical bug or a truly unrepresentable row.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/querykit/common/OffsetLimitFrameProcessor.java:153

    } else if (startRow == 0
               && endRow == frame.numRows()
               && inputSignatureMatchesOutputSignature
               && frameWriterFactory.frameType().equals(frame.type())) {
      // Want the whole frame; emit it as-is.
      rowsProcessedSoFar += frame.numRows();
      return frame;
    }

    final Cursor cursor = FrameProcessors.makeCursor(frame, frameReader);

    long rowsProcessedSoFarInFrame = 0;

    try (final FrameWriter frameWriter = frameWriterFactory.newFrameWriter(cursor.getColumnSelectorFactory())) {
      while (!cursor.isDone() && rowsProcessedSoFarInFrame < endRow) {
        if (rowsProcessedSoFarInFrame >= startRow && !frameWriter.addSelection()) {
          // Don't retry; it can't work because the allocator is unlimited anyway.
          // Also, I don't think this line can be reached, because the allocator is unlimited.
          throw new FrameRowTooLargeException(frameWriterFactory.allocatorCapacity());
        }

        cursor.advance();
        rowsProcessedSoFarInFrame++;
      }

      rowsProcessedSoFar += rowsProcessedSoFarInFrame;
      return Frame.wrap(frameWriter.toByteArray());
    }
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

Retry the query; if reproducible, file a Druid bug with the query and stack trace
Reduce row width (fewer/smaller columns) before the limit/offset stage
Increase MSQ memory limits in case the allocator is not truly unlimited in your configuration
Check Druid version for known fixes to OffsetLimitFrameProcessor

Defensive patterns

Strategy: retry

Try / catch

try {
  runMsqQuery(query);
} catch (MSQException e) {
  if (e.getFault() instanceof FrameTooLarge && stageIsOffsetLimit(e)) {
    // not user-fixable; capture query+trace and report to Druid
  }
}

Prevention

Avoid pathological ultra-wide rows in limit/offset stages
Keep Druid upgraded to pick up frame-processor fixes
Reduce projections before offset/limit stages

When it happens

Trigger: An offset/limit stage over a cursor whose current row cannot be added to the frame despite the unlimited frame allocator — normally an internal invariant breach (row larger than any allocation) reached while processing rows between startRow and endRow.

Common situations: Very large single rows flowing through an MSQ limit/offset stage; typically surfaces as an engine bug report rather than user misconfiguration, but can accompany extremely wide rows near allocator limits.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ee621fa67131b14c. Report an issue: GitHub.