apache/druid · error · FrameRowTooLargeException

Row too large to add to frame (max frame size = %,d)

Error message

Row too large to add to frame (max frame size = %,d)

What it means

GroupByPostShuffleFrameProcessor.writeOutputRow throws FrameRowTooLargeException when a single aggregation output row cannot be added to the current frame because it exceeds the frame writer's allocator capacity. MSQ frames have a fixed maximum size, and one row that does not fit in an empty frame cannot be split across frames.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/querykit/groupby/GroupByPostShuffleFrameProcessor.java:255

    // Finalize aggregators after checking if they are passing the havingSpec, because havingSpec expects the
    // unfinalized row (and finalizes it internally after making a copy of it)
    finalizeFn.accept(outputRow);

    if (frameWriter.addSelection()) {
      incrementBoostColumn();
      outputRow = null;
      return false;
    } else if (frameWriter.getNumRows() > 0) {
      writeCurrentFrameIfNeeded();
      setUpFrameWriterIfNeeded();

      if (frameWriter.addSelection()) {
        incrementBoostColumn();
        outputRow = null;
        return true;
      } else {
        throw new FrameRowTooLargeException(frameWriterFactory.allocatorCapacity());
      }
    } else {
      throw new FrameRowTooLargeException(frameWriterFactory.allocatorCapacity());
    }
  }

  private void writeCurrentFrameIfNeeded() throws IOException
  {
    if (frameWriter != null && frameWriter.getNumRows() > 0) {
      final Frame frame = Frame.wrap(frameWriter.toByteArray());
      outputChannel.write(frame);
      frameWriter.close();
      frameWriter = null;
      outputRows += frame.numRows();
    }
  }

  /**

View on GitHub (pinned to 9b90983fd2)

Solutions

  1. Increase per-frame capacity: raise the frame memory allocation / maxRows config (druid.msq.* frame settings) so a row fits.
  2. Reduce row width: select fewer dimensions/columns, shorten string cardinality, or use smaller sketch types.
  3. Increase worker memory or reduce worker capacity so more memory is available for frames.
  4. Split the query (e.g. aggregate in stages with fewer output columns) to shrink each row.

Example fix

// before (druid config)
druid.msq.frame.maxRows=100000  // frames too small for wide rows
// after
druid.msq.frame.maxRows=10000  // smaller rows-per-frame => larger per-row capacity with same memory
Defensive patterns

Strategy: try-catch

Validate before calling

// Estimate row size against frame capacity before running wide group-bys
const estimatedRowBytes = dimCount * 64 + aggCount * 256;
if (estimatedRowBytes > frameCapacityBytes) throw new Error('Row likely exceeds MSQ frame capacity; increase frame memory or reduce row width');

Try / catch

try {
  runMsqQuery(query);
} catch (FrameRowTooLargeException e) {
  // e reports max frame size; retry with fewer columns or bigger frames
  retryWithWiderFramesOrNarrowerQuery(query, e.getMaxFrameSize());
}

Prevention

When it happens

Trigger: A group-by query executed via MSQ produces a post-shuffle output row whose serialized size (dimensions + aggregators, including any boost/virtual columns) exceeds the configured max frame size (druid.msq.frame.maxRows or memory allocation per frame).

Common situations: Queries with very wide result rows (many dimensions, large string values, complex sketches like HLL/quantiles), or overly small frame memory allocation in the MSQ worker config.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/c9de4eb1df59e552. Report an issue: GitHub.