{"record":{"id":"2f8c587e724c6be5","repo":"apache/druid","slug":"row-too-large-to-add-to-frame-max-frame-size-2f8c58","errorCode":null,"errorMessage":"Row too large to add to frame (max frame size = %,d)","messagePattern":"Row too large to add to frame \\(max frame size = %,d\\)","errorType":"exception","errorClass":"FrameRowTooLargeException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/querykit/groupby/GroupByPreShuffleFrameProcessor.java","lineNumber":328,"sourceCode":"  public void cleanup() throws IOException\n  {\n    closer.register(this::closeAndDiscardResultYielder);\n    closer.register(frameWriter);\n    closer.register(super::cleanup);\n    closer.close();\n  }\n\n  private void populateFrameWriterAndFlushIfNeeded() throws IOException\n  {\n    createFrameWriterIfNeeded();\n\n    while (!resultYielder.isDone()) {\n      final boolean didAddToFrame = frameWriter.addSelection();\n\n      if (didAddToFrame) {\n        resultYielder = resultYielder.next(null);\n      } else if (frameWriter.getNumRows() == 0) {\n        throw new FrameRowTooLargeException(currentAllocatorCapacity);\n      } else {\n        flushFrameWriterIfNeeded();\n        return;\n      }\n    }\n\n    flushFrameWriterIfNeeded();\n    closeAndDiscardResultYielder();\n  }\n\n  private void createFrameWriterIfNeeded()\n  {\n    if (frameWriter == null) {\n      final FrameWriterFactory frameWriterFactory = getFrameWriterFactory();\n      frameWriter = frameWriterFactory.newFrameWriter(frameWriterColumnSelectorFactory);\n      currentAllocatorCapacity = frameWriterFactory.allocatorCapacity();\n    }\n  }","sourceCodeStart":310,"sourceCodeEnd":346,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/groupby/GroupByPreShuffleFrameProcessor.java#L310-L346","documentation":"GroupByPreShuffleFrameProcessor.populateFrameWriterAndFlushIfNeeded throws FrameRowTooLargeException when a single pre-shuffle group-by result row cannot fit into a completely empty frame (frameWriter.getNumRows() == 0 and addSelection() fails). MSQ frames cannot hold a row larger than the frame's allocator capacity, so processing aborts.","triggerScenarios":"Pre-shuffle group-by execution where one output row's serialized size exceeds the current frame allocator capacity; triggered when reading from a data server query, segment, or input channel yields rows too wide for the configured frame size.","commonSituations":"Very wide dimension sets or long strings in group-by keys; sketch aggregators with large state; MSQ workers configured with too little frame memory (druid.msq.frame settings).","solutions":["Increase frame memory/allocation in MSQ worker configuration so the largest row fits.","Reduce group-by key width: fewer dimensions, shorter strings, hashed keys.","Use more compact aggregators or reduce aggregator count.","Increase per-task worker memory or lower concurrency to free memory for frames."],"exampleFix":"// before\n// worker: 1GiB memory, default frame size -> row of 2MB does not fit\n// after\n// raise frame capacity or worker memory, e.g. give each MSQ worker 4GiB and raise druid.msq.frame allocation","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  runMsqQuery(groupBy);\n} catch (FrameRowTooLargeException e) {\n  // Retry with reduced key width or increased frame capacity\n  retry(withNarrowerKeys(withLargerFrames(groupBy, e.getMaxFrameSize())));\n}","preventionTips":["Hash or reduce high-cardinality group-by keys.","Keep string dimension values bounded (cap lengths at ingestion).","Match frame memory config to the largest expected pre-shuffle row."],"tags":["msq","memory","frame"],"backgroundTag":"payload-too-large","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}