{"record":{"id":"477ac1d279a40dc5","repo":"apache/druid","slug":"frametoolarge-477ac1","errorCode":"FrameTooLarge","errorMessage":"FrameRowTooLargeException: row too large for frame allocator capacity %s","messagePattern":"FrameRowTooLargeException: row too large for frame allocator capacity (.+?)","errorType":"error_code","errorClass":"FrameRowTooLargeException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/querykit/common/SortMergeJoinFrameProcessor.java","lineNumber":277,"sourceCode":"   * @param markCmp    result of {@link #compareMarks()}\n   * @param marksMatch whether the marks actually matched, taking nulls into account\n   *\n   * @return true if cursors should be advanced, false if we should run again without moving cursors\n   */\n  private boolean emitRowIfNeeded(final int markCmp, final boolean marksMatch) throws IOException\n  {\n    if (marksMatch || (markCmp <= 0 && joinType.isLefty()) || (markCmp >= 0 && joinType.isRighty())) {\n      // Emit row, if there's room in the current frameWriter.\n      joinColumnSelectorFactory.cmp = markCmp;\n      joinColumnSelectorFactory.match = marksMatch;\n\n      if (!frameWriter.addSelection()) {\n        if (frameWriter.getNumRows() > 0) {\n          // Out of space in the current frame. Run again without moving cursors.\n          flushCurrentFrame();\n          return false;\n        } else {\n          throw new FrameRowTooLargeException(frameWriterFactory.allocatorCapacity());\n        }\n      }\n    }\n\n    return true;\n  }\n\n  /**\n   * Advance one or both trackers after emitting a row.\n   *\n   * @param markCmp    result of {@link #compareMarks()}\n   * @param marksMatch whether the marks actually matched, taking nulls into account\n   */\n  private void advanceTrackersAfterEmittingRow(final int markCmp, final boolean marksMatch)\n  {\n    if (marksMatch) {\n      // Matching keys. First advance the tracker with the complete set.\n      final Tracker completeSetTracker = trackers.get(trackerWithCompleteSetForCurrentKey);","sourceCodeStart":259,"sourceCodeEnd":295,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/common/SortMergeJoinFrameProcessor.java#L259-L295","documentation":"In the sort-merge join frame processor, emitRowIfNeeded appends the joined row to the current frame. If the frame already has rows, it is flushed and processing resumes; but if the frame is empty and addSelection() still fails, the single joined row exceeds the frame allocator capacity and FrameRowTooLargeException is thrown. The query cannot buffer even one output row within the frame memory budget.","triggerScenarios":"Sort-merge join stage emits a merged row (left+right columns concatenated) whose encoded size exceeds frameWriterFactory.allocatorCapacity() when the current frame is empty; reached via runIncrementally.","commonSituations":"Joining tables with many/wide string columns so combined rows are huge; low frame allocator capacity after aggressive memory tuning; large ARRAY/COMPLEX columns from either side of the join.","solutions":["Increase the MSQ frame allocator capacity (frame size / worker memory settings)","Project only required columns from each side of the JOIN before the join stage","Shorten or hash large join payload columns; move big columns to a post-join lookup","Increase peon/task memory to allow bigger frames"],"exampleFix":"// before\nSELECT a.*, b.* FROM tbl1 a JOIN tbl2 b ON a.k = b.k\n// after\nSELECT a.k, a.val1, b.val2 FROM tbl1 a JOIN tbl2 b ON a.k = b.k","handlingStrategy":"validation","validationCode":"// Ensure joined row width fits frame capacity\nlong joinedRowBytes = approxRowBytes(leftCols) + approxRowBytes(rightCols);\nif (joinedRowBytes >= frameAllocatorCapacityBytes) {\n  throw new IllegalStateException(\"Join output row (\" + joinedRowBytes + \"B) exceeds frame capacity\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  runMsqQuery(query);\n} catch (MSQException e) {\n  if (e.getFault() instanceof FrameTooLarge) {\n    // narrow join projection or raise frame allocator capacity, then retry\n  }\n}","preventionTips":["Project only needed columns from each join side","Avoid SELECT * across large joins","Increase frame size for wide schemas"],"tags":["msq","join","memory","frame-allocator"],"backgroundTag":"payload-too-large","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}