{"record":{"id":"a88c46883480df8c","repo":"apache/druid","slug":"row-too-large-to-add-to-frame-max-frame-size-a88c46","errorCode":null,"errorMessage":"Row too large to add to frame (max frame size = %,d)","messagePattern":"Row too large to add to frame \\(max frame size = %,d\\)","errorType":"exception","errorClass":"FrameRowTooLargeException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/querykit/scan/ScanQueryFrameProcessor.java","lineNumber":491,"sourceCode":"  private void populateFrameWriterAndFlushIfNeeded() throws IOException\n  {\n    createFrameWriterIfNeeded();\n\n    while (!cursor.isDone()) {\n      boolean flush;\n\n      if (frameWriter.addSelection()) {\n        cursorRowsRead++;\n        cursor.advance();\n        cursorOffset.increment();\n        partitionBoostVirtualColumn.setValue(partitionBoostVirtualColumn.getValue() + 1);\n\n        // Flush if we reached cursorPushDownLimit.\n        flush = cursorPushDownLimit >= 0 && cursorRowsRead >= cursorPushDownLimit;\n      } else {\n        // addSelection failed because the frame is full.\n        if (frameWriter.getNumRows() == 0) {\n          throw new FrameRowTooLargeException(currentAllocatorCapacity);\n        }\n\n        flush = true;\n      }\n\n      if (flush) {\n        final long numRowsWritten = flushFrameWriter();\n\n        if (sharedRunningCountForLimit != null) {\n          sharedRunningCountForLimit.addAndGet(numRowsWritten);\n        }\n\n        break;\n      }\n    }\n  }\n\n  private void createFrameWriterIfNeeded()","sourceCodeStart":473,"sourceCodeEnd":509,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/scan/ScanQueryFrameProcessor.java#L473-L509","documentation":"ScanQueryFrameProcessor.populateFrameWriterAndFlushIfNeeded throws FrameRowTooLargeException when frameWriter.addSelection() fails and the current frame is still empty, meaning a single scan result row exceeds the frame's allocator capacity. MSQ cannot split a row across frames, so the processor aborts.","triggerScenarios":"Scanning segments or input frames via MSQ where one row (all selected columns serialized) is larger than the current frame allocator capacity — wide selects, long string values, or undersized frame memory config.","commonSituations":"SELECT * on wide datasources with large strings; ingestion of oversized rows later scanned with MSQ; MSQ workers with low frame memory limits.","solutions":["Increase MSQ frame allocator capacity / worker memory configuration.","Select only needed columns to shrink each row.","Fix upstream ingestion to avoid storing oversized rows.","Reduce per-task concurrency to allow larger frames within the same memory budget."],"exampleFix":"// before\nSELECT * FROM huge_table  // row > frame capacity\n// after\nSELECT col1, col2 FROM huge_table  // narrower rows fit the frame","handlingStrategy":"try-catch","validationCode":"// Pre-check: estimated scan row size vs frame capacity\nif (selectedColumnsBytesEstimate > frameCapacityBytes) {\n  throw new Error('Scan row exceeds frame capacity; select fewer columns or raise frame memory');\n}","typeGuard":null,"tryCatchPattern":"try {\n  runMsqScan(scan);\n} catch (FrameRowTooLargeException e) {\n  // Retry with a narrower column list or larger frame allocation\n  retry(selectSubset(scan, e.getMaxFrameSize()));\n}","preventionTips":["Avoid SELECT * on wide datasources via MSQ.","Bound string column lengths at ingestion time.","Provide adequate frame memory per MSQ worker."],"tags":["msq","scan","frame","memory"],"backgroundTag":"payload-too-large","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}