{"record":{"id":"3650fdef8ebfaa33","repo":"apache/druid","slug":"toomanyrowsinawindow","errorCode":"TooManyRowsInAWindow","errorMessage":"TooManyRowsInAWindowFault: too many rows in a window, %s > max %s","messagePattern":"TooManyRowsInAWindowFault: too many rows in a window, (.+?) > max (.+?)","errorType":"error_code","errorClass":"MSQException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/querykit/WindowOperatorQueryFrameProcessor.java","lineNumber":328,"sourceCode":"    final RowSignature signature = frameReader.signature();\n    RowBasedFrameRowsAndColumns frameRowsAndColumns = new RowBasedFrameRowsAndColumns(frame, signature);\n    LazilyDecoratedRowsAndColumns ldrc = new LazilyDecoratedRowsAndColumns(\n        frameRowsAndColumns,\n        null,\n        null,\n        null,\n        OffsetLimit.limit(Integer.MAX_VALUE),\n        null,\n        null,\n        frameWriterFactory.allocatorCapacity()\n    );\n    return ldrc;\n  }\n\n  private static void ensureMaxRowsInAWindowConstraint(int numRowsInWindow, int maxRowsMaterialized)\n  {\n    if (numRowsInWindow > maxRowsMaterialized) {\n      throw new MSQException(new TooManyRowsInAWindowFault(\n          numRowsInWindow,\n          maxRowsMaterialized\n      ));\n    }\n  }\n\n  private boolean needToProcessBatch()\n  {\n    return frameRowsAndColsBuilder.getNumRows() >= maxRowsMaterialized / 2; // Can this be improved further?\n  }\n\n  private static class RowsAndColumnsBuilder\n  {\n    private final List<RowsAndColumns> racList;\n    private int totalRows;\n    private final int maxRowsMaterialized;\n\n    public RowsAndColumnsBuilder(int maxRowsMaterialized)","sourceCodeStart":310,"sourceCodeEnd":346,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/WindowOperatorQueryFrameProcessor.java#L310-L346","documentation":"MSQ window processing materializes the rows of a window partition in memory and enforces a cap (maxRowsMaterialized). ensureMaxRowsInAWindowConstraint throws TooManyRowsInAWindowFault when a single window partition contains more rows than this configured maximum. The query is deliberately stopped instead of exhausting worker memory.","triggerScenarios":"A window function (OVER clause) whose partition-by key groups so many rows that numRowsInWindow exceeds maxRowsInWindow/maxRowsMaterialized (default typically 100,000), checked from goOrContinue.","commonSituations":"A partition column with very low cardinality (e.g. all rows share one partition key) while running window functions over a large table; copying example context settings that cap rows in a window.","solutions":["Increase the context parameter maxRowsInWindow / maxRowsMaterialized for the query (mind memory)","Add more columns to PARTITION BY so each window partition contains fewer rows","Pre-aggregate rows before applying the window function","Split the data (e.g. by time or additional key) into smaller window partitions"],"exampleFix":"// before\n\"context\": {\"maxRowsInWindow\": 100000}\n// after\n\"context\": {\"maxRowsInWindow\": 1000000}","handlingStrategy":"validation","validationCode":"// Check partition cardinality against maxRowsInWindow before submitting\nlong maxPartitionRows = estimateMaxRowsPerPartition(table, partitionKeys);\nif (maxPartitionRows > maxRowsInWindow) {\n  throw new IllegalStateException(\"Partition has \" + maxPartitionRows + \" rows > maxRowsInWindow \" + maxRowsInWindow);\n}","typeGuard":null,"tryCatchPattern":"try {\n  runMsqQuery(query);\n} catch (MSQException e) {\n  if (e.getFault() instanceof TooManyRowsInAWindow) {\n    TooManyRowsInAWindowFault f = (TooManyRowsInAWindowFault) e.getFault();\n    // resubmit with maxRowsInWindow >= f.getNumRowsInWindow() or refine PARTITION BY\n  }\n}","preventionTips":["Choose high-cardinality PARTITION BY keys","Raise maxRowsInWindow only with proportional worker memory","Pre-aggregate low-cardinality dimensions"],"tags":["msq","window-function","memory","limits"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}