{"record":{"id":"2b63c05ff63da191","repo":"apache/beam","slug":"order-by-is-only-supported-for-s-actual-windowing-strategy-s","errorCode":null,"errorMessage":"`ORDER BY` is only supported for %s, actual windowing strategy: %s","messagePattern":"`ORDER BY` is only supported for (.+?), actual windowing strategy: (.+?)","errorType":"exception","errorClass":"java.lang.UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamSortRel.java","lineNumber":204,"sourceCode":"\n      // There is a need to separate ORDER BY LIMIT and LIMIT:\n      //  - GroupByKey (used in Top) is not allowed on unbounded data in global window so ORDER BY\n      // ... LIMIT\n      //    works only on bounded data.\n      //  - Just LIMIT operates on unbounded data, but across windows.\n      if (fieldIndices.isEmpty()) {\n        // TODO(https://github.com/apache/beam/issues/19075)\n        // Figure out which operations are per-window and which are not.\n\n        return upstream\n            .apply(Window.into(new GlobalWindows()))\n            .apply(new LimitTransform<>(startIndex))\n            .setRowSchema(CalciteUtils.toSchema(getRowType()));\n      } else {\n\n        WindowingStrategy<?, ?> windowingStrategy = upstream.getWindowingStrategy();\n        if (!(windowingStrategy.getWindowFn() instanceof GlobalWindows)) {\n          throw new UnsupportedOperationException(\n              String.format(\n                  \"`ORDER BY` is only supported for %s, actual windowing strategy: %s\",\n                  GlobalWindows.class.getSimpleName(), windowingStrategy));\n        }\n\n        // When no limit is specified (count == -1), we must sort the entire dataset.\n        // To achieve this globally, we key all rows by a single dummy key, group them together\n        // using GroupByKey to ensure they are processed together, and then sort them in-memory\n        // via SortInMemoryFn. Note: This can be memory-intensive for large datasets. It should\n        // only be done as a final step when the remaining data is small\n        if (count == -1) {\n          BeamSqlRowComparator comparator =\n              new BeamSqlRowComparator(fieldIndices, orientation, nullsFirst);\n          return upstream\n              .apply(\"WithDummyKey\", WithKeys.of(\"DummyKey\"))\n              .apply(\"GroupByKey\", GroupByKey.create())\n              .apply(\"SortInMemory\", ParDo.of(new SortInMemoryFn(comparator)))\n              .setRowSchema(CalciteUtils.toSchema(getRowType()));","sourceCodeStart":186,"sourceCodeEnd":222,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamSortRel.java#L186-L222","documentation":"Beam SQL's ORDER BY (BeamSortRel) can only operate on globally sorted data. When the upstream PCollection's windowing strategy is not GlobalWindows (e.g. fixed or sliding windows), there is no meaningful total order across windows, so the transform throws this UnsupportedOperationException during expand(). With non-global windows it only works when an explicit LIMIT with a fetch path is used.","triggerScenarios":"Running a SQL query with ORDER BY over a streaming/windowed PCollection whose windowFn is not GlobalWindows and without a limit allowing the early-limit path; thrown from BeamSortRel.expand().","commonSituations":"ORDER BY on a Pub/Sub/Kafka-backed table (streaming, windowed), or on a batch table that was re-windowed before the sort; users expect SQL ORDER BY to work anywhere but it needs a single global window (or a LIMIT).","solutions":["Remove ORDER BY, or apply it after re-windowing the data into GlobalWindows (e.g. via a windowing transform / aggregate first).","Add a LIMIT (ORDER BY ... LIMIT n) so BeamSortRel can use the supported limited path if applicable.","Aggregate/window the streaming data into a bounded result (e.g. GlobalWindows via Window.into(GlobalWindows()) after a trigger) before sorting.","Perform the sort downstream in a batch job or in the sink instead of inside the streaming SQL query."],"exampleFix":"// before\nPCollection<Row> sorted = windowed.apply(SqlTransform.query(\"SELECT * FROM t ORDER BY x\"));\n// after\nPCollection<Row> global = windowed.apply(Window.<Row>into(new GlobalWindows()).triggering(AfterWatermark.pastEndOfWindow()).withAllowedLateness(Duration.ZERO).discardingFiredPanes());\nPCollection<Row> sorted = global.apply(SqlTransform.query(\"SELECT * FROM PCOLLECTION ORDER BY x\"));","handlingStrategy":"validation","validationCode":"WindowingStrategy<?, ?> ws = upstream.getWindowingStrategy(); if (hasOrderBy && !(ws.getWindowFn() instanceof GlobalWindows) && !hasLimit) { throw new IllegalArgumentException(\"ORDER BY requires GlobalWindows or a LIMIT\"); }","typeGuard":"boolean sortable = !(windowingStrategy.getWindowFn() instanceof GlobalWindows) ? hasLimit : true;","tryCatchPattern":"try { result = pc.apply(SqlTransform.query(sql)); } catch (UnsupportedOperationException e) { if (e.getMessage().contains(\"ORDER BY\")) { /* re-window to GlobalWindows or add LIMIT */ } else { throw e; } }","preventionTips":["Only use ORDER BY on GlobalWindows PCollections (default for batch)","Re-window to GlobalWindows before sorting streaming data","Use ORDER BY ... LIMIT n for the supported limited path in streaming","Aggregate first, then sort the small bounded result"],"tags":["sql","order-by","windowing","streaming"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}