{"record":{"id":"b52e7e9661eaa288","repo":"apache/beam","slug":"does-not-support-queries-with-limit-in-torowlist","errorCode":null,"errorMessage":"Does not support queries with LIMIT in toRowList.","messagePattern":"Does not support queries with LIMIT in toRowList\\.","errorType":"exception","errorClass":"java.lang.UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamEnumerableConverter.java","lineNumber":162,"sourceCode":"  }\n\n  public static PipelineOptions createPipelineOptions(Map<String, String> map) {\n    final String[] args = new String[map.size()];\n    int i = 0;\n    for (Map.Entry<String, String> entry : map.entrySet()) {\n      args[i++] = \"--\" + entry.getKey() + \"=\" + entry.getValue();\n    }\n    PipelineOptions options = PipelineOptionsFactory.fromArgs(args).withValidation().create();\n    FileSystems.setDefaultPipelineOptions(options);\n    options.as(ApplicationNameOptions.class).setAppName(\"BeamSql\");\n    return options;\n  }\n\n  static List<Row> toRowList(PipelineOptions options, BeamRelNode node) {\n    if (node instanceof BeamIOSinkRel) {\n      throw new UnsupportedOperationException(\"Does not support BeamIOSinkRel in toRowList.\");\n    } else if (isLimitQuery(node)) {\n      throw new UnsupportedOperationException(\"Does not support queries with LIMIT in toRowList.\");\n    }\n    return collectRows(options, node).stream().collect(Collectors.toList());\n  }\n\n  static Enumerable<Object> toEnumerable(PipelineOptions options, BeamRelNode node) {\n    if (node instanceof BeamIOSinkRel) {\n      return count(options, node);\n    } else if (isLimitQuery(node)) {\n      return limitCollect(options, node);\n    }\n    return Linq4j.asEnumerable(rowToAvaticaAndUnboxValues(collectRows(options, node)));\n  }\n\n  private static PipelineResult limitRun(\n      PipelineOptions options,\n      BeamRelNode node,\n      DoFn<Row, Void> doFn,\n      Queue<Row> values,","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamEnumerableConverter.java#L144-L180","documentation":"toRowList materializes the full result of a Beam SQL plan into memory, which cannot honor a LIMIT clause (the limit is not pushed through collectRows in this path). If isLimitQuery(node) detects LIMIT in the plan, it throws UnsupportedOperationException telling the caller this execution mode does not support LIMIT. Use the toEnumerable path (or remove the LIMIT) instead.","triggerScenarios":"Calling BeamEnumerableConverter.toRowList(options, node) on a BeamRelNode whose plan contains a Sort/Limit (isLimitQuery returns true) — typically a SELECT ... LIMIT N statement executed through the row-list collection path.","commonSituations":"Running 'SELECT * FROM t LIMIT 10' via BeamSqlCli/JDBC paths configured to use toRowList; adding LIMIT to a query in tests that use the toRowList helper; library versions where LIMIT was only supported in the enumerable (toEnumerable) execution mode.","solutions":["Use toEnumerable(options, node) instead of toRowList for queries with LIMIT","Remove the LIMIT clause from the SQL and limit the resulting list yourself after collection","Push the limit into the source (e.g. a filtered/limited PCollection or read with a bounded count) instead of relying on SQL LIMIT with toRowList","Upgrade/patch BeamEnumerableConverter to support LIMIT by applying the limit to the collected list (or use the enumerable path internally)"],"exampleFix":"// before\nList<Row> rows = BeamEnumerableConverter.toRowList(options, relNode); // SELECT ... LIMIT 10\n// after\nList<Row> rows = new ArrayList<>();\nBeamEnumerableConverter.toEnumerable(options, relNode).forEach(r -> { if (rows.size() < 10) rows.add(r); }); // or drop LIMIT and slice after collection","handlingStrategy":"type-guard","validationCode":"static boolean planHasLimit(BeamRelNode node) {\n  for (RelNode n : node.getInputs()) { if (n instanceof BeamSortRel) return true; }\n  return node instanceof BeamSortRel;\n}","typeGuard":"boolean isLimitQuery(BeamRelNode node) { return node instanceof BeamSortRel || node.getInputs().stream().anyMatch(i -> i instanceof BeamSortRel); }","tryCatchPattern":"try {\n  rows = BeamEnumerableConverter.toRowList(options, node);\n} catch (UnsupportedOperationException e) {\n  if (e.getMessage().contains(\"LIMIT\")) {\n    rows = collectViaEnumerable(node); // LIMIT-capable path, or re-run without LIMIT\n  } else { throw e; }\n}","preventionTips":["Prefer toEnumerable for any query that might contain LIMIT/OFFSET","Validate the SQL plan (look for BeamSortRel) before selecting the toRowList execution path","Apply limits client-side on the collected rows instead of using SQL LIMIT with toRowList","Document in test helpers which execution modes support which SQL features"],"tags":["java","beam-sql","unsupported-operation","limit","execution"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}