{"record":{"id":"5d16be2ee4428fdf","repo":"apache/beam","slug":"it-is-required-to-set-usecursorfetch-true-in-the-jdbc-url","errorCode":null,"errorMessage":"It is required to set useCursorFetch=true in the JDBC URL when using fetchSize for MySQL","messagePattern":"It is required to set useCursorFetch=true in the JDBC URL when using fetchSize for MySQL","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/providers/ReadFromMySqlSchemaTransformProvider.java","lineNumber":73,"sourceCode":"  @Override\n  public @UnknownKeyFor @NonNull @Initialized SchemaTransform from(\n      JdbcReadSchemaTransformConfiguration configuration) {\n    String jdbcType = configuration.getJdbcType();\n    if (jdbcType != null && !jdbcType.isEmpty() && !jdbcType.equals(jdbcType())) {\n      LOG.warn(\n          \"Wrong JDBC type. Expected '{}' but got '{}'. Overriding with '{}'.\",\n          jdbcType(),\n          jdbcType,\n          jdbcType());\n      configuration = configuration.toBuilder().setJdbcType(jdbcType()).build();\n    }\n\n    Integer fetchSize = configuration.getFetchSize();\n    if (fetchSize != null\n        && fetchSize > 0\n        && configuration.getJdbcUrl() != null\n        && !configuration.getJdbcUrl().contains(\"useCursorFetch=true\")) {\n      throw new IllegalArgumentException(\n          \"It is required to set useCursorFetch=true\"\n              + \" in the JDBC URL when using fetchSize for MySQL\");\n    }\n    return new MySqlReadSchemaTransform(configuration);\n  }\n\n  public static class MySqlReadSchemaTransform extends JdbcReadSchemaTransform {\n    public MySqlReadSchemaTransform(JdbcReadSchemaTransformConfiguration config) {\n      super(config, MYSQL);\n    }\n  }\n}\n","sourceCodeStart":55,"sourceCodeEnd":86,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/providers/ReadFromMySqlSchemaTransformProvider.java#L55-L86","documentation":"This IllegalArgumentException is thrown by the MySQL JDBC read provider's from() configuration method in ReadFromMySqlSchemaTransformProvider. Beam's MySQL reader requires the MySQL JDBC driver's cursor-based fetch mode to actually honor a small fetchSize; without useCursorFetch=true in the JDBC URL, the driver buffers the entire result set in memory regardless of fetchSize, so the provider refuses the configuration. It is a fail-fast validation to prevent silent full-result-set materialization and OOM.","triggerScenarios":"Calling ReadFromMySqlSchemaTransformProvider.from() (or its builder read()) with a configuration where fetchSize is set to a positive integer while the jdbcUrl does not contain the substring 'useCursorFetch=true'.","commonSituations":"Developers copy a working Postgres/SQL Server JDBC URL into the MySQL read config and then set fetchSize to bound memory usage; or they set fetchSize in a pipeline sent to a Dataflow/expansion service without updating the URL parameters. Also common after upgrading Beam where fetchSize support for MySQL was added with this requirement.","solutions":["Append useCursorFetch=true to the JDBC URL, e.g. jdbc:mysql://host:3306/db?useCursorFetch=true","If you do not need bounded fetching, remove the fetchSize setting from the configuration so the check passes","Ensure the URL is passed with exact spelling useCursorFetch=true (case-sensitive substring check) and that it is the URL actually supplied to the provider, not another config layer"],"exampleFix":"// before\nJdbcReadSchemaTransformConfiguration cfg = JdbcReadSchemaTransformConfiguration.builder()\n    .setJdbcUrl(\"jdbc:mysql://host:3306/db\")\n    .setFetchSize(1000)\n    .build();\n// after\nJdbcReadSchemaTransformConfiguration cfg = JdbcReadSchemaTransformConfiguration.builder()\n    .setJdbcUrl(\"jdbc:mysql://host:3306/db?useCursorFetch=true\")\n    .setFetchSize(1000)\n    .build();","handlingStrategy":"validation","validationCode":"if (fetchSize != null && fetchSize > 0 && jdbcUrl != null && !jdbcUrl.contains(\"useCursorFetch=true\")) {\n  throw new IllegalArgumentException(\"MySQL read requires useCursorFetch=true in the JDBC URL when fetchSize is set\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always include useCursorFetch=true in MySQL JDBC URLs used with Beam","Centralize URL construction in one helper so required params are never missing","When setting fetchSize, review provider-specific JDBC URL requirements per dialect"],"tags":["jdbc","mysql","beam","configuration"],"backgroundTag":"invalid-config-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}