{"record":{"id":"615d9c3ea2b72836","repo":"apache/beam","slug":"length-of-schema-field-field-getname-data-exceeds-database","errorCode":null,"errorMessage":"Length of Schema.Field[${field.getName()}] data exceeds database column capacity","messagePattern":"Length of Schema\\.Field\\[(.+?)\\] data exceeds database column capacity","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java","lineNumber":423,"sourceCode":"      }\n      ps.setString(i + 1, value);\n    };\n  }\n\n  private static void validateLogicalTypeLength(Schema.Field field, Integer length) {\n    try {\n      if (!field.getType().getTypeName().isLogicalType()) {\n        return;\n      }\n\n      Integer maxLimit =\n          (Integer) checkArgumentNotNull(field.getType().getLogicalType()).getArgument();\n      if (maxLimit == null) {\n        return;\n      }\n\n      if (length > maxLimit) {\n        throw new RuntimeException(\n            String.format(\n                \"Length of Schema.Field[%s] data exceeds database column capacity\",\n                field.getName()));\n      }\n    } catch (NumberFormatException e) {\n      // if argument is not set or not integer then do nothing and proceed with the insertion\n    }\n  }\n\n  private static Calendar getDateOrTimeOnly(DateTime dateTime, boolean wantDateOnly) {\n    Calendar cal = Calendar.getInstance();\n    cal.setTimeZone(TimeZone.getTimeZone(dateTime.getZone().getID()));\n\n    if (wantDateOnly) { // return date only\n      cal.set(Calendar.YEAR, dateTime.getYear());\n      cal.set(Calendar.MONTH, dateTime.getMonthOfYear() - 1);\n      cal.set(Calendar.DATE, dateTime.getDayOfMonth());\n","sourceCodeStart":405,"sourceCodeEnd":441,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java#L405-L441","documentation":"Pre-write validation in JdbcUtil that checks whether a row field's data length exceeds the declared maximum capacity (from the field's logical type argument, e.g. VARCHAR/CHAR/DECIMAL max length). It throws a RuntimeException before any SQL is executed when length > maxLimit, preventing a driver-level overflow error.","triggerScenarios":"Using JdbcIo.write with a schema whose field has a logical type with an integer argument (max length, e.g. VARCHAR(n) or DECIMAL(p,s)) and supplying a value whose length (string chars or numeric precision) exceeds that argument.","commonSituations":"User declares VARCHAR(10) in the Beam schema but feeds strings of 50 chars; mapping an existing Avro/Beam schema with smaller limits than the data; after someone lowered the logical-type argument in code.","solutions":["Increase the logical type argument (e.g. VARCHAR length) in the schema to cover the actual data","Truncate or validate the data upstream before the JdbcIO write","Remove the length argument from the logical type if the DB column is unlimited (validation is skipped when maxLimit is null)","Align the Beam schema with the real table DDL via SchemaUtil.toBeamSchema on read side"],"exampleFix":"// before\nFieldType t = FieldType.STRING.withLogicalType(Records.sqlTypeToLogicalType(\"VARCHAR\", 10));\n// after\nFieldType t = FieldType.STRING.withLogicalType(Records.sqlTypeToLogicalType(\"VARCHAR\", 255));","handlingStrategy":"validation","validationCode":"boolean validateLengths(Row row, Schema schema) {\n  for (int i = 0; i < schema.getFieldCount(); i++) {\n    Schema.Field f = schema.getField(i);\n    if (f.getType().getLogicalType() == null) continue;\n    Object arg = f.getType().getLogicalType().getArgument();\n    if (!(arg instanceof Integer)) continue;\n    int max = (Integer) arg;\n    Object v = row.getValue(i);\n    if (v instanceof String && ((String) v).length() > max) return false;\n  }\n  return true;\n}","typeGuard":null,"tryCatchPattern":"try {\n  rows.apply(JdbcIO.<Row>write()...);\n} catch (RuntimeException e) {\n  if (e.getMessage().contains(\"exceeds database column capacity\")) {\n    LOG.error(\"Row value longer than schema logical-type limit\");\n  }\n  throw e;\n}","preventionTips":["Generate the Beam schema from the database via SchemaUtil.toBeamSchema rather than hand-writing it","Keep logical-type length arguments at least as large as the widest expected value","Add a length-checking ParDo before the write for early failure"],"tags":["jdbc","beam","schema","validation"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}