{"record":{"id":"45dfea4833f4be31","repo":"apache/beam","slug":"expect-d-fields-but-actually-d","errorCode":null,"errorMessage":"Expect %d fields, but actually %d","messagePattern":"Expect (.+?) fields, but actually (.+?)","errorType":"validation","errorClass":"java.lang.IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamTableUtils.java","lineNumber":75,"sourceCode":"  /**\n   * Decode zero or more CSV records from the given string, according to the specified {@link\n   * CSVFormat}, and converts them to {@link Row Rows} with the specified {@link Schema}.\n   *\n   * <p>A single \"line\" read from e.g. {@link TextIO} can have zero or more records, depending on\n   * whether the line was split on the same characters that delimite CSV records, and whether the\n   * {@link CSVFormat} ignores blank lines.\n   */\n  public static Iterable<Row> csvLines2BeamRows(CSVFormat csvFormat, String line, Schema schema) {\n    // Empty lines can result in empty strings after Beam splits the file,\n    // which are not empty records to CSVParser unless they have a record terminator.\n    if (!line.endsWith(csvFormat.getRecordSeparator())) {\n      line += csvFormat.getRecordSeparator();\n    }\n    try (CSVParser parser = CSVParser.parse(line, csvFormat)) {\n      List<Row> rows = new ArrayList<>();\n      for (CSVRecord rawRecord : parser.getRecords()) {\n        if (rawRecord.size() != schema.getFieldCount()) {\n          throw new IllegalArgumentException(\n              String.format(\n                  \"Expect %d fields, but actually %d\", schema.getFieldCount(), rawRecord.size()));\n        }\n        rows.add(\n            IntStream.range(0, schema.getFieldCount())\n                .mapToObj(idx -> autoCastField(schema.getField(idx), rawRecord.get(idx)))\n                .collect(toRow(schema)));\n      }\n      return rows;\n    } catch (IOException e) {\n      throw new IllegalArgumentException(\n          String.format(\"Could not parse CSV records from %s with format %s\", line, csvFormat), e);\n    }\n  }\n\n  public static String beamRow2CsvLine(Row row, CSVFormat csvFormat) {\n    StringWriter writer = new StringWriter();\n    try (CSVPrinter printer = csvFormat.print(writer)) {","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamTableUtils.java#L57-L93","documentation":"When converting CSV lines to Beam Rows, BeamTableUtils validates that each parsed CSV record has exactly as many fields as the target schema. A mismatch throws IllegalArgumentException with the expected and actual counts.","triggerScenarios":"Parsing a CSV line whose number of columns differs from schema.getFieldCount() — extra or missing delimiters, ragged rows, wrong delimiter configured in CSVFormat, or a trailing separator producing an empty extra field.","commonSituations":"Malformed input files with unquoted embedded commas; users changing CSVFormat delimiter without updating the schema; header rows accidentally parsed as data (or vice versa).","solutions":["Fix the source CSV so each row has the schema's field count","Set the correct delimiter/quote/escape in the CSVFormat to match the data","Skip header rows or filter malformed lines before csvLines2BeamRows","Adjust the schema to match the actual data layout"],"exampleFix":"// before\nCSVFormat.DEFAULT // data is semicolon-separated\n// after\nCSVFormat.DEFAULT.withDelimiter(';')","handlingStrategy":"validation","validationCode":"long expected = schema.getFieldCount();\nlong actual = line.chars().filter(c -> c == ',').count() + 1;\nif (actual != expected) throw new IllegalArgumentException(\"field count mismatch\");","typeGuard":null,"tryCatchPattern":"try { rows = BeamTableUtils.csvLines2BeamRows(line, schema, format); } catch (IllegalArgumentException e) { log.warn(\"skipping malformed row\"); }","preventionTips":["Validate CSV column counts against schema at pipeline start","Match CSVFormat delimiters to the data","Configure header handling explicitly","Skip/repair ragged rows before parsing"],"tags":["csv","schema","parsing"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}