{"record":{"id":"9728c5a30625964a","repo":"apache/beam","slug":"could-not-parse-csv-records-from-s-with-format-s","errorCode":null,"errorMessage":"Could not parse CSV records from %s with format %s","messagePattern":"Could not parse CSV records from (.+?) with format (.+?)","errorType":"exception","errorClass":"java.lang.IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamTableUtils.java","lineNumber":86,"sourceCode":"    if (!line.endsWith(csvFormat.getRecordSeparator())) {\n      line += csvFormat.getRecordSeparator();\n    }\n    try (CSVParser parser = CSVParser.parse(line, csvFormat)) {\n      List<Row> rows = new ArrayList<>();\n      for (CSVRecord rawRecord : parser.getRecords()) {\n        if (rawRecord.size() != schema.getFieldCount()) {\n          throw new IllegalArgumentException(\n              String.format(\n                  \"Expect %d fields, but actually %d\", schema.getFieldCount(), rawRecord.size()));\n        }\n        rows.add(\n            IntStream.range(0, schema.getFieldCount())\n                .mapToObj(idx -> autoCastField(schema.getField(idx), rawRecord.get(idx)))\n                .collect(toRow(schema)));\n      }\n      return rows;\n    } catch (IOException e) {\n      throw new IllegalArgumentException(\n          String.format(\"Could not parse CSV records from %s with format %s\", line, csvFormat), e);\n    }\n  }\n\n  public static String beamRow2CsvLine(Row row, CSVFormat csvFormat) {\n    StringWriter writer = new StringWriter();\n    try (CSVPrinter printer = csvFormat.print(writer)) {\n      for (int i = 0; i < row.getFieldCount(); i++) {\n        printer.print(row.getBaseValue(i, Object.class).toString());\n      }\n      printer.println();\n    } catch (IOException e) {\n      throw new IllegalArgumentException(\"encodeRecord failed!\", e);\n    }\n    return writer.toString();\n  }\n\n  /**","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/schema/BeamTableUtils.java#L68-L104","documentation":"BeamTableUtils.csvLines2BeamRows wraps IOExceptions from the Commons CSV parser in an IllegalArgumentException indicating the CSV records could not be parsed from the given line with the given format.","triggerScenarios":"CSVParser.parse/getRecords throwing IOException for the input line under the configured CSVFormat — malformed quoting, invalid characters relative to the format settings, or IO problems reading the parsed content.","commonSituations":"Unbalanced quotes in CSV fields; mismatched CSVFormat settings (delimiter/escape/quote) versus actual data; corrupt or binary data fed into a CSV pipeline.","solutions":["Fix malformed quoting/characters in the CSV data","Align CSVFormat settings (delimiter, quote, escape) with the actual file format","Pre-validate/sanitize lines before parsing","Catch the IllegalArgumentException upstream to log and skip bad records"],"exampleFix":"// before\nCSVFormat.DEFAULT // data contains embedded quotes\n// after\nCSVFormat.DEFAULT.withEscape('\\\\').withQuote('\"')","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { rows = BeamTableUtils.csvLines2BeamRows(line, schema, format); } catch (IllegalArgumentException e) { log.error(\"CSV parse failed: \" + line, e); }","preventionTips":["Sanitize quotes and escapes in CSV input","Keep CSVFormat settings consistent with producer format","Reject binary/corrupt lines early","Add dead-letter output for unparseable records"],"tags":["csv","parsing","io"],"backgroundTag":"invalid-argument-format","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}