{"record":{"id":"9a61b3acd53dcc5f","repo":"pentaho/pentaho-kettle","slug":"error-serializing-rows-of-data-to-the-monetdb-api-mapi","errorCode":null,"errorMessage":"Error serializing rows of data to the MonetDB API (MAPI).","messagePattern":"Error serializing rows of data to the MonetDB API \\(MAPI\\)\\.","errorType":"exception","errorClass":"KettleException","httpStatus":null,"severity":"error","filePath":"plugins/monet-db-bulk-loader/impl/src/main/java/org/pentaho/di/trans/steps/monetdbbulkloader/MonetDBBulkLoader.java","lineNumber":377,"sourceCode":"              break;\n            default:\n              break;\n          }\n        } else {\n          line.append( data.nullrepresentation );\n        }\n      }\n\n      // finally write a newline\n      //\n      line.append( data.newline );\n\n      // Now that we have the line, grab the content and store it in the buffer...\n      //\n      data.rowBuffer[data.bufferIndex] = line.toString(); // line.toByteArray();\n      data.bufferIndex++;\n    } catch ( Exception e ) {\n      throw new KettleException( \"Error serializing rows of data to the MonetDB API (MAPI).\", e );\n    }\n\n  }\n\n  public void truncate() throws KettleException {\n    String cmd;\n    String table = data.schemaTable;\n    String truncateStatement = meta.getDatabaseMeta().getTruncateTableStatement( null, table );\n    if ( truncateStatement == null ) {\n      throw new KettleException( \"Truncate table is not supported!\" );\n    }\n    cmd = truncateStatement + \";\";\n\n    try {\n      executeSql( cmd );\n    } catch ( Exception e ) {\n      throw new KettleException( \"Error while truncating table \" + table, e );\n    }","sourceCodeStart":359,"sourceCodeEnd":395,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/plugins/monet-db-bulk-loader/impl/src/main/java/org/pentaho/di/trans/steps/monetdbbulkloader/MonetDBBulkLoader.java#L359-L395","documentation":"addRowToBuffer serializes one incoming row into the MAPI bulk-load line format (escaping delimiters, building a line string) and stores it in data.rowBuffer. Any exception during serialization is wrapped in KettleException with this message. It indicates a row's data could not be converted into the text format MonetDB's bulk load expects.","triggerScenarios":"writeRowToMonetDB calls addRowToBuffer and the row conversion throws — typically a ValueMeta conversion failure (malformed date/number), null handling issues, or string content breaking the line format.","commonSituations":"Source fields not matching the target column types (text in a numeric column); date formats not matching the expected load format; special characters/newlines in string data.","solutions":["Inspect the wrapped cause to find which field failed to convert","Fix the field type mapping or add a Select Values step to convert types before the bulk loader","Escape or clean problematic characters (newlines, delimiters) in string columns","Check the target table's column formats and adjust date/number format settings on the step"],"exampleFix":"// before: string date into DATE column with wrong format\nrow[4] = \"12/31/2024\"\n// after: convert upstream\n// Select Values step: field 4, Date, format yyyy-MM-dd","handlingStrategy":"validation","validationCode":"// Sanitize/convert fields before the bulk loader\nfor (int i = 0; i < rowMeta.size(); i++) {\n  ValueMetaInterface vm = rowMeta.getValueMeta(i);\n  Object v = row[i];\n  if (vm.isString() && v != null && v.toString().contains(\"\\n\")) {\n    throw new IllegalArgumentException(\"Field \" + vm.getName() + \" contains newline; escape or strip it\");\n  }\n}","typeGuard":"boolean isMapiSafe(ValueMetaInterface vm, Object v) {\n  if (v == null) return true;\n  if (vm.isString()) return !v.toString().contains(\"\\n\");\n  return true;\n}","tryCatchPattern":"try {\n  writeRowToMonetDB(rowMeta, row);\n} catch (KettleException e) {\n  if (e.getMessage().contains(\"serializing rows\")) {\n    logError(\"Row conversion failed: \" + e.getCause() + \" — check field types/formats\", e);\n  }\n}","preventionTips":["Match source field types to target column types with an upstream Select Values step","Normalize date/number formats to the expected bulk load format","Strip or escape control characters and delimiters in text data","Preview the flow on sample data including nulls and edge-case strings"],"tags":["monetdb","serialization","bulk-loader","data-conversion"],"backgroundTag":"database-write-failed","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}