{"record":{"id":"7285d4904af65da8","repo":"apache/beam","slug":"failed-to-convert-the-row-to-json","errorCode":null,"errorMessage":"Failed to convert the row to JSON","messagePattern":"Failed to convert the row to JSON","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java","lineNumber":1164,"sourceCode":"\n        int strideIndex = 0;\n        // Upload in batches.\n        List<TableDataInsertAllRequest.Rows> rows = new ArrayList<>();\n        long dataSize = 0L;\n\n        List<Future<List<TableDataInsertAllResponse.InsertErrors>>> futures = new ArrayList<>();\n        List<Integer> strideIndices = new ArrayList<>();\n        // Store the longest throttled time across all parallel threads\n        final AtomicLong maxThrottlingMsec = new AtomicLong();\n\n        int rowIndex = 0;\n        while (rowIndex < rowsToPublish.size()) {\n          TableRow row = rowsToPublish.get(rowIndex).getValue();\n          long nextRowSize = 0L;\n          try {\n            nextRowSize = TableRowJsonCoder.of().getEncodedElementByteSize(row);\n          } catch (Exception ex) {\n            throw new RuntimeException(\"Failed to convert the row to JSON\", ex);\n          }\n\n          // The following scenario must be *extremely* rare.\n          // If this row's encoding by itself is larger than the maximum row payload, then it's\n          // impossible to insert into BigQuery, and so we send it out through the dead-letter\n          // queue.\n          if (nextRowSize >= MAX_BQ_ROW_PAYLOAD_BYTES) {\n            InsertErrors error =\n                new InsertErrors()\n                    .setErrors(ImmutableList.of(new ErrorProto().setReason(\"row-too-large\")));\n            // We verify whether the retryPolicy parameter expects us to retry. If it does, then\n            // it will return true. Otherwise it will return false.\n            if (retryPolicy.shouldRetry(new InsertRetryPolicy.Context(error))) {\n              // Obtain table schema\n              TableSchema tableSchema = null;\n              try {\n                String tableSpec = BigQueryHelpers.toTableSpec(ref);\n                if (tableSchemaCache.containsKey(tableSpec)) {","sourceCodeStart":1146,"sourceCodeEnd":1182,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L1146-L1182","documentation":"DatasetServiceImpl measures each TableRow's encoded JSON size using TableRowJsonCoder before batching it for insertAll. If encoding the row throws, the code wraps the exception in a RuntimeException with this message. It means the row cannot be serialized to the JSON representation BigQuery streaming insert requires.","triggerScenarios":"A TableRow contains values that TableRowJsonCoder cannot encode (e.g. unsupported nested value types, non-serializable objects placed in the row map, or a corrupted row produced upstream).","commonSituations":"User DoFns putting arbitrary Java objects (BigDecimal variants, byte arrays, custom POJOs) into TableRows instead of supported JSON-compatible values; schema/row construction bugs in dynamic destinations.","solutions":["Inspect getCause() to see which field/value fails to encode","Ensure all row values are JSON-compatible (String, Long, Double, Boolean, List, Map, or nested TableRow)","Convert custom types explicitly (e.g. use String or Double representations) before building the TableRow","Use TableRowJsonCoder.of().getEncodedElementByteSize(row) in a unit test to reproduce the failing row"],"exampleFix":"// before\nrow.set(\"payload\", myCustomObject);\n// after\nrow.set(\"payload\", objectMapper.writeValueAsString(myCustomObject));","handlingStrategy":"validation","validationCode":"// validate rows before writing\nfor (TableRow row : rows) {\n  try {\n    TableRowJsonCoder.of().getEncodedElementByteSize(row);\n  } catch (Exception e) {\n    throw new IllegalArgumentException(\"Row not JSON-encodable: \" + e.getMessage());\n  }\n}","typeGuard":"boolean isEncodable(TableRow row) {\n  return row.entrySet().stream()\n      .allMatch(e -> e.getValue() instanceof String\n          || e.getValue() instanceof Number\n          || e.getValue() instanceof Boolean\n          || e.getValue() instanceof TableRow);\n}","tryCatchPattern":"try {\n  rows.forEach(this::write);\n} catch (RuntimeException e) {\n  if (\"Failed to convert the row to JSON\".equals(e.getMessage())) {\n    // route row to DLQ, log e.getCause()\n  }\n}","preventionTips":["Only put JSON-compatible values in TableRows","Unit-test row construction with TableRowJsonCoder","Serialize custom objects to String explicitly","Avoid passing raw POJOs through row maps"],"tags":["bigquery","gcp","serialization","json"],"backgroundTag":"json-marshal-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}