apache/beam · error · RuntimeException

Failed to convert BSON Document to Beam Row: " +…

Error message

Failed to convert BSON Document to Beam Row: " + doc.toJson()

What it means

In MongoDbReadSchemaTransformProvider's read DoFn, each BSON Document is converted to a Beam Row via MongoDbUtils.toRow. If conversion throws and error handling (handleErrors) is disabled, the DoFn wraps the failure in a RuntimeException that includes the document's JSON for debugging.

Solutions

Fix the configured schema to match the actual BSON field types in the collection
Enable error handling (handleErrors true) so bad documents are routed to the error output instead of failing the pipeline
Pre-clean/transform the collection data or filter incompatible documents before the read transform

Example fix

// before
MongoDb.read(schemaConfig).withHandleErrors(false);
// after
MongoDb.read(schemaConfig).withHandleErrors(true); // bad docs go to error output
// or fix schema so doc types match:
// schema field "age" INT64 but docs store STRING -> change schema to STRING or migrate data

Defensive patterns

Strategy: validation

Validate before calling

// before running, sample documents and validate against schema
for (Document doc : sampleDocs) {
  try {
    MongoDbUtils.toRow(doc, schema);
  } catch (Exception e) {
    throw new IllegalStateException("Schema mismatch: " + doc.toJson(), e);
  }
}

Try / catch

try {
  Row row = MongoDbUtils.toRow(doc, schema);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Failed to convert BSON Document to Beam Row")) {
    // enable handleErrors or quarantine document by JSON
  }
}

Prevention

Enable handleErrors so bad documents flow to the error output instead of failing the pipeline
Sample and validate collection documents against the configured schema before deployment
Watch for BSON type drift in MongoDB collections (fields changing type over time)

When it happens

Trigger: A Document from MongoDB whose BSON shape does not match the configured schema (e.g., mismatched field types, unsupported BSON types) passed to toRow while handleErrors is false.

Common situations: MongoDB collections with heterogeneous documents or fields that changed type over time; schema configured in the transform that doesn't match live collection data.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/746c7a4ed8692a2c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbReadSchemaTransformProvider.java:137

  /** Converts a MongoDB BSON {@link Document} to a Beam {@link Row}. */
  static class DocumentToRowFn extends DoFn<Document, Row> {
    private final Schema schema;
    private final boolean handleErrors;
    private final Schema errorSchema;

    DocumentToRowFn(Schema schema, boolean handleErrors, Schema errorSchema) {
      this.schema = schema;
      this.handleErrors = handleErrors;
      this.errorSchema = errorSchema;
    }

    @ProcessElement
    public void processElement(@Element Document doc, MultiOutputReceiver receiver) {
      try {
        receiver.get(OUTPUT_TAG).output(MongoDbUtils.toRow(doc, schema));
      } catch (Exception e) {
        if (!handleErrors) {
          throw new RuntimeException(
              "Failed to convert BSON Document to Beam Row: " + doc.toJson(), e);
        }
        errorCounter.inc();
        byte[] docBytes;
        try {
          docBytes = doc.toJson().getBytes(java.nio.charset.StandardCharsets.UTF_8);
        } catch (Exception jsonEx) {
          docBytes = doc.toString().getBytes(java.nio.charset.StandardCharsets.UTF_8);
        }
        receiver.get(ERROR_TAG).output(ErrorHandling.errorRecord(errorSchema, docBytes, e));
      }
    }
  }
}

View on GitHub (pinned to 12126d8942)