apache/beam · error · IllegalArgumentException

Document id field '{documentIdField}' must be set on input r

Error message

Document id field '{documentIdField}' must be set on input rows.

What it means

rowToDocument builds a Firestore Document from a Beam Row and requires the document ID to be present in the row under the configured documentIdField column. It throws IllegalArgumentException when that field is null or the empty string, because the document path cannot be constructed without an ID.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreUtils.java:83

    for (Map.Entry<String, Value> entry : document.getFieldsMap().entrySet()) {
      values.put(entry.getKey(), valueToJava(entry.getValue()));
    }
    if (documentIdField != null && schema.hasField(documentIdField)) {
      values.put(documentIdField, documentIdFromName(document.getName()));
    }
    return toRow(values, schema);
  }

  static Document rowToDocument(
      Row row,
      Schema schema,
      String projectId,
      String databaseId,
      String collectionId,
      String documentIdField) {
    String documentId = row.getString(documentIdField);
    if (documentId == null || documentId.isEmpty()) {
      throw new IllegalArgumentException(
          "Document id field '" + documentIdField + "' must be set on input rows.");
    }

    Document.Builder builder =
        Document.newBuilder()
            .setName(documentPath(projectId, databaseId, collectionId, documentId));
    for (Field field : schema.getFields()) {
      String fieldName = field.getName();
      if (fieldName.equals(documentIdField)) {
        continue;
      }
      Object fieldValue = row.getValue(fieldName);
      if (fieldValue != null) {
        builder.putFields(fieldName, javaToValue(fieldValue, field.getType()));
      }
    }
    return builder.build();
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure every input Row has a non-empty string in the documentIdField column before writing
2. Verify the documentIdField option matches the exact schema field name (case-sensitive)
3. Populate the ID field upstream with a map transform if the source doesn't provide it
4. Filter out or repair rows with null/empty IDs before the write

Example fix

// before
Row row = Row.withSchema(schema).addValues(null, "Alice").build(); // id field null
// after
Row row = Row.withSchema(schema).addValues("user42", "Alice").build(); // id populated
// or guard upstream:
rows.apply(Filter.by(r -> r.getString("id") != null && !r.getString("id").isEmpty()))

if (row.getSchema().getFieldNames().contains(documentIdField)
    && row.getString(documentIdField) != null
    && !row.getString(documentIdField).isEmpty()) {
  firestoreIo.write(row);
}

try {
  return FirestoreUtils.rowToDocument(row, projectId, databaseId, collectionId, documentIdField);
} catch (IllegalArgumentException e) {
  log.error("Row missing document id field '{}': {}", documentIdField, row); 
  throw e;
}

When it happens

Trigger: Writing Rows via the Firestore IO connector when the row's value at row.getString(documentIdField) is null or "" — e.g. the documentIdField option names a field the schema doesn't populate, or upstream transforms left it unset.

Common situations: Misconfigured documentIdField name (typo or field added after schema evolution); a source query that doesn't select the ID column; rows produced from a merge/join where the ID field is nullable and missing for some records.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Collection element type cannot be null.
• Map value type cannot be null.
• Row schema cannot be null.
• Cast isn't compatible using +validator()+: +reason
• Invalid Firestore document name: {documentName}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c379747801682ad8. Report an issue: GitHub.