apache/beam · error · NullPointerException

Null row schema at column

Error message

Null row schema at column 

What it means

In BigtableRowToBeamRow.columnToRow, a column whose type declares a Row schema (nested struct) must have that nested schema available. If columnType.getRowSchema() is null, a NullPointerException('Null row schema at column <name>') is thrown. The mapping declared the column as a nested row but omitted the nested Schema.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableRowToBeamRow.java:125

    // Returns Simple type, List<Simple type> or Row
    private Object columnToRow(Column column, Schema schema) {
      String columnName = column.getQualifier().toStringUtf8();
      Schema.FieldType columnType = schema.getField(columnName).getType();
      List<Cell> cells = column.getCellsList();
      switch (columnType.getTypeName()) {
        case ARRAY:
          Schema.FieldType collectionElementType = columnType.getCollectionElementType();
          if (collectionElementType != null) {
            return cells.stream()
                .map(cell -> getCellValue(cell, collectionElementType))
                .collect(toList());
          } else {
            throw new NullPointerException("Null collectionElementType at column " + columnName);
          }
        case ROW:
          @Nullable Schema rowSchema = columnType.getRowSchema();
          if (rowSchema == null) {
            throw new NullPointerException("Null row schema at column " + columnName);
          } else {
            return cellToRow(getLastCell(cells), rowSchema);
          }
        default:
          return getCellValue(getLastCell(cells), columnType);
      }
    }

    private Row familyToRow(Family family, Schema schema) {
      Map<String, Object> columns =
          family.getColumnsList().stream()
              .filter(column -> schema.hasField(column.getQualifier().toStringUtf8()))
              .map(
                  column -> {
                    String columnName = column.getQualifier().toStringUtf8();
                    return KV.of(columnName, columnToRow(column, schema));
                  })
              .collect(

View on GitHub (pinned to 12126d8942)

Solutions

1. Attach the nested schema: use Schema.FieldType.row(nestedSchema) for the column
2. If the column is not a nested structure, map it to a scalar type instead of ROW
3. Rebuild the schema from the source mapping rather than reusing a serialized schema that lost nested definitions
4. Add a pre-flight check: assert columnType.getRowSchema() != null for all ROW-typed columns before running the pipeline

Example fix

// before
Schema.FieldType colType = Schema.FieldType.row(null);
// after
Schema.FieldType colType = Schema.FieldType.row(nestedSchema);

if (columnType.getTypeName() == Schema.TypeName.ROW && columnType.getRowSchema() == null) {
  throw new IllegalArgumentException("Column " + columnName + " of type ROW needs a nested schema");
}

boolean hasRowSchema(Schema.FieldType t) {
  return t.getTypeName() != Schema.TypeName.ROW || t.getRowSchema() != null;
}

When it happens

Trigger: A Bigtable column is mapped with FieldType ROW (component type 'row' / nested struct) but no nested Schema is attached to the field type in the schema mapping.

Common situations: Deserialized schemas where nested row schemas were dropped, hand-built field types with Schema.FieldType.row(null)-like constructions, schema evolution/round-tripping through JSON that lost the nested schema.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Null collectionElementType at column
• Null value at column
• Null family schema at family
• Null column list at family
• Null value found for field that doesn't support nulls.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1af291a0561dea3d. Report an issue: GitHub.