apache/beam · error · NullPointerException

Null family schema at family

Error message

Null family schema at family 

What it means

In BigtableRowToBeamRow.bigtableRowToBeamRow, each Bigtable family present in the row and in the target schema must map to a nested row; schema.getField(family).getType().getRowSchema() returning null means the family field was declared without a nested row schema. A NullPointerException('Null family schema at family <name>') is thrown. The schema declares the family as a struct type but omits its fields.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableRowToBeamRow.java:166

                        if (value == null) {
                          throw new NullPointerException("Null value at column " + kv.getKey());
                        } else {
                          return value;
                        }
                      }));
      return Row.withSchema(schema).withFieldValues(columns).build();
    }

    private Row bigtableRowToBeamRow(com.google.bigtable.v2.Row bigtableRow) {
      Row.FieldValueBuilder rowBuilder =
          Row.withSchema(schema).withFieldValue(KEY, bigtableRow.getKey().toStringUtf8());
      bigtableRow.getFamiliesList().stream()
          .filter(family -> schema.hasField(family.getName()))
          .forEach(
              family -> {
                Schema familySchema = schema.getField(family.getName()).getType().getRowSchema();
                if (familySchema == null) {
                  throw new NullPointerException(
                      "Null family schema at family " + family.getName());
                } else {
                  rowBuilder.withFieldValue(family.getName(), familyToRow(family, familySchema));
                }
              });
      return rowBuilder.build();
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Attach the family's nested schema when building the field: Schema.FieldType.row(familySchema)
2. Verify the schema used in bigtableRowToBeamRow matches the one created by the schema-building helpers (BigtableRowToBeamRow.getSchema/buildBigtableRowSchema)
3. Reconstruct the schema rather than deserializing a possibly-truncated one
4. Add validation: for each family field, assert getType().getRowSchema() != null before starting the read

Example fix

// before
fields.add(Field.of(familyName, Schema.FieldType.row(null)));
// after
fields.add(Field.of(familyName, Schema.FieldType.row(familySchema)));

for (Field f : schema.getFields()) {
  if (f.getType().getTypeName() == Schema.TypeName.ROW && f.getType().getRowSchema() == null) {
    throw new IllegalArgumentException("Family field " + f.getName() + " missing nested row schema");
  }
}

boolean hasFamilySchema(Schema schema, String family) {
  return schema.getField(family).getType().getRowSchema() != null;
}

When it happens

Trigger: Building the row schema with a field per Bigtable family using Schema.FieldType.row(...) without attaching the family's nested Schema, then reading rows from that family.

Common situations: Schema constructed programmatically where the family field type is ROW but getRowSchema() was never set, schema round-tripped through serialization losing nested schemas, mapping mismatch between writer schema and reader schema versions.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Null collectionElementType at column
• Null row schema at column
• Null value at column
• Null column list at family
• Null value found for field that doesn't support nulls.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7b2b70d8a9dd92c7. Report an issue: GitHub.