apache/beam · error · NullPointerException

Null column list at family

Error message

Null column list at family 

What it means

In BigtableRowToBeamRowFlat.setFamily, the columnsMapping maps family names to the set of column qualifiers to include. If a family encountered in the Bigtable row has no entry in columnsMapping, a NullPointerException('Null column list at family <name>') is thrown. This is an internal invariant: every family in the row must have been pre-registered in the flat mapping.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableRowToBeamRowFlat.java:94

      super(schema);
      this.columnsMapping = columnsMapping;
    }

    @Override
    public Row apply(com.google.bigtable.v2.Row bigtableRow) {
      Row.FieldValueBuilder rowBuilder =
          Row.withSchema(schema).withFieldValue(KEY, bigtableRow.getKey().toStringUtf8());

      bigtableRow.getFamiliesList().stream()
          .filter(family -> columnsMapping.containsKey(family.getName()))
          .forEach(family -> setFamily(rowBuilder, family));
      return rowBuilder.build();
    }

    private void setFamily(Row.FieldValueBuilder rowBuilder, Family family) {
      Set<String> columns = columnsMapping.get(family.getName());
      if (columns == null) {
        throw new NullPointerException("Null column list at family " + family.getName());
      } else {
        family.getColumnsList().stream()
            .filter(column -> columns.contains(column.getQualifier().toStringUtf8()))
            .forEach(column -> setColumn(rowBuilder, column));
      }
    }

    private void setColumn(Row.FieldValueBuilder rowBuilder, Column column) {
      String columnName = column.getQualifier().toStringUtf8();
      Schema.FieldType type = schema.getField(columnName).getType();
      rowBuilder.withFieldValue(columnName, getCellValue(getLastCell(column.getCellsList()), type));
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Regenerate the schema/column mapping so it includes all families present in the table (or restrict the read to the mapped families)
2. Filter the read to only the mapped families (e.g. use RowFilters on family names) so unmapped families never reach the converter
3. Check family-name casing/spelling differences between the table and the mapping
4. Guard at the apply level: skip families missing from columnsMapping instead of assuming all families are mapped

Example fix

// before
Set<String> columns = columnsMapping.get(family.getName());
if (columns == null) { throw ... }
// after
Set<String> columns = columnsMapping.get(family.getName());
if (columns == null) { return; } // ignore unmapped families

for (String family : tableFamilies) {
  if (!columnsMapping.containsKey(family)) {
    throw new IllegalArgumentException("Family " + family + " missing from columnsMapping; regenerate mapping");
  }
}

boolean isMappedFamily(Map<String, Set<String>> mapping, Family family) {
  return mapping.containsKey(family.getName());
}

When it happens

Trigger: A Bigtable row contains a family not present in the schema/column mapping used to build columnsMapping — e.g. new families added to the table after the mapping was generated, or case/name mismatches between family names and mapping keys.

Common situations: Table schema evolved (family added) while the Beam pipeline's mapping was built from an older table schema; read filtered by family but the flat converter receives unfiltered rows; typo in family name in the mapping.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Null collectionElementType at column
• Null row schema at column
• Null value at column
• Null family schema at family
• Null value found for field that doesn't support nulls.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a8d75c6b5d2058e3. Report an issue: GitHub.