apache/beam · error · RuntimeException

Null type descriptor on input.

Error message

Null type descriptor on input.

What it means

WithKeys (the schema-based variant keying a PCollection by a Row built from its schema) needs the input's TypeDescriptor to construct the keyed output coder. If the PCollection's type descriptor is null — meaning the input has a schema but no attached Java TypeDescriptor — expand() throws.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/WithKeys.java:52

import org.apache.beam.sdk.values.TypeDescriptor;

public class WithKeys<T> extends PTransform<PCollection<T>, PCollection<KV<Row, T>>> {
  private final FieldAccessDescriptor fieldAccessDescriptor;

  public static <T> WithKeys<T> of(FieldAccessDescriptor fieldAccessDescriptor) {
    return new WithKeys<>(fieldAccessDescriptor);
  }

  private WithKeys(FieldAccessDescriptor fieldAccessDescriptor) {
    this.fieldAccessDescriptor = fieldAccessDescriptor;
  }

  @Override
  public PCollection<KV<Row, T>> expand(PCollection<T> input) {
    Schema schema = input.getSchema();
    TypeDescriptor<T> typeDescriptor = input.getTypeDescriptor();
    if (typeDescriptor == null) {
      throw new RuntimeException("Null type descriptor on input.");
    }
    SerializableFunction<T, Row> toRowFunction = input.getToRowFunction();
    SerializableFunction<Row, T> fromRowFunction = input.getFromRowFunction();

    FieldAccessDescriptor resolved = fieldAccessDescriptor.resolve(schema);
    RowSelector rowSelector = new RowSelectorContainer(schema, resolved, true);
    Schema keySchema = SelectHelpers.getOutputSchema(schema, resolved);

    return input
        .apply(
            "selectKeys",
            ParDo.of(
                new DoFn<T, KV<Row, T>>() {
                  @ProcessElement
                  public void process(
                      @Element Row row, // Beam will convert the element to a row.
                      @Element T element, // Beam will return the original element.
                      OutputReceiver<KV<Row, T>> o) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Set a schema coder with a type descriptor: p.setCoder(SchemaCoder.of(schema, typeDescriptor, toRow, fromRow)).
2. Materialize rows into a concrete Java type (e.g. convert Row to a POJO/Row wrapper class) before WithKeys.
3. Use keyBy on a typed PCollection so schema inference attaches a TypeDescriptor.
4. If input comes from Beam SQL output, use a schema-aware key transform instead of WithKeys.

Example fix

// before
PCollection<Row> rows = ...; rows.apply(WithKeys.of("key"))
// after
PCollection<MyPojo> typed = rows.apply(Select.fieldnames(...)).setCoder(SchemaCoder.of(schema, TypeDescriptor.of(MyPojo.class), toRow, fromRow));
typed.apply(WithKeys.of("key"))

if (input.getTypeDescriptor() == null) { /* set a SchemaCoder with a TypeDescriptor or convert to a typed PCollection first */ }

boolean keyable(PCollection<?> p) { return p.getTypeDescriptor() != null && p.getSchema() != null; }

try { keyed = input.apply(WithKeys.of("k")); } catch (RuntimeException e) { if (e.getMessage().equals("Null type descriptor on input.")) { /* attach typed SchemaCoder */ } throw e; }

When it happens

Trigger: Applying WithKeys with field-access-based keys to a PCollection whose getTypeDescriptor() returns null, which happens for PCollections carrying a SchemaCoder without a fully-qualified encoded type descriptor (e.g. rows decoded from portable/SQL stages or rows created via Create.of(Row) with a row schema).

Common situations: Beam SQL / portable pipelines producing Row PCollections then re-entering Java schema transforms; crossing language/portability boundaries where the Java TypeDescriptor is unavailable.

Related errors

• Cannot provide a coder for a Beam Row. Please provide a sche
• Failed to decode Schema due to an error decoding Field proto
• Encountered UNSPECIFIED AtomicType
• Convert requires a schema on the input.
• Exploded field %s must be an iterable type, got %s.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fbc2ab1c2f47a4d8. Report an issue: GitHub.