{"record":{"id":"fbc2ab1c2f47a4d8","repo":"apache/beam","slug":"null-type-descriptor-on-input","errorCode":null,"errorMessage":"Null type descriptor on input.","messagePattern":"Null type descriptor on input\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/WithKeys.java","lineNumber":52,"sourceCode":"import org.apache.beam.sdk.values.TypeDescriptor;\n\npublic class WithKeys<T> extends PTransform<PCollection<T>, PCollection<KV<Row, T>>> {\n  private final FieldAccessDescriptor fieldAccessDescriptor;\n\n  public static <T> WithKeys<T> of(FieldAccessDescriptor fieldAccessDescriptor) {\n    return new WithKeys<>(fieldAccessDescriptor);\n  }\n\n  private WithKeys(FieldAccessDescriptor fieldAccessDescriptor) {\n    this.fieldAccessDescriptor = fieldAccessDescriptor;\n  }\n\n  @Override\n  public PCollection<KV<Row, T>> expand(PCollection<T> input) {\n    Schema schema = input.getSchema();\n    TypeDescriptor<T> typeDescriptor = input.getTypeDescriptor();\n    if (typeDescriptor == null) {\n      throw new RuntimeException(\"Null type descriptor on input.\");\n    }\n    SerializableFunction<T, Row> toRowFunction = input.getToRowFunction();\n    SerializableFunction<Row, T> fromRowFunction = input.getFromRowFunction();\n\n    FieldAccessDescriptor resolved = fieldAccessDescriptor.resolve(schema);\n    RowSelector rowSelector = new RowSelectorContainer(schema, resolved, true);\n    Schema keySchema = SelectHelpers.getOutputSchema(schema, resolved);\n\n    return input\n        .apply(\n            \"selectKeys\",\n            ParDo.of(\n                new DoFn<T, KV<Row, T>>() {\n                  @ProcessElement\n                  public void process(\n                      @Element Row row, // Beam will convert the element to a row.\n                      @Element T element, // Beam will return the original element.\n                      OutputReceiver<KV<Row, T>> o) {","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/WithKeys.java#L34-L70","documentation":"WithKeys (the schema-based variant keying a PCollection by a Row built from its schema) needs the input's TypeDescriptor to construct the keyed output coder. If the PCollection's type descriptor is null — meaning the input has a schema but no attached Java TypeDescriptor — expand() throws.","triggerScenarios":"Applying WithKeys with field-access-based keys to a PCollection whose getTypeDescriptor() returns null, which happens for PCollections carrying a SchemaCoder without a fully-qualified encoded type descriptor (e.g. rows decoded from portable/SQL stages or rows created via Create.of(Row) with a row schema).","commonSituations":"Beam SQL / portable pipelines producing Row PCollections then re-entering Java schema transforms; crossing language/portability boundaries where the Java TypeDescriptor is unavailable.","solutions":["Set a schema coder with a type descriptor: p.setCoder(SchemaCoder.of(schema, typeDescriptor, toRow, fromRow)).","Materialize rows into a concrete Java type (e.g. convert Row to a POJO/Row wrapper class) before WithKeys.","Use keyBy on a typed PCollection so schema inference attaches a TypeDescriptor.","If input comes from Beam SQL output, use a schema-aware key transform instead of WithKeys."],"exampleFix":"// before\nPCollection<Row> rows = ...; rows.apply(WithKeys.of(\"key\"))\n// after\nPCollection<MyPojo> typed = rows.apply(Select.fieldnames(...)).setCoder(SchemaCoder.of(schema, TypeDescriptor.of(MyPojo.class), toRow, fromRow));\ntyped.apply(WithKeys.of(\"key\"))","handlingStrategy":"validation","validationCode":"if (input.getTypeDescriptor() == null) { /* set a SchemaCoder with a TypeDescriptor or convert to a typed PCollection first */ }","typeGuard":"boolean keyable(PCollection<?> p) { return p.getTypeDescriptor() != null && p.getSchema() != null; }","tryCatchPattern":"try { keyed = input.apply(WithKeys.of(\"k\")); } catch (RuntimeException e) { if (e.getMessage().equals(\"Null type descriptor on input.\")) { /* attach typed SchemaCoder */ } throw e; }","preventionTips":["Keep PCollections typed with concrete Java classes before schema transforms","Avoid passing Row-typed outputs of SQL/portable stages directly to WithKeys","Set SchemaCoder with an explicit TypeDescriptor when schemas cross boundaries"],"tags":["java","apache-beam","withkeys","type-descriptor","schema"],"backgroundTag":"null-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}