What causes "Parse can't be used for reading as GenericRecord." in apache/beam?

Change the transform's element type: use ParquetIO. readFrom(...) with .withParseFn(parse -> MyType) instead of GenericRecord.

apache/beam · error · IllegalArgumentException

Parse can't be used for reading as GenericRecord.

Error message

Parse can't be used for reading as GenericRecord.

What it means

ParquetIO's inferCoder is invoked from expand() when reading with a parse function. If the output type is GenericRecord AND a parseFn has been set, the transform cannot both parse and emit GenericRecords — parsing necessarily produces the parseFn's output type — so it throws IllegalArgumentException telling you Parse is incompatible with GenericRecord output.

Solutions

Change the transform's element type: use ParquetIO.<MyType>readFrom(...) with .withParseFn(parse -> MyType) instead of GenericRecord.
Or drop .withParseFn(...) and read raw GenericRecords via readGenericRecords().
Ensure generics are explicit so T is the parsed type, not GenericRecord.

Example fix

// before
ParquetIO.<GenericRecord>readFrom(path).withParseFn(record -> parse(record)) // illegal
// after
ParquetIO.<MyPojo>readFrom(path).withParseFn(record -> parse(record))

Defensive patterns

Strategy: validation

Validate before calling

if (parseFn != null && elementClass.equals(GenericRecord.class)) throw new IllegalStateException("withParseFn requires a non-GenericRecord element type");

Prevention

Use readGenericRecords() when you want GenericRecord; use typed read + withParseFn otherwise.
Always specify explicit generics on ParquetIO.<T>readFrom.

When it happens

Trigger: Calling .withParseFn(...) on a ParquetIO.Read whose element type T is GenericRecord (isGenericRecordOutput() true), then expanding the transform.

Common situations: Starting from GenericRecord-based reading code and adding a parseFn without changing the element type; generics erasure hiding that T is still GenericRecord; copy-paste between readGenericRecords() and read() flows.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1967ad82d0d77f33. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java:582

    }

    /** Returns true if expected output is {@code PCollection<GenericRecord>}. */
    private boolean isGenericRecordOutput() {
      String outputType = TypeDescriptors.outputOf(getParseFn()).getType().getTypeName();
      return outputType.equals(GenericRecord.class.getTypeName());
    }

    /**
     * Identifies the {@code Coder} to be used for the output PCollection.
     *
     * <p>throws an exception if expected output is of type {@link GenericRecord}.
     *
     * @param coderRegistry the {@link org.apache.beam.sdk.Pipeline}'s CoderRegistry to identify
     *     Coder for expected output type of {@link #getParseFn()}
     */
    private Coder<T> inferCoder(CoderRegistry coderRegistry) {
      if (isGenericRecordOutput()) {
        throw new IllegalArgumentException("Parse can't be used for reading as GenericRecord.");
      }

      // Use explicitly provided coder
      if (getCoder() != null) {
        return getCoder();
      }

      // If not GenericRecord infer it from ParseFn.
      try {
        return coderRegistry.getCoder(TypeDescriptors.outputOf(getParseFn()));
      } catch (CannotProvideCoderException e) {
        throw new IllegalArgumentException(
            "Unable to infer coder for output of parseFn. Specify it explicitly using .withCoder().",
            e);
      }
    }
  }

View on GitHub (pinned to 12126d8942)