apache/beam · error · IllegalStateException

Reader returned null element

Error message

Reader returned null element

What it means

Thrown when a BigQuery source reader reports start()==true (indicating an element is available) but getCurrent() returns null. This violates the BoundedSource contract and is surfaced as IllegalStateException.

Solutions

1. Retry the pipeline (often transient per-worker)
2. Check for known Beam issues and upgrade to a newer Beam version
3. Verify the source data/schema integrity in BigQuery

if (reader.start()) {
  T current = reader.getCurrent();
  if (current == null) {
    // treat as transient worker failure; runner will retry the work item
    throw new IllegalStateException("Reader returned null element");
  }
}

When it happens

Trigger: During Storage API / Avro record iteration, reader.start() succeeds but the underlying record/element is null — usually an internal reader bug or malformed source data handling.

Common situations: Rare; seen with corrupted Avro blocks, runner/source incompatibilities, or Beam bugs in specific versions.

Related errors

• Received null value for non-nullable field " +…
• A function must be provided to convert the input type into…
• BigQuery %1$s not found for table "%2$s" . Please create…
• BigQuery data contained value
• BigQuery temp location expected a valid 'gs://' path, but…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0e335451245f1dbe. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:2167

    public static <T> void readSource(
        PipelineOptions options,
        TupleTag<T> rowTag,
        MultiOutputReceiver outputReceiver,
        BoundedSource<T> streamSource,
        ErrorHandlingParseFn<T> errorHandlingParseFn,
        BadRecordRouter badRecordRouter)
        throws Exception {
      // Read all the data from the stream. In the event that this work
      // item fails and is rescheduled, the same rows will be returned in
      // the same order.
      BoundedSource.BoundedReader<T> reader = streamSource.createReader(options);

      @Nullable T current = null;
      try {
        if (reader.start()) {
          current = reader.getCurrent();
          if (current == null) {
            throw new IllegalStateException("Reader returned null element");
          }
        } else {
          return;
        }
      } catch (ParseException e) {
        GenericRecord record = errorHandlingParseFn.getSchemaAndRecord().getRecord();
        badRecordRouter.route(
            outputReceiver,
            record,
            AvroCoder.of(record.getSchema()),
            (Exception) e.getCause(),
            "Unable to parse record reading from BigQuery");
      }
      if (current != null) {
        outputReceiver.get(rowTag).output(current);
      }

      while (true) {

View on GitHub (pinned to 12126d8942)