What causes "There are no more Rows." in apache/beam?

Always guard with hasNext() before next(), or consume via a for-each/Iterator loop

apache/beam · error · IllegalStateException

There are no more Rows.

Error message

There are no more Rows.

What it means

RecordBatchRowIterator.next() checks hasNext() (currRowIndex < vectorSchemaRoot.getRowCount()) and throws IllegalStateException if no rows remain. This guards Iterator contract violations: calling next() past the end of the current batch instead of checking hasNext first.

Solutions

Always guard with hasNext() before next(), or consume via a for-each/Iterator loop
Fix loop bounds to use vectorSchemaRoot.getRowCount() / iterator.hasNext() rather than assumed sizes
Check whether upstream code stopped loading further RecordBatches early and load the next batch before continuing
Catch IllegalStateException at the boundary as a programming-bug signal and fix the caller rather than suppress it

Example fix

// before
while (true) { Row r = it.next(); } // throws at end
// after
while (it.hasNext()) { Row r = it.next(); ... }

Defensive patterns

Strategy: try-catch

Validate before calling

if (iterator.hasNext()) {
  Row r = iterator.next();
}

Type guard

java.util.Iterator<Row> it = ...;
boolean canAdvance = it.hasNext();

Try / catch

try {
  Row r = iterator.next();
} catch (IllegalStateException e) {
  // iterator exhausted: fix caller to check hasNext(); treat as bug, do not retry
  throw new AssertionError("next() called past end of Arrow batch", e);
}

Prevention

Always iterate with hasNext() or for-each instead of bare next() calls
Never assume batch row counts; derive loop bounds from getRowCount()/hasNext()
Wrap the iterator in a well-tested collector (e.g. Iterators.concat) rather than manual loops

When it happens

Trigger: Calling next() on the iterator when all rows in the current VectorSchemaRoot have been consumed; loops that call next() rowCount+1 times; iterators reused across batches without checking hasNext per batch.

Common situations: Hand-rolled row consumption without an Iterator wrapper; code that assumed more batches would be loaded before the iterator was exhausted; off-by-one in batch-size calculations.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/73242019d32be5bc. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/arrow/src/main/java/org/apache/beam/sdk/extensions/arrow/ArrowConversion.java:586

          new CachingFactory<>(
              FieldVectorListValueGetterFactory.of(vectorSchemaRoot.getFieldVectors()));
      this.currRowIndex = 0;
    }

    @Override
    public void close() {
      this.vectorSchemaRoot.close();
    }

    @Override
    public boolean hasNext() {
      return currRowIndex < vectorSchemaRoot.getRowCount();
    }

    @Override
    public Row next() {
      if (!hasNext()) {
        throw new IllegalStateException("There are no more Rows.");
      }
      Row result =
          Row.withSchema(schema)
              .withFieldValueGetters(
                  this.fieldValueGetters, this.currRowIndex, TypeDescriptor.of(Integer.class));
      this.currRowIndex += 1;
      return result;
    }
  }

  private ArrowConversion() {}

  /** Converts Arrow schema to Beam row schema. */
  public static class ArrowSchemaTranslator {

    /** Converts a supported Beam row schema to an Arrow schema. */
    public static org.apache.arrow.vector.types.pojo.Schema toArrowSchema(Schema schema) {
      return new org.apache.arrow.vector.types.pojo.Schema(

View on GitHub (pinned to 12126d8942)