apache/beam · error · IllegalStateException

Trying to return record

Error message

Trying to return record [starting at %d] which is before the last-returned record [starting at %d]

What it means

OffsetRangeTracker tracks progress through an offset range for an IO source. tryReturnRecordAt enforces monotonic progress: a record's start offset must never go backwards. The library throws IllegalStateException when a record is returned at an offset earlier than the previously returned record, because this would indicate a broken source producing out-of-order records.

Solutions

1. Verify the source reader restores its offset correctly in start() after splitAt/resume (lastRecordStart, not startOffset, must be respected).
2. Ensure records are yielded in strictly increasing start-offset order; drop or skip duplicates before calling tryReturnRecordAt.
3. If the reader was re-created, recompute the correct resume offset instead of re-reading already-returned records.
4. Check for split logic that assigns overlapping ranges, causing re-emission of earlier records.

Example fix

// before
tracker.tryReturnRecordAt(record.getStartOffset(), record.isAtSplitPoint());
// after
if (record.getStartOffset() >= lastReturnedStart) {
  tracker.tryReturnRecordAt(record.getStartOffset(), record.isAtSplitPoint());
}

if (recordStart < tracker.lastRecordStart) { skipOrLog(recordStart); return; }

try { tracker.tryReturnRecordAt(offset, atSplit); } catch (IllegalStateException e) { LOG.warn("out-of-order record at offset " + offset, e); }

When it happens

Trigger: Calling tryReturnRecordAt(recordStart, isAtSplitPoint) with a recordStart value strictly less than the lastRecordStart recorded by the previous call; typically caused by a custom Source/reader restarting from a wrong offset or yielding records out of order.

Common situations: Custom Beam IO implementations that restore readers from a stale checkpoint, offset math bugs after range splitting, or readers that yield the same record twice during resumption after a worker failure.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Record at a split point has same offset as the previous…
• A BigQuery table or a query must be specified
• A cluster_identifier should be Optional[Union[str…
• A context manager constructor (not a fully constructed…
• A function must be provided to convert the input type into…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/11646e792e7d7690. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java:98

  @Override
  public boolean tryReturnRecordAt(boolean isAtSplitPoint, Long recordStart) {
    return tryReturnRecordAt(isAtSplitPoint, recordStart.longValue());
  }

  public synchronized boolean tryReturnRecordAt(boolean isAtSplitPoint, long recordStart) {
    if (!isStarted() && !isAtSplitPoint) {
      throw new IllegalStateException(
          String.format("The first record [starting at %d] must be at a split point", recordStart));
    }
    if (recordStart < startOffset) {
      throw new IllegalStateException(
          String.format(
              "Trying to return record [starting at %d] which is before the start offset [%d]",
              recordStart, startOffset));
    }
    if (recordStart < lastRecordStart) {
      throw new IllegalStateException(
          String.format(
              "Trying to return record [starting at %d] "
                  + "which is before the last-returned record [starting at %d]",
              recordStart, lastRecordStart));
    }

    if (lastRecordStart == -1) {
      startOffset = recordStart;
    }
    lastRecordStart = recordStart;

    if (isAtSplitPoint) {
      if (recordStart == offsetOfLastSplitPoint) {
        throw new IllegalStateException(
            String.format(
                "Record at a split point has same offset as the previous split point: "
                    + "previous split point at %d, current record starts at %d",
                offsetOfLastSplitPoint, recordStart));

View on GitHub (pinned to 12126d8942)