apache/beam · error · IllegalStateException

Trying to return record [starting at %d] which is before the

Error message

Trying to return record [starting at %d] which is before the start offset [%d]

What it means

OffsetRangeTracker.tryReturnRecordAt rejects any record whose start offset is before the range's startOffset. The tracker represents the half-open range [startOffset, stopOffset); a record starting before startOffset belongs to a different (already completed or never-assigned) range, and returning it would cause duplicate or out-of-bounds processing, so it throws IllegalStateException.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java:92

  }

  @Override
  public synchronized Long getStopPosition() {
    return stopOffset;
  }

  @Override
  public boolean tryReturnRecordAt(boolean isAtSplitPoint, Long recordStart) {
    return tryReturnRecordAt(isAtSplitPoint, recordStart.longValue());
  }

  public synchronized boolean tryReturnRecordAt(boolean isAtSplitPoint, long recordStart) {
    if (!isStarted() && !isAtSplitPoint) {
      throw new IllegalStateException(
          String.format("The first record [starting at %d] must be at a split point", recordStart));
    }
    if (recordStart < startOffset) {
      throw new IllegalStateException(
          String.format(
              "Trying to return record [starting at %d] which is before the start offset [%d]",
              recordStart, startOffset));
    }
    if (recordStart < lastRecordStart) {
      throw new IllegalStateException(
          String.format(
              "Trying to return record [starting at %d] "
                  + "which is before the last-returned record [starting at %d]",
              recordStart, lastRecordStart));
    }

    if (lastRecordStart == -1) {
      startOffset = recordStart;
    }
    lastRecordStart = recordStart;

    if (isAtSplitPoint) {

View on GitHub (pinned to 12126d8942)

Solutions

Ensure the reader skips any record whose start is before startOffset and only claims records beginning at or after it.
Align split points with record boundaries so a claimed record's start always equals or exceeds startOffset.
Fix checkpoint persistence so the resumed startOffset matches the record boundary the reader actually resumes at.

Example fix

// before
tracker.tryReturnRecordAt(isAtSplitPoint, recordStart); // may be < startOffset

// after
if (recordStart >= tracker.startOffset()) {
  tracker.tryReturnRecordAt(isAtSplitPoint, recordStart);
} // else skip: record belongs to previous range

Defensive patterns

Strategy: validation

Validate before calling

if (recordStart < tracker.startOffset()) { return true; /* skip record belonging to previous range */ }

Try / catch

try {
  tracker.tryReturnRecordAt(isAtSplitPoint, recordStart);
} catch (IllegalStateException e) {
  // drop the record or resynchronize the reader at startOffset
}

Prevention

Skip records whose start offset precedes the tracker's startOffset instead of claiming them.
Persist checkpoints with record-boundary-accurate offsets.
Handle reshard/resume so the reader never re-emits records from before its assigned range.

When it happens

Trigger: A reader calls tryReturnRecordAt with recordStart < startOffset — e.g. after resuming from a checkpoint the reader re-reads a record that began before the resumed offset, or split positions were computed on different boundaries than record boundaries.

Common situations: File/text sources that split at byte offsets but whose reader re-reads the last record overlapping the previous range and returns it without adjusting recordStart; miscomputed checkpoint offsets stored in CheckpointMarks; reading a shard with a stale/incorrect start offset after autoscaling resharding.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2292919a60327e39. Report an issue: GitHub.