apache/beam · error · IllegalStateException

Record at a split point has same offset as the previous…

Error message

Record at a split point has same offset as the previous split point: previous split point at %d, current record starts at %d

What it means

OffsetRangeTracker considers some offsets 'split points' (records that should be split around). tryReturnRecordAt throws IllegalStateException if two consecutive split-point records start at the same offset, which would break the defer/split accounting invariant in the range tracker.

Solutions

Fix the isAtSplitPoint computation so only true boundary records are flagged.
Deduplicate records so the same offset is never returned twice as a split point.
Review how offsets are advanced in the reader loop to ensure each offset is consumed once.
Compare with an existing Beam source (e.g.AvroIO/BoundedSource readers) for correct split-point semantics.

Example fix

// before
boolean isSplit = true; // wrong: every record flagged
tracker.tryReturnRecordAt(offset, isSplit);
// after
boolean isSplit = (offset % recordInterval == 0);
tracker.tryReturnRecordAt(offset, isSplit);

Defensive patterns

Strategy: validation

Validate before calling

boolean isSplit = isTrueSplitPoint(offset); if (isSplit && offset == lastSplitOffset) { isSplit = false; }

Try / catch

try { tracker.tryReturnRecordAt(offset, atSplit); } catch (IllegalStateException e) { LOG.error("duplicate split point at " + offset, e); throw e; }

Prevention

Only mark genuine boundary records as split points
Deduplicate offsets before returning records
Mirror split-point logic from an existing Beam source

When it happens

Trigger: Calling tryReturnRecordAt(recordStart, true) twice with identical recordStart values, i.e. two records both flagged as at a split point at the same offset; usually from a reader that marks every record as a split point or yields duplicates.

Common situations: Custom IO implementations where isAtSplitPoint is incorrectly computed (e.g. always true), or offset-based sources where the same boundary record is consumed twice.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/81fee9f94f1196b8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/range/OffsetRangeTracker.java:112

              "Trying to return record [starting at %d] which is before the start offset [%d]",
              recordStart, startOffset));
    }
    if (recordStart < lastRecordStart) {
      throw new IllegalStateException(
          String.format(
              "Trying to return record [starting at %d] "
                  + "which is before the last-returned record [starting at %d]",
              recordStart, lastRecordStart));
    }

    if (lastRecordStart == -1) {
      startOffset = recordStart;
    }
    lastRecordStart = recordStart;

    if (isAtSplitPoint) {
      if (recordStart == offsetOfLastSplitPoint) {
        throw new IllegalStateException(
            String.format(
                "Record at a split point has same offset as the previous split point: "
                    + "previous split point at %d, current record starts at %d",
                offsetOfLastSplitPoint, recordStart));
      }
      if (recordStart >= stopOffset) {
        done = true;
        return false;
      }
      offsetOfLastSplitPoint = recordStart;
      ++splitPointsSeen;
    }

    return true;
  }

  @Override
  public boolean trySplitAtPosition(Long splitOffset) {

View on GitHub (pinned to 12126d8942)