apache/beam · error

Pool - unknown case which is likely a bug: state= seqnum=

Error message

Pool %s - unknown case which is likely a bug: state=%s seqnum=%s

What it means

During EFO (Enhanced Fan-Out) shard subscription, the subscriber's internal state machine reached an unexpected combination of subscription state and continuation sequence number. Rather than continuing with wrong assumptions about shard position, the subscriber completes its future exceptionally with an IllegalStateException, halting consumption for that shard.

Solutions

1. Upgrade to the latest Beam release for aws2 kinesis EFO, as this path marks a likely bug fixed in newer versions
2. Report/reproduce with the pool id, state and seqnum logged in the warning; check your checkpoint store for malformed or half-written sequence numbers
3. As a workaround, reset the shard's checkpoint so the subscriber starts from a clean position (e.g. TRIM_HORIZON or LATEST)
4. Verify you are not sharing/mutating the checkpoint mark across threads while EFO subscription is active

Example fix

// before: checkpoint restored with a seqnum but state machine expects none
// after: discard stale checkpoint for the shard so subscription restarts cleanly
// CopyOnExtendedInMemoryStateInternals: clear shard checkpoint or use InitialPositionInStream.LATEST
config.setInitialPosition(InitialPositionInStream.LATEST);

// Verify checkpoint sequence numbers are well-formed before restart
for (ShardCheckpoint cp : checkpoints) {
  if (cp.getSequenceNumber() != null && !ShardCheckpoint.isSentinel(cp.getSequenceNumber())) {
    Preconditions.checkState(!cp.getSequenceNumber().isEmpty(), "empty seqnum for %s", cp.getShardId());
  }
}

boolean hasValidContinuation(String state, String seqnum) {
  return ("RECEIVED".equals(state)) == (seqnum != null); // state and seqnum must agree
}

try {
  subscriber.start().get();
} catch (ExecutionException e) {
  if (e.getCause() instanceof IllegalStateException) {
    // reset shard checkpoint and resubscribe from a sentinel position
    checkpointStore.reset(shardId);
  } else { throw e; }
}

When it happens

Trigger: The internal state machine in EFOShardSubscriber has cases for states like INITIALIZING/RECEIVED with-or-without a continuation sequence number; a combination falls through to the default case in the switch. This is a library-internal invariant violation, typically surfaced after resubscribing a shard or when checkpoint/sequence number handling races with state transitions.

Common situations: Seen when a Kinesis checkpoint contains a sequence number but the subscriber state says it should not have one (or vice versa), often after restarting from a checkpoint, shard splits/merges, or upgrades between Beam versions that changed the state machine.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Attempting to add message
• Executor service is taking long time to shutdown, will…
• Executor service was not completely terminated after
• Expected input schema with a 'data' (BYTES) field, a…
• Expected state to be STARTED, but was COMPLETE_ERROR

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e48330a173bec74e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/EFOShardSubscriber.java:183

          String lastContinuationSequenceNumber = eventsSubscriber.sequenceNumber;

          // happy-path re-subscribe, subscription was complete by the SDK after 5 min
          if (error == null && state != STOPPED && lastContinuationSequenceNumber != null) {
            internalReSubscribe(lastContinuationSequenceNumber);
            return;
          }

          // shard is fully consumed - re-shard happened
          if (error == null && state != STOPPED && lastContinuationSequenceNumber == null) {
            done.complete(null);
            return;
          }

          String msg =
              String.format(
                  "Pool %s - unknown case which is likely a bug: state=%s seqnum=%s",
                  pool.getPoolId(), state, lastContinuationSequenceNumber);
          LOG.warn("{}", msg);
          done.completeExceptionally(new IllegalStateException(msg));
        };
  }

  /**
   * Subscribes to shard {@link #shardId} at starting position and automatically re-subscribes when
   * necessary using {@link #reSubscriptionHandler}.
   *
   * <p>Note:
   * <li>{@link #subscribe} may only ever be invoked once by an external caller.
   * <li>The re-subscription is hidden from the external caller. To the outside it looks this
   *     subscriber is always subscribed to the shard once {@link #subscribe} was called.
   *
   * @return {@link #done} to signal completion of this subscriber, normally (stopped or shard is
   *     completely consumed) or exceptionally due to a non retry-able error.
   */
  CompletableFuture<Void> subscribe(StartingPosition position) {
    checkState(state == INITIALIZED, "Subscriber was already started");

View on GitHub (pinned to 12126d8942)