apache/beam · error · IllegalArgumentException

Unrecognized ' ' position to create shard filter with

Error message

Unrecognized '%s' position to create shard filter with

What it means

buildShardFilterForStartingPoint in ShardListingUtils maps an InitialPositionInStream (LATEST, TRIM_HORIZON, AT_TIMESTAMP) to an AWS SDK v2 ShardFilter used by listShards. Any position value outside that switch's known cases falls into the default branch and throws this IllegalArgumentException. It effectively means the StartingPoint supplied to the Kinesis IO contains a position the shard-filter mapping does not handle.

Solutions

1. Use only InitialPositionInStream.LATEST, TRIM_HORIZON or AT_TIMESTAMP when building the StartingPoint for Kinesis IO.
2. If a new position constant was added upstream, upgrade the Beam AWS2 SDK module (or your fork) so the switch maps it to a ShardFilter type.
3. Check the serialized pipeline/options for a corrupted or foreign InitialPositionInStream value and reconfigure the consumer with a valid position.

Example fix

// before
StartingPoint sp = new StartingPoint(myCustomPosition);
// after
StartingPoint sp = new StartingPoint(InitialPositionInStream.AT_TIMESTAMP, ReadableInstant timestamp);

if (position != InitialPositionInStream.LATEST
    && position != InitialPositionInStream.TRIM_HORIZON
    && position != InitialPositionInStream.AT_TIMESTAMP) {
  throw new IllegalArgumentException("Unsupported position: " + position);
}

When it happens

Trigger: Calling ShardListingUtils.buildShardFilterForStartingPoint (indirectly via shardFilter/listShardsAtPoint) with a StartingPoint whose InitialPositionInStream is neither LATEST, TRIM_HORIZON nor AT_TIMESTAMP — typically a newly added enum constant or a custom/deserialized position value.

Common situations: Beam pipeline configured with a custom InitialPositionInStream extension; a Beam version upgrade introduced a new position constant while code (or a copied fork) still uses the old switch; deserialization of a serialized StartingPoint from an older pipeline representation yielding an unexpected enum value.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Invalid initial position in stream
• Invalid watermark policy
• A 'datagen' table requires either 'rows-per-second' (for…
• A schema was provided without a data format (or viceversa)…
• Batch size is too large! It should be smaller or equal than

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/35c842bb38de9ccd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardListingUtils.java:74

    ShardFilter shardFilter =
        buildShardFilterForStartingPoint(kinesisClient, streamName, startingPoint);
    return listShards(kinesisClient, streamName, shardFilter);
  }

  static ShardFilter buildShardFilterForStartingPoint(
      KinesisClient kinesisClient, String streamName, StartingPoint startingPoint)
      throws InterruptedException {
    InitialPositionInStream position = startingPoint.getPosition();
    switch (position) {
      case LATEST:
        return ShardFilter.builder().type(ShardFilterType.AT_LATEST).build();
      case TRIM_HORIZON:
        return ShardFilter.builder().type(ShardFilterType.AT_TRIM_HORIZON).build();
      case AT_TIMESTAMP:
        return buildShardFilterForTimestamp(
            kinesisClient, streamName, startingPoint.getTimestamp());
      default:
        throw new IllegalArgumentException(
            String.format("Unrecognized '%s' position to create shard filter with", position));
    }
  }

  private static ShardFilter buildShardFilterForTimestamp(
      KinesisClient kinesisClient, String streamName, Instant startingPointTimestamp)
      throws InterruptedException {
    StreamDescriptionSummary streamDescription = describeStreamSummary(kinesisClient, streamName);

    Instant streamCreationTimestamp = TimeUtil.toJoda(streamDescription.streamCreationTimestamp());
    if (streamCreationTimestamp.isAfter(startingPointTimestamp)) {
      return ShardFilter.builder().type(ShardFilterType.AT_TRIM_HORIZON).build();
    }

    Duration retentionPeriod = Duration.standardHours(streamDescription.retentionPeriodHours());

    Instant streamTrimHorizonTimestamp =
        Instant.now()

View on GitHub (pinned to 12126d8942)