{"record":{"id":"4b456c351cc05828","repo":"apache/beam","slug":"no-shard-found-for-shards","errorCode":null,"errorMessage":"No shard found for {} [shards={}]","messagePattern":"No shard found for (.+?) \\[shards=(.+?)\\]","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisIO.java","lineNumber":1230,"sourceCode":"\n      class ShardRangesImpl implements ShardRanges {\n        private static final Logger LOG = LoggerFactory.getLogger(ShardRanges.class);\n\n        private final String streamName;\n\n        private final AtomicBoolean running = new AtomicBoolean(false);\n        private NavigableSet<BigInteger> shardBounds = ImmutableSortedSet.of();\n        private Instant nextRefresh = Instant.EPOCH;\n\n        private ShardRangesImpl(String streamName) {\n          this.streamName = streamName;\n        }\n\n        @Override\n        public @Nullable BigInteger shardAwareHashKey(BigInteger hashedPartitionKey) {\n          BigInteger lowerBound = shardBounds.floor(hashedPartitionKey);\n          if (!shardBounds.isEmpty() && lowerBound == null) {\n            LOG.warn(\"No shard found for {} [shards={}]\", hashedPartitionKey, shardBounds.size());\n          }\n          return lowerBound;\n        }\n\n        @Override\n        public void refreshPeriodically(\n            KinesisAsyncClient client, Supplier<Instant> nextRefreshFn) {\n          if (nextRefresh.isBeforeNow() && running.compareAndSet(false, true)) {\n            refresh(client, nextRefreshFn, new TreeSet<>(), null);\n          }\n        }\n\n        @SuppressWarnings(\"FutureReturnValueIgnored\") // safe to ignore\n        private void refresh(\n            KinesisAsyncClient client,\n            Supplier<Instant> nextRefreshFn,\n            NavigableSet<BigInteger> bounds,\n            @Nullable String nextToken) {","sourceCodeStart":1212,"sourceCodeEnd":1248,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisIO.java#L1212-L1248","documentation":"KinesisIO's shard-aware partitioner looks up which shard bounds contain the hashed partition key. When the cached shard map is non-empty but no shard's hash range covers the hashed key, it logs this warning and returns null, meaning the record cannot be mapped to a shard and will follow the fallback routing.","triggerScenarios":"Writing to a Kinesis stream with shard-aware hashing enabled (withShardAwareRandomization / refreshPeriodically) while the cached lower hash-key bounds are stale relative to the live stream, e.g. right after resharding (split/merge) so a hash key falls into a gap between stale bounds.","commonSituations":"Streams that recently split or merged shards while the writer still holds an old shard snapshot; explicit partition keys whose hash lands outside all cached ranges during the window before the next refresh tick.","solutions":["Enable/shorten withRefreshPeriod so shard bounds refresh promptly after resharding","Ensure the stream's shards cover the full 0..2^128 hash range (a reshard in progress leaves gaps); wait or trigger a manual refresh","Reduce producers' reliance on explicit partition keys that hash into shrinking shard ranges, or use a wider key space","Upgrade Beam — shard map refresh handling has been improved in later versions"],"exampleFix":"// before\nKinesisIO.write().withStreamName(\"s\").withNumShards(10)\n// after: enable periodic shard-boundary refresh so bounds stay current\nKinesisIO.write().withStreamName(\"s\")\n  .withShardAwareRandomization(true)\n  .withRefreshPeriod(Duration.standardSeconds(30));","handlingStrategy":"fallback","validationCode":"// check stream shard coverage before writing\nDescribeStreamSummary s = client.describeStreamSummary(b -> b.streamName(stream)).streamDescriptionSummary();\nList<Shard> shards = listAllShards(client, stream);\nBigInteger min = shards.stream().map(shard -> lowerHashKey(shard)).min(BigInteger::compareTo).orElse(null);\nif (min == null || min.signum() != 0) log.warn(\"stream {} not fully covered by shards\", stream);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Configure withRefreshPeriod shorter than expected reshard frequency","Avoid resharding streams while high-throughput producers with shard-aware hashing are running","Use uniform partition keys so hashes spread across the full range","Monitor this warning as a signal that shard bounds are stale"],"tags":["kinesis","sharding","partition-key","resharding"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}