{"record":{"id":"416712cc9a7a404b","repo":"apache/beam","slug":"transient-exception-occurred","errorCode":null,"errorMessage":"Transient exception occurred.","messagePattern":"Transient exception occurred\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardReadersPool.java","lineNumber":187,"sourceCode":"          // Wait until all records from already closed shard are taken from the buffer and only\n          // then start reading successive shards. This guarantees that checkpoints will contain\n          // either parent or child shard and never both. Such approach allows for more\n          // straightforward checkpoint restoration than in a case when new shards are read\n          // immediately.\n          waitUntilAllShardRecordsRead(shardRecordsIterator);\n          readFromSuccessiveShards(shardRecordsIterator);\n          break;\n        }\n      } catch (KinesisClientThrottledException e) {\n        try {\n          rateLimiter.onThrottle(e);\n        } catch (InterruptedException ex) {\n          LOG.warn(\"Thread was interrupted, finishing the read loop\", ex);\n          Thread.currentThread().interrupt();\n          break;\n        }\n      } catch (TransientKinesisException e) {\n        LOG.warn(\"Transient exception occurred.\", e);\n      } catch (InterruptedException e) {\n        LOG.warn(\"Thread was interrupted, finishing the read loop\", e);\n        Thread.currentThread().interrupt();\n        break;\n      } catch (Throwable e) {\n        LOG.error(\"Unexpected exception occurred\", e);\n      }\n    }\n    LOG.info(\"Kinesis Shard read loop has finished\");\n  }\n\n  CustomOptional<KinesisRecord> nextRecord() {\n    try {\n      KinesisRecord record = recordsQueue.poll(QUEUE_POLL_TIMEOUT_MS, MILLISECONDS);\n      if (record == null) {\n        return CustomOptional.absent();\n      }\n      shardIteratorsMap.get().get(record.getShardId()).ackRecord(record);","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardReadersPool.java#L169-L205","documentation":"readLoop caught a TransientKinesisException from a Kinesis call (getRecords/iterator refresh). The exception is logged and the loop continues, retrying the read on the next iteration — the SDK classifies it as retryable. Persistent occurrences indicate ongoing Kinesis-side throttling or limit issues.","triggerScenarios":"ShardReadersPool.readLoop calls Kinesis getRecords / getShardIterator and the AWS client returns a retryable error (LimitExceededException, ProvisionedThroughputExceeded, 5xx) wrapped as TransientKinesisException.","commonSituations":"Shard hot-spots exceeding Kinesis throughput; too many concurrent consumers on one stream; transient AWS outages; shard iterator expiry scenarios.","solutions":["Monitor the frequency; occasional occurrences can be ignored due to automatic retry.","Increase stream shard count / consumer capacity if throughput errors recur.","Add or strengthen retry backoff configuration in the Kinesis IO / AWS client.","Check AWS service health and region status if errors cluster in time."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Pre-check stream capacity:\nawsCli kinesis describe-stream-summary --stream-name ... // compare shards vs consumer load","typeGuard":null,"tryCatchPattern":"try {\n  records = kinesis.getRecords(request);\n} catch (TransientKinesisException e) {\n  LOG.warn(\"Transient exception occurred.\", e); // loop retries automatically\n}","preventionTips":["Size stream shards for peak consumer throughput","Enable AWS client retry with exponential backoff","Watch AWS health dashboards during job windows","Alert on frequent TransientKinesisException rates"],"tags":["aws","kinesis","transient","retry"],"backgroundTag":"http-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}