What causes "Thread was interrupted, finishing the read loop" in apache/beam?

No fix needed if this happens at shutdown — the loop exits cleanly.

Thread was interrupted, finishing the read loop

Error message

Thread was interrupted, finishing the read loop

What it means

In ShardReadersPool.readLoop, when a KinesisClientThrottledException occurs the rate limiter's onThrottle() sleeps; if that sleep is interrupted, the thread's interrupt status is restored and the read loop is terminated gracefully. This is the shutdown path, not a failure — it usually indicates the reader thread was asked to stop while throttled.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardReadersPool.java:182

        } catch (KinesisShardClosedException e) {
          LOG.info(
              "Shard iterator for {} shard is closed, finishing the read loop",
              shardRecordsIterator.getShardId(),
              e);
          // Wait until all records from already closed shard are taken from the buffer and only
          // then start reading successive shards. This guarantees that checkpoints will contain
          // either parent or child shard and never both. Such approach allows for more
          // straightforward checkpoint restoration than in a case when new shards are read
          // immediately.
          waitUntilAllShardRecordsRead(shardRecordsIterator);
          readFromSuccessiveShards(shardRecordsIterator);
          break;
        }
      } catch (KinesisClientThrottledException e) {
        try {
          rateLimiter.onThrottle(e);
        } catch (InterruptedException ex) {
          LOG.warn("Thread was interrupted, finishing the read loop", ex);
          Thread.currentThread().interrupt();
          break;
        }
      } catch (TransientKinesisException e) {
        LOG.warn("Transient exception occurred.", e);
      } catch (InterruptedException e) {
        LOG.warn("Thread was interrupted, finishing the read loop", e);
        Thread.currentThread().interrupt();
        break;
      } catch (Throwable e) {
        LOG.error("Unexpected exception occurred", e);
      }
    }
    LOG.info("Kinesis Shard read loop has finished");
  }

  CustomOptional<KinesisRecord> nextRecord() {
    try {

View on GitHub (pinned to 12126d8942)

Solutions

No fix needed if this happens at shutdown — the loop exits cleanly.
If it happens unexpectedly, check which component interrupts the reader thread (runner checkpoint/cancel).
Reduce throttling: lower consumer parallelism or increase the rate limiter's capacity/burst settings.
Retry the pipeline; throttling is transient.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  rateLimiter.onThrottle(e);
} catch (InterruptedException ex) {
  Thread.currentThread().interrupt(); // always restore flag before exiting
  break;
}

Prevention

Configure rate limiters so throttle sleeps are short relative to shutdown time
Reduce shard contention to avoid KinesisClientThrottledException
Coordinate stop() with read loop lifecycle to avoid interrupts mid-throttle
Scale shards/consumers to stay under Kinesis quotas

When it happens

Trigger: readLoop gets throttled by Kinesis, calls rateLimiter.onThrottle(e) which blocks, and the thread is interrupted (pool stop()/shutdown or runner teardown) during that wait.

Common situations: Pipeline stoppage/drain while Kinesis is throttling the consumer; aggressive record limiter configuration causing long throttle sleeps that overlap shutdown; too many consumers sharing a shard.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9c078806cc591d1c. Report an issue: GitHub.