apache/beam · error · RuntimeException

Unknown kinesis failure, when trying to reach kinesis

Error message

Unknown kinesis failure, when trying to reach kinesis

What it means

The final catch-all in wrapExceptions wraps any Exception not previously classified (not an ExpiredIteratorException, limit/throughput, SdkServiceException or SdkClientException) into a RuntimeException with 'Unknown kinesis failure, when trying to reach kinesis'. It is a last-resort guard so callers always get a Kinesis-related wrapper exception.

Solutions

1. Inspect the cause chain (the wrapped exception) to find the real root cause — this message itself is not diagnostic.
2. Validate inputs (stream name, shard id, iterator) before calling SimplifiedKinesisClient methods.
3. If caused by a library bug, report to Apache Beam with the full stack trace.

// validate before calling
if (streamName == null || streamName.isEmpty()) throw new IllegalArgumentException("streamName required");
if (shardId == null || shardId.isEmpty()) throw new IllegalArgumentException("shardId required");

catch (RuntimeException e) {
  log.error("Kinesis call failed", e.getCause()); // inspect the wrapped cause
}

When it happens

Trigger: Any unchecked or unexpected exception thrown inside getShardIterator, listShardsFollowingClosedShard, getRecords or getBacklogBytes — e.g. NPEs, IllegalArgumentException from argument validation, or a third-party exception escaping the AWS SDK.

Common situations: Passing invalid stream names or iterator arguments; bugs in custom watermark/client configuration; unexpected runtime errors during client invocation that the AWS SDK lets propagate.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• A transform with label
• bad type
• Cannot access the output of an error handler until it has…
• Database operation failed
• Executor service is taking long time to shutdown, will…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a07464b6eec09476. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/SimplifiedKinesisClient.java:241

      return callable.call();
    } catch (ExpiredIteratorException e) {
      throw e;
    } catch (LimitExceededException | ProvisionedThroughputExceededException e) {
      throw new KinesisClientThrottledException(
          "Too many requests to Kinesis. Wait some time and retry.", e);
    } catch (SdkServiceException e) {
      if (e.isThrottlingException()
          || SdkDefaultRetrySetting.RETRYABLE_STATUS_CODES.contains(e.statusCode())) {
        throw new TransientKinesisException("Kinesis backend failed. Wait some time and retry.", e);
      }
      throw e; // others, such as 4xx, are not retryable
    } catch (SdkClientException e) {
      if (SdkDefaultRetrySetting.RETRYABLE_EXCEPTIONS.contains(e.getClass())) {
        throw new TransientKinesisException("Retryable failure", e);
      }
      throw e;
    } catch (Exception e) {
      throw new RuntimeException("Unknown kinesis failure, when trying to reach kinesis", e);
    }
  }

  @Override
  public void close() throws Exception {
    try (AutoCloseable c1 = kinesis;
        AutoCloseable c2 = cloudWatch) {
      // nothing to do
    }
  }

  /** Memoizing supplier that closes resources appropriately. */
  private static class LazyResource<T extends AutoCloseable> implements Supplier<T>, AutoCloseable {
    private final Supplier<T> initializer;
    private volatile T resource = null;

    private LazyResource(Supplier<T> initializer) {
      this.initializer = initializer;

View on GitHub (pinned to 12126d8942)