apache/beam · warning

SpannerIO.ReadAll( ) is being applied to an unbounded…

Error message

SpannerIO.ReadAll({}) is being applied to an unbounded input. This is not supported and can lead to runtime failures.

What it means

SpannerIO.ReadAll.expand warns when its input PCollection of ReadOperations is UNBOUNDED. SpannerIO.ReadAll is designed for batch-style reads; applying it downstream of an unbounded source (streaming) is unsupported and the transform's watermark/truncation assumptions can break, producing runtime failures or incorrect results. The warning does not stop the pipeline.

Solutions

1. Restructure the pipeline so SpannerIO reads happen on bounded inputs, or perform point lookups per element using a DoFn with a DatabaseClient instead.
2. Use SpannerIO.read() only after windowing/triggering with an explicit bounded collection of ReadOperations if the runtime semantics allow it.
3. For streaming lookups, consider caching side inputs or using a stateful DoFn that issues direct reads.

Example fix

// before (streaming input)
unboundedKeys.apply("Read", SpannerIO.read().withTable("Users"));

// after: per-element lookup in a DoFn
@ProcessElement
public void process(ProcessContext c) {
  try (ResultSet rs = dbClient.singleUse()
      .readRow("Users", Key.of(c.element()))) {
    if (rs != null) c.output(rs.getCurrentRowAsStruct());
  }
}

if (input.isBounded() == PCollection.IsBounded.UNBOUNDED) {
  throw new IllegalStateException(
      "SpannerIO.ReadAll requires a bounded input; restructure streaming lookups into a DoFn with DatabaseClient.");
}

When it happens

Trigger: Applying SpannerIO.read() (ReadAll) to a PCollection produced by an unbounded source such as PubsubIO, KafkaIO, or a streaming BigQuery source — e.g. pcollectionOfKeysFromPubsub.apply(SpannerIO.read().withQuery(...)).

Common situations: Streaming pipelines that receive Spanner keys/queries from a message bus and try to fan them out through SpannerIO.ReadAll; users migrating a batch lookup pattern into a streaming job.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Average partition bytes size has not been initialized…
• Both query and table cannot be specified at the same time…
• Cannot find Spanner table.
• Column has array of arrays which is prohibited in Spanner.
• Could not get schema for configuration

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/05ea63e7fbdb763a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java:763

    public ReadAll withLowPriority() {
      SpannerConfig config = getSpannerConfig();
      return withSpannerConfig(config.withRpcPriority(RpcPriority.LOW));
    }

    public ReadAll withHighPriority() {
      SpannerConfig config = getSpannerConfig();
      return withSpannerConfig(config.withRpcPriority(RpcPriority.HIGH));
    }

    abstract boolean getBatching();

    @Override
    public PCollection<Struct> expand(PCollection<ReadOperation> input) {

      if (PCollection.IsBounded.UNBOUNDED == input.isBounded()) {
        // Warn that SpannerIO.ReadAll should not be used on unbounded inputs.
        LOG.warn(
            "SpannerIO.ReadAll({}) is being applied to an unbounded input. "
                + "This is not supported and can lead to runtime failures.",
            this.getName());
      }

      PTransform<PCollection<ReadOperation>, PCollection<Struct>> readTransform;
      if (getBatching()) {
        readTransform =
            BatchSpannerRead.create(getSpannerConfig(), getTransaction(), getTimestampBound());
      } else {
        readTransform =
            NaiveSpannerRead.create(getSpannerConfig(), getTransaction(), getTimestampBound());
      }
      return input
          .apply("Reshuffle", Reshuffle.viaRandomKey())
          .apply("Read from Cloud Spanner", readTransform);
    }

View on GitHub (pinned to 12126d8942)