{"record":{"id":"05ea63e7fbdb763a","repo":"apache/beam","slug":"spannerio-readall-is-being-applied-to-an-unbounded-input","errorCode":null,"errorMessage":"SpannerIO.ReadAll({}) is being applied to an unbounded input. This is not supported and can lead to runtime failures.","messagePattern":"SpannerIO\\.ReadAll\\((.+?)\\) is being applied to an unbounded input\\. This is not supported and can lead to runtime failures\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java","lineNumber":763,"sourceCode":"\n    public ReadAll withLowPriority() {\n      SpannerConfig config = getSpannerConfig();\n      return withSpannerConfig(config.withRpcPriority(RpcPriority.LOW));\n    }\n\n    public ReadAll withHighPriority() {\n      SpannerConfig config = getSpannerConfig();\n      return withSpannerConfig(config.withRpcPriority(RpcPriority.HIGH));\n    }\n\n    abstract boolean getBatching();\n\n    @Override\n    public PCollection<Struct> expand(PCollection<ReadOperation> input) {\n\n      if (PCollection.IsBounded.UNBOUNDED == input.isBounded()) {\n        // Warn that SpannerIO.ReadAll should not be used on unbounded inputs.\n        LOG.warn(\n            \"SpannerIO.ReadAll({}) is being applied to an unbounded input. \"\n                + \"This is not supported and can lead to runtime failures.\",\n            this.getName());\n      }\n\n      PTransform<PCollection<ReadOperation>, PCollection<Struct>> readTransform;\n      if (getBatching()) {\n        readTransform =\n            BatchSpannerRead.create(getSpannerConfig(), getTransaction(), getTimestampBound());\n      } else {\n        readTransform =\n            NaiveSpannerRead.create(getSpannerConfig(), getTransaction(), getTimestampBound());\n      }\n      return input\n          .apply(\"Reshuffle\", Reshuffle.viaRandomKey())\n          .apply(\"Read from Cloud Spanner\", readTransform);\n    }\n","sourceCodeStart":745,"sourceCodeEnd":781,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java#L745-L781","documentation":"SpannerIO.ReadAll.expand warns when its input PCollection of ReadOperations is UNBOUNDED. SpannerIO.ReadAll is designed for batch-style reads; applying it downstream of an unbounded source (streaming) is unsupported and the transform's watermark/truncation assumptions can break, producing runtime failures or incorrect results. The warning does not stop the pipeline.","triggerScenarios":"Applying SpannerIO.read() (ReadAll) to a PCollection produced by an unbounded source such as PubsubIO, KafkaIO, or a streaming BigQuery source — e.g. pcollectionOfKeysFromPubsub.apply(SpannerIO.read().withQuery(...)).","commonSituations":"Streaming pipelines that receive Spanner keys/queries from a message bus and try to fan them out through SpannerIO.ReadAll; users migrating a batch lookup pattern into a streaming job.","solutions":["Restructure the pipeline so SpannerIO reads happen on bounded inputs, or perform point lookups per element using a DoFn with a DatabaseClient instead.","Use SpannerIO.read() only after windowing/triggering with an explicit bounded collection of ReadOperations if the runtime semantics allow it.","For streaming lookups, consider caching side inputs or using a stateful DoFn that issues direct reads."],"exampleFix":"// before (streaming input)\nunboundedKeys.apply(\"Read\", SpannerIO.read().withTable(\"Users\"));\n\n// after: per-element lookup in a DoFn\n@ProcessElement\npublic void process(ProcessContext c) {\n  try (ResultSet rs = dbClient.singleUse()\n      .readRow(\"Users\", Key.of(c.element()))) {\n    if (rs != null) c.output(rs.getCurrentRowAsStruct());\n  }\n}","handlingStrategy":"validation","validationCode":"if (input.isBounded() == PCollection.IsBounded.UNBOUNDED) {\n  throw new IllegalStateException(\n      \"SpannerIO.ReadAll requires a bounded input; restructure streaming lookups into a DoFn with DatabaseClient.\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only apply SpannerIO.read() downstream of batch (bounded) sources.","For streaming key/query fan-out, use a per-element lookup DoFn or side inputs.","Review pipeline topology: any PubsubIO/KafkaIO upstream makes the PCollection unbounded."],"tags":["java","spanner","streaming","unbounded-input"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}