{"record":{"id":"51963f1170150856","repo":"apache/beam","slug":"unable-to-split-tablesource","errorCode":null,"errorMessage":"Unable to split TableSource","messagePattern":"Unable to split TableSource","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java","lineNumber":1819,"sourceCode":"                  getProjectionPushdownApplied(),\n                  getDirectReadPicosTimestampPrecision());\n          List<? extends BoundedSource<T>> sources;\n          try {\n            // This splitting logic taken from the SDF implementation of Read\n            long estimatedSize = source.getEstimatedSizeBytes(bqOptions);\n            // Split into pieces as close to the default desired bundle size but if that would cause\n            // too few splits then prefer to split up to the default desired number of splits.\n            long desiredChunkSize;\n            if (estimatedSize <= 0) {\n              desiredChunkSize = 64 << 20; // 64mb\n            } else {\n              // 1mb --> 1 shard; 1gb --> 32 shards; 1tb --> 1000 shards, 1pb --> 32k shards\n              desiredChunkSize =\n                  Math.max(1 << 20, (long) (1000 * Math.sqrt((double) estimatedSize)));\n            }\n            sources = source.split(desiredChunkSize, bqOptions);\n          } catch (Exception e) {\n            throw new RuntimeException(\"Unable to split TableSource\", e);\n          }\n          TupleTag<T> rowTag = new TupleTag<>();\n          PCollectionTuple resultTuple =\n              p.apply(Create.of(sources))\n                  .apply(\n                      \"Read Storage Table Source\",\n                      ParDo.of(new ReadTableSource<T>(rowTag, parseFn, getBadRecordRouter()))\n                          .withOutputTags(rowTag, TupleTagList.of(BAD_RECORD_TAG)));\n          getBadRecordErrorHandler()\n              .addErrorCollection(\n                  resultTuple\n                      .get(BAD_RECORD_TAG)\n                      .setCoder(BadRecord.getCoder(input.getPipeline())));\n\n          return resultTuple.get(rowTag).setCoder(outputCoder);\n        }\n      }\n","sourceCodeStart":1801,"sourceCodeEnd":1837,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1801-L1837","documentation":"Thrown when splitting a BigQuery Storage API TableSource into readable bounded sources fails. Beam estimates table size, computes a desired chunk size, and calls source.split(); any exception there is wrapped in this RuntimeException.","triggerScenarios":"Using BigQueryIO.read with the STORAGE_API (DIRECT_READ) method and source.split(desiredChunkSize, options) throws — typically a Storage API session creation failure.","commonSituations":"Storage Read API not enabled on the project; insufficient permissions for storage reads; transient gRPC/API errors; table too small or estimated size issues.","solutions":["Enable the BigQuery Storage Read API and grant storage read permissions","Retry — split failures are often transient gRPC errors","Fall back to the EXPORT/FILE_LOADS method if the Storage API is unavailable"],"exampleFix":"// before\nBigQueryIO.read().from(\"proj:ds.tbl\").withMethod(TypedRead.Method.DIRECT_READ);\n// after\nBigQueryIO.read().from(\"proj:ds.tbl\").withMethod(TypedRead.Method.EXPORT);","handlingStrategy":"retry","validationCode":"// Pre-check Storage API access\ntry (BigQueryReadClient client = BigQueryReadClient.create()) {\n  // a session creation against a tiny table verifies access\n} catch (Exception e) {\n  throw new IllegalStateException(\"Storage Read API unavailable: \" + e.getMessage(), e);\n}","typeGuard":null,"tryCatchPattern":"try {\n  List<BoundedSource<T>> parts = source.split(chunkSize, options);\n} catch (Exception e) {\n  // transient gRPC/API error: back off and retry, or fall back to EXPORT method\n  throw new RuntimeException(\"Unable to split TableSource\", e);\n}","preventionTips":["Enable the BigQuery Storage Read API on the project","Grant roles/bigquery.readSessionUser and dataViewer","Fall back to EXPORT method when Storage API is unavailable in the region"],"tags":["bigquery","storage-api","split","java"],"backgroundTag":"api-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}