{"record":{"id":"83c6d4dcab83a23f","repo":"apache/beam","slug":"could-not-get-the-row-count-for-the-table","errorCode":null,"errorMessage":"Could not get the row count for the table {}","messagePattern":"Could not get the row count for the table (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryTable.java","lineNumber":259,"sourceCode":"        .withMethod(method)\n        .from(bqLocation)\n        .withCoder(SchemaCoder.of(schema));\n  }\n\n  private static BeamTableStatistics getRowCountFromBQ(PipelineOptions o, String bqLocation) {\n    try {\n      BigInteger rowCount =\n          BigQueryHelpers.getNumRows(\n              o.as(BigQueryOptions.class), BigQueryHelpers.parseTableSpec(bqLocation));\n\n      if (rowCount == null) {\n        return BeamTableStatistics.BOUNDED_UNKNOWN;\n      }\n\n      return BeamTableStatistics.createBoundedTableStatistics(rowCount.doubleValue());\n\n    } catch (IOException | InterruptedException e) {\n      LOG.warn(\"Could not get the row count for the table {}\", bqLocation, e);\n    }\n\n    return BeamTableStatistics.BOUNDED_UNKNOWN;\n  }\n\n  public static class InvalidPropertyException extends UnsupportedOperationException {\n    private InvalidPropertyException(String s) {\n      super(s);\n    }\n  }\n}\n","sourceCodeStart":241,"sourceCodeEnd":271,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BigQueryTable.java#L241-L271","documentation":"BigQueryTable queries BigQuery for the table's row count to produce Calcite table statistics. If that lookup fails with IOException or InterruptedException, the error is logged and BOUNDED_UNKNOWN statistics are returned instead. Query planning still proceeds; only the optimizer's row-count estimate is degraded.","triggerScenarios":"Calling getRowCountFromBQ (via getTableStatistics) when the BigQuery tables.get/query API call throws IOException (network, auth, permission) or the calling thread is interrupted while waiting for the response.","commonSituations":"Missing BigQuery permissions (bigquery.tables.get / bigquery.jobs.create); network or quota issues; pipeline worker shutdown interrupting planning; misconfigured bqLocation (project:dataset.table).","solutions":["Check the logged exception cause for auth/network/permission errors and fix (credentials, IAM roles: roles/bigquery.dataViewer + jobUser).","Verify the bqLocation identifier points to an existing table.","Re-run planning without worker interruption; statistics will populate on next use.","Ignore safely if approximate statistics are acceptable — behavior falls back to BOUNDED_UNKNOWN."],"exampleFix":"// before\n} catch (IOException | InterruptedException e) {\n  LOG.warn(\"Could not get the row count for the table {}\", bqLocation, e);\n}\n// after\n// No change needed by caller; ensure permissions via gcloud:\n// gcloud projects add-iam-policy-binding PROJECT --member=user:ME --role=roles/bigquery.jobUser","handlingStrategy":"fallback","validationCode":"// Verify table access before planning:\nbq.getTable(TableId.from(project, dataset, table)) // throws on missing permission/table","typeGuard":null,"tryCatchPattern":"try {\n  BeamTableStatistics stats = bigQueryTable.getTableStatistics();\n} catch (Exception e) {\n  // library already falls back to BOUNDED_UNKNOWN; optionally supply own estimate\n  stats = BeamTableStatistics.BOUNDED_UNKNOWN;\n}","preventionTips":["Grant bigquery.dataViewer and bigquery.jobUser roles to pipeline service accounts","Validate bqLocation strings (project:dataset.table) before planning","Avoid interrupting workers during SQL planning","Accept BOUNDED_UNKNOWN when exact stats aren't needed"],"tags":["bigquery","sql","statistics","permissions"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}