{"record":{"id":"a722928072b809f7","repo":"apache/beam","slug":"exception-while-trying-to-retrieve-schema","errorCode":null,"errorMessage":"Exception while trying to retrieve schema","messagePattern":"Exception while trying to retrieve schema","errorType":"exception","errorClass":"BigQuerySchemaRetrievalException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableSourceDef.java","lineNumber":112,"sourceCode":"      String stepUuid,\n      Coder<T> coder,\n      SerializableFunction<TableSchema, AvroSource.DatumReaderFactory<T>> readerFactory,\n      boolean useAvroLogicalTypes) {\n    return BigQueryTableSource.create(\n        stepUuid, this, bqServices, coder, readerFactory, useAvroLogicalTypes);\n  }\n\n  /** {@inheritDoc} */\n  @Override\n  public TableSchema getTableSchema(BigQueryOptions bqOptions) {\n    try {\n      try (DatasetService datasetService = bqServices.getDatasetService(bqOptions)) {\n        TableReference tableRef = getTableReference(bqOptions);\n        Table table = datasetService.getTable(tableRef);\n        return Preconditions.checkStateNotNull(table).getSchema();\n      }\n    } catch (Exception e) {\n      throw new BigQuerySchemaRetrievalException(\"Exception while trying to retrieve schema\", e);\n    }\n  }\n}\n","sourceCodeStart":94,"sourceCodeEnd":116,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableSourceDef.java#L94-L116","documentation":"BigQueryTableSourceDef.getTableSchema retrieves the table and returns its schema. Any exception during retrieval (network, permission, missing table, null table failing checkStateNotNull) is wrapped in a BigQuerySchemaRetrievalException with message 'Exception while trying to retrieve schema' and the original cause attached. The real cause is always in getCause().","triggerScenarios":"Calling getTableSchema (directly or via schema-aware BigQueryIO reads) when the DatasetService.getTable call fails: table missing, permissions denied, transient BigQuery API errors, or a null table tripping checkStateNotNull.","commonSituations":"Service account lacks bigquery.tables.get on the dataset; table dropped/renamed between pipeline validation and execution; transient 429/5xx from BigQuery; cross-project access without proper ACLs.","solutions":["Inspect the cause chain (e.getCause()) to see the underlying BigQuery error and fix that specifically","Verify table existence and access with `bq show --schema project:dataset.table` using pipeline credentials","Add retry with backoff around schema retrieval for transient BigQuery errors","Grant roles/bigquery.metadataViewer to the runner service account on the dataset"],"exampleFix":"// before\nSchema s = tableSourceDef.getTableSchema(); // wraps any failure\n// after\ntry {\n  Schema s = tableSourceDef.getTableSchema();\n} catch (BigQuerySchemaRetrievalException e) {\n  Throwable cause = e.getCause(); // e.g. permission-denied / not-found\n  throw new IllegalStateException(\"Fix BQ schema access: \" + cause.getMessage(), e);\n}","handlingStrategy":"try-catch","validationCode":"// Pre-check retrievability before asking the source for its schema:\nBigQuery bq = BigQueryOptions.getDefaultInstance().getService();\nTable t = bq.getTable(\"proj\", \"ds\", \"tbl\");\nif (t == null || t.getSchema() == null) throw new IllegalStateException(\"Schema not retrievable\");","typeGuard":null,"tryCatchPattern":"try {\n  Schema schema = tableSourceDef.getTableSchema();\n} catch (BigQuerySchemaRetrievalException e) {\n  // always inspect the wrapped cause for the real BigQuery error\n  throw new IllegalStateException(\"Schema retrieval failed: \" + e.getCause(), e);\n}","preventionTips":["Read e.getCause() to find the true failure (not-found vs permission vs transient)","Retry with backoff for transient 429/5xx causes","Grant bigquery.tables.get via roles/bigquery.metadataViewer","Avoid reading schemas of tables subject to TTL/retention deletion"],"tags":["bigquery","schema","exception-wrapping","gcp"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}