{"record":{"id":"b201b8df9908a39c","repo":"apache/beam","slug":"unable-to-get-bigquery-response-after-retrying-d-times-for-b201b8","errorCode":null,"errorMessage":"Unable to get BigQuery response after retrying %d times for table (%s)","messagePattern":"Unable to get BigQuery response after retrying (.+?) times for table \\((.+?)\\)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/BigqueryClient.java","lineNumber":535,"sourceCode":"              new IOException(\"Expected valid response from create table job, but received null.\");\n        }\n      } catch (GoogleJsonResponseException e) {\n        if (e.getStatusCode() == 409) {\n          LOG.info(\n              \"Table {}:{}.{} already exists, treating as success.\",\n              projectId,\n              datasetId,\n              newTable.getTableReference().getTableId());\n          return;\n        }\n        lastException = e;\n      } catch (IOException e) {\n        // ignore and retry\n        lastException = e;\n      }\n    } while (BackOffUtils.next(sleeper, backoff));\n\n    throw new RuntimeException(\n        String.format(\n            \"Unable to get BigQuery response after retrying %d times for table (%s)\",\n            MAX_QUERY_RETRIES, newTable.getId()),\n        lastException);\n  }\n\n  /** Inserts rows to a table using a BigQuery streaming write. */\n  public void insertDataToTable(\n      String projectId, String datasetId, String tableName, List<Map<String, Object>> rows)\n      throws IOException, InterruptedException {\n    Sleeper sleeper = Sleeper.DEFAULT;\n    BackOff backoff = BackOffAdapter.toGcpBackOff(BACKOFF_FACTORY.backoff());\n    IOException lastException = null;\n    do {\n      if (lastException != null) {\n        LOG.warn(\"Retrying insert table ({}) after exception\", tableName, lastException);\n      }\n      try {","sourceCodeStart":517,"sourceCodeEnd":553,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/BigqueryClient.java#L517-L553","documentation":"createNewTable inserts a BigQuery table with bounded retries over IOExceptions. When all attempts fail, it throws 'Unable to get BigQuery response after retrying %d times for table (%s)' with newTable.getId() in the message and the last exception attached.","triggerScenarios":"bigquery.tables().insert throwing IOException on every attempt while creating the test table — persistent network errors, invalid table schema never accepted, missing permissions, or quota limits exceeded throughout the retry loop.","commonSituations":"Schema JSON in test fixtures rejected server-side repeatedly; CI without egress to Google APIs; service account lacking bigquery.tables.create; parallel ITs hammering the same project's quota.","solutions":["Read the wrapped lastException for the true cause (403 vs 400 vs network).","Validate the table schema/definition is accepted by BigQuery (test a manual insert via bq CLI).","Confirm the credential can create tables in the target dataset.","Check network/proxy access to bigquery.googleapis.com.","Check quotas and reduce concurrent table creation in test harnesses."],"exampleFix":"// before\nclient.createNewTable(datasetId, newTable); // throws after retries\n\n// after\ntry {\n  client.createNewTable(datasetId, newTable);\n} catch (RuntimeException e) {\n  throw new RuntimeException(\"Table \" + newTable.getId() + \" creation failed; check schema/permissions\", e);\n}","handlingStrategy":"validation","validationCode":"// validate schema JSON is parseable and non-empty before insert\nboolean schemaLooksValid(com.google.api.services.bigquery.model.Table t) {\n  return t != null && t.getSchema() != null\n      && t.getSchema().getFields() != null && !t.getSchema().getFields().isEmpty();\n}","typeGuard":null,"tryCatchPattern":"try {\n  client.createNewTable(datasetId, newTable);\n} catch (RuntimeException e) {\n  LOG.error(\"Table creation failed for %s; cause:\", newTable.getId(), e.getCause());\n  throw e;\n}","preventionTips":["Validate table schema fields before calling createNewTable.","Ensure the credential has bigquery.tables.create in the dataset.","Test the same schema manually with the bq CLI first.","Avoid hammering the same project with parallel table creation."],"tags":["java","gcp","bigquery","retry","table-creation"],"backgroundTag":"api-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}