apache/iceberg · error · NotAuthorizedException

Not authorized to call the BigQuery API or access this…

Error message

Not authorized to call the BigQuery API or access this resource: %s

What it means

convertExceptionIfUnsuccessful maps HTTP 401 responses from the BigQuery API to NotAuthorizedException. The BigQuery client call succeeded at the transport level but Google rejected the request because the caller is not authenticated or not authorized to access the dataset/table resource.

Solutions

1. Re-authenticate: run `gcloud auth application-default login` or refresh the service-account key used via GOOGLE_APPLICATION_CREDENTIALS
2. Grant the caller (service account or user) BigQuery IAM permissions (e.g. roles/bigquery.dataEditor) on the target dataset/project
3. Verify the configured project matches the project where the credentials are valid

Example fix

// before
// no credentials configured
BigQueryMetastoreCatalog catalog = new BigQueryMetastoreCatalog();
// after
// export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json
// grant roles/bigquery.dataEditor to the SA on the dataset
catalog.initialize("bq", conf);

// Java: verify credentials resolve before building the catalog
GoogleCredentials creds = GoogleCredentials.getApplicationDefault();
Preconditions.checkNotNull(creds, "No GCP credentials found");

try {
  Table t = catalog.loadTable(id);
} catch (NotAuthorizedException e) {
  LOG.error("BigQuery auth failed; check credentials/IAM for {}", id, e);
  throw e;
}

When it happens

Trigger: Any BigQueryMetastoreClientImpl call (internalCreate, load, delete, result, internalUpdate) where the underlying HTTP request returns status 401 — expired/missing OAuth credentials, a service account lacking BigQuery permissions on the dataset, or the default credentials not being picked up.

Common situations: Expired GOOGLE_APPLICATION_CREDENTIALS token; service account without roles/bigquery.dataOwner or dataEditor on the dataset; running locally without `gcloud auth application-default login`; wrong project configured.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors

• Failed to create impersonated credentials for
• Failed to get application default credentials
• Not authorized
• Cannot specify both and
• Creating BigQuery client failed

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/aced31f8db3b2d5c. Report an issue: GitHub.

Appendix: source

Thrown at bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryMetastoreClientImpl.java:608

  /**
   * Converts BigQuery generic API errors to Iceberg exceptions, *without* handling the
   * resource-specific exceptions like NoSuchTableException, NoSuchNamespaceException, etc.
   */
  private static HttpResponse convertExceptionIfUnsuccessful(HttpResponse response)
      throws IOException {
    if (response.isSuccessStatusCode()) {
      return response;
    }

    GoogleJsonResponseException exception =
        GoogleJsonResponseException.from(GsonFactory.getDefaultInstance(), response);
    String errorMessage =
        exception.getStatusMessage()
            + (exception.getContent() != null ? "\n" + exception.getContent() : "");

    switch (response.getStatusCode()) {
      case HttpStatusCodes.STATUS_CODE_UNAUTHORIZED ->
          throw new NotAuthorizedException(
              "Not authorized to call the BigQuery API or access this resource: %s", errorMessage);
      case HttpStatusCodes.STATUS_CODE_BAD_REQUEST -> {
        GoogleJsonError errorDetails = exception.getDetails();
        if (errorDetails != null) {
          List<GoogleJsonError.ErrorInfo> errors = errorDetails.getErrors();
          if (errors != null) {
            for (GoogleJsonError.ErrorInfo errorInfo : errors) {
              if (errorInfo.getReason().equals("resourceInUse")) {
                throw new NamespaceNotEmptyException("%s", errorInfo.getMessage());
              }
            }
          }
        }
        throw new BadRequestException("%s", errorMessage);
      }
      case HttpStatusCodes.STATUS_CODE_FORBIDDEN ->
          throw new ForbiddenException("%s", errorMessage);
      case HttpStatusCodes.STATUS_CODE_PRECONDITION_FAILED ->

View on GitHub (pinned to 86d9c8fc54)