apache/iceberg · error

Failed to commit updates to table

Error message

Failed to commit updates to table {}

What it means

BigQueryTableOperations.doCommit() commits table metadata to BigQuery metastore. If the commit status resolves to FAILURE, cleanup deletes the just-written metadata file and logs this WARN naming the table. If the status is UNKNOWN, a CommitStateUnknownException is thrown instead. A subsequent ERROR is logged if the cleanup itself fails.

Solutions

1. Check the preceding exception in logs for the root commit cause (permissions, conflict, quota)
2. Retry the commit after resolving conflicts — re-read current metadata and reapply
3. Verify the BigQuery identity has permissions on the metadata tables/dataset
4. If commit state is unknown, query the table's current metadata location before retrying to avoid divergence

// before committing, confirm identity permissions and current metadata location
TableMetadata current = ops.current();
// verify BigQuery dataset/table exists and writer IAM role is intact

try {
  ops.commit(base, metadata);
} catch (CommitStateUnknownException e) {
  // refresh table and check whether the commit actually landed before retrying
  ops.refresh();
  if (!metadataFileApplied(ops.current(), metadata)) { /* safe retry */ }
}

When it happens

Trigger: A commit to BigQuery fails definitively (e.g. the metadata table row was concurrently updated, permission denied, or BigQuery API error) so the new metadata location file must be rolled back.

Common situations: Concurrent writers causing optimistic-concurrency conflicts; BigQuery IAM changes removing write access; transient BigQuery API outages during commits.

Related errors

• Creating BigQuery client failed
• Creating BigQuery client failed due to a security issue
• Failed to commit
• Failed to create impersonated credentials for
• Failed to get application default credentials

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/099f78513d59c410. Report an issue: GitHub.

Appendix: source

Thrown at bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryTableOperations.java:117

      throw e;
    } catch (Throwable e) {
      LOG.error("Exception thrown on commit: ", e);
      if (e instanceof AlreadyExistsException) {
        throw e;
      }
      commitStatus =
          BaseMetastoreOperations.CommitStatus.valueOf(
              checkCommitStatus(newMetadataLocation, metadata).name());
      if (commitStatus == BaseMetastoreOperations.CommitStatus.FAILURE) {
        throw new CommitFailedException(e, "Failed to commit");
      }
      if (commitStatus == BaseMetastoreOperations.CommitStatus.UNKNOWN) {
        throw new CommitStateUnknownException(e);
      }
    } finally {
      try {
        if (commitStatus == BaseMetastoreOperations.CommitStatus.FAILURE) {
          LOG.warn("Failed to commit updates to table {}", tableName());
          io().deleteFile(newMetadataLocation);
        }
      } catch (RuntimeException e) {
        LOG.error(
            "Failed to cleanup metadata file at {} for table {}",
            newMetadataLocation,
            tableName(),
            e);
      }
    }
  }

  @Override
  public String tableName() {
    return String.format("%s.%s", tableReference.getDatasetId(), tableReference.getTableId());
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)