apache/iceberg · error · ValidationException

Etag of legacy table %s is empty, manually update the table

Error message

Etag of legacy table %s is empty, manually update the table via the BigQuery API or recreate and retry

What it means

updateTable requires a non-empty etag on the loaded legacy BigQuery Metastore table to perform an etag-based conditional update. If the etag is empty the library cannot do a safe compare-and-swap, so it throws ValidationException telling you to update the table manually via the BigQuery API or recreate it.

Source

Thrown at bigquery/src/main/java/org/apache/iceberg/gcp/bigquery/BigQueryTableOperations.java:165

  }

  private void addConnectionIfProvided(Table tableBuilder, Map<String, String> metadataProperties) {
    if (metadataProperties.containsKey(TABLE_PROPERTIES_BQ_CONNECTION)) {
      tableBuilder
          .getExternalCatalogTableOptions()
          .setConnectionId(metadataProperties.get(TABLE_PROPERTIES_BQ_CONNECTION));
    }
  }

  /** Update table properties with concurrent update detection using etag. */
  private void updateTable(String newMetadataLocation, TableMetadata metadata) {
    Preconditions.checkState(
        metastoreTable != null,
        "Table %s must be loaded during refresh before commit",
        tableName());

    if (metastoreTable.getEtag().isEmpty()) {
      throw new ValidationException(
          "Etag of legacy table %s is empty, manually update the table via the BigQuery API or"
              + " recreate and retry",
          tableName());
    }
    ExternalCatalogTableOptions options = metastoreTable.getExternalCatalogTableOptions();
    addConnectionIfProvided(metastoreTable, metadata.properties());

    options.setParameters(buildTableParameters(newMetadataLocation, metadata));
    client.update(tableReference, metastoreTable);
    this.metastoreTable = null;
  }

  // To make the table queryable from Hive, the user would likely be setting the HIVE_ENGINE_ENABLED
  // parameter.
  //
  // TODO: We need to make a decision on how to make the table queryable from Hive.
  // (could be a server side change or a client side change - that's TBD).
  private Table makeNewTable(TableMetadata metadata, String metadataFileLocation) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Update the table manually through the BigQuery API to fix its state, as the message suggests, then retry the commit.
2. Drop and recreate the Iceberg table in the BigQuery Metastore so a fresh etag is populated.
3. Refresh the table (doRefresh) before committing so the latest etag is loaded.
4. Check whether another tool is stripping the etag and stop that mutation path.

// Before committing, ensure a refreshed table with a usable etag:
TableOperations ops = ((HasTableOperations) table).operations();
ops.refresh();
BigQueryMetastoreTable metastoreTable = /* loaded via ops.refresh() */;
if (metastoreTable == null || metastoreTable.getEtag() == null || metastoreTable.getEtag().isEmpty()) {
  throw new IllegalStateException("Table has no etag; fix via BigQuery API or recreate before committing");
}

try {
  table.refresh();
} catch (ValidationException e) {
  if (e.getMessage().contains("Etag")) {
    throw new IllegalStateException("Recreate or manually fix the BigQuery Metastore table: " + e.getMessage(), e);
  }
  throw e;
}

When it happens

Trigger: doCommit -> updateTable on a metastore table whose ExternalCatalogTableOptions / etag field came back empty from the BigQuery Metastore API — typically a table created outside Iceberg's normal flow or mutated by another tool.

Common situations: A legacy table migrated/created without etag populated; the table was modified by non-Iceberg tooling that cleared it; the metastore returned a degraded/incomplete table resource.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• Failed to commit
• Table %s is not a valid BigQuery Metastore Iceberg table, me
• Cannot commit file that conflicts with existing partition: %
• Found conflicting files that can contain records matching pa
• Failed to validate no appends matching %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f0e0a18684a083a0. Report an issue: GitHub.