apache/iceberg · error · RuntimeException

Metadata file might have been modified. DEK length %s differ

Error message

Metadata file might have been modified. DEK length %s differs from HMS value %s

What it means

During refresh, HiveTableOperations verifies that the DEK (data encryption key) length recorded in the Iceberg metadata file matches the DEK length stored in HMS table parameters. A mismatch means the metadata file may have been altered outside Iceberg's commit protocol, so a RuntimeException is thrown. This prevents loading a table whose encryption configuration cannot be trusted.

Source

Thrown at hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:606

    String encryptionKeyIdFromMetadata =
        propertiesFromMetadata.get(TableProperties.ENCRYPTION_TABLE_KEY);
    if (!Objects.equals(encryptionKeyIdFromHMS, encryptionKeyIdFromMetadata)) {
      String errMsg =
          String.format(
              "Metadata file might have been modified. Encryption key id %s differs from HMS value %s",
              encryptionKeyIdFromMetadata, encryptionKeyIdFromHMS);
      throw new RuntimeException(errMsg);
    }

    String dekLengthFromMetadata =
        propertiesFromMetadata.get(TableProperties.ENCRYPTION_DEK_LENGTH);
    if (!Objects.equals(dekLengthFromHMS, dekLengthFromMetadata)) {
      String errMsg =
          String.format(
              "Metadata file might have been modified. DEK length %s differs from HMS value %s",
              dekLengthFromMetadata, dekLengthFromHMS);
      throw new RuntimeException(errMsg);
    }
  }

  @VisibleForTesting
  HiveLock lockObject(TableMetadata metadata) {
    if (hiveLockEnabled(metadata, conf)) {
      return new MetastoreLock(conf, metaClients, catalogName, database, tableName);
    } else {
      return new NoLock();
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Compare TableProperties.ENCRYPTION_DEK_LENGTH in the metadata file with the HMS table parameter value and re-align them via a proper Iceberg property commit.
Restore the correct metadata file version so it matches HMS, or update HMS through the same Iceberg commit that wrote the metadata.
Ensure all writers use the same encryption/DEK configuration; check for versions or tools that set a different DEK length.
Stop any manual mutation of HMS table parameters for Iceberg tables.

Example fix

// before: metadata says dek length 32, HMS says 16
// after: fix through Iceberg
Table table = catalog.loadTable(identifier);
table.updateProperties().set(TableProperties.ENCRYPTION_DEK_LENGTH, "32").commit();
table.refresh();

Defensive patterns

Strategy: validation

Validate before calling

String dekInMetadata = table.operations().current().properties().get(TableProperties.ENCRYPTION_DEK_LENGTH);
String dekInHms = hmsTable.getParameters().get(TableProperties.ENCRYPTION_DEK_LENGTH);
if (!Objects.equals(dekInMetadata, dekInHms)) {
  throw new IllegalStateException("DEK length mismatch between metadata and HMS");
}

Try / catch

try {
  table.refresh();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("DEK length")) {
    // re-align DEK length via updateProperties, then retry
  } else { throw e; }
}

Prevention

Use a single, consistent ENCRYPTION_DEK_LENGTH configuration across all writers.
Avoid restoring old metadata files without updating HMS in the same operation.
Set encryption properties only via Iceberg APIs.
Document and control who may modify encryption settings for the table.

When it happens

Trigger: Any refresh path (doRefresh via table refresh/load) where TableProperties.ENCRYPTION_DEK_LENGTH in the metadata JSON differs from the dekLength value in HMS parameters.

Common situations: Out-of-band edits to HMS parameters or metadata files; restoring old metadata versions; configuring DEK length differently on two writer versions or tools; migration between encryption configurations done manually.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/00bb884f160bc072. Report an issue: GitHub.