{"record":{"id":"3424186a1e73e36e","repo":"apache/iceberg","slug":"metadata-file-might-have-been-modified-encryption","errorCode":null,"errorMessage":"Metadata file might have been modified. Encryption key id %s differs from HMS value %s","messagePattern":"Metadata file might have been modified\\. Encryption key id (.+?) differs from HMS value (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java","lineNumber":596,"sourceCode":"      HMSTablePropertyHelper.verifyMetadataHash(metadata, metadataHashFromHMS);\n      return;\n    }\n\n    LOG.warn(\n        \"Full metadata integrity check skipped because no metadata hash was recorded in HMS for table {}.\"\n            + \" Falling back to encryption property based check.\",\n        tableName);\n\n    Map<String, String> propertiesFromMetadata = metadata.properties();\n\n    String encryptionKeyIdFromMetadata =\n        propertiesFromMetadata.get(TableProperties.ENCRYPTION_TABLE_KEY);\n    if (!Objects.equals(encryptionKeyIdFromHMS, encryptionKeyIdFromMetadata)) {\n      String errMsg =\n          String.format(\n              \"Metadata file might have been modified. Encryption key id %s differs from HMS value %s\",\n              encryptionKeyIdFromMetadata, encryptionKeyIdFromHMS);\n      throw new RuntimeException(errMsg);\n    }\n\n    String dekLengthFromMetadata =\n        propertiesFromMetadata.get(TableProperties.ENCRYPTION_DEK_LENGTH);\n    if (!Objects.equals(dekLengthFromHMS, dekLengthFromMetadata)) {\n      String errMsg =\n          String.format(\n              \"Metadata file might have been modified. DEK length %s differs from HMS value %s\",\n              dekLengthFromMetadata, dekLengthFromHMS);\n      throw new RuntimeException(errMsg);\n    }\n  }\n\n  @VisibleForTesting\n  HiveLock lockObject(TableMetadata metadata) {\n    if (hiveLockEnabled(metadata, conf)) {\n      return new MetastoreLock(conf, metaClients, catalogName, database, tableName);\n    } else {","sourceCodeStart":578,"sourceCodeEnd":614,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java#L578-L614","documentation":"During refresh, HiveTableOperations compares the Iceberg table's encryption key id (from the metadata file properties) against the value stored in the Hive metastore table parameters. If they differ, the metadata file may have been modified or swapped outside of Iceberg's optimistic concurrency protocol, and a RuntimeException is thrown to prevent reading from untrusted metadata. This integrity check protects Iceberg's encryption key tracking from being bypassed by direct HMS edits.","triggerScenarios":"Calling any table refresh (e.g., table.refresh() or a load that triggers doRefresh) on a table where TableProperties.ENCRYPTION_TABLE_KEY in the Iceberg metadata JSON does not equal the encryption key id recorded in the HMS table parameters, typically because someone modified the metadata file or the HMS parameters out-of-band.","commonSituations":"Manual edits to HMS table parameters or metadata JSON files; concurrent or divergent writes to encryption properties from different tools; restoring an old metadata file without restoring HMS properties; using non-Iceberg tooling to mutate encrypted tables.","solutions":["Inspect the HMS table parameters and the current Iceberg metadata file; identify which side was modified and restore consistency via a proper Iceberg commit (never by hand-editing).","If a stale/mismatched metadata file was manually restored, re-commit through Iceberg so HMS metadata_location and encryption properties are rewritten together.","Audit for concurrent writers or external tools (ETL scripts, CLI tools) modifying HMS parameters directly and stop them.","If encryption properties are intentionally not used, ensure they are consistently absent or equal on both sides before refreshing."],"exampleFix":"// before: manually patched metadata file left HMS params stale\n// after: refresh through Iceberg after verifying both sides match\nTable table = catalog.loadTable(identifier);\ntable.refresh(); // fails while key id differs\n// Fix by re-applying the property through Iceberg:\ntable.updateProperties().set(TableProperties.ENCRYPTION_TABLE_KEY, correctKeyId).commit();","handlingStrategy":"validation","validationCode":"TableMetadata tm = table.operations().current();\nString keyIdInMetadata = tm.properties().get(TableProperties.ENCRYPTION_TABLE_KEY);\nString keyIdInHms = hmsTable.getParameters().get(TableProperties.ENCRYPTION_TABLE_KEY);\nif (!Objects.equals(keyIdInMetadata, keyIdInHms)) {\n  throw new IllegalStateException(\"Encryption key id mismatch between metadata and HMS; reconcile before refresh\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  table.refresh();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Encryption key id\")) {\n    // reconcile metadata vs HMS through a proper Iceberg commit, not manual edits\n  } else { throw e; }\n}","preventionTips":["Never edit HMS table parameters or Iceberg metadata files manually.","Change encryption properties only through Table.updateProperties().","Audit for external tools writing HMS parameters directly on Iceberg tables.","Keep all writers on consistent Iceberg versions and encryption configuration."],"tags":["hive-metastore","encryption","integrity-check","concurrency"],"backgroundTag":"checksum-mismatch","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}