{"record":{"id":"9f9afce5944da7c3","repo":"apache/iceberg","slug":"fail-to-acquire-lock-s-to-commit-new-metadata-at","errorCode":null,"errorMessage":"Fail to acquire lock %s to commit new metadata at %s","messagePattern":"Fail to acquire lock (.+?) to commit new metadata at (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java","lineNumber":258,"sourceCode":"                      .build())\n              .build());\n      return true;\n    }\n\n    return false;\n  }\n\n  private void cleanupGlueTempTableIfNecessary(\n      boolean glueTempTableCreated, CommitStatus commitStatus) {\n    if (glueTempTableCreated && commitStatus != CommitStatus.SUCCESS) {\n      glue.deleteTable(\n          DeleteTableRequest.builder().databaseName(databaseName).name(tableName).build());\n    }\n  }\n\n  private void lock(String newMetadataLocation) {\n    if (lockManager != null && !lockManager.acquire(commitLockEntityId, newMetadataLocation)) {\n      throw new IllegalStateException(\n          String.format(\n              \"Fail to acquire lock %s to commit new metadata at %s\",\n              commitLockEntityId, newMetadataLocation));\n    }\n  }\n\n  private void checkMetadataLocation(Table glueTable, TableMetadata base) {\n    String glueMetadataLocation =\n        glueTable != null ? glueTable.parameters().get(METADATA_LOCATION_PROP) : null;\n    String baseMetadataLocation = base != null ? base.metadataFileLocation() : null;\n    if (!Objects.equals(baseMetadataLocation, glueMetadataLocation)) {\n      throw new CommitFailedException(\n          \"Cannot commit %s because base metadata location '%s' is not same as the current Glue location '%s'\",\n          tableName(), baseMetadataLocation, glueMetadataLocation);\n    }\n  }\n\n  private Table getGlueTable() {","sourceCodeStart":240,"sourceCodeEnd":276,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/glue/GlueTableOperations.java#L240-L276","documentation":"GlueTableOperations.lock throws IllegalStateException when the configured LockManager (custom commit lock implementation) fails to acquire the commit lock for the table before writing new metadata. Iceberg requires the lock to guard against concurrent commits; failing to acquire it aborts the commit before any metadata is written.","triggerScenarios":"doCommit calls lock(newMetadataLocation) and lockManager.acquire(commitLockEntityId, location) returns false — the lock is already held by another process, or the LockManager backend (e.g. DynamoDB) is unavailable/misconfigured.","commonSituations":"Two Spark jobs committing to the same table simultaneously; a crashed job that never released its lock (stale lock); LockManager table unavailable or IAM denied; lock TTL/heartbeat misconfiguration.","solutions":["Find and stop the competing writer holding the lock, or wait for it to finish and retry the commit.","Clear the stale lock left by a crashed job — use the LockManager's break/force-release mechanism (or delete the lock record, e.g. the DynamoDB item) once you confirm the holder is dead.","Configure lock timeouts/TTL so locks from dead processes expire automatically.","Verify the LockManager backend is reachable and IAM permissions allow read/write on the lock entity."],"exampleFix":"// before\n// custom LockManager without TTL; crashed job left the lock forever\nnew InMemoryLockManager();\n\n// after\n// use a lock backend with TTL so stale locks expire\nDynamoDbLockManager lockManager = new DynamoDbLockManager();\nlockManager.initialize(\"iceberg-commit-locks\"); // locks expire if holder dies","handlingStrategy":"retry","validationCode":"// before committing, verify the lock entity is free\nif (!lockManager.isAcquired(commitLockEntityId)) {\n  LOG.info(\"Lock {} is held by another writer, deferring commit\", commitLockEntityId);\n}","typeGuard":null,"tryCatchPattern":"try {\n  table.refresh(); // triggers doCommit -> lock internally\n} catch (IllegalStateException e) {\n  if (e.getMessage().startsWith(\"Fail to acquire lock\")) {\n    LOG.error(\"Commit lock {} held elsewhere; check for stale locks or competing writers\", commitLockEntityId);\n  }\n  throw e;\n}","preventionTips":["Use a LockManager backend with TTLs (e.g. DynamoDB) so crashed jobs release locks automatically","Ensure only one writer pipeline commits to a table at a time","Implement a documented stale-lock break procedure for dead holders","Verify IAM permissions on the lock table before starting commits"],"tags":["aws","glue","lock","concurrency","commit"],"backgroundTag":"lock-acquisition-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}