{"record":{"id":"b1c81d9cdc146b2a","repo":"apache/iceberg","slug":"optimistic-locking-is-not-available-in-the-environ","errorCode":null,"errorMessage":"Optimistic locking is not available in the environment. Using in-memory lock manager. To ensure atomic transaction, please configure a distributed lock manager such as the DynamoDB lock manager.","messagePattern":"Optimistic locking is not available in the environment\\. Using in-memory lock manager\\. To ensure atomic transaction, please configure a distributed lock manager such as the DynamoDB lock manager\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java","lineNumber":159,"sourceCode":"\n    initialize(\n        name,\n        properties.get(CatalogProperties.WAREHOUSE_LOCATION),\n        new AwsProperties(properties),\n        new S3FileIOProperties(properties),\n        awsClientFactory.glue(),\n        initializeLockManager(properties),\n        PropertyUtil.propertyAsBoolean(\n            properties,\n            CatalogProperties.UNIQUE_TABLE_LOCATION,\n            CatalogProperties.UNIQUE_TABLE_LOCATION_DEFAULT));\n  }\n\n  private LockManager initializeLockManager(Map<String, String> properties) {\n    if (properties.containsKey(CatalogProperties.LOCK_IMPL)) {\n      return LockManagers.from(properties);\n    } else if (SET_VERSION_ID.isNoop()) {\n      LOG.warn(\n          \"Optimistic locking is not available in the environment. Using in-memory lock manager.\"\n              + \" To ensure atomic transaction, please configure a distributed lock manager\"\n              + \" such as the DynamoDB lock manager.\");\n      return LockManagers.defaultLockManager();\n    } else {\n      LOG.debug(\"Using optimistic locking for Glue Data Catalog tables.\");\n    }\n    return null;\n  }\n\n  @VisibleForTesting\n  void initialize(\n      String name,\n      String path,\n      AwsProperties properties,\n      S3FileIOProperties s3Properties,\n      GlueClient client,\n      LockManager lock,","sourceCodeStart":141,"sourceCodeEnd":177,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java#L141-L177","documentation":"A warning from GlueCatalog.initializeLockManager: no custom lock-impl was configured and the Glue client cannot perform optimistic locking (the SET_VERSION_ID operation is a no-op in this environment), so the catalog falls back to an in-memory lock manager. In-memory locking is only process-local, so concurrent commits from different jobs/engines are not safely serialized against each other.","triggerScenarios":"Initializing GlueCatalog without catalog_properties.lock_impl while the AWS SDK/Glue environment does not support the version-id conditional update used for optimistic locking (e.g. Glue without LockTable support or restricted permissions).","commonSituations":"Multi-cluster or multi-engine deployments where two Spark/Flink jobs commit to the same Glue table concurrently; missing IAM permissions causing the version-id operation to be unavailable; forgetting to configure a DynamoDB LockManager in production.","solutions":["Set warehouse/catalog property lock.impl to org.apache.iceberg.aws.dynamodb.DynamoDbLockManager with a DynamoDB lock table for distributed locking","Check why SET_VERSION_ID is no-op: verify AWS SDK version supports Glue table version locking and that no environment flag disabled it","If truly single-process, the in-memory manager is acceptable — silence the warning by accepting the risk or configuring a lock explicitly","Ensure IAM permissions permit the Glue calls that back optimistic locking"],"exampleFix":"// before\nMap<String, String> props = Map.of(\"type\", \"glue\");\nGlueCatalog catalog = new GlueCatalog();\ncatalog.initialize(\"glue\", props); // falls back to in-memory lock manager\n// after\nMap<String, String> props = Map.of(\n    \"type\", \"glue\",\n    CatalogProperties.LOCK_IMPL, \"org.apache.iceberg.aws.dynamodb.DynamoDbLockManager\",\n    \"lock.table\", \"iceberg_lock_table\");\ncatalog.initialize(\"glue\", props);","handlingStrategy":"fallback","validationCode":"// warn on startup if lock-impl missing in multi-writer deployments\nif (!props.containsKey(CatalogProperties.LOCK_IMPL) && multiWriter) {\n  LOG.warn(\"Distributed lock manager not configured for Glue catalog\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always configure lock.impl=DynamoDbLockManager in production","Provision a DynamoDB lock table before deploying","Check Glue/SDK support for optimistic locking in your environment","Accept in-memory locks only for single-process usage"],"tags":["aws","glue","locking","concurrency"],"backgroundTag":"missing-config-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}