apache/druid · error · IllegalStateException

Could not create role [%s] due to concurrent update contenti

Error message

Could not create role [%s] due to concurrent update contention.

What it means

Creation of a basic authorizer role failed after numRetries compare-and-swap attempts because a concurrent writer kept updating the role/user metadata maps first. The updater retries the whole createRoleInternal operation with randomized delays, then throws this ISE naming the contested role. It indicates contention on metadata storage, not an invalid role name.

Source

Thrown at extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authorization/db/updater/CoordinatorBasicAuthorizerMetadataStorageUpdater.java:745

  }

  private void createRoleInternal(String prefix, String roleName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (createRoleOnce(prefix, roleName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }
    throw new ISE("Could not create role [%s] due to concurrent update contention.", roleName);
  }

  private void deleteRoleInternal(String prefix, String roleName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (deleteRoleOnce(prefix, roleName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

Retry createRole after a delay; contention is usually transient.
Confirm only one coordinator leader writes basic-security metadata.
Serialize role provisioning operations across scripts.
Increase numRetries to tolerate bursts of concurrent updates.
Investigate metadata storage performance/latency.

Example fix

// before
CompletableFuture.allOf(roles.map(r -> supplyAsync(() -> client.createRole(prefix, r))))
                .join();
// after: sequential creation with retry
roles.forEach(r -> RetryUtils.retry(() -> client.createRole(prefix, r),
                                    e -> e instanceof IllegalStateException,
                                    MAX_ATTEMPTS));

Defensive patterns

Strategy: retry

Validate before calling

// Skip creation if the role already exists
Map<String, BasicAuthorizerRole> roles =
    BasicAuthUtils.deserializeAuthorizerRoleMap(mapper, getCurrentRoleMapBytes(prefix));
if (roles.containsKey(roleName)) return;

Try / catch

try {
  updater.createRole(prefix, roleName);
} catch (IJSE e) {
  RetryUtils.retry(() -> updater.createRole(prefix, roleName),
                   ex -> ex instanceof IllegalStateException, MAX_ATTEMPTS);
}

Prevention

Create roles sequentially in provisioning scripts
Verify leadership before writes
Use retry-with-backoff wrappers around role mutations
Tune numRetries upward for concurrent environments
Keep metadata store latency low

When it happens

Trigger: Calling createRole while another client repeatedly mutates authorization metadata (users, roles, group mappings) for the same authorizer prefix, so every CAS in tryUpdateGroupMappingMap/userMap fails through all retries.

Common situations: Provisioning scripts creating many roles in parallel from several machines; concurrent role creation and user-role assignment from different admin tools; leader election races in multi-coordinator deployments.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/f0a3809796faf40e. Report an issue: GitHub.