apache/druid · error · IllegalStateException

Could not create group mapping [%s] due to concurrent update

Error message

Could not create group mapping [%s] due to concurrent update contention.

What it means

The coordinator-side basic authorizer metadata storage updater failed to create an authorization group mapping after exhausting all CAS retry attempts (numRetries). Each attempt re-reads the group mapping map from metadata storage and does a compare-and-swap update; if another writer keeps winning the race, the retries run out and this ISE is thrown. It signals persistent concurrent update contention on the group mapping map in metadata storage, not a validation problem with the mapping itself.

Source

Thrown at extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authorization/db/updater/CoordinatorBasicAuthorizerMetadataStorageUpdater.java:707

  }

  private void createGroupMappingInternal(String prefix, BasicAuthorizerGroupMapping groupMapping)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (createGroupMappingOnce(prefix, groupMapping)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }
    throw new ISE("Could not create group mapping [%s] due to concurrent update contention.", groupMapping);
  }

  private void deleteGroupMappingInternal(String prefix, String groupMappingName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (deleteGroupMappingOnce(prefix, groupMappingName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

Retry the createGroupMapping call after a delay; the failure is contention-based, so a subsequent attempt usually succeeds.
Verify only one coordinator is acting as leader/performing metadata writes for this authorizer at a time.
Reduce write concurrency: serialize admin/automation requests against the basic security API for this authorizer prefix.
Increase the updater's retry budget (numRetries) if bursts of concurrent updates are expected.
Inspect metadata storage latency/health; slow CAS operations widen the contention window.

Example fix

// before: fire-and-forget parallel setup
users.forEach(u -> client.createGroupMapping(prefix, u));
// after: serialize mutations with bounded retry
for (final String u : users) {
  boolean ok = retryUntil(() -> { try { client.createGroupMapping(prefix, u); return true; }
                            catch (ISE e) { return false; } }, 5);
  if (!ok) throw new IllegalStateException("group mapping " + u + " still contended");
}

Defensive patterns

Strategy: retry

Validate before calling

// Verify no concurrent writers before mutating
boolean isLeader = coordinatorClient.isCurrentLeader();
if (!isLeader) throw new IllegalStateException("Not the leader; skip metadata writes");

Try / catch

try {
  updater.createGroupMapping(prefix, groupMapping);
} catch (IJSE e) {
  // contention exhausted; back off and retry once
  Thread.sleep(RETRY_BACKOFF_MS);
  updater.createGroupMapping(prefix, groupMapping);
}

Prevention

Only the leader coordinator should write basic-security metadata
Serialize admin mutations; avoid parallel provisioning scripts
Use randomized backoff between attempts
Increase numRetries for bursty workloads
Monitor metadata store latency to shrink the contention window

When it happens

Trigger: Calling createGroupMapping (or startup via initSuperUsersAndGroupMapping) while another process keeps updating the same authorizer prefix's group mapping map so every tryUpdateGroupMappingMap compare-and-swap fails for numRetries consecutive attempts.

Common situations: Multiple coordinator overlord/leader instances concurrently mutating basic-security authorization entities; automation scripts or CI repeatedly creating the same group mapping in parallel; a hot metadata store (e.g. slow/contended ZooKeeper or metadata store) making each CAS retry fail; unusually low numRetries configuration under bursty admin traffic.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/222a406a30ed2990. Report an issue: GitHub.