apache/pulsar · error · ManagedLedgerException

Timeout during clear backlog operation

Error message

Timeout during clear backlog operation

What it means

The Azure Blob offload provider obtains credentials from the environment variables AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_ACCESS_KEY. buildCredentials throws IllegalArgumentException when AZURE_STORAGE_ACCOUNT is empty or unset, because no storage account name is available to build the Credentials object.

Source

Thrown at managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java:2008

        final Result result = new Result();
        final CountDownLatch counter = new CountDownLatch(1);

        asyncClearBacklog(new ClearBacklogCallback() {
            @Override
            public void clearBacklogComplete(Object ctx) {
                counter.countDown();
            }

            @Override
            public void clearBacklogFailed(ManagedLedgerException exception, Object ctx) {
                result.exception = exception;
                counter.countDown();
            }
        }, null);

        if (!counter.await(ManagedLedgerImpl.AsyncOperationTimeoutSeconds, TimeUnit.SECONDS)) {
            throw new ManagedLedgerException("Timeout during clear backlog operation");
        }

        if (result.exception != null) {
            throw result.exception;
        }
    }

    @Override
    public void asyncClearBacklog(final ClearBacklogCallback callback, Object ctx) {
        asyncMarkDelete(ledger.getLastPosition(), new MarkDeleteCallback() {
            @Override
            public void markDeleteComplete(Object ctx) {
                callback.clearBacklogComplete(ctx);
            }

            @Override
            public void markDeleteFailed(ManagedLedgerException exception, Object ctx) {
                if (exception.getCause() instanceof IllegalArgumentException) {

View on GitHub (pinned to 820761864e)

Solutions

1. Set AZURE_STORAGE_ACCOUNT in the process environment of the broker/offloader before startup (export or deployment manifest env entry).
2. Also set AZURE_STORAGE_ACCESS_KEY, otherwise the next check will fail with the access-key error.
3. Verify with `tr '\0' '\n' < /proc/<broker-pid>/environ | grep AZURE` that the running process actually sees it.
4. Check spelling and casing of the variable name in your deployment configuration.

Example fix

// before (docker-compose): env var missing
broker:
  image: apachepulsar/pulsar

// after:
broker:
  image: apachepulsar/pulsar
  environment:
    - AZURE_STORAGE_ACCOUNT=mystorageacct
    - AZURE_STORAGE_ACCESS_KEY=<base64-key>

if (System.getenv("AZURE_STORAGE_ACCOUNT") == null || System.getenv("AZURE_STORAGE_ACCOUNT").isEmpty()) {
    throw new IllegalStateException("AZURE_STORAGE_ACCOUNT must be set for azureblob offload");
}
if (System.getenv("AZURE_STORAGE_ACCESS_KEY") == null || System.getenv("AZURE_STORAGE_ACCESS_KEY").isEmpty()) {
    throw new IllegalStateException("AZURE_STORAGE_ACCESS_KEY must be set for azureblob offload");
}

try {
    provider.buildCredentials(tieringConfig);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("azure storage account")) {
        log.error("Set AZURE_STORAGE_ACCOUNT (and AZURE_STORAGE_ACCESS_KEY) in the broker process environment");
    }
    throw e;
}

When it happens

Trigger: Starting offload with driver=azureblob while the AZURE_STORAGE_ACCOUNT environment variable is not set in the broker/offloader process environment.

Common situations: Env var not exported in the systemd unit / Docker container / K8s deployment spec, variable set only in an interactive shell but not in the service environment, name typo (AZURE_STORAGE_ACCOUNT_NAME), or running the offloader on a different host than where the variable was configured.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Timeout during skip messages operation
• Timeout during mark-delete operation
• Both ${S3_ID_FIELD} and ${S3_SECRET_FIELD} must be set when
• Couldn't get the access key id.
• Couldn't get the access key secret.

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/04bbb342f59f7ec5. Report an issue: GitHub.