apache/pulsar · error · ManagedLedgerException

Timeout during close operation

Error message

Timeout during close operation

What it means

The shared VALIDATION lambda applied to generic jcloud drivers requires at least one of region (managedLedgerOffloadRegion) or service endpoint (managedLedgerOffloadServiceEndpoint) to identify the storage service, and throws IllegalArgumentException when both are empty. The message includes the configured driver name so you can see which provider failed validation.

Source

Thrown at managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java:2911

        final Result result = new Result();
        final CountDownLatch latch = new CountDownLatch(1);
        asyncClose(new AsyncCallbacks.CloseCallback() {
            @Override
            public void closeComplete(Object ctx) {
                log.debug("Successfully closed ledger");
                latch.countDown();
            }

            @Override
            public void closeFailed(ManagedLedgerException exception, Object ctx) {
                log.warn().exception(exception).log("Closing ledger failed");
                result.exception = exception;
                latch.countDown();
            }
        }, null);

        if (!latch.await(ManagedLedgerImpl.AsyncOperationTimeoutSeconds, TimeUnit.SECONDS)) {
            throw new ManagedLedgerException("Timeout during close operation");
        }

        if (result.exception != null) {
            throw result.exception;
        }
    }

    /**
     * Persist given markDelete position to cursor-ledger or zk-metaStore based on max number of allowed unack-range
     * that can be persist in zk-metastore. If current unack-range is higher than configured threshold then broker
     * persists mark-delete into cursor-ledger else into zk-metastore.
     *
     * @param position
     * @param properties
     * @param callback
     * @param ctx
     */
    void persistPositionWhenClosing(Position position, Map<String, Long> properties,

View on GitHub (pinned to 820761864e)

Solutions

1. Set managedLedgerOffloadRegion=<region> (e.g. us-west-2) for regional cloud providers.
2. For S3-compatible storage without a region, set managedLedgerOffloadServiceEndpoint=<endpoint-url> instead.
3. Confirm the properties are read by the broker/offloader (right conf file, not shadowed by an empty override).

Example fix

// before: broker.conf
managedLedgerOffloadDriver=s3
managedLedgerOffloadBucket=my-bucket

// after: region supplied
managedLedgerOffloadDriver=s3
managedLedgerOffloadBucket=my-bucket
managedLedgerOffloadRegion=us-west-2

boolean hasRegion = config.getRegion() != null && !config.getRegion().isEmpty();
boolean hasEndpoint = config.getServiceEndpoint() != null && !config.getServiceEndpoint().isEmpty();
if (!hasRegion && !hasEndpoint) {
    throw new IllegalStateException("Set managedLedgerOffloadRegion or managedLedgerOffloadServiceEndpoint for driver " + config.getDriver());
}

try {
    provider.validate(tieringConfig);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("Either Region or ServiceEndpoint")) {
        log.error("Configure managedLedgerOffloadRegion (cloud) or managedLedgerOffloadServiceEndpoint (S3-compatible) for {}", tieringConfig.getDriver());
    }
    throw e;
}

When it happens

Trigger: getBlobStore/initiating offload with a driver using the common VALIDATION (e.g. aws, gcs, azureblob) while both TieredStorageConfiguration.getRegion() and getServiceEndpoint() are null/empty.

Common situations: Omitting managedLedgerOffloadRegion when using AWS S3, forgetting managedLedgerOffloadServiceEndpoint for S3-compatible endpoints (MinIO, COS, BOS), property placed in the wrong config file, or topic-level offload policy missing the region field.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Timeout during delete operation
• Timeout during open-cursor operation
• Timeout during delete-cursors operation
• Timeout during managed ledger terminate
• The '${offloaderName}' offloader does not provide an offload

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/98a5516a55d79b5e. Report an issue: GitHub.