apache/pulsar · error · IllegalArgumentException

The retention size must > the backlog quota limit size, but

Error message

The retention size must > the backlog quota limit size, but the configured backlog quota limit bytes is %d, the retention size is %d

What it means

This IllegalArgumentException is thrown at broker startup when the default backlog quota size limit is greater than or equal to the default retention size. Pulsar requires retention size to be strictly larger than the backlog quota limit, otherwise retention (which can only run when a topic is under quota) would never be able to delete data before the backlog quota stops producers. It is a fail-fast configuration sanity check performed by PulsarService.start().

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java:890

            }

            if (config.getWebServicePort().isEmpty()
                    && config.getWebServicePortTls().isEmpty()
                    && BindAddressValidator.validateBindAddresses(config, Arrays.asList("http", "https")).isEmpty()) {
                throw new IllegalArgumentException(
                        "webServicePort/webServicePortTls or http/https bindAddresses must be present");
            }

            if (config.isAuthorizationEnabled() && !config.isAuthenticationEnabled()) {
                throw new IllegalStateException("Invalid broker configuration. Authentication must be enabled with "
                        + "authenticationEnabled=true when authorization is enabled with authorizationEnabled=true.");
            }

            if (config.getDefaultRetentionSizeInMB() > 0
                    && config.getBacklogQuotaDefaultLimitBytes() > 0
                    && config.getBacklogQuotaDefaultLimitBytes()
                    >= (config.getDefaultRetentionSizeInMB() * 1024L * 1024L)) {
                throw new IllegalArgumentException(String.format("The retention size must > the backlog quota limit "
                                + "size, but the configured backlog quota limit bytes is %d, the retention size is %d",
                        config.getBacklogQuotaDefaultLimitBytes(),
                        config.getDefaultRetentionSizeInMB() * 1024L * 1024L));
            }

            if (config.getDefaultRetentionTimeInMinutes() > 0
                    && config.getBacklogQuotaDefaultLimitSecond() > 0
                    && config.getBacklogQuotaDefaultLimitSecond() >= config.getDefaultRetentionTimeInMinutes() * 60) {
                throw new IllegalArgumentException(String.format("The retention time must > the backlog quota limit "
                                + "time, but the configured backlog quota limit time duration is %d, "
                                + "the retention time duration is %d",
                        config.getBacklogQuotaDefaultLimitSecond(),
                        config.getDefaultRetentionTimeInMinutes() * 60));
            }

            if (config.isBrokerDeleteInactiveTopicsEnabled() && config.isBrokerCloseInactiveTopicsEnabled()) {
                throw new IllegalArgumentException(
                        "brokerDeleteInactiveTopicsEnabled and brokerCloseInactiveTopicsEnabled are mutually "

View on GitHub (pinned to 820761864e)

Solutions

Increase defaultRetentionSizeInMB (or lower backlogQuotaDefaultLimitBytes) so retention size in bytes is strictly greater than the backlog quota limit bytes.
If retention should be effectively unlimited, set backlogQuotaDefaultLimitBytes to -1 (disabled) so the check is skipped.
Set defaultRetentionSizeInMB to -1 (disabled) if you want quota-only eviction without size-based retention.
Align the defaults in broker.conf / pulsar.znc configuration and restart the broker.

Example fix

// before (broker.conf)
defaultRetentionSizeInMB=1
backlogQuotaDefaultLimitBytes=1073741824
// after
defaultRetentionSizeInMB=2048
backlogQuotaDefaultLimitBytes=1073741824

Defensive patterns

Strategy: validation

Validate before calling

// before broker start (Java)
if (conf.getDefaultRetentionSizeInMB() > 0 && conf.getBacklogQuotaDefaultLimitBytes() > 0
        && conf.getBacklogQuotaDefaultLimitBytes() >= conf.getDefaultRetentionSizeInMB() * 1024L * 1024L) {
    throw new IllegalArgumentException(
        "backlogQuotaDefaultLimitBytes must be < defaultRetentionSizeInMB * 1024 * 1024 (or disabled)");
}

Try / catch

try {
    pulsar.start();
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("retention size must > the backlog quota limit")) {
        // fix broker.conf: raise retention size or lower/disable backlogQuotaDefaultLimitBytes, then retry
    } else {
        throw e;
    }
}

Prevention

Keep defaultRetentionSizeInMB strictly larger than backlogQuotaDefaultLimitBytes / 1MB in broker.conf.
Add the size comparison to your configuration CI/lint checks before deploying broker configs.
Disable one side (-1) when you intend quota-only or retention-only behavior.
Re-check the pair whenever you tune backlog quotas; they are validated against retention at startup.

When it happens

Trigger: Starting a broker whose service configuration has defaultRetentionSizeInMB > 0, backlogQuotaDefaultLimitBytes > 0, and backlogQuotaDefaultLimitBytes >= defaultRetentionSizeInMB * 1024 * 1024 (e.g. retentionSizeInMB=1 with backlogQuotaDefaultLimitBytes=1073741824).

Common situations: Operators set a generous default backlog quota (e.g. 1GB) while leaving a small default retention size (e.g. 1MB), or tune backlogQuotaDefaultLimitBytes without re-checking retentionSizeInMB, so the broker refuses to boot.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/df89ba645239f36e. Report an issue: GitHub.