apache/pulsar · error · RestException

Backlog Quota exceeds configured retention quota for…

Error message

Backlog Quota exceeds configured retention quota for namespace. Please increase retention quota and retry

What it means

Thrown when setting a backlog quota (setBacklogQuota) for a namespace whose quota type is destination_storage: the broker validates that the new backlog quota does not exceed the namespace's configured retention quota. The check (checkBacklogQuota) failed, so the update is rejected with HTTP 412 PRECONDITION_FAILED to prevent a backlog policy that conflicts with retention policies.

Solutions

Increase the namespace retention policies (setRetention) so retention quota exceeds the requested backlog quota, then retry
Lower the backlog quota limit/retentionTime to a value at or below the configured retention quota
Check tenant/broker-level defaults that may inject retention limits conflicting with your value
Set both retention and backlog quota in a consistent order: retention first, then backlog quota

Example fix

// before
admin.namespaces().setRetention("my-tenant/my-ns", new RetentionPolicies(1, 10)); // 1MB/10min
admin.namespaces().setBacklogQuota("my-tenant/my-ns",
    new BacklogQuota(1_000_000_000L, RetentionPolicy.producer_exception), BacklogQuotaType.destination_storage); // 412
// after
admin.namespaces().setRetention("my-tenant/my-ns", new RetentionPolicies(2_000_000_000L, 60));
admin.namespaces().setBacklogQuota("my-tenant/my-ns",
    new BacklogQuota(1_000_000_000L, RetentionPolicy.producer_exception), BacklogQuotaType.destination_storage); // passes

Defensive patterns

Strategy: validation

Validate before calling

RetentionPolicies retention = admin.namespaces().getRetention(ns);
BacklogQuota quota = new BacklogQuota(limitBytes, policy);
if (retention != null && quota.getLimitSize() > retention.getRetentionSizeInMB() * 1024L * 1024L) {
    throw new IllegalArgumentException("Backlog quota exceeds retention quota; raise retention first");
}

Try / catch

try {
    admin.namespaces().setBacklogQuota(ns, quota, BacklogQuotaType.destination_storage);
} catch (PulsarAdminException.PreconditionFailedException e) {
    admin.namespaces().setRetention(ns, largerRetention);
    admin.namespaces().setBacklogQuota(ns, quota, BacklogQuotaType.destination_storage);
}

Prevention

Set retention policies before (or together with) backlog quotas
Keep a documented ratio (retention > backlog) in your config pipeline
Read current retention/backlog via the admin API before changes

When it happens

Trigger: POST/PUT to /namespaces/{ns}/backlogQuota with a backlogQuota (limit in bytes or retentionTimeInMinutes) larger than the retention quota configured for the namespace while quotaType is destination_storage; setting a very large backlog limit when retention size/time is small.

Common situations: Ops teams raising backlog limits to tolerate slow consumers without realizing retention policies cap it; copy-pasted quota configs across namespaces where retention differs; namespace-level retention lowered earlier, making previously valid backlog values invalid.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/1beffbe06921a0ae. Report an issue: GitHub.

Appendix: source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:1891

                + " backlog quota failed because the data validation failed. %s", namespaceName, e.getMessage()));
            return CompletableFuture.failedFuture(restException);
        }
        return namespaceResources().setPoliciesAsync(namespaceName, policies -> {
            RetentionPolicies retentionPolicies = policies.retention_policies;
            final BacklogQuotaType quotaType = backlogQuotaType != null ? backlogQuotaType
                    : BacklogQuotaType.destination_storage;
            if (retentionPolicies == null) {
                policies.backlog_quota_map.put(quotaType, quota);
                return policies;
            }
            // If we have retention policies, we have to check the conflict.
            BacklogQuota needCheckQuota = null;
            if (quotaType == BacklogQuotaType.destination_storage) {
                needCheckQuota = quota;
            }
            boolean passCheck = checkBacklogQuota(needCheckQuota, retentionPolicies);
            if (!passCheck) {
                throw new RestException(Response.Status.PRECONDITION_FAILED,
                        "Backlog Quota exceeds configured retention quota for namespace."
                                + " Please increase retention quota and retry");
            }
            policies.backlog_quota_map.put(quotaType, quota);
            return policies;
        });
    }

    protected void internalSetRetention(RetentionPolicies retention) {
        validateRetentionPolicies(retention);
        validateNamespacePolicyOperation(namespaceName, PolicyName.RETENTION, PolicyOperation.WRITE);
        validatePoliciesReadOnlyAccess();

        try {
            Policies policies = namespaceResources().getPolicies(namespaceName)
                    .orElseThrow(() -> new RestException(Status.NOT_FOUND,
                    "Namespace policies does not exist"));
            if (!checkQuotas(policies, retention)) {

View on GitHub (pinned to 820761864e)