apache/pulsar · error · RestException

Backlog Quota exceeds configured retention quota for namespa

Error message

Backlog Quota exceeds configured retention quota for namespace. Please increase retention quota and retry

What it means

Thrown when setting a backlog quota (setBacklogQuota) for a namespace whose quota type is destination_storage: the broker validates that the new backlog quota does not exceed the namespace's configured retention quota. The check (checkBacklogQuota) failed, so the update is rejected with HTTP 412 PRECONDITION_FAILED to prevent a backlog policy that conflicts with retention policies.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:1891

                + " backlog quota failed because the data validation failed. %s", namespaceName, e.getMessage()));
            return CompletableFuture.failedFuture(restException);
        }
        return namespaceResources().setPoliciesAsync(namespaceName, policies -> {
            RetentionPolicies retentionPolicies = policies.retention_policies;
            final BacklogQuotaType quotaType = backlogQuotaType != null ? backlogQuotaType
                    : BacklogQuotaType.destination_storage;
            if (retentionPolicies == null) {
                policies.backlog_quota_map.put(quotaType, quota);
                return policies;
            }
            // If we have retention policies, we have to check the conflict.
            BacklogQuota needCheckQuota = null;
            if (quotaType == BacklogQuotaType.destination_storage) {
                needCheckQuota = quota;
            }
            boolean passCheck = checkBacklogQuota(needCheckQuota, retentionPolicies);
            if (!passCheck) {
                throw new RestException(Response.Status.PRECONDITION_FAILED,
                        "Backlog Quota exceeds configured retention quota for namespace."
                                + " Please increase retention quota and retry");
            }
            policies.backlog_quota_map.put(quotaType, quota);
            return policies;
        });
    }

    protected void internalSetRetention(RetentionPolicies retention) {
        validateRetentionPolicies(retention);
        validateNamespacePolicyOperation(namespaceName, PolicyName.RETENTION, PolicyOperation.WRITE);
        validatePoliciesReadOnlyAccess();

        try {
            Policies policies = namespaceResources().getPolicies(namespaceName)
                    .orElseThrow(() -> new RestException(Status.NOT_FOUND,
                    "Namespace policies does not exist"));
            if (!checkQuotas(policies, retention)) {

View on GitHub (pinned to 820761864e)

Solutions

1. Increase the namespace retention policies (setRetention) so retention quota exceeds the requested backlog quota, then retry
2. Lower the backlog quota limit/retentionTime to a value at or below the configured retention quota
3. Check tenant/broker-level defaults that may inject retention limits conflicting with your value
4. Set both retention and backlog quota in a consistent order: retention first, then backlog quota

Example fix

// before
admin.namespaces().setRetention("my-tenant/my-ns", new RetentionPolicies(1, 10)); // 1MB/10min
admin.namespaces().setBacklogQuota("my-tenant/my-ns",
    new BacklogQuota(1_000_000_000L, RetentionPolicy.producer_exception), BacklogQuotaType.destination_storage); // 412
// after
admin.namespaces().setRetention("my-tenant/my-ns", new RetentionPolicies(2_000_000_000L, 60));
admin.namespaces().setBacklogQuota("my-tenant/my-ns",
    new BacklogQuota(1_000_000_000L, RetentionPolicy.producer_exception), BacklogQuotaType.destination_storage); // passes

RetentionPolicies retention = admin.namespaces().getRetention(ns);
BacklogQuota quota = new BacklogQuota(limitBytes, policy);
if (retention != null && quota.getLimitSize() > retention.getRetentionSizeInMB() * 1024L * 1024L) {
    throw new IllegalArgumentException("Backlog quota exceeds retention quota; raise retention first");
}

try {
    admin.namespaces().setBacklogQuota(ns, quota, BacklogQuotaType.destination_storage);
} catch (PulsarAdminException.PreconditionFailedException e) {
    admin.namespaces().setRetention(ns, largerRetention);
    admin.namespaces().setBacklogQuota(ns, quota, BacklogQuotaType.destination_storage);
}

When it happens

Trigger: POST/PUT to /namespaces/{ns}/backlogQuota with a backlogQuota (limit in bytes or retentionTimeInMinutes) larger than the retention quota configured for the namespace while quotaType is destination_storage; setting a very large backlog limit when retention size/time is small.

Common situations: Ops teams raising backlog limits to tolerate slow consumers without realizing retention policies cap it; copy-pasted quota configs across namespaces where retention differs; namespace-level retention lowered earlier, making previously valid backlog values invalid.

Related errors

• Retention Quota must exceed configured backlog quota for nam
• The retention size must > the backlog quota limit size, but
• The retention time must > the backlog quota limit time, but
• Topic name is not valid
• Partitioned Topic Name should not contain '-partition-'

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/1beffbe06921a0ae. Report an issue: GitHub.