Hmbown/CodeWhale · error · PersistenceBacklogError

estimated_retained_payload_bytes is smaller than the frozen

Q: What causes "estimated_retained_payload_bytes is smaller than the frozen retained content" in Hmbown/CodeWhale?

Verify the fixture generator still produces exactly 64 KiB of content per request.

Error message

estimated_retained_payload_bytes is smaller than the frozen retained content

What it means

Raised in validate_receipt (scripts/check-persistence-backlog-budget.py:200-204) when estimated_retained_payload_bytes < retained_queued_requests * 65536 (FIXTURE content_bytes_per_request). Each fixture request carries 64 KiB of content, so the retained payload estimate can never be smaller than retained count times that floor — otherwise the estimator is under-measuring.

Source

Thrown at scripts/check-persistence-backlog-budget.py:202

    accepted = non_negative_integer(receipt["accepted_requests"], "accepted_requests")
    if accepted != attempted:
        raise PersistenceBacklogError(
            "accepted_requests must equal requests_attempted; sender rejection is not backlog improvement"
        )
    retained = non_negative_integer(
        receipt["retained_queued_requests"], "retained_queued_requests"
    )
    if retained > accepted:
        raise PersistenceBacklogError("retained_queued_requests exceeds accepted_requests")
    for field in ("estimated_retained_payload_bytes", "enqueue_elapsed_ns"):
        non_negative_integer(receipt[field], field)
    if retained == 0 or receipt["estimated_retained_payload_bytes"] == 0:
        raise PersistenceBacklogError(
            "the paused channel must retain the newest request and its payload"
        )
    minimum_payload_bytes = retained * FIXTURE["content_bytes_per_request"]
    if receipt["estimated_retained_payload_bytes"] < minimum_payload_bytes:
        raise PersistenceBacklogError(
            "estimated_retained_payload_bytes is smaller than the frozen retained content"
        )
    applied = non_negative_integer(
        receipt["applied_version"], "applied_version"
    )
    if applied != FIXTURE["expected_applied_version"]:
        raise PersistenceBacklogError("applied_version is not the final sent version")
    if receipt["final_version_applied"] is not True:
        raise PersistenceBacklogError("final_version_applied must be true")

    limitations = receipt["limitations"]
    if not isinstance(limitations, list) or not limitations or not all(
        isinstance(item, str) and item for item in limitations
    ):
        raise PersistenceBacklogError("limitations must be a non-empty string array")

    if not isinstance(receipt["rss_supported"], bool):
        raise PersistenceBacklogError("rss_supported must be boolean")

View on GitHub (pinned to 8880682c63)

Solutions

Verify the fixture generator still produces exactly 64 KiB of content per request.
Fix the estimator to measure the serialized retained JSON byte length, then regenerate the receipt.
Cross-check estimated_retained_payload_bytes against retained_queued_requests * 65536 in the new receipt.

Example fix

// receipt (before)
"retained_queued_requests": 128, "estimated_retained_payload_bytes": 1000
// receipt (after)
"retained_queued_requests": 128, "estimated_retained_payload_bytes": 8388608

Defensive patterns

Strategy: validation

Validate before calling

min_bytes = receipt["retained_queued_requests"] * 64 * 1024
if receipt["estimated_retained_payload_bytes"] < min_bytes:
    sys.exit("payload estimator under-measures the 64 KiB-per-request fixture content")

Try / catch

try:
    validate_receipt(receipt)
except PersistenceBacklogError as e:
    if "smaller than the frozen retained content" in str(e):
        raise RuntimeError("estimator returns compressed/truncated size; measure serialized bytes") from e
    raise

Prevention

Keep the fixture generator at exactly 64 KiB content per request and assert it in tests.
Measure serialized JSON byte length, not compressed size or a stored length field.
Re-verify the estimator whenever the session serialization format changes.

When it happens

Trigger: The estimator measuring compressed or truncated size instead of serialized JSON bytes; payloads generated smaller than 64 KiB after a fixture change; counting bytes of only the first request; unit confusion (KB vs bytes).

Common situations: Session serialization switched to a compact encoding; content generator changed; estimator reading a length field instead of the actual buffer length.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@8880682c63 (2026-08-16). Data as JSON: /api/errors/1002e64ad3b1b748. Report an issue: GitHub.