What causes "Timeout during reset cursor" in apache/pulsar?

Re-read the blob from storage — the input is truncated; delete and re-offload the ledger if the stored object is incomplete.

apache/pulsar · error · ManagedLedgerException

Timeout during reset cursor

Error message

Timeout during reset cursor

What it means

fromStream computes toSkip = headerLen - bytes already consumed and calls dis.skip(toSkip); if the underlying stream cannot skip that many bytes it throws EOFException("Header was too small"). This means the InputStream contains fewer bytes than the header's declared headerLen, so the header cannot be fully consumed to reach the block payload. It is an end-of-data condition caused by truncated input or a malformed/inflated headerLen field.

Source

Thrown at managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java:1791

                counter.countDown();
            }

            @Override
            public void resetFailed(ManagedLedgerException exception, Object ctx) {
                result.exception = exception;
                counter.countDown();

            }
        });

        if (!counter.await(ManagedLedgerImpl.AsyncOperationTimeoutSeconds, TimeUnit.SECONDS)) {
            if (result.exception != null) {
                log.warn()
                        .attr("position", newPos)
                        .attr("error", result.exception)
                        .log("Reset cursor timed out");
            }
            throw new ManagedLedgerException("Timeout during reset cursor");
        }

        if (result.exception != null) {
            throw result.exception;
        }
    }

    @Override
    public List<Entry> replayEntries(Set<? extends Position> positions)
            throws InterruptedException, ManagedLedgerException {
        final CountDownLatch counter = new CountDownLatch(1);
        class Result {
            ManagedLedgerException exception = null;
            List<Entry> entries = null;
        }

        final Result result = new Result();

View on GitHub (pinned to 820761864e)

Solutions

Re-read the blob from storage — the input is truncated; delete and re-offload the ledger if the stored object is incomplete.
Validate the blob size on download (compare against stored object metadata / Content-Length) before parsing.
Check headerLen for sanity (e.g. < HEADER_MAX_SIZE) before skipping; an absurd value indicates header corruption.
Ensure reader and writer versions agree on the header format so headerLen is parsed from the right offset.

Example fix

// before: no size check, truncated stream surfaces as EOF deep in parse
StreamingDataBlockHeaderImpl.fromStream(truncatedStream);

// after: verify declared header fits in available bytes first
StreamingDataBlockHeaderImpl hdr = StreamingDataBlockHeaderImpl.of(headerLen, blockLen, ledgerId, firstEntryId);
if (headerLen > availableBytes) {
    throw new IOException("Truncated block: headerLen=" + headerLen + " available=" + availableBytes);
}

Defensive patterns

Strategy: validation

Validate before calling

long declaredHeaderLen = readLongLE(bytes, offset); // once parsed
if (bytesRemaining(stream) < declaredHeaderLen) {
    throw new IOException("Truncated blob: headerLen=" + declaredHeaderLen + " > available bytes");
}

Try / catch

try {
    return StreamingDataBlockHeaderImpl.fromStream(in);
} catch (EOFException e) {
    log.warn("Truncated offload data block header", e);
    return retryFromFreshDownload(); // re-open stream from storage, not the same stream
}

Prevention

Compare downloaded blob size against stored object Content-Length/metadata before parsing.
Use fully-buffered reads for small blobs so a mid-read network failure fails cleanly.
Sanity-check headerLen against a max bound to catch corrupted header fields.
Monitor offload writes for completion; failed uploads leave truncated blobs.

When it happens

Trigger: Calling fromStream with a stream carrying fewer than headerLen readable bytes: a truncated blob download, a header whose headerLen field was corrupted to a huge value, or a HEADER_MAX_SIZE-bounded stream where the declared header exceeds the available bytes.

Common situations: Interrupted or partial reads from object storage (network cut mid-read), offload blobs written by a crashed producer, corrupted headerLen bytes flipping to a large value, or reading a blob with a different format version whose header layout differs.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/cdf014818ca1bf3a. Report an issue: GitHub.