apache/druid · error · IllegalArgumentException

Cannot delete [%,d] bytes, only have [%,d] buffered

Error message

Cannot delete [%,d] bytes, only have [%,d] buffered

What it means

deleteFromQueuedChunks removes numBytes from the front of the buffered chunk queue. If fewer bytes are buffered than requested, it throws this IAE. Like its copy counterpart, this is an internal invariant: callers must only delete bytes known to be fully buffered. Violations imply races or misuse.

Solutions

1. Use the channel's public API from a single consumer only.
2. Audit any code that calls updateStreamState/delete paths concurrently.
3. Upgrade/report if reproducible on stock Druid paths.
4. Capture the id and buffered counts in logs to diagnose the race.

try { rac = channel.read(); } catch (IAE e) { if (e.getMessage().startsWith("Cannot delete")) { reportInternalBug(e); } else { throw e; } }

When it happens

Trigger: deleteFromQueuedChunks(n) invoked when bytesBuffered < n — e.g. deleting the magic header or a consumed frame when buffering has already been drained by another caller.

Common situations: Concurrent readers consuming the same channel; custom wrappers bypassing the lock; internal bugs in stream-state accounting.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Cannot copy [%,d] bytes, only have [%,d] buffered
• bitmapEncoding[ ] does not match indexSpec.bitmap[ ]
• Cannot read frame of size [%,d] bytes
• Cleaned partition map
• Frame of size [%,d] not yet ready to read

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/435ee31bc7f8be89. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java:500

      final int chunkPosition = chunkNumber == 0 ? positionInFirstChunk : 0;
      final int len = Math.min(chunk.length - chunkPosition, numBytes - bufPos);

      buf.putByteArray(bufPos, chunk, chunkPosition, len);
      bufPos += len;

      if (bufPos == numBytes) {
        break;
      }
    }

    return buf;
  }

  @GuardedBy("lock")
  private void deleteFromQueuedChunks(final long numBytes)
  {
    if (bytesBuffered < numBytes) {
      throw new IAE("Cannot delete [%,d] bytes, only have [%,d] buffered", numBytes, bytesBuffered);
    }

    long toDelete = numBytes;

    while (toDelete > 0) {
      final byte[] chunk = chunks.get(0).valueOrThrow();
      final int bytesRemainingInChunk = chunk.length - positionInFirstChunk;

      if (toDelete >= bytesRemainingInChunk) {
        toDelete -= bytesRemainingInChunk;
        positionInFirstChunk = 0;
        chunks.remove(0);
      } else {
        positionInFirstChunk = Ints.checkedCast(positionInFirstChunk + toDelete);
        toDelete = 0;
      }
    }

View on GitHub (pinned to 9b90983fd2)