apache/druid · error · IllegalStateException

Frame of size [%,d] not yet ready to read

Error message

Frame of size [%,d] not yet ready to read

What it means

ReadableByteChunksFrameChannel buffers bytes from a producer and decodes complete frames into RowsAndColumns. When read() is invoked, it attempts to extract the next frame; if the buffered stream does not yet contain a full frame (canReadFrame() is false, meaning the announced length nextCompressedFrameLength has not fully arrived), it throws this ISE. It signals reading ahead of the writer.

Solutions

1. Check readyToRead() (or canRead via the channel's state) before calling read().
2. Wait for the producer to finish writing: use a completion signal or await() on the channel instead of polling read().
3. Inspect the producer side for stalled/crashed tasks if the frame never arrives.
4. Enable debug logging on ReadableByteChunksFrameChannel to trace bytesAdded/bytesBuffered progress.

Example fix

// before
RowsAndColumns rac = channel.read();
// after
if (channel.readyToRead()) {
  RowsAndColumns rac = channel.read();
} else {
  // wait for writer or idle() await
}

if (!channel.readyToRead()) { /* await writer or idle */ } else { RowsAndColumns rac = channel.read(); }

boolean isReady(ReadableByteChunksFrameChannel ch) { return ch.readyToRead(); }

try { rac = channel.read(); } catch (ISE e) { if (e.getMessage().contains("not yet ready")) { /* defer read */ } else { throw e; } }

When it happens

Trigger: Calling read() before addChunk() has delivered all bytes of the current frame, i.e. bytesBuffered < FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength after a length was announced but not fully received.

Common situations: MSQ/stage-exchange consumers polling a channel whose producer task is still writing or has stalled; race between producer completion and consumer read; consumer error handling that ignores readyToRead() checks.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Attempt to add row to swapped-out sink for segment
• Cannot copy [%,d] bytes, only have [%,d] buffered
• Cannot delete [%,d] bytes, only have [%,d] buffered
• Cannot read frame of size [%,d] bytes
• Could not assign role

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b94d46ea73a6ecf6. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java:356

    }
  }

  @VisibleForTesting
  long getBytesBuffered()
  {
    synchronized (lock) {
      return bytesBuffered;
    }
  }

  private RowsAndColumns nextRAC()
  {
    final Memory compressedMemory;
    final byte markerType;

    synchronized (lock) {
      if (!canReadFrame()) {
        throw new ISE("Frame of size [%,d] not yet ready to read", nextCompressedFrameLength);
      }

      if (nextCompressedFrameLength > Integer.MAX_VALUE - FRAME_MARKER_BYTES - FrameCompression.COMPRESSED_DATA_ENVELOPE_SIZE) {
        throw new ISE("Cannot read frame of size [%,d] bytes", nextCompressedFrameLength);
      }

      markerType = nextMarkerType;
      final int numBytes = Ints.checkedCast(FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength);
      compressedMemory = copyFromQueuedChunks(numBytes).region(
          FRAME_MARKER_BYTES,
          FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength - FRAME_MARKER_BYTES
      );
      deleteFromQueuedChunks(numBytes);
      updateStreamState();
    }

    final byte[] decompressedBytes =
        FrameCompression.decompress(compressedMemory, 0, compressedMemory.getCapacity());

View on GitHub (pinned to 9b90983fd2)