apache/druid · error · IllegalStateException

Frame of size [%,d] not yet ready to read

Error message

Frame of size [%,d] not yet ready to read

What it means

ReadableByteChunksFrameChannel buffers bytes from a producer and decodes complete frames into RowsAndColumns. When read() is invoked, it attempts to extract the next frame; if the buffered stream does not yet contain a full frame (canReadFrame() is false, meaning the announced length nextCompressedFrameLength has not fully arrived), it throws this ISE. It signals reading ahead of the writer.

Source

Thrown at processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java:356

    }
  }

  @VisibleForTesting
  long getBytesBuffered()
  {
    synchronized (lock) {
      return bytesBuffered;
    }
  }

  private RowsAndColumns nextRAC()
  {
    final Memory compressedMemory;
    final byte markerType;

    synchronized (lock) {
      if (!canReadFrame()) {
        throw new ISE("Frame of size [%,d] not yet ready to read", nextCompressedFrameLength);
      }

      if (nextCompressedFrameLength > Integer.MAX_VALUE - FRAME_MARKER_BYTES - FrameCompression.COMPRESSED_DATA_ENVELOPE_SIZE) {
        throw new ISE("Cannot read frame of size [%,d] bytes", nextCompressedFrameLength);
      }

      markerType = nextMarkerType;
      final int numBytes = Ints.checkedCast(FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength);
      compressedMemory = copyFromQueuedChunks(numBytes).region(
          FRAME_MARKER_BYTES,
          FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength - FRAME_MARKER_BYTES
      );
      deleteFromQueuedChunks(numBytes);
      updateStreamState();
    }

    final byte[] decompressedBytes =
        FrameCompression.decompress(compressedMemory, 0, compressedMemory.getCapacity());

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Check readyToRead() (or canRead via the channel's state) before calling read().
2. Wait for the producer to finish writing: use a completion signal or await() on the channel instead of polling read().
3. Inspect the producer side for stalled/crashed tasks if the frame never arrives.
4. Enable debug logging on ReadableByteChunksFrameChannel to trace bytesAdded/bytesBuffered progress.

Example fix

// before
RowsAndColumns rac = channel.read();
// after
if (channel.readyToRead()) {
  RowsAndColumns rac = channel.read();
} else {
  // wait for writer or idle() await
}

if (!channel.readyToRead()) { /* await writer or idle */ } else { RowsAndColumns rac = channel.read(); }

boolean isReady(ReadableByteChunksFrameChannel ch) { return ch.readyToRead(); }

try { rac = channel.read(); } catch (ISE e) { if (e.getMessage().contains("not yet ready")) { /* defer read */ } else { throw e; } }

When it happens

Trigger: Calling read() before addChunk() has delivered all bytes of the current frame, i.e. bytesBuffered < FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength after a length was announced but not fully received.

Common situations: MSQ/stage-exchange consumers polling a channel whose producer task is still writing or has stalled; race between producer completion and consumer read; consumer error handling that ignores readyToRead() checks.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Could not create group mapping [%s] due to concurrent update
• Could not delete group mapping [%s] due to concurrent update
• Could not create role [%s] due to concurrent update contenti
• Could not delete role [%s] due to concurrent update contenti
• Could not assign role [%s] to user [%s] due to concurrent up

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b94d46ea73a6ecf6. Report an issue: GitHub.