apache/druid · error · IllegalStateException

Unexpected marker type

Error message

Unexpected marker type[%d]

What it means

After the stream header, each frame begins with a marker byte (frame data or no-more-frames). The switch over nextMarkerType hit a value the deserializer does not understand, so the reader throws. This means stream state desynchronized or the marker byte was corrupted.

Solutions

1. Regenerate the frame data (rerun the stage/query) to rule out transient corruption.
2. Confirm writer and reader use the same Druid version/frame format.
3. Check disk/filesystem health where frame files are spooled.
4. Report the marker byte value and channel id with logs if persistent.

try { rac = channel.read(); } catch (ISE e) { if (e.getMessage().startsWith("Unexpected marker type")) { failStage("stream desync/corruption", e); } else { throw e; } }

When it happens

Trigger: markerType read from the stream is neither MARKER_FRAME nor MARKER_NO_MORE_FRAMES (the switch default) when decoding a frame in nextRAC.

Common situations: Corrupted frame file or transport; reading a stream whose format differs between writer and reader versions; earlier decoding bug shifted the byte position.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• Cannot read frame of size [%,d] bytes
• Invalid frame size (size = %,d B)
• Invalid midstream marker
• Invalid stream header
• Cannot copy [%,d] bytes, only have [%,d] buffered

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/366d6e62ed84f698. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java:391

    final byte[] decompressedBytes =
        FrameCompression.decompress(compressedMemory, 0, compressedMemory.getCapacity());

    switch (markerType) {
      case FrameFileWriter.MARKER_FRAME:
        final Frame frame = Frame.wrap(decompressedBytes);
        log.debug("Read frame with rows[%,d] and bytes[%,d].", frame.numRows(), frame.numBytes());
        return frame.asRAC();

      case FrameFileWriter.MARKER_RAC:
        if (wtContext == null) {
          throw DruidException.defensive("Cannot read RAC, no WireTransferableContext");
        }
        final RowsAndColumns rac = wtContext.deserialize(ByteBufferUtils.wrapLE(decompressedBytes));
        log.debug("Read RAC with rows[%,d] and bytes[%,d].", rac.numRows(), decompressedBytes.length);
        return rac;

      default:
        throw new ISE("Unexpected marker type[%d]", markerType);
    }
  }

  @GuardedBy("lock")
  private void updateStreamState()
  {
    if (streamPart == StreamPart.MAGIC) {
      if (bytesBuffered >= FrameFileWriter.MAGIC.length) {
        final Memory memory = copyFromQueuedChunks(FrameFileWriter.MAGIC.length);

        if (memory.equalTo(0, Memory.wrap(FrameFileWriter.MAGIC), 0, FrameFileWriter.MAGIC.length)) {
          streamPart = StreamPart.FRAMES;
          deleteFromQueuedChunks(FrameFileWriter.MAGIC.length);
        } else {
          throw new ISE("Invalid stream header (id = %s, position = %d)", id, bytesAdded - bytesBuffered);
        }
      }
    }

View on GitHub (pinned to 9b90983fd2)