apache/druid · error · IllegalStateException

Unexpected null byte

Error message

Unexpected null byte [%s]

What it means

ComplexFieldReader.readFieldFromByteArray decodes a complex value from a byte array using a one-byte header (nullByte) that encodes null/inline/pointer layout. It throws ISE when the header byte is none of the recognized values, meaning the serialized bytes do not match the expected ComplexFieldWriter format.

Solutions

1. Verify the bytes come from a ComplexFieldWriter-produced column and position points at the start of a field (the header byte).
2. Check that writer and reader Druid versions agree on the complex field layout.
3. Log/dump the offending nullByte value to identify which unexpected marker was found.
4. Rebuild or re-ingest the affected frame/segment if the data is corrupted.

Example fix

// before
byte[] value = reader.readFieldFromByteArray(bytes, arbitraryOffset, serde);
// after
int offset = fieldPointer.position(); // must point at the ComplexFieldWriter header byte
byte[] value = reader.readFieldFromByteArray(bytes, offset, serde);

if (position < 0 || position >= bytes.length) { throw new IllegalArgumentException("position not at field start"); }

try { v = reader.readFieldFromByteArray(bytes, pos, serde); } catch (IllegalStateException e) { log.error("corrupt complex field, nullByte={}", bytes[pos]); throw e; }

When it happens

Trigger: Reading a complex field from a byte array whose first header byte is not one of the documented NULL/inline/offset markers — typically corrupted data, a file written by a different writer version, or misaligned position into the byte array.

Common situations: Frames or segments written by an incompatible Druid version; manual byte manipulation that skips the header; memory corruption or truncated file being read at a wrong offset.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• bucketSize must be a power of two but was[%,d]
• bucketSize must be a power of two but was[%,d]
• Bad null-marker byte, delegate class
• bucketSize must be a power of two (from 1 up to 128) but…
• Cannot find strategy for type

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/9165d2b2ab89b1dd. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/frame/field/ComplexFieldReader.java:124

      final ComplexMetricSerde serde,
      final byte[] bytes,
      final int position
  )
  {
    final byte nullByte = bytes[position];

    if (nullByte == ComplexFieldWriter.NULL_BYTE) {
      return null;
    } else if (nullByte == ComplexFieldWriter.NOT_NULL_BYTE) {
      // Reads length in little-endian format
      int length;
      length = (bytes[position + 4] & 0xFF) << 24;
      length |= (bytes[position + 3] & 0xFF) << 16;
      length |= (bytes[position + 2] & 0xFF) << 8;
      length |= (bytes[position + 1] & 0xFF);
      return serde.fromBytes(bytes, position + ComplexFieldWriter.HEADER_SIZE, length);
    } else {
      throw new ISE("Unexpected null byte [%s]", nullByte);
    }
  }

  /**
   * Alternative interface to read the field from the memory without creating a selector and field pointer
   */
  @Nullable
  public static <T> T readFieldFromMemory(
      final ComplexMetricSerde serde,
      final Memory memory,
      final long position
  )
  {
    final byte nullByte = memory.getByte(position);

    if (nullByte == ComplexFieldWriter.NULL_BYTE) {
      return null;
    } else if (nullByte == ComplexFieldWriter.NOT_NULL_BYTE) {

View on GitHub (pinned to 9b90983fd2)