apache/druid · error · IllegalStateException

Could not write [ ] bytes into smooshChannel.

Error message

Could not write [%d] bytes into smooshChannel.

What it means

writeBytesIntoSmooshedChannel streams a serialized GenericIndexed from an in-memory holder into a smooshed file channel and tracks remaining bytes. If the input stream reaches EOF before all expected bytes are written, it throws ISE, meaning the serialized data was truncated relative to its recorded size.

Solutions

  1. Check for disk space/full-disk errors on the segment store and retry the persist/merge.
  2. Verify the serialized holder size matches numBytesWritten/declared sizes; fix size accounting if custom code writes the holder.
  3. Retry the operation; if reproducible on valid inputs, report as a bug with the failing segment.
Defensive patterns

Strategy: try-catch

Validate before calling

// confirm holder size matches declared byte count before streaming
long expected = numBytesWritten;
long available = holderInputStream.available(); // heuristic pre-check

Try / catch

try { writer.writeToMultiFiles(...); } catch (ISE e) {
  if (e.getMessage().contains("smooshChannel")) { /* retry persist; check disk space */ }
  throw e;
}

Prevention

When it happens

Trigger: writeToMultiFiles / writeBytesIntoSmooshedChannel when is.read() returns -1 while numBytesToPutInFile > 0, i.e. the holder's data ended sooner than the declared byte count.

Common situations: Disk-full or partial-write during segment persist, a bug producing inconsistent size metadata, or corrupted intermediate files during merge.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a2200849728fd861. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/data/GenericIndexedWriter.java:228

    return StringUtils.format("%s_header", fileNameBase);
  }

  private static void writeBytesIntoSmooshedChannel(
      long numBytesToPutInFile,
      final byte[] buffer,
      final SegmentFileChannel smooshChannel,
      final InputStream is
  )
      throws IOException
  {
    ByteBuffer holderBuffer = ByteBuffer.wrap(buffer);
    while (numBytesToPutInFile > 0) {
      int bytesRead = is.read(buffer, 0, Math.min(buffer.length, Ints.saturatedCast(numBytesToPutInFile)));
      if (bytesRead != -1) {
        smooshChannel.write((ByteBuffer) holderBuffer.clear().limit(bytesRead));
        numBytesToPutInFile -= bytesRead;
      } else {
        throw new ISE("Could not write [%d] bytes into smooshChannel.", numBytesToPutInFile);
      }
    }
  }

  @Override
  public void open() throws IOException
  {
    headerOut = segmentWriteOutMedium.makeWriteOutBytes();
    valuesOut = segmentWriteOutMedium.makeWriteOutBytes();
  }

  public void setObjectsNotSorted()
  {
    objectsSorted = false;
  }

  @Override
  public boolean isSorted()

View on GitHub (pinned to 9b90983fd2)