apache/druid · error · ColumnCapacityExceededException

Column capacity exceeded

Error message

Column capacity exceeded

What it means

Thrown by EntireLayoutColumnarFloatsSerializer.add when numInserted overflows past Integer.MAX_VALUE. Float columns are indexed by int, so a serializer can never hold more than 2^31 - 1 values; adding more would corrupt offsets. ColumnCapacityExceededException aborts the write instead of producing an unreadable segment.

Source

Thrown at processing/src/main/java/org/apache/druid/segment/data/EntireLayoutColumnarFloatsSerializer.java:79

  @Override
  public int size()
  {
    return numInserted;
  }

  @Override
  public void add(float value) throws IOException
  {
    int valueBits = Float.floatToRawIntBits(value);
    // WriteOutBytes are always big-endian, so need to reverse bytes
    if (isLittleEndian) {
      valueBits = Integer.reverseBytes(valueBits);
    }
    valuesOut.writeInt(valueBits);
    ++numInserted;
    if (numInserted < 0) {
      throw new ColumnCapacityExceededException(columnName);
    }
  }

  @Override
  public long getSerializedSize()
  {
    return META_SERDE_HELPER.size(this) + valuesOut.size();
  }

  @Override
  public void writeTo(WritableByteChannel channel, SegmentFileBuilder fileBuilder) throws IOException
  {
    META_SERDE_HELPER.writeTo(channel, this);
    valuesOut.writeTo(channel);
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Lower maxRowsPerSegment / maxTotalRows so segments stay well below 2^31 rows
2. Enable dynamic or hash partitioning to split the data across segments
3. Reduce maxRowsInMemory so intermediate persists happen before the limit
4. If unavoidable, split the data source across multiple ingestion tasks

Example fix

// before
tuningConfig.maxRowsInMemory = Integer.MAX_VALUE;
// after
tuningConfig.maxRowsInMemory = 1000000; // persist before capacity overflow

if (rowsInSegment >= Integer.MAX_VALUE) {
  throw new IllegalStateException("Float column exceeds max int-indexed capacity");
}

try {
  serializer.add(value);
} catch (ColumnCapacityExceededException e) {
  throw new RuntimeException("Float column capacity exceeded; repartition data", e);
}

When it happens

Trigger: Calling add(float) after Integer.MAX_VALUE inserts into a single EntireLayoutColumnarFloatsSerializer, detected when numInserted wraps negative.

Common situations: Unbounded ingestion building one giant segment (missing or too-large partitioning settings); very long-running tasks appending to one column; import jobs that bypass row-count limits.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Column capacity exceeded
• Column capacity exceeded
• Column capacity exceeded
• Column capacity exceeded
• Column capacity exceeded

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d336f794a3d13546. Report an issue: GitHub.