{"record":{"id":"ccd330adf28467d1","repo":"apache/druid","slug":"column-capacity-exceeded-ccd330","errorCode":null,"errorMessage":"Column capacity exceeded","messagePattern":"Column capacity exceeded","errorType":"exception","errorClass":"ColumnCapacityExceededException","httpStatus":null,"severity":"critical","filePath":"processing/src/main/java/org/apache/druid/segment/data/CompressedVSizeColumnarIntsSerializer.java","lineNumber":167,"sourceCode":"  public void addValue(int val) throws IOException\n  {\n    if (endBuffer == null) {\n      throw new IllegalStateException(\"written out already\");\n    }\n    if (!endBuffer.hasRemaining()) {\n      endBuffer.rewind();\n      flattener.write(endBuffer);\n      endBuffer.clear();\n    }\n    intBuffer.putInt(0, val);\n    if (isBigEndian) {\n      endBuffer.put(intBuffer.array(), Integer.BYTES - numBytes, numBytes);\n    } else {\n      endBuffer.put(intBuffer.array(), 0, numBytes);\n    }\n    numInserted++;\n    if (numInserted < 0) {\n      throw new ColumnCapacityExceededException(columnName);\n    }\n  }\n\n  @Override\n  public long getSerializedSize() throws IOException\n  {\n    writeEndBuffer();\n    return META_SERDE_HELPER.size(this) + flattener.getSerializedSize();\n  }\n\n  @Override\n  public void writeTo(WritableByteChannel channel, SegmentFileBuilder fileBuilder) throws IOException\n  {\n    writeEndBuffer();\n    META_SERDE_HELPER.writeTo(channel, this);\n    flattener.writeTo(channel, fileBuilder);\n  }\n","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/data/CompressedVSizeColumnarIntsSerializer.java#L149-L185","documentation":"CompressedVSizeColumnarIntsSerializer increments numInserted per addValue call and throws ColumnCapacityExceededException when the int counter overflows (numInserted < 0), i.e. more than Integer.MAX_VALUE values were inserted. Druid columns cannot exceed ~2.1 billion entries.","triggerScenarios":"Adding more than Integer.MAX_VALUE values to a single CompressedVSizeColumnarIntsSerializer - typically a column with more than 2^31-1 rows (or values for multi-value rows) in one segment.","commonSituations":"Oversized segments from coarse partitioning; huge multi-value dimensions whose total value count exceeds the limit even when row count does not; ingestion tasks without maxTotalRows bounds.","solutions":["Configure partitionsSpec/maxTotalRows so segments stay far below 2^31 rows.","For multi-value dimensions, reduce the number of values per row or shard the data.","Split the workload across more ingestion tasks/segments."],"exampleFix":"// before: dynamic partitioning with no maxTotalRows\n// after\n\"partitionsSpec\": { \"type\": \"hashed\", \"maxTotalRows\": 5000000 }","handlingStrategy":"validation","validationCode":"// Java\nif ((long) numExpectedValues >= Integer.MAX_VALUE) {\n  throw new IllegalArgumentException(\"Column would exceed 2^31 values; re-partition\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  serializer.addValue(val);\n} catch (ColumnCapacityExceededException e) {\n  throw new ISE(\"Column capacity exceeded; split segment\", e);\n}","preventionTips":["Bound segments with maxTotalRows in partitionsSpec.","For multi-value dimensions, estimate total value count, not just row count.","Alert on segments approaching hundreds of millions of rows."],"tags":["capacity","overflow","segment-limits"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}