{"record":{"id":"b84b598e9b70771a","repo":"apache/druid","slug":"column-capacity-exceeded-b84b59","errorCode":null,"errorMessage":"Column capacity exceeded","messagePattern":"Column capacity exceeded","errorType":"exception","errorClass":"ColumnCapacityExceededException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/data/BlockLayoutColumnarFloatsSerializer.java","lineNumber":104,"sourceCode":"  {\n    return numInserted;\n  }\n\n  @Override\n  public void add(float value) throws IOException\n  {\n    if (endBuffer == null) {\n      throw new IllegalStateException(\"written out already\");\n    }\n    if (!endBuffer.hasRemaining()) {\n      endBuffer.rewind();\n      flattener.write(endBuffer);\n      endBuffer.clear();\n    }\n    endBuffer.putFloat(value);\n    ++numInserted;\n    if (numInserted < 0) {\n      throw new ColumnCapacityExceededException(columnName);\n    }\n  }\n\n  @Override\n  public long getSerializedSize() throws IOException\n  {\n    writeEndBuffer();\n    return META_SERDE_HELPER.size(this) + flattener.getSerializedSize();\n  }\n\n  @Override\n  public void writeTo(WritableByteChannel channel, SegmentFileBuilder fileBuilder) throws IOException\n  {\n    writeEndBuffer();\n    META_SERDE_HELPER.writeTo(channel, this);\n    flattener.writeTo(channel, fileBuilder);\n  }\n","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/data/BlockLayoutColumnarFloatsSerializer.java#L86-L122","documentation":"ColumnCapacityExceededException is thrown by BlockLayoutColumnarFloatsSerializer.add(float) when the number of inserted rows (numInserted, an int) overflows past Integer.MAX_VALUE. At that point the column's offset encoding cannot address further rows, so Druid aborts to avoid a corrupt segment.","triggerScenarios":"Inserting more than 2^31-1 float values into a single float column of one segment via add(float).","commonSituations":"Oversized segments in batch ingestion (no maxRowsPerSegment), streaming tasks running far beyond intended interval/row limits, re-indexing very wide inputs without partitioning.","solutions":["Reduce maxRowsPerSegment in the partitionsSpec so segments are cut before 2^31 rows","Use finer segmentGranularity or hash partitioning to distribute rows across segments","Enable roll-up where appropriate to reduce per-segment row counts","Review task failure logs for which column overflowed and adjust spec accordingly"],"exampleFix":"// before\n\"partitionsSpec\": { \"type\": \"dynamic\", \"maxRowsPerSegment\": 10000000000 }\n// after\n\"partitionsSpec\": { \"type\": \"dynamic\", \"maxRowsPerSegment\": 5000000 }","handlingStrategy":"validation","validationCode":"if (numRows >= Integer.MAX_VALUE) {\n  throw new IllegalArgumentException(\"Would exceed float column capacity; cut segment\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  serializer.add(value);\n} catch (ColumnCapacityExceededException e) {\n  logger.warn(\"Column %s exceeded capacity; rolling segment\", e.getColumnName());\n  segmentRoller.roll();\n}","preventionTips":["Cap maxRowsPerSegment in partitionsSpec","Partition data across segments","Monitor row counts per task","Use roll-up where semantics allow"],"tags":["druid","segment","ingestion","capacity","overflow"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}