{"record":{"id":"36ecc27b5a8bf79b","repo":"apache/druid","slug":"column-capacity-exceeded-36ecc2","errorCode":null,"errorMessage":"Column capacity exceeded","messagePattern":"Column capacity exceeded","errorType":"exception","errorClass":"ColumnCapacityExceededException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/data/BlockLayoutColumnarLongsSerializer.java","lineNumber":116,"sourceCode":"  @Override\n  public void add(long value) throws IOException\n  {\n    if (endBuffer == null) {\n      throw new IllegalStateException(\"written out already\");\n    }\n    if (numInserted == numInsertedForNextFlush) {\n      numInsertedForNextFlush += sizePer;\n      writer.flush();\n      endBuffer.flip();\n      flattener.write(endBuffer);\n      endBuffer.clear();\n      writer.setBuffer(endBuffer);\n    }\n\n    writer.write(value);\n    ++numInserted;\n    if (numInserted < 0) {\n      throw new ColumnCapacityExceededException(columnName);\n    }\n  }\n\n  @Override\n  public long getSerializedSize() throws IOException\n  {\n    writeEndBuffer();\n    return META_SERDE_HELPER.size(this) + flattener.getSerializedSize();\n  }\n\n  @Override\n  public void writeTo(WritableByteChannel channel, SegmentFileBuilder fileBuilder) throws IOException\n  {\n    writeEndBuffer();\n    META_SERDE_HELPER.writeTo(channel, this);\n    flattener.writeTo(channel, fileBuilder);\n  }\n","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/data/BlockLayoutColumnarLongsSerializer.java#L98-L134","documentation":"ColumnCapacityExceededException from BlockLayoutColumnarLongsSerializer.add(long) fires when numInserted overflows Integer.MAX_VALUE. Long columns are row-addressed by int offsets, so beyond 2^31-1 rows Druid cannot encode more values and aborts segment creation to prevent corruption.","triggerScenarios":"Inserting more than 2^31-1 rows into a single long-typed column of one segment via add(long).","commonSituations":"Very high-volume ingestion without row-count partitioning; metric columns in billion-row segments; task retries that keep appending to the same serializer.","solutions":["Set maxRowsPerSegment well below Integer.MAX_VALUE in partitionsSpec","Partition by time or hash so no single segment exceeds ~2.1B rows","Use roll-up/aggregation to shrink row counts","Upgrade Druid if a newer segment format lifts the limit"],"exampleFix":"// before\n// unbounded dynamic partitioning\n\"partitionsSpec\": { \"type\": \"dynamic\" }\n// after\n\"partitionsSpec\": { \"type\": \"dynamic\", \"maxRowsPerSegment\": 5000000 }","handlingStrategy":"validation","validationCode":"if (numRows >= Integer.MAX_VALUE) {\n  throw new IllegalArgumentException(\"Would exceed long column capacity; cut segment\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  serializer.add(value);\n} catch (ColumnCapacityExceededException e) {\n  // cut the segment and restart ingestion at the failed row\n  cutSegmentAndRetry(e.getColumnName());\n}","preventionTips":["Keep segments far below 2^31 rows via partitionsSpec","Use hash/time partitioning","Enable roll-up for high-cardinality-free aggregates","Alert on segment row-count metrics"],"tags":["druid","segment","ingestion","capacity","overflow"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}