{"record":{"id":"d336f794a3d13546","repo":"apache/druid","slug":"column-capacity-exceeded-d336f7","errorCode":null,"errorMessage":"Column capacity exceeded","messagePattern":"Column capacity exceeded","errorType":"exception","errorClass":"ColumnCapacityExceededException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/data/EntireLayoutColumnarFloatsSerializer.java","lineNumber":79,"sourceCode":"\n  @Override\n  public int size()\n  {\n    return numInserted;\n  }\n\n  @Override\n  public void add(float value) throws IOException\n  {\n    int valueBits = Float.floatToRawIntBits(value);\n    // WriteOutBytes are always big-endian, so need to reverse bytes\n    if (isLittleEndian) {\n      valueBits = Integer.reverseBytes(valueBits);\n    }\n    valuesOut.writeInt(valueBits);\n    ++numInserted;\n    if (numInserted < 0) {\n      throw new ColumnCapacityExceededException(columnName);\n    }\n  }\n\n  @Override\n  public long getSerializedSize()\n  {\n    return META_SERDE_HELPER.size(this) + valuesOut.size();\n  }\n\n  @Override\n  public void writeTo(WritableByteChannel channel, SegmentFileBuilder fileBuilder) throws IOException\n  {\n    META_SERDE_HELPER.writeTo(channel, this);\n    valuesOut.writeTo(channel);\n  }\n}\n","sourceCodeStart":61,"sourceCodeEnd":96,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/data/EntireLayoutColumnarFloatsSerializer.java#L61-L96","documentation":"Thrown by EntireLayoutColumnarFloatsSerializer.add when numInserted overflows past Integer.MAX_VALUE. Float columns are indexed by int, so a serializer can never hold more than 2^31 - 1 values; adding more would corrupt offsets. ColumnCapacityExceededException aborts the write instead of producing an unreadable segment.","triggerScenarios":"Calling add(float) after Integer.MAX_VALUE inserts into a single EntireLayoutColumnarFloatsSerializer, detected when numInserted wraps negative.","commonSituations":"Unbounded ingestion building one giant segment (missing or too-large partitioning settings); very long-running tasks appending to one column; import jobs that bypass row-count limits.","solutions":["Lower maxRowsPerSegment / maxTotalRows so segments stay well below 2^31 rows","Enable dynamic or hash partitioning to split the data across segments","Reduce maxRowsInMemory so intermediate persists happen before the limit","If unavoidable, split the data source across multiple ingestion tasks"],"exampleFix":"// before\ntuningConfig.maxRowsInMemory = Integer.MAX_VALUE;\n// after\ntuningConfig.maxRowsInMemory = 1000000; // persist before capacity overflow","handlingStrategy":"validation","validationCode":"if (rowsInSegment >= Integer.MAX_VALUE) {\n  throw new IllegalStateException(\"Float column exceeds max int-indexed capacity\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  serializer.add(value);\n} catch (ColumnCapacityExceededException e) {\n  throw new RuntimeException(\"Float column capacity exceeded; repartition data\", e);\n}","preventionTips":["Partition input data so segments stay far below 2^31 rows","Lower maxRowsInMemory to trigger intermediate persists","Watch segment row metrics during long-running ingestion"],"tags":["capacity","segment","serialization"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}