{"record":{"id":"1551fa664ef66eb2","repo":"apache/druid","slug":"no-value-split-found-with-filesizelimit-d-avgo","errorCode":null,"errorMessage":"no value split found with fileSizeLimit [%d], avgObjectSize [%d]","messagePattern":"no value split found with fileSizeLimit \\[(.+?)\\], avgObjectSize \\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/data/GenericIndexedWriter.java","lineNumber":453,"sourceCode":"  /**\n   * Tries to get best value split(number of elements in each value file) which can be expressed as power of 2.\n   *\n   * @return Returns the size of value file splits as power of 2.\n   *\n   * @throws IOException\n   */\n  private int bagSizePower()\n  {\n    long avgObjectSize = (valuesOut.size() + numWritten - 1) / numWritten;\n\n    for (int i = 31; i >= 0; --i) {\n      if ((1L << i) * avgObjectSize <= fileSizeLimit) {\n        if (actuallyFits(i)) {\n          return i;\n        }\n      }\n    }\n    throw new ISE(\n        \"no value split found with fileSizeLimit [%d], avgObjectSize [%d]\",\n        fileSizeLimit,\n        avgObjectSize\n    );\n  }\n\n  /**\n   * Checks if candidate value splits can divide value file in such a way no object/element crosses the value splits.\n   *\n   * @param powerTwo candidate value split expressed as power of 2.\n   *\n   * @return true if candidate value split can hold all splits.\n   *\n   * @throws IOException\n   */\n  private boolean actuallyFits(int powerTwo)\n  {\n    long lastValueOffset = 0;","sourceCodeStart":435,"sourceCodeEnd":471,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/data/GenericIndexedWriter.java#L435-L471","documentation":"GenericIndexedWriter converts generic-indexed data into files by choosing a power-of-two 'bags per file' split so each file stays under fileSizeLimit. bagSizePower() iterates candidate splits and verifies each actually fits; if no split satisfies the limit it throws ISE. In practice this means a single object is larger than the file size limit, so it can never fit.","triggerScenarios":"writeToMultiFiles when avgObjectSize is so large that even (1 << 0) * avgObjectSize > fileSizeLimit, or actuallyFits(i) fails for every candidate — i.e. one serialized value exceeds maxFileSize.","commonSituations":"Very large individual dimension values (huge strings/blobs) combined with a small druid.segment.cache.initialSize / maxBytesPerFileBytes tuning, or misconfigured maxBytesPerFile smaller than any single value.","solutions":["Increase the segment max file size config (e.g. druid.segment.progressPeriod/maxBytesPerFileBytes) so the largest value fits.","Reduce the size of individual values (truncate, hash, or split oversized dimension values before ingestion).","Verify serialization isn't inflating a single object unexpectedly (e.g. accidental concatenation of all values into one)."],"exampleFix":"// before\nmaxBytesPerFileBytes = 10_000; // single value is 50 KB\n// after\nmaxBytesPerFileBytes = Math.max(maxBytesPerFileBytes, largestSerializedValueSize);","handlingStrategy":"validation","validationCode":"long largest = maxSizeOfSerializedValue;\nif (largest > maxBytesPerFileBytes) {\n  throw new IllegalArgumentException(\"maxBytesPerFileBytes (\" + maxBytesPerFileBytes + \") must exceed largest value (\" + largest + \")\");\n}","typeGuard":null,"tryCatchPattern":"try { writer.writeToMultiFiles(...); } catch (ISE e) {\n  if (e.getMessage().startsWith(\"no value split found\")) { /* raise maxBytesPerFileBytes and retry */ }\n  throw e;\n}","preventionTips":["Set maxBytesPerFileBytes comfortably above the largest serialized value you ingest","Reject or truncate oversized dimension values at ingestion time","Re-estimate file-size tuning after schema changes that widen values"],"tags":["java","segment-writer","file-size"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}