{"record":{"id":"9345a2945e3c57fa","repo":"apache/druid","slug":"values-must-be-sorted-and-unique-element-s-wit-9345a2","errorCode":null,"errorMessage":"Values must be sorted and unique. Element [%s] with value [%s] is before or equivalent to [%s]","messagePattern":"Values must be sorted and unique\\. Element \\[(.+?)\\] with value \\[(.+?)\\] is before or equivalent to \\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/data/FrontCodedIntArrayIndexedWriter.java","lineNumber":119,"sourceCode":"    this.byteOrder = byteOrder;\n    this.bucketBuffer = new int[bucketSize][];\n    this.getOffsetBuffer = ByteBuffer.allocate(Integer.BYTES).order(byteOrder);\n    this.div = Integer.numberOfTrailingZeros(bucketSize);\n  }\n\n  @Override\n  public void open() throws IOException\n  {\n    headerOut = segmentWriteOutMedium.makeWriteOutBytes();\n    valuesOut = segmentWriteOutMedium.makeWriteOutBytes();\n  }\n\n  @Override\n  public int write(@Nullable int[] value) throws IOException\n  {\n\n    if (prevObject != null && ARRAY_COMPARATOR.compare(prevObject, value) >= 0) {\n      throw new ISE(\n          \"Values must be sorted and unique. Element [%s] with value [%s] is before or equivalent to [%s]\",\n          numWritten,\n          value == null ? null : Arrays.toString(value),\n          Arrays.toString(prevObject)\n      );\n    }\n\n    if (value == null) {\n      if (numWritten != 0) {\n        throw DruidException.defensive(\"Null must come first, got it at numWritten[%,d]!=0\", numWritten);\n      }\n      hasNulls = true;\n      return 0;\n    }\n\n    // if the bucket buffer is full, write the bucket\n    if (numWritten > 0 && (numWritten % bucketSize) == 0) {\n      resetScratch();","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/data/FrontCodedIntArrayIndexedWriter.java#L101-L137","documentation":"FrontCodedIntArrayIndexedWriter.write() enforces that int[] values are supplied in strictly ascending, unique order (compared with ARRAY_COMPARATOR). Front coding relies on lexicographic prefix sharing between consecutive sorted values; writing a value that is <= the previous one would corrupt the encoding, so it throws ISE. Null is allowed only before the first non-null value.","triggerScenarios":"Calling write(int[]) with an array that compares equal to or sorts before the previously written value; interleaving writers; re-writing the same value; or a sorter upstream that emits duplicates.","commonSituations":"Dictionary/dimension value encoding where the input column was not sorted before encoding, duplicate dimension values slipping past a dedup step, or using the writer directly on unsorted data.","solutions":["Sort and deduplicate all int[] values lexicographically before writing them.","If the writer receives data from a stream, buffer and sort first, or assert ascending order before calling write().","Check that only one writer instance writes sequentially and prevObject state is not bypassed."],"exampleFix":"// before\nfor (int[] v : values) writer.write(v);\n// after\nArrays.sort(values, FrontCodedIntArrayIndexedWriter.ARRAY_COMPARATOR);\nfor (int i = 0; i < values.length; i++) {\n  if (i > 0 && ARRAY_COMPARATOR.compare(values[i-1], values[i]) == 0) continue;\n  writer.write(values[i]);\n}","handlingStrategy":"validation","validationCode":"for (int i = 1; i < values.length; i++) {\n  if (FrontCodedIntArrayIndexedWriter.ARRAY_COMPARATOR.compare(values[i-1], values[i]) >= 0)\n    throw new IllegalStateException(\"values not strictly increasing at \" + i);\n}","typeGuard":null,"tryCatchPattern":"try { writer.write(value); } catch (IllegalStateException e) { throw new IOException(\"unsorted values supplied to FrontCodedIntArrayIndexedWriter\", e); }","preventionTips":["Sort and deduplicate values before feeding any front-coded writer","Use a sorted-set/sorted stream upstream so ordering is guaranteed","Never reuse a writer after writing a larger value"],"tags":["java","segment-writer","ordering"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}